Fair Value of Financial Instruments Level 3 (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2020 USD ($)
Non-Agency MBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	$ 1,690
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Other Comprehensive Income (Loss)	(116)	[1]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(105)
Balance at the end of the period	1,562
Loans Receivable
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	8,857	[2]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings	312	[1]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(615)
Balance at the end of the period	7,922
Interest Income | Non-Agency MBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Accretion (amortization) (3)	93	[3]
Interest Income | Loans Receivable
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Accretion (amortization) (3)	$ (8)	[3]

[1] Change in fair value for other non-Agency MBS is recorded as unrealized gain (loss) in “other comprehensive income” and change in fair value for mortgage loans is net of chargeoffs and recorded as unrealized gain (loss) in “fair value adjustments,net“

[2] See Note 1 for information regarding the Company’s election of the fair value option for its mortgage loans effective January 1, 2020 which were previously reported at amortized cost. The beginning balance shown in this table represents the fair value of the mortgage loans after the cumulative adjustment of $(548) made to the amortized cost of $9,405 as of December 31, 2019.

[3] Accretion (amortization) represents the amount of (premium) discount recognized in “interest income” during the period indicated.