Fair Value of Financial Instruments Level 3 (Details) - Non-Agency MBS - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	$ 2,274	$ 7,243
Unrealized gain included in OCI	117	(1,850)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Settlements	(1,958)	(5,071)
Balance at the end of the period	1,690	2,274
Interest Income
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Accretion	$ 1,257	$ 1,952