Fair Value of Financial Instruments Level 3 (Details) - Fair Value, Measurements, Recurring $ in Thousands	9 Months Ended
	Sep. 30, 2017 USD ($)
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	$ 10,927
Unrealized (loss) gain included in OCI	(1,406)	[1]
Principal payments	(4,029)
Accretion	2,075
Balance at the end of the period	7,567
Non-Agency CMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	9,669
Unrealized (loss) gain included in OCI	(1,422)	[1]
Principal payments	(3,896)
Accretion	2,075
Balance at the end of the period	6,426
Non-Agency RMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	1,258
Unrealized (loss) gain included in OCI	16	[1]
Principal payments	(133)
Accretion	0
Balance at the end of the period	$ 1,141

[1] Amount included in "unrealized gain on available-for-sale investments, net" on consolidated statements of comprehensive income (loss).