Fair Value of Financial Instruments Level 3 (Details) - Fair Value, Measurements, Recurring $ in Thousands	3 Months Ended
	Mar. 31, 2017 USD ($)
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	$ 10,927
Unrealized (loss) gain included in OCI	(380)	[1]
Principal payments	(427)
Accretion	382
Balance at the end of the period	10,502
Non-Agency CMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	9,669
Unrealized (loss) gain included in OCI	(377)	[1]
Principal payments	(383)
Accretion	382
Balance at the end of the period	9,291
Non-Agency RMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	1,258
Unrealized (loss) gain included in OCI	(3)	[1]
Principal payments	(44)
Accretion	0
Balance at the end of the period	$ 1,211

[1] Amount included in "unrealized gain on available-for-sale investments, net" on consolidated statements of comprehensive income (loss).