Fair Value of Financial Instruments Level 3 (Details) - Fair Value, Measurements, Recurring - Level 3 $ in Thousands	6 Months Ended
	Jun. 30, 2016 USD ($)
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	$ 16,435
Unrealized (loss) gain included in OCI	(142)
Principal payments	(2,920)
Accretion	522
Balance at the end of the period	13,895
Non-Agency CMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	14,903
Unrealized (loss) gain included in OCI	(157)
Principal payments	(2,780)
Accretion	522
Balance at the end of the period	12,488
Non-Agency RMBS
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	1,532
Unrealized (loss) gain included in OCI	15
Principal payments	(140)
Accretion	0
Balance at the end of the period	$ 1,407