Fair Value of Financial Instruments Level 3 (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2013	Dec. 31, 2012
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		$ (4,670)
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	105,245	134,024
Purchases, Level 3 Assets	26,021	7,500
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings	(25)
Unrealized (loss) gain included in OCI	(6,021)	27
Principal payments	51,588	31,241
Amortization	(152)	(395)
Balance at the end of the period	73,480	105,245
Non-Agency CMBS [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	100,102	123,703
Purchases, Level 3 Assets	26,021	0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		(4,670)
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings	0
Unrealized (loss) gain included in OCI	(6,026)	(59)
Principal payments	49,167	18,437
Amortization	(197)	(435)
Balance at the end of the period	70,733	100,102
Non-Agency RMBS [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	5,118	10,296
Purchases, Level 3 Assets	0	7,500
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3		0
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings	0
Unrealized (loss) gain included in OCI	5	86
Principal payments	2,421	12,804
Amortization	45	40
Balance at the end of the period	2,747	5,118
Other Investments [Member] | Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at the beginning of the period	25	25
Purchases, Level 3 Assets
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Transfers out of Level 3
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset, Gain (Loss) Included in Earnings	(25)
Unrealized (loss) gain included in OCI
Principal payments
Amortization
Balance at the end of the period	$ 0	$ 25