Fair Value of Financial Instruments (Details) (USD $) In Thousands, unless otherwise specified			9 Months Ended																																																				3 Months Ended	9 Months Ended					3 Months Ended	9 Months Ended			3 Months Ended	9 Months Ended		3 Months Ended	9 Months Ended
	Sep. 30, 2012	Dec. 31, 2011	Sep. 30, 2012 Level 3 [Member] Non-Agency RMBS [Member]		Sep. 30, 2012 Level 3 [Member] Non-Agency CMBS [Member]		Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Derivative liabilities [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Derivative liabilities [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Derivative liabilities [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Repurchase agreements [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Repurchase agreements [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Repurchase agreements [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Repurchase agreements [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Non-recourse collateralized financing [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Non-recourse collateralized financing [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Non-recourse collateralized financing [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Non-recourse collateralized financing [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Agency MBS [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Agency MBS [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Agency MBS [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Agency MBS [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Non-Agency CMBS and CMBS IOs [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Non-Agency CMBS and CMBS IOs [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Non-Agency CMBS and CMBS IOs [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Non-Agency CMBS and CMBS IOs [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Non-Agency RMBS [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Non-Agency RMBS [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Non-Agency RMBS [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Non-Agency RMBS [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Other Investments [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Other Investments [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Other Investments [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Other Investments [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Securitized mortgage loans, net [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Securitized mortgage loans, net [Member] Carrying (Reported) Amount, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Securitized mortgage loans, net [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Dec. 31, 2011 Fair Value, Measurements, Recurring [Member] Securitized mortgage loans, net [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Estimate of Fair Value, Fair Value Disclosure [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Estimate of Fair Value, Fair Value Disclosure [Member] Agency MBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Estimate of Fair Value, Fair Value Disclosure [Member] Non-Agency CMBS and CMBS IOs [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Estimate of Fair Value, Fair Value Disclosure [Member] Non-Agency RMBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Estimate of Fair Value, Fair Value Disclosure [Member] Other Investments [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 1 [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 1 [Member] Derivative liabilities [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 1 [Member] Agency MBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 1 [Member] Non-Agency CMBS and CMBS IOs [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 1 [Member] Non-Agency RMBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 1 [Member] Other Investments [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 2 [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 2 [Member] Derivative liabilities [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 2 [Member] Agency MBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 2 [Member] Non-Agency CMBS and CMBS IOs [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 2 [Member] Non-Agency RMBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 2 [Member] Other Investments [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member]	Mar. 31, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Derivative liabilities [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Agency MBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency CMBS and CMBS IOs [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency RMBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency RMBS [Member]	Mar. 31, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency RMBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Other Investments [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Other financial instruments [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Other financial instruments [Member]	Mar. 31, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Other financial instruments [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency CMBS [Member]	Sep. 30, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency CMBS [Member]	Mar. 31, 2012 Fair Value, Measurements, Recurring [Member] Level 3 [Member] Non-Agency CMBS [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Financial Instruments, Owned, Mortgages, Mortgage-backed and Asset-backed Securities, at Fair Value	$ (586,931)	$ (421,096)																$ (3,650,672)	$ (1,965,159)	$ (3,650,672)	$ (1,965,159)	$ (572,690)	$ (405,826)	$ (572,690)	$ (405,826)	$ (14,241)	$ (15,270)	$ (14,241)	$ (15,270)										$ (3,650,672)	$ (572,690)	$ (14,241)				$ 0	$ 0	$ 0				$ (3,650,672)	$ (472,374)	$ (5,640)						$ 0	$ (100,316)	$ (8,601)	$ (8,601)
Other investments																														896	1,018	842	892									25						0						0										25
Assets, Fair Value Disclosure																																						4,237,628					0						4,128,686						108,942	108,942
Derivative liabilities	46,496	27,997					27,997	46,496	27,997																																			0						46,496								0
Liabilities, Fair Value Disclosure																																						46,496					0						46,496						0	0
Securitized mortgage loans, net	77,748	113,703																																77,748	113,703	66,879	101,116
Repurchase agreements										3,670,972	2,093,793	3,670,972	2,093,793
Non recourse collateralized financing														31,295	70,895	31,013	69,752
Fair Value Inputs, Quantitative Information [Abstract]
Fair Value Inputs, Prepayment Rate			5.00%	[1]	20.00%	[1]
Fair Value Inputs, Probability of Default			0.50%	[1]	2.50%	[1]
Fair Value Inputs, Loss Severity			9.00%	[1]	35.00%	[1]
Fair Value Inputs, Discount Rate			5.10%	[1]	4.40%	[1]
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance as of June 30, 2012																																																								134,024	117,197					10,296	12,657			25	25		123,703	104,515
Purchases																																																								7,500						7,500				0			0
Transfers out to Level 2																																																								(4,670)						0				0			(4,670)
Included in other comprehensive income																																																							131	(1,817)					41	117			0	0		90	(1,934)
Principal payments																																																							(8,302)	(25,758)					(4,108)	(9,349)			0	0		(4,194)	(16,409)
Amortization																																																							(84)	(337)					11	37			0	0		(95)	(374)
Balance as of September 30, 2012																																																							$ 108,942	$ 108,942	$ 117,197				$ 8,601	$ 8,601	$ 12,657		$ 25	$ 25	$ 25	$ 100,316	$ 100,316	$ 104,515

[1] Data presented are weighted averages.