Summary of Significant Accounting Policies (Details 2) (USD $) In Thousands, unless otherwise specified	3 Months Ended		9 Months Ended
	Sep. 30, 2013	Sep. 30, 2012	Sep. 30, 2013	Sep. 30, 2012	Dec. 31, 2012
Consolidated Balance Sheet
Investment securities	$ 566,793		$ 566,793		$ 884,254
Residential real estate, net	548,022		548,022		99,115
Non-performing residential loans	197,716		197,716		68,883
Investment in unconsolidated entities	138,168		138,168		32,318
Accounts payable, accrued expenses and other liabilities combined from Other liabilities	189,172		189,172		30,094
Condensed Consolidated Statements of Operations
Other revenues	2,098	66	4,077	180
Earnings from unconsolidated entities	4,577	787	10,915	2,739
Rental income	5,080	59	8,733	59
Residential segment, other operating costs	6,023	327	10,622	327
Depreciation and amortization	4,988	78	8,644	78
Other income	3,705	180	3,744	530
Loss on derivative financial instruments	(22,451)	(7,561)	(65)	(9,784)
Unrealized foreign currency remeasurement (losses) gains			3,481	11,516
Foreign currency gain, net	9,580	6,725	3,495	11,222
Condensed Consolidated Statements of Cash Flows
Gain on sale of investments				(19,274)
Change in fair value of derivatives			(2,196)	3,328
Unrealized foreign currency remeasurement (losses) gains			3,481	11,516
Origination and purchase of loans held-for-investment			(1,658,240)	(943,330)
Proceeds from principal collections on loans			394,616	494,307
Amount As Previously Reported
Consolidated Balance Sheet
Mortgage-backed securities, available-for-sale, at fair value					862,587
Other investments					221,983
Accounts payable and accrued expenses					8,890
Other liabilities					21,204
Condensed Consolidated Statements of Operations
Interest income from cash balances		66		180
Other income		621		2,923
Net gains (losses) on currency hedges		(7,510)		(10,392)
Net gains (losses) on interest rate hedges		(51)		608
Net realized foreign currency gains (losses)		(337)		8,515
Unrealized foreign currency remeasurement (losses) gains		7,062		2,707
Condensed Consolidated Statements of Cash Flows
Gain on sale of available-for-sale securities				(12,097)
Gain on sale of loans				(7,177)
Unrealized gains (losses) on interest rate hedges				(9,991)
Unrealized losses on currency hedges				13,319
Gain on foreign currency remeasurement				(8,809)
Unrealized foreign currency remeasurement (losses) gains		7,062		2,707
Purchased interest on investments				(638)
Origination and purchase of loans held-for-investment				(942,692)
Loan maturities				460,789
Loan investment principal repayments				33,518
Reclassification Adjustment
Consolidated Balance Sheet
Mortgage-backed securities, available-for-sale, at fair value					(862,587)
Investment securities reclassified from Mortgage-backed securities, available-for-sale, at fair value					862,587
Investment securities reclassified from Other investments					21,667
Other investments reclassified to Investment securities					(21,667)
Other investments reclassified to residential real estate and non-performing residential loans					(167,998)
Other investments reclassified to Investments in unconsolidated entities					32,318
Residential real estate, net					99,115
Non-performing residential loans					68,883
Investment in unconsolidated entities					32,318
Accounts payable, accrued expenses and other liabilities combined from Accounts payable and accrued expenses					21,204
Accounts payable, accrued expenses and other liabilities combined from Other liabilities					8,890
Accounts payable and accrued expenses					(8,890)
Other liabilities					(21,204)
Condensed Consolidated Statements of Operations
Other revenues		66		180
Interest income from cash balances		(66)		(180)
Earnings from unconsolidated entities		787		2,739
Rental income		59		59
Residential segment, other operating costs		327		327
Depreciation and amortization		78		78
Other income		(441)
Net gains (losses) on currency hedges		7,510		10,392
Net gains (losses) on interest rate hedges		51		(608)
Loss on derivative financial instruments		(7,561)		(9,784)
Net realized foreign currency gains (losses)		337		(8,515)
Unrealized foreign currency remeasurement (losses) gains		(7,062)		(2,707)
Foreign currency gain, net		6,725		11,222
Condensed Consolidated Statements of Cash Flows
Gain on sale of available-for-sale securities				12,097
Gain on sale of loans				7,177
Gain on sale of investments				(19,274)
Unrealized gains (losses) on interest rate hedges				9,991
Unrealized losses on currency hedges				(13,319)
Change in fair value of derivatives				3,328
Gain on foreign currency remeasurement				8,809
Unrealized foreign currency remeasurement (losses) gains		(7,062)		(2,707)
Purchased interest on investments				638
Origination and purchase of loans held-for-investment				(638)
Loan maturities				(460,789)
Loan investment principal repayments				(33,518)
Proceeds from principal collections on loans				$ 494,307