Fair Value of Financial Instruments - Valuation for Level 3 Assets (Details) - USD ($) $ in Thousands		1 Months Ended	12 Months Ended
		Feb. 28, 2015	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period			$ 8,442,604	$ 8,209,702	$ 5,514,065
Total gains (realized/unrealized)
Included in earnings	[1]		(90,662)	384,826	(470,334)
Included in other comprehensive income (loss)			(360)	(5,863)	15,532
Purchases/(Sales)	[2]		(1,075,529)	(93,578)	3,218,301
Contributions			26,461	33,075	5,969
Paydowns			(85,979)	(80,451)	(73,831)
Distributions			(2,894)	(1,712)	0
Sale of real estate owned			0	(3,395)	0
Balance at the end of period			$ 7,213,641	$ 8,442,604	$ 8,209,702
PO Security
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Proceeds from sale of investments		$ 44,300
Gain on sale of investments		1,500
Multi-Family Collateralized Mortgage Backed Securities
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Deconsolidation of assets		1,100,000
Multi-family collateralized debt obligations, at fair value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Deconsolidation of assets		$ 1,000,000

[1] Amounts included in interest income from multi-family loans held in securitization trusts, unrealized gain on multi-family loans and debt held in securitization trusts, realized gain (loss) on investment securities and related hedges, net, other income and gain on deconsolidation.

[2] In February 2015, the Company sold a first loss PO security from one of the Company’s Consolidated K-Series securitizations obtaining total proceeds of approximately $44.3 million and realizing a gain of approximately $1.5 million. The sale resulted in a de-consolidation of $1.1 billion in Multi-Family loans held in a securitization trust and $1.0 billion in Multi-Family CDOs.