Fair Value of Financial Instruments - Valuation for Level 3 Assets (Details) - USD ($) $ in Thousands		6 Months Ended
		Jun. 30, 2016	Jun. 30, 2015
Fair Value, Assets Measured on Recurring Basis, Unobservable Input Reconciliation, Calculation [Roll Forward]
Balance at beginning of period		$ 7,213,641	$ 8,442,604
Total gains/(losses) (realized/unrealized)
Included in earnings	[1]	240,755	(12,863)
Included in other comprehensive income		124	422
Sales	[2]	0	(1,073,029)
Transfers in	[3]	52,176	0
Transfers out	[4]	(56,756)	0
Contributions		1,500	12,701
Paydowns		(58,993)	(38,475)
Distributions		(4,976)	(382)
Balance at the end of period		$ 7,387,471	$ 7,330,978

[1] Amounts included in interest income from multi-family loans held in securitization trusts, unrealized gain on multi-family loans and debt held in securitization trusts, realized gain (loss) on investment securities and related hedges, gain on de-consolidation, and other income.

[2] In February 2015, the Company sold a first loss PO security from one of the Company’s Consolidated K-Series securitizations obtaining total proceeds of approximately $44.3 million and realizing a gain of approximately $1.5 million. The sale resulted in a de-consolidation of $1.1 billion in multi-family loans held in a securitization trust and $1.0 billion in Multi-Family CDOs.

[3] (3) Transfers into Level 3 are investments in unconsolidated entities held by RiverBanc and RBMI for which the Company accounts under the equity method of accounting with a fair value election. These transfers in are a result of the Company’s acquisition of the outstanding membership interests in RiverBanc and RBMI that were not previously owned by the Company on May 16, 2016, which resulted in consolidation of these entities into the Company's financial statements. (see Note 19).

[4] (4) Transfers out of Level 3 are the Company’s previously held membership interests in RBMI and RBDHC that were accounted for under the equity method of accounting with a fair value election. These transfers out are a result of the Company’s acquisition of the outstanding membership interests in RBMI and RBDHC that were not previously owned by the Company on May 16, 2016, which resulted in consolidation of these entities into the Company's financial statements. (see Note 19).