VARIABLE INTEREST ENTITIES - Statement of Comprehensive Income (Loss) of VIEs Reflected in Consolidated Statements of Comprehensive Income (Loss) (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2018	Sep. 30, 2017	Sep. 30, 2018	Sep. 30, 2017
Variable Interest Entity [Line Items]
Interest income	$ 816,596	$ 622,550	$ 2,472,889	$ 1,747,703
Interest expense	500,973	268,937	1,311,086	689,643
Net interest income	315,623	353,613	1,161,803	1,058,060
Net gains (losses) on other derivatives	94,827	154,208	81,871	140,104
Less: General and administrative expenses	126,509	57,016	252,800	164,867
Net income (loss)	385,429	367,315	2,309,020	822,245
Consolidated VIEs | FREMF Trusts
Variable Interest Entity [Line Items]
Interest income	36,387	28,841	85,325	81,508
Interest expense	25,068	15,791	54,265	42,046
Net interest income	11,319	13,050	31,060	39,462
Net realized gains (losses) on disposal of investments	0	0	0	0
Net gains (losses) on trading assets		0		0
Net gains (losses) on other derivatives			0
Net unrealized gains (losses) on instruments measured at fair value through earnings	220	(2,256)	1,332	2,833
Other income (loss)	(4,217)	(6,073)	(12,986)	(18,595)
Less: General and administrative expenses	0	(1)	0	0
Net income (loss)	7,322	4,722	19,406	23,700
Consolidated VIEs | Residential Mortgage Loan Trusts
Variable Interest Entity [Line Items]
Interest income	1,723	1,145	6,280	3,685
Interest expense	773	282	3,621	854
Net interest income	950	863	2,659	2,831
Net realized gains (losses) on disposal of investments	147	(229)	2,049	(611)
Net gains (losses) on trading assets		0		0
Net gains (losses) on other derivatives			0
Net unrealized gains (losses) on instruments measured at fair value through earnings	242	(20)	(925)	1,682
Other income (loss)	(70)	(89)	(221)	(280)
Less: General and administrative expenses	21	34	50	71
Net income (loss)	1,248	491	3,512	3,551
Consolidated VIEs | Mortgage Servicing Rights Silos
Variable Interest Entity [Line Items]
Interest income	2,678	514	4,126	1,005
Interest expense	438	121	875	243
Net interest income	2,240	393	3,251	762
Net realized gains (losses) on disposal of investments	(516)	(1,430)	(1,826)	(1,915)
Net gains (losses) on trading assets		(19)		(17)
Net gains (losses) on other derivatives			70
Net unrealized gains (losses) on instruments measured at fair value through earnings	(13,364)	(36,226)	3,101	(83,340)
Other income (loss)	26,866	32,001	83,924	99,927
Less: General and administrative expenses	481	560	1,408	2,500
Net income (loss)	$ 14,745	$ (5,841)	$ 87,112	$ 12,917