VARIABLE INTEREST ENTITIES - Statement of Comprehensive Income (Loss) of VIEs Reflected in Consolidated Statements of Comprehensive Income (Loss) (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Dec. 31, 2017	Sep. 30, 2017	Jun. 30, 2017	Mar. 31, 2017	Dec. 31, 2016	Sep. 30, 2016	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Net interest income:
Interest income	$ 745,423	$ 622,550	$ 537,426	$ 587,727	$ 807,022	$ 558,668	$ 457,118	$ 388,143	$ 2,493,126	$ 2,210,951	$ 2,170,697
Interest expense	318,711	268,937	222,281	198,425	183,396	174,154	152,755	147,447	1,008,354	657,752	471,596
Net interest income	426,712	353,613	315,145	389,302	623,626	384,514	304,363	240,696	1,484,772	1,553,199	1,699,101
Realized gain (loss) on disposal of investments									0	0	(12,450)
Other income (loss)									115,857	44,144	(13,717)
General and administration expenses									224,124	250,356	200,240
Net income (loss)	$ 746,771	$ 367,315	$ 14,522	$ 440,408	$ 1,848,483	$ 730,880	$ (278,497)	$ (868,080)	1,569,016	$ 1,432,786	$ 465,747
Consolidated VIEs | FREMF Trusts
Net interest income:
Interest income									110,712
Interest expense									58,583
Net interest income									52,129
Realized gain (loss) on disposal of investments									0
Net gains (losses) on trading assets									0
Unrealized gain (loss) on investments at fair value									4,273
Other income (loss)									(24,541)
General and administration expenses									1
Net income (loss)									31,860
Consolidated VIEs | Residential Mortgage Loan Trusts
Net interest income:
Interest income									5,436
Interest expense									1,723
Net interest income									3,713
Realized gain (loss) on disposal of investments									(831)
Net gains (losses) on trading assets									0
Unrealized gain (loss) on investments at fair value									7,865
Other income (loss)									(361)
General and administration expenses									97
Net income (loss)									10,289
Consolidated VIEs | MSR Silo
Net interest income:
Interest income									1,500
Interest expense									374
Net interest income									1,126
Realized gain (loss) on disposal of investments									(2,044)
Net gains (losses) on trading assets									14
Unrealized gain (loss) on investments at fair value									(71,613)
Other income (loss)									129,325
General and administration expenses									2,567
Net income (loss)									$ 54,241