OTHER INCOME (LOSS), GENERAL AND ADMINISTRATIVE, OTHER ASSETS AND LIABILITIES - Schedule of Change in Fair Value of Investments (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2020	Jun. 30, 2019	Jun. 30, 2020	Jun. 30, 2019
Other Income and Expenses [Abstract]
Change in fair value of investments in excess mortgage servicing rights	$ (85)	$ (8,455)	$ (11,109)	$ (3,828)
Change in fair value of investments in excess mortgage servicing rights, equity method investees	(2,052)	(3,276)	(2,509)	(664)
Change in fair value of investments in mortgage servicing rights financing receivables	(121,520)	(55,411)	(225,631)	(91,790)
Change in fair value of servicer advance investments	16,037	1,388	(2,712)	9,291
Change in fair value of real estate and other securities	58,598	7,385	(28,194)	14,064
Change in fair value of residential mortgage loans	99,998	72,393	(165,246)	81,607
Change in fair value of investments in consumer loans held-for-investment	30,694	0	(9,223)	0
Change in fair value of derivative instruments	21,106	(40,666)	(18,876)	(66,426)
Change in fair value of investments	$ 102,776	$ (26,642)	$ (463,500)	$ (57,746)