CONSOLIDATED VARIABLE INTEREST ENTITIES - Variable Interest Entities, Assets and Liabilities (Details) - USD ($) $ in Thousands		Sep. 30, 2020	Dec. 31, 2019
Assets
Servicer advance investments, at fair value	[1]	$ 535,760	$ 581,777
Residential mortgage loans, held-for-sale		4,936,826	6,042,664
Cash and cash equivalents	[1]	841,022	528,737
Restricted cash		180,554	162,197
Other assets		1,161,933	1,395,800
Total assets		30,406,364	44,863,454
Liabilities
Accrued expenses and other liabilities	[1]	497,838	600,790
Total liabilities		24,977,733	37,627,194
VIE, consolidated
Assets
Servicer advance investments, at fair value		521,882	565,271
Residential mortgage loans, held-for-investment, at fair value		390,890	913,030
Residential mortgage loans, held-for-sale		352,225
Residential mortgage loans, held-for-sale, at fair value		622,269
Consumer loans, held-for-investment, at fair value		718,355	818,943
Cash and cash equivalents		76,295	53,867
Restricted cash		10,934	14,423
Other assets		25,935	18,379
Total assets		2,718,785	2,383,913
Liabilities
Secured notes and bonds payable		2,175,283	1,913,696
Accrued expenses and other liabilities		14,700	20,038
Total liabilities		2,189,983	1,933,734
VIE, consolidated | The Buyer
Assets
Servicer advance investments, at fair value		521,882	565,271
Residential mortgage loans, held-for-investment, at fair value		0	0
Residential mortgage loans, held-for-sale		0
Residential mortgage loans, held-for-sale, at fair value		0
Consumer loans, held-for-investment, at fair value		0	0
Cash and cash equivalents		40,653	30,065
Restricted cash		5,044	5,350
Other assets		6	2,414
Total assets		567,585	603,100
Liabilities
Secured notes and bonds payable		404,907	433,300
Accrued expenses and other liabilities		2,237	1,593
Total liabilities		407,144	434,893
VIE, consolidated | Shelter Joint Ventures
Assets
Servicer advance investments, at fair value		0	0
Residential mortgage loans, held-for-investment, at fair value		0	0
Residential mortgage loans, held-for-sale		0
Residential mortgage loans, held-for-sale, at fair value		0
Consumer loans, held-for-investment, at fair value		0	0
Cash and cash equivalents		35,642	23,802
Restricted cash		0	0
Other assets		9,896	3,556
Total assets		45,538	27,358
Liabilities
Secured notes and bonds payable		0	0
Accrued expenses and other liabilities		8,906	4,187
Total liabilities		8,906	4,187
VIE, consolidated | Residential Mortgage Loans
Assets
Servicer advance investments, at fair value		0	0
Residential mortgage loans, held-for-investment, at fair value		390,890	913,030
Residential mortgage loans, held-for-sale		352,225
Residential mortgage loans, held-for-sale, at fair value		622,269
Consumer loans, held-for-investment, at fair value		0	0
Cash and cash equivalents		0	0
Restricted cash		0	0
Other assets		6,183	0
Total assets		1,371,567	913,030
Liabilities
Secured notes and bonds payable		1,091,426	659,738
Accrued expenses and other liabilities		1,871	10,132
Total liabilities		1,093,297	669,870
VIE, consolidated | Consumer Loan Companies
Assets
Servicer advance investments, at fair value		0	0
Residential mortgage loans, held-for-investment, at fair value		0	0
Residential mortgage loans, held-for-sale		0
Residential mortgage loans, held-for-sale, at fair value		0
Consumer loans, held-for-investment, at fair value		718,355	818,943
Cash and cash equivalents		0	0
Restricted cash		5,890	9,073
Other assets		9,850	12,409
Total assets		734,095	840,425
Liabilities
Secured notes and bonds payable		678,950	820,658
Accrued expenses and other liabilities		1,686	4,126
Total liabilities		$ 680,636	$ 824,784

[1] See Note 13 regarding consolidated VIEs.