RESIDENTIAL MORTGAGE LOANS - Residential Mortgage Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2021 USD ($) loan	Dec. 31, 2020 USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year 3 months 18 days
Residential mortgage loans held-for-investment, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 623,937
Carrying Value	$ 569,933	$ 674,179
Loan Count | loan	9,718,000
Weighted Average Yield	7.10%
Weighted Average Life (Years)	5 years 1 month 6 days
Total residential mortgage loans, held-for-sale, at lower of cost or market
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 144,967
Carrying Value	$ 132,921	509,887
Loan Count | loan	2,873,000
Weighted Average Yield	6.60%
Weighted Average Life (Years)	4 years 7 months 6 days
Acquired reverse residential mortgage loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 0
Carrying Value	$ 0	5,884
Loan Count | loan	0
Weighted Average Yield	0.00%
Weighted Average Life (Years)	0 years
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 142,142
Carrying Value	$ 130,634	129,345
Loan Count | loan	2,839,000
Weighted Average Yield	6.60%
Weighted Average Life (Years)	4 years 7 months 6 days
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 2,825
Carrying Value	$ 2,287	374,658
Loan Count | loan	34,000
Weighted Average Yield	7.50%
Weighted Average Life (Years)	4 years 8 months 12 days
Total residential mortgage loans, held-for-sale, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 10,955,534
Carrying Value	$ 11,214,924	4,705,816
Loan Count | loan	27,485,000
Weighted Average Yield	3.30%
Weighted Average Life (Years)	24 years 4 months 24 days
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 2,046,945
Carrying Value	$ 2,070,262	1,423,159
Loan Count | loan	12,757,000
Weighted Average Yield	3.50%
Weighted Average Life (Years)	12 years 4 months 24 days
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 343,133
Carrying Value	$ 315,063	335,544
Loan Count | loan	2,249,000
Weighted Average Yield	4.80%
Weighted Average Life (Years)	6 years 1 month 6 days
Originated loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 8,565,456
Carrying Value	$ 8,829,599	2,947,113
Loan Count | loan	12,479,000
Weighted Average Yield	3.20%
Weighted Average Life (Years)	28 years
Total residential mortgage loans, held-for-sale, at fair value/lower of cost or market
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 11,100,501
Carrying Value	11,347,845	$ 5,215,703
Performing Loans | Ginnie Mae EBO
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	860,400
Non-Performing Loans | Ginnie Mae EBO
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 221,900