RESIDENTIAL MORTGAGE LOANS - Residential Mortgage Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2022 USD ($) loan	Dec. 31, 2021 USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year 3 months 18 days
Residential mortgage loans held-for-investment, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 605,417
Carrying Value	$ 547,404	$ 569,933
Loan Count | loan	10,891
Weighted Average Yield	7.10%
Weighted Average Life (Years)	5 years 1 month 6 days
Acquired residential mortgage loans held-for-sale
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 138,377
Carrying Value	$ 125,559	132,921
Loan Count | loan	2,746
Weighted Average Yield	7.00%
Weighted Average Life (Years)	4 years 8 months 12 days
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 135,025
Carrying Value	$ 122,889	130,634
Loan Count | loan	2,705
Weighted Average Yield	7.00%
Weighted Average Life (Years)	4 years 8 months 12 days
Acquired performing loans | Ginnie Mae
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 725,300
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	3,352
Carrying Value	$ 2,670	2,287
Loan Count | loan	41
Weighted Average Yield	7.40%
Weighted Average Life (Years)	4 years 6 months
Acquired non-performing loans | Ginnie Mae
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 363,900
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	1,592,694
Carrying Value	$ 1,550,278	2,070,262
Loan Count | loan	9,535
Weighted Average Yield	3.90%
Weighted Average Life (Years)	11 years 8 months 12 days
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 597,699
Carrying Value	$ 559,201	315,063
Loan Count | loan	3,278
Weighted Average Yield	3.40%
Weighted Average Life (Years)	4 years 10 months 24 days
Originated loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 4,984,128
Carrying Value	$ 4,967,437	8,829,599
Loan Count | loan	8,113
Weighted Average Yield	3.80%
Weighted Average Life (Years)	27 years 6 months
Total residential mortgage loans, held-for-sale, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 7,174,521
Carrying Value	$ 7,076,916	11,214,924
Loan Count | loan	20,926
Weighted Average Yield	3.80%
Weighted Average Life (Years)	22 years 1 month 6 days
Total residential mortgage loans, held-for-sale, at fair value/lower of cost or market
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 7,312,898
Carrying Value	$ 7,202,475	$ 11,347,845