RESIDENTIAL MORTGAGE LOANS - Residential Mortgage Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	6 Months Ended
	Jun. 30, 2022 USD ($) loan	Dec. 31, 2021 USD ($)
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year 3 months 18 days
Residential mortgage loans held-for-investment, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 576,750
Carrying Value	$ 510,744	$ 569,933
Loan Count | loan	10,273
Weighted Average Yield	7.10%
Weighted Average Life (Years)	4 years 4 months 24 days
Acquired residential mortgage loans held-for-sale
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 131,937
Carrying Value	$ 117,053	132,921
Loan Count | loan	2,906
Weighted Average Yield	6.60%
Weighted Average Life (Years)	5 years
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 102,734
Carrying Value	$ 92,430	130,634
Loan Count | loan	2,689
Weighted Average Yield	6.70%
Weighted Average Life (Years)	4 years 10 months 24 days
Acquired performing loans | Ginnie Mae
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 672,100
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	29,203
Carrying Value	$ 24,623	2,287
Loan Count | loan	217
Weighted Average Yield	6.30%
Weighted Average Life (Years)	5 years 2 months 12 days
Acquired non-performing loans | Ginnie Mae
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 208,200
Acquired performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	1,759,976
Carrying Value	$ 1,661,890	2,070,262
Loan Count | loan	10,370
Weighted Average Yield	4.70%
Weighted Average Life (Years)	13 years 3 months 18 days
Acquired non-performing loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 308,514
Carrying Value	$ 282,734	315,063
Loan Count | loan	1,546
Weighted Average Yield	4.40%
Weighted Average Life (Years)	12 years 6 months
Originated loans
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 3,319,543
Carrying Value	$ 3,349,312	8,829,599
Loan Count | loan	4,751
Weighted Average Yield	5.10%
Weighted Average Life (Years)	28 years 10 months 24 days
Total residential mortgage loans, held-for-sale, at fair value
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 5,388,033
Carrying Value	$ 5,293,936	11,214,924
Loan Count | loan	16,667
Weighted Average Yield	4.90%
Weighted Average Life (Years)	22 years 10 months 24 days
Total residential mortgage loans, held-for-sale, at fair value/lower of cost or market
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 5,519,970
Carrying Value	$ 5,410,989	$ 11,347,845