RESIDENTIAL MORTGAGE LOANS - Carrying Value of Mortgage Loans (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2022	Dec. 31, 2021
SEC Schedule, 12-29, Real Estate Companies, Investment in Movement in Mortgage Loans on Real Estate [Roll Forward]
Balance, beginning	$ 11,347,845
Transfer of loans to real estate owned	(14,936)	$ (30,015)
Balance, ending	3,398,298	11,347,845
Residential Portfolio Segment
SEC Schedule, 12-29, Real Estate Companies, Investment in Movement in Mortgage Loans on Real Estate [Roll Forward]
Balance, beginning	11,917,778	5,889,882
Caliber acquisition (Note 3)		7,685,681
Originations	67,406,228	123,059,895
Sales	(81,653,129)	(132,335,618)
Purchases/additional fundings	6,887,407	8,102,055
Proceeds from repayments	(493,051)	(673,407)
Transfer of loans to other assets	(25,375)	21,527
Transfer of loans to real estate owned	(7,094)	(26,268)
Transfers of loans to held-for-sale	(1,580)
Transfers of loans to from held-for-investment	1,582
Valuation provision on loans	(8,305)	38,273
Changes in instrument-specific credit risk	(69,290)	(20,119)
Other factors	(104,354)	175,877
Balance, ending	3,850,817	11,917,778
Residential Mortgage Loans, Held-for-Investment | Residential Portfolio Segment
SEC Schedule, 12-29, Real Estate Companies, Investment in Movement in Mortgage Loans on Real Estate [Roll Forward]
Balance, beginning	569,933	674,179
Caliber acquisition (Note 3)		0
Originations	0	0
Sales	0	0
Purchases/additional fundings	7,182	0
Proceeds from repayments	(80,661)	(120,247)
Transfer of loans to other assets	0	0
Transfer of loans to real estate owned	(4,956)	(15,165)
Transfers of loans to held-for-sale	(1,580)
Transfers of loans to from held-for-investment	0
Valuation provision on loans	0	0
Changes in instrument-specific credit risk	(33,086)	(2,020)
Other factors	(4,313)	33,186
Balance, ending	452,519	569,933
Residential Mortgage Loans, Held-for-Sale | Loans Held-for-Sale, at Lower Cost or Fair Value | Residential Portfolio Segment
SEC Schedule, 12-29, Real Estate Companies, Investment in Movement in Mortgage Loans on Real Estate [Roll Forward]
Balance, beginning	132,921	509,887
Caliber acquisition (Note 3)		0
Originations	0	0
Sales	(4,426)	(374,683)
Purchases/additional fundings	0	0
Proceeds from repayments	(17,777)	(32,826)
Transfer of loans to other assets	0	(585)
Transfer of loans to real estate owned	(1,386)	(7,145)
Transfers of loans to held-for-sale	0
Transfers of loans to from held-for-investment	0
Valuation provision on loans	(8,305)	38,273
Changes in instrument-specific credit risk	0	0
Other factors	0	0
Balance, ending	101,027	132,921
Residential Mortgage Loans, Held-for-Sale | Loans Held-for-Sale, at Fair Value | Residential Portfolio Segment
SEC Schedule, 12-29, Real Estate Companies, Investment in Movement in Mortgage Loans on Real Estate [Roll Forward]
Balance, beginning	11,214,924	4,705,816
Caliber acquisition (Note 3)		7,685,681
Originations	67,406,228	123,059,895
Sales	(81,648,703)	(131,960,935)
Purchases/additional fundings	6,880,225	8,102,055
Proceeds from repayments	(394,613)	(520,334)
Transfer of loans to other assets	(25,375)	22,112
Transfer of loans to real estate owned	(752)	(3,958)
Transfers of loans to held-for-sale	0
Transfers of loans to from held-for-investment	1,582
Valuation provision on loans	0	0
Changes in instrument-specific credit risk	(36,204)	(18,099)
Other factors	(100,041)	142,691
Balance, ending	$ 3,297,271	$ 11,214,924