Residential Loans - Activity of Residential Loans Held-for-Investment (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2022	Dec. 31, 2021	Dec. 31, 2020
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Transfers from loans held-for-sale to loans held-for-investment	$ 2,949,262	$ 5,026,723	$ 1,868,656
Residential loans, held-for-investment, at fair value | Sequoia
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Valuation adjustments	(675,659)	(66,727)
Residential loans, held-for-investment, at fair value | Freddie Mac SLST
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Valuation adjustments	(215,687)	(14,735)
Residential loans, held-for-investment, at fair value | Legacy Sequoia
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Valuation adjustments	12,956	12,125
Residential loans, held-for-investment, at fair value | Sequoia
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Transfers from loans held-for-sale to loans held-for-investment	$ 684,491	$ 3,035,100