Residential Loans - Quarterly Activity of Residential Loans Held-for-Investment (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2023	Mar. 31, 2022
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Transfers from loans held-for-sale to loans held-for-investment	$ 873,093	$ 1,098,459
Held-for-investment at fair value | Sequoia
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Net market valuation gain (loss)	61,867	(270,731)
Held-for-investment at fair value | Freddie Mac SLST
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Net market valuation gain (loss)	32,437	(43,768)
Held-for-investment at fair value | Legacy Sequoia
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Net market valuation gain (loss)	(463)	6,325
Held-for-investment at fair value | Sequoia
SEC Schedule, 12-29, Real Estate Companies, Investment in Mortgage Loans on Real Estate [Line Items]
Transfers from loans held-for-sale to loans held-for-investment	$ 657,295	$ 684,491