Principles of Consolidation - Income (Loss) from Consolidated VIEs Accounted for as Collateralized Financing Entities (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2023	Mar. 31, 2022
Variable Interest Entity [Line Items]
Interest income	$ 178,516	$ 189,400
Interest expense	(152,079)	(136,298)
Net Interest Income	26,437	53,102
Investment fair value changes, net	(127)	(6,120)
Total non-interest income, net	21,098	18,759
General and administrative expenses	(35,555)	(33,276)
Net Income	4,620	30,915
Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	119,549	140,316
Interest expense	(91,408)	(107,470)
Net Interest Income	28,141	32,846
Investment fair value changes, net	1,341	1,371
Other income	172	90
Total non-interest income, net	1,513	1,461
General and administrative expenses	(10)
General and administrative expenses		(31)
Other expenses	(577)	(551)
Net Income	29,087	33,725
Legacy Sequoia | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	2,543	1,012
Interest expense	(2,504)	(701)
Net Interest Income	39	311
Investment fair value changes, net	(94)	(714)
Other income	0	0
Total non-interest income, net	(94)	(714)
General and administrative expenses	0
General and administrative expenses		0
Other expenses	0	0
Net Income	(55)	(403)
Sequoia | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	34,644	32,098
Interest expense	(30,055)	(28,171)
Net Interest Income	4,589	3,927
Investment fair value changes, net	2,442	(3,822)
Other income	0	0
Total non-interest income, net	2,442	(3,822)
General and administrative expenses	0
General and administrative expenses		0
Other expenses	0	0
Net Income	7,031	105
CAFL | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	54,437	77,334
Interest expense	(39,542)	(58,480)
Net Interest Income	14,895	18,854
Investment fair value changes, net	(9,682)	2,664
Other income	172	90
Total non-interest income, net	(9,510)	2,754
General and administrative expenses	0
General and administrative expenses		0
Other expenses	0	0
Net Income	5,385	21,608
Freddie Mac SLST | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	15,493	17,200
Interest expense	(11,218)	(14,085)
Net Interest Income	4,275	3,115
Investment fair value changes, net	8,934	3,036
Other income	0	0
Total non-interest income, net	8,934	3,036
General and administrative expenses	0
General and administrative expenses		0
Other expenses	0	0
Net Income	13,209	6,151
Freddie Mac K-Series | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	4,618	4,753
Interest expense	(4,241)	(4,371)
Net Interest Income	377	382
Investment fair value changes, net	363	264
Other income	0	0
Total non-interest income, net	363	264
General and administrative expenses	0
General and administrative expenses		0
Other expenses	0	0
Net Income	740	646
Servicing Investment | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	7,814	7,919
Interest expense	(3,848)	(1,662)
Net Interest Income	3,966	6,257
Investment fair value changes, net	(1,047)	(3,468)
Other income	0	0
Total non-interest income, net	(1,047)	(3,468)
General and administrative expenses	(10)
General and administrative expenses		(31)
Other expenses	(577)	(551)
Net Income	2,352	2,207
HEI | Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Interest income	0	0
Interest expense	0	0
Net Interest Income	0	0
Investment fair value changes, net	425	3,411
Other income	0	0
Total non-interest income, net	425	3,411
General and administrative expenses	0
General and administrative expenses		0
Other expenses	0	0
Net Income	$ 425	$ 3,411