Investment Securities - Purchases, Sales and Redemptions, and Principal Collections for All Investment Securities (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	$ 11,578	$ 86,512	$ 198,358
Sales and redemptions	3,042	0	0
Principal collections	90,348	115,947	87,450
HTM Securities
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	11,578	86,512	198,358
Sales and redemptions	0	0	0
Principal collections	80,083	94,868	54,725
Equity Security
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	0	0
Sales and redemptions	2,493	0	0
Principal collections	0	0	0
Securitization VIEs
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	(48,011)	(289,833)	(240,301)
Sales and redemptions	0	0	(120,585)
Principal collections	(169,642)	(93,412)	(68,873)
RMBS | Available-for-sale
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	0	0
Sales and redemptions	549	0	0
Principal collections	9,475	19,087	30,722
RMBS | Fair value option
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	226,152	168,825
Sales and redemptions	0	0	81,871
Principal collections	53,332	74,542	63,144
CMBS | Fair value option
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	48,011	63,681	71,476
Sales and redemptions	0	0	38,714
Principal collections	$ 117,100	$ 20,862	$ 7,732