Investment Securities - Purchases and Sales (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2023	Mar. 31, 2022
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	$ 591	$ 18,139
Sales	0	0
Principal collections	40,778	7,784
Held-to-maturity
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	591	18,139
Sales	0
Principal collections	38,198	664
VIE eliminations
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	(84,357)
Sales	0	0
Principal collections	(15,329)	(22,014)
RMBS | Available-for-sale
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	0
Sales	0	0
Principal collections	2,435	6,895
RMBS | Fair value option
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	84,357
Sales	0	0
Principal collections	14,220	21,604
CMBS | Fair value option
Debt Securities, Available-for-sale [Line Items]
Purchases/fundings	0	0
Sales	0	0
Principal collections	$ 1,254	$ 635