Investment Securities - Investment Securities (Details) - USD ($) $ in Thousands	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021	Dec. 31, 2020
Debt Securities, Available-for-sale [Line Items]
Available-for-sale securities	$ 102,368	$ 113,386
Credit loss allowance	13,143	3,182	$ 8,610	$ 5,675
HTM debt securities	619,397	676,652
Total investment securities	735,562	815,804
VIE eliminations
Debt Securities, Available-for-sale [Line Items]
Equity security	(1,578,859)	(1,666,921)
CMBS
Debt Securities, Available-for-sale [Line Items]
Credit loss allowance	164	172	$ 3,140	$ 0
HTM debt securities	580,704	577,681
CMBS | Non- Controlling Interests
Debt Securities, Available-for-sale [Line Items]
Equity security	177,300	198,900
Before consolidation of securitization VIEs
Debt Securities, Available-for-sale [Line Items]
HTM debt securities	606,254	673,470
Equity security, fair value	8,340	9,840
Total investment securities	2,314,421	2,482,725
Before consolidation of securitization VIEs | RMBS
Debt Securities, Available-for-sale [Line Items]
Available-for-sale securities	102,368	113,386
Equity security	449,909	423,183
Before consolidation of securitization VIEs | CMBS
Debt Securities, Available-for-sale [Line Items]
Equity security	$ 1,147,550	$ 1,262,846