Investment Securities - Summary of Investment Securities (Details) - USD ($) $ in Thousands	Sep. 30, 2021	Dec. 31, 2020
Debt Securities, Available-for-sale [Line Items]
Available-for-sale securities	$ 148,583	$ 167,349
HTM debt securities	493,123	544,280
Total investment securities	670,887	736,658
Credit loss allowance	6,389	5,675
VIE eliminations
Debt Securities, Available-for-sale [Line Items]
Equity security	(1,418,768)	(1,425,570)
CMBS
Debt Securities, Available-for-sale [Line Items]
HTM debt securities	339,190	339,059
Credit loss allowance	1,593	0
CMBS | Non- Controlling Interests
Debt Securities, Available-for-sale [Line Items]
Equity security	179,800	179,500
Before consolidation of securitization VIEs
Debt Securities, Available-for-sale [Line Items]
HTM debt securities	486,734	538,605
Equity security, fair value	12,067	11,247
Total investment securities	2,089,655	2,162,228
Before consolidation of securitization VIEs | RMBS
Debt Securities, Available-for-sale [Line Items]
Available-for-sale securities	148,583	167,349
Equity security	215,152	235,997
Before consolidation of securitization VIEs | CMBS
Debt Securities, Available-for-sale [Line Items]
Equity security	$ 1,227,119	$ 1,209,030