Fair Value Measurements - Carrying Value and Fair Value of Financial Instruments Not Carried at Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Jun. 30, 2023	Dec. 31, 2022
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Securitized debt, collateralized by Non-Agency RMBS	$ 77,195	$ 78,542
Carrying Amount | Level 2
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Secured financing agreements	2,332,705	3,060,592
Carrying Amount | Level 3
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Equity method investments	25,221	25,538
Securitized debt, collateralized by Non-Agency RMBS	77,195	78,542
Fair Value | Level 2
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Secured financing agreements	2,404,473	3,080,982
Fair Value | Level 3
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Equity method investments	25,221	25,538
Securitized debt, collateralized by Non-Agency RMBS	$ 52,264	$ 54,590