Fair Value Measurements - Impact of Change in Fair Value on Each of the Financial Instruments Carried at Fair Value (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2023	Jun. 30, 2022	Jun. 30, 2023	Jun. 30, 2022
Non-Agency RMBS | Senior
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	$ 181	$ (784)	$ 218	$ (1,385)
Non-Agency RMBS | Subordinated
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	(3,471)	(24,694)	(2,533)	(45,274)
Non-Agency RMBS | Senior, interest-only
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	(12,777)	(8,639)	(1,133)	(35,599)
Agency MBS - Residential | Interest-only
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	(2,195)	(4,312)	(2,410)	(9,924)
Agency MBS - Project Loans | Interest-only
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	14,033	(1,964)	13,977	(8,834)
Agency MBS - Project Loans | Project loans
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	(2,054)	(12,078)	4,826	(35,032)
Loans held for investment
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	(39,592)	(450,653)	74,546	(1,049,031)
Secured Financing Agreements, at fair value
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	10,286	0	12,446	0
Securitized debt
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Gain/(Loss) on Change in Fair Value	$ 42,543	$ 263,878	$ (28,391)	$ 520,616