Fair Value Measurements - Financial Assets and Liabilities Carried at Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Mar. 31, 2024	Dec. 31, 2023
Assets:
Non-Agency RMBS, at fair value	$ 1,059,863	$ 1,043,806
Agency MBS, at fair value	65,999	102,484
Loans held for investment, at fair value	11,074,519	11,397,046
Derivative assets, at fair value	0	0
Liabilities:
Securitized debt at fair value, collateralized by Loans held for investment	7,336,345	7,601,881
Derivative liabilities, at fair value	0	0
Fair Value, Measurements, Recurring
Assets:
Non-Agency RMBS, at fair value	1,059,863	1,043,806
Agency MBS, at fair value	65,999	102,484
Loans held for investment, at fair value	11,074,519	11,397,046
Counterparty and Cash Collateral, netting	(26,806)	(18,817)
Derivative assets, at fair value	0	0
Liabilities:
Secured Financing Agreements, at fair value	339,581	350,238
Securitized debt at fair value, collateralized by Loans held for investment	7,336,345	7,601,881
Counterparty and Cash Collateral, netting	0	0
Derivative liabilities, at fair value	0	0
Fair Value, Measurements, Recurring | Level 1
Assets:
Non-Agency RMBS, at fair value	0	0
Agency MBS, at fair value	0	0
Loans held for investment, at fair value	0	0
Derivatives, at fair value	0	0
Liabilities:
Secured Financing Agreements, at fair value	0	0
Securitized debt at fair value, collateralized by Loans held for investment	0	0
Derivatives, at fair value	0	0
Fair Value, Measurements, Recurring | Level 2
Assets:
Non-Agency RMBS, at fair value	0	0
Agency MBS, at fair value	65,999	102,484
Loans held for investment, at fair value	161,915	271,994
Derivatives, at fair value	26,806	18,817
Liabilities:
Secured Financing Agreements, at fair value	339,581	350,238
Securitized debt at fair value, collateralized by Loans held for investment	0	0
Derivatives, at fair value	0	0
Fair Value, Measurements, Recurring | Level 3
Assets:
Non-Agency RMBS, at fair value	1,059,863	1,043,806
Agency MBS, at fair value	0	0
Loans held for investment, at fair value	10,912,604	11,125,052
Derivatives, at fair value	0	0
Liabilities:
Secured Financing Agreements, at fair value	0	0
Securitized debt at fair value, collateralized by Loans held for investment	7,336,345	7,601,881
Derivatives, at fair value	$ 0	$ 0