Fair Value Measurements - Financial Assets and Liabilities Carried at Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Jun. 30, 2023	Dec. 31, 2022
Assets:
Non-Agency RMBS, at fair value	$ 1,092,205	$ 1,147,481
Agency MBS, at fair value	136,326	430,944
Loans held for investment, at fair value	11,929,537	11,359,236
Derivatives, at fair value	6,328	4,096
Liabilities:
Securitized debt at fair value, collateralized by Loans held for investment	8,041,276	7,100,742
Derivative liabilities, at fair value	0	0
Fair Value, Measurements, Recurring
Assets:
Non-Agency RMBS, at fair value	1,092,205	1,147,481
Agency MBS, at fair value	136,326	430,944
Loans held for investment, at fair value	11,929,537	11,359,236
Counterparty and Cash Collateral, netting	(28,343)	(14,697)
Derivatives, at fair value	6,328	4,096
Liabilities:
Secured Financing Agreements, at fair value	353,817	374,172
Securitized debt at fair value, collateralized by Loans held for investment	8,041,276	7,100,742
Counterparty and Cash Collateral, netting	0	(14,074)
Derivative liabilities, at fair value	0	0
Fair Value, Measurements, Recurring | Level 1
Assets:
Non-Agency RMBS, at fair value	0	0
Agency MBS, at fair value	0	0
Loans held for investment, at fair value	0	0
Derivatives, at fair value	0	0
Liabilities:
Secured Financing Agreements, at fair value	0	0
Securitized debt at fair value, collateralized by Loans held for investment	0	0
Derivatives	0	0
Fair Value, Measurements, Recurring | Level 2
Assets:
Non-Agency RMBS, at fair value	0	0
Agency MBS, at fair value	136,326	430,944
Loans held for investment, at fair value	159,411	204,636
Derivatives, at fair value	34,671	18,793
Liabilities:
Secured Financing Agreements, at fair value	353,817	374,172
Securitized debt at fair value, collateralized by Loans held for investment	0	0
Derivatives	0	14,074
Fair Value, Measurements, Recurring | Level 3
Assets:
Non-Agency RMBS, at fair value	1,092,205	1,147,481
Agency MBS, at fair value	0	0
Loans held for investment, at fair value	11,770,126	11,154,600
Derivatives, at fair value	0	0
Liabilities:
Secured Financing Agreements, at fair value	0	0
Securitized debt at fair value, collateralized by Loans held for investment	8,041,276	7,100,742
Derivatives	$ 0	$ 0