Mortgage-Backed Securities - Gross Unrealized Gains (Losses) (Details) - USD ($) $ in Thousands	Jun. 30, 2023	Dec. 31, 2022
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	$ 229,734	$ 251,727
Gross Unrealized Gain Included in Cumulative Earnings	25,461	26,428
Total Gross Unrealized Gain	255,195	278,155
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	(19,058)	(22,382)
Gross Unrealized Loss Included in Cumulative Earnings	(126,231)	(140,137)
Total Gross Unrealized Loss	(145,289)	(162,519)
Non-Agency RMBS - Senior
Debt Securities, Available-for-sale [Line Items]
Total Gross Unrealized Gain	216,269	237,127
Total Gross Unrealized Loss	(6,974)	(6,933)
Non-Agency RMBS - Subordinated
Debt Securities, Available-for-sale [Line Items]
Total Gross Unrealized Gain	20,952	22,035
Total Gross Unrealized Loss	(38,494)	(36,677)
Non-Agency RMBS - Senior interest-only
Debt Securities, Available-for-sale [Line Items]
Total Gross Unrealized Gain	16,640	15,968
Total Gross Unrealized Loss	(81,832)	(80,024)
Agency RMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Total Gross Unrealized Gain	0	1,371
Total Gross Unrealized Loss	(6,031)	(4,991)
Agency MBS - Commercial
Debt Securities, Available-for-sale [Line Items]
Total Gross Unrealized Gain	0	0
Total Gross Unrealized Loss	(11,241)	(18,880)
Agency CMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Total Gross Unrealized Gain	1,334	1,654
Total Gross Unrealized Loss	(717)	(15,014)
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	216,269	237,127
Gross Unrealized Gain Included in Cumulative Earnings	0	0
Total Gross Unrealized Gain	216,269	237,127
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	(5,391)	(5,132)
Gross Unrealized Loss Included in Cumulative Earnings	(1,583)	(1,801)
Total Gross Unrealized Loss	(6,974)	(6,933)
Residential Mortgage-Backed Securities | Non-Agency RMBS - Subordinated
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	13,465	14,600
Gross Unrealized Gain Included in Cumulative Earnings	7,487	7,435
Total Gross Unrealized Gain	20,952	22,035
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	(13,649)	(14,418)
Gross Unrealized Loss Included in Cumulative Earnings	(24,845)	(22,259)
Total Gross Unrealized Loss	(38,494)	(36,677)
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior interest-only
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	16,640	15,968
Total Gross Unrealized Gain	16,640	15,968
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(81,832)	(80,024)
Total Gross Unrealized Loss	(81,832)	(80,024)
Residential Mortgage-Backed Securities | Agency RMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	0	1,371
Total Gross Unrealized Gain	0	1,371
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(6,031)	(4,991)
Total Gross Unrealized Loss	(6,031)	(4,991)
Commercial Mortgage Backed Securities | Agency MBS - Commercial
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	0	0
Total Gross Unrealized Gain	0	0
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	(18)	(2,832)
Gross Unrealized Loss Included in Cumulative Earnings	(11,223)	(16,048)
Total Gross Unrealized Loss	(11,241)	(18,880)
Commercial Mortgage Backed Securities | Agency CMBS - Interest-only
Debt Securities, Available-for-sale [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	1,334	1,654
Total Gross Unrealized Gain	1,334	1,654
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(717)	(15,014)
Total Gross Unrealized Loss	$ (717)	$ (15,014)