Mortgage-Backed Securities - Gross Unrealized Gains (Losses) (Details) - USD ($) $ in Thousands	Sep. 30, 2021	Dec. 31, 2020
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	$ 439,105	$ 558,413
Gross Unrealized Gain Included in Cumulative Earnings	184,583	242,587
Total Gross Unrealized Gain	623,688	801,000
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	(190)	(317)
Gross Unrealized Loss Included in Cumulative Earnings	(116,986)	(121,993)
Total Gross Unrealized Loss	(117,176)	(122,310)
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	400,801	487,963
Gross Unrealized Gain Included in Cumulative Earnings	0	0
Total Gross Unrealized Gain	400,801	487,963
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	(190)	(248)
Gross Unrealized Loss Included in Cumulative Earnings	0	0
Total Gross Unrealized Loss	(190)	(248)
Residential Mortgage-Backed Securities | Non-Agency RMBS - Subordinated
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	35,171	65,043
Gross Unrealized Gain Included in Cumulative Earnings	58,155	17,964
Total Gross Unrealized Gain	93,326	83,007
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(21,584)	(38,029)
Total Gross Unrealized Loss	(21,584)	(38,029)
Residential Mortgage-Backed Securities | Non-Agency RMBS - Senior interest-only
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	40,268	67,868
Total Gross Unrealized Gain	40,268	67,868
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(54,558)	(55,219)
Total Gross Unrealized Loss	(54,558)	(55,219)
Residential Mortgage-Backed Securities | Agency RMBS - Interest-only
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	106	270
Total Gross Unrealized Gain	106	270
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(40,694)	(28,399)
Total Gross Unrealized Loss	(40,694)	(28,399)
Commercial Mortgage Backed Securities | Agency MBS - Commercial
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	3,133	5,407
Gross Unrealized Gain Included in Cumulative Earnings	81,813	153,826
Total Gross Unrealized Gain	84,946	159,233
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	(69)
Gross Unrealized Loss Included in Cumulative Earnings	0	0
Total Gross Unrealized Loss	0	(69)
Commercial Mortgage Backed Securities | Agency CMBS - Interest-only
Investment Holdings [Line Items]
Gross Unrealized Gain Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Gain Included in Cumulative Earnings	4,241	2,659
Total Gross Unrealized Gain	4,241	2,659
Gross Unrealized Loss Included in Accumulated Other Comprehensive Income	0	0
Gross Unrealized Loss Included in Cumulative Earnings	(150)	(346)
Total Gross Unrealized Loss	$ (150)	$ (346)