Investments in RMBS (Details) $ in Thousands		6 Months Ended	12 Months Ended
		Jun. 30, 2021 USD ($) Security	Dec. 31, 2020 USD ($) Security
Residential Mortgage-Backed Securities [Abstract]
Carrying value		$ 998,799	$ 1,228,251
Carrying value of collateral for repurchase agreements		933,100	1,164,416
RMBS [Member]
Residential Mortgage-Backed Securities [Abstract]
Original face value		1,225,005	1,411,705
Book value		984,441	1,192,539
Gross unrealized gains		18,181	35,833
Gross unrealized losses		(3,823)	(121)
Carrying value	[1]	$ 998,799	$ 1,228,251
Number of securities | Security		119	135
Weighted average coupon		3.07%	3.18%
Weighted average yield	[2]	2.94%	3.05%
Weighted average maturity		28 years	28 years
Carrying value of collateral for repurchase agreements		$ 933,100	$ 1,164,400
RMBS [Member] | Fannie Mae [Member]
Residential Mortgage-Backed Securities [Abstract]
Original face value		776,856	840,175
Book value		606,853	692,665
Gross unrealized gains		11,739	22,530
Gross unrealized losses		(1,548)	(39)
Carrying value	[1],[3]	$ 617,044	$ 715,156
Number of securities | Security		76	81
Weighted average coupon		3.13%	3.31%
Weighted average yield	[2]	2.99%	3.17%
Weighted average maturity		28 years	28 years
RMBS [Member] | Freddie Mac [Member]
Residential Mortgage-Backed Securities [Abstract]
Original face value		$ 448,149	$ 549,530
Book value		377,588	493,930
Gross unrealized gains		6,442	13,106
Gross unrealized losses		(2,275)	(82)
Carrying value	[1],[3]	$ 381,755	$ 506,954
Number of securities | Security		43	49
Weighted average coupon		2.98%	2.99%
Weighted average yield	[2]	2.85%	2.87%
Weighted average maturity		28 years	28 years
RMBS [Member] | Private Label MBS [Member]
Residential Mortgage-Backed Securities [Abstract]
Original face value			$ 22,000
Book value			5,944
Gross unrealized gains			197
Gross unrealized losses			0
Carrying value	[1],[3]		$ 6,141
Number of securities | Security			5
Weighted average coupon			4.08%
Weighted average yield	[2]		4.08%
Weighted average maturity			28 years

[1] See Note 9 regarding the estimation of fair value, which approximates carrying value for all securities.

[2] The weighted average yield is based on the most recent gross monthly interest income, which is then annualized and divided by the book value of settled securities.

[3] The Company used an implied AAA rating for the Agency RMBS. Private label RMBS are rated investment grade or better by at least one NRSRO as of December 31, 2020.