Fair Value of Financial Instruments (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	$ 109,746	$ 11,342
Derivative liabilities	19,866	1,634
Fair Value, Measurements, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	2,380,373	2,596,255
Assets, Fair Value Disclosure	2,495,868	2,613,861
Financial Liabilities Fair Value Disclosure	19,866	1,634
Mortgage loans held for investment, at fair value	5,749	6,264
Fair Value, Measurements, Recurring | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	0	0
Assets, Fair Value Disclosure	81,903	1,094
Financial Liabilities Fair Value Disclosure	0	1,527
Mortgage loans held for investment, at fair value	0	0
Fair Value, Measurements, Recurring | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	2,379,230	2,594,980
Assets, Fair Value Disclosure	2,407,073	2,605,228
Financial Liabilities Fair Value Disclosure	19,866	107
Mortgage loans held for investment, at fair value	0	0
Fair Value, Measurements, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	1,143	1,275
Assets, Fair Value Disclosure	6,892	7,539
Financial Liabilities Fair Value Disclosure	0	0
Mortgage loans held for investment, at fair value	5,749	6,264
Fair Value, Measurements, Recurring | Options on U.S. Treasury futures
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	9,180	1,094
Fair Value, Measurements, Recurring | Options on U.S. Treasury futures | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	9,180	1,094
Fair Value, Measurements, Recurring | Options on U.S. Treasury futures | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Fair Value, Measurements, Recurring | Options on U.S. Treasury futures | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Fair Value, Measurements, Recurring | Interest rate swaption
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	27,843	1,360
Derivative liabilities		107
Fair Value, Measurements, Recurring | Interest rate swaption | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Derivative liabilities		0
Fair Value, Measurements, Recurring | Interest rate swaption | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	27,843	1,360
Derivative liabilities		107
Fair Value, Measurements, Recurring | Interest rate swaption | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Derivative liabilities		0
Fair Value, Measurements, Recurring | TBA securities | Long position
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		8,888
Derivative liabilities	19,866
Fair Value, Measurements, Recurring | TBA securities | Level 1 | Long position
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0
Derivative liabilities	0
Fair Value, Measurements, Recurring | TBA securities | Level 2 | Long position
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		8,888
Derivative liabilities	19,866
Fair Value, Measurements, Recurring | TBA securities | Level 3 | Long position
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0
Derivative liabilities	0
Fair Value, Measurements, Recurring | Futures
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	72,723
Derivative liabilities		1,527
Fair Value, Measurements, Recurring | Futures | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	72,723
Derivative liabilities		1,527
Fair Value, Measurements, Recurring | Futures | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0
Derivative liabilities		0
Fair Value, Measurements, Recurring | Futures | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	$ 0
Derivative liabilities		$ 0