Fair Value of Financial Instruments (Details) - USD ($) $ in Thousands	Mar. 31, 2018	Dec. 31, 2017
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value	$ 6,883	$ 7,243
Mortgage-backed securities	2,864,822	3,026,989
U.S. Treasuries	204,535	146,530
Derivative asset	4,754	2,940
Total assets carried at fair value	3,074,111	3,176,459
Derivative liabilities	0	269
Total liabilities carried at fair value	0	269
Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	0	0
U.S. Treasuries	204,535	146,530
Total assets carried at fair value	206,209	147,196
Total liabilities carried at fair value	0	0
Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	2,857,939	3,019,746
U.S. Treasuries	0	0
Total assets carried at fair value	2,861,019	3,022,020
Total liabilities carried at fair value	0	269
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	6,883	7,243
U.S. Treasuries	0	0
Total assets carried at fair value	6,883	7,243
Total liabilities carried at fair value	0	0
Interest rate swaps
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	1,081	791
Interest rate swaps | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Interest rate swaps | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	1,081	791
Interest rate swaps | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Eurodollar futures
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	1,674	666
Eurodollar futures | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	1,674	666
Eurodollar futures | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Eurodollar futures | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
TBA securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	1,999	1,483
Derivative liabilities		269
TBA securities | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Derivative liabilities		0
TBA securities | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	1,999	1,483
Derivative liabilities		269
TBA securities | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	$ 0	0
Derivative liabilities		$ 0