Fair Value of Financial Instruments (Details) - USD ($) $ in Thousands		Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities		$ 3,026,989	$ 3,212,084
U.S. Treasuries		146,530	0
Derivative asset		2,940	28,534
Total assets carried at fair value		3,176,459	3,240,618
Derivative liabilities		269	6,922
Total liabilities carried at fair value		269	6,922
Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities		0	0
Total assets carried at fair value		147,196	0
Total liabilities carried at fair value		0	0
Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities		3,019,746	3,201,157
U.S. Treasuries		0
Total assets carried at fair value		3,022,020	3,229,691
Total liabilities carried at fair value		269	6,922
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities		7,243	10,927
U.S. Treasuries		0
Total assets carried at fair value		7,243	10,927
Total liabilities carried at fair value		0	0
Interest rate swaps
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		791	28,534
Derivative liabilities			6,922
Interest rate swaps | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0	0
Derivative liabilities			0
Interest rate swaps | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		791	28,534
Derivative liabilities			6,922
Interest rate swaps | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0	0
Derivative liabilities			0
Eurodollar futures
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		666
Eurodollar futures | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		666
Eurodollar futures | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0
Eurodollar futures | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0
TBA securities
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	[1]	1,483
Derivative liabilities		269
TBA securities | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0
Derivative liabilities		0
TBA securities | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		1,483
Derivative liabilities		269
TBA securities | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		0
Derivative liabilities		0
Fair Value, Measurements, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Fair Value, Measurement with Unobservable Inputs Reconciliation, Recurring Basis, Asset Value		$ 7,243	$ 10,927	$ 16,435	$ 43,957

[1] Refer to Note 1 regarding information on a change in the CME rulebook. Amounts reported on the consolidated balance sheet as of December 31, 2017 for its interest rate swaps reflect the netting of the derivative asset or liability with the related collateral received or posted, respectively. The net amounts comparable to December 31, 2017 for the derivative asset and derivative liabilities as of December 31, 2016 were $104 and $(576), respectively.