Fair Value of Financial Instruments (Details) - USD ($) $ in Thousands	Sep. 30, 2017	Dec. 31, 2016
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	$ 2,921,444	$ 3,212,084
Derivative Instruments Not Designated as Hedging Instruments, Asset, at Fair Value	368	28,534	[1]
Derivative asset	368	28,534
Total assets carried at fair value	2,921,812	3,240,618
Derivative liabilities	133	6,922
Total liabilities carried at fair value	133	6,922
Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	0	0
Total assets carried at fair value	0	0
Total liabilities carried at fair value	0	0
Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	2,913,877	3,201,157
Total assets carried at fair value	2,914,245	3,229,691
Total liabilities carried at fair value	133	6,922
Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	7,567	10,927
Total assets carried at fair value	7,567	10,927
Total liabilities carried at fair value	0	0
TBA security
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative liabilities	133
TBA security | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative liabilities	0
TBA security | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative liabilities	133
TBA security | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative liabilities	0
Interest Rate Swap
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset		28,534
Derivative liabilities		6,922
Interest Rate Swap | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Derivative liabilities		0
Interest Rate Swap | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	368	28,534
Derivative liabilities		6,922
Interest Rate Swap | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative asset	0	0
Derivative liabilities		0
Not Designated as Hedging Instrument, Economic Hedge [Member] | Interest Rate Swap
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Derivative Instruments Not Designated as Hedging Instruments, Asset, at Fair Value	$ 368	$ 28,534	[1]

[1] Refer to Note 1 regarding information on a change in the CME rulebook. Amounts reported on the consolidated balance sheet as of September 30, 2017 for its interest rate swaps reflect the netting of the derivative asset or liability with the related collateral received or posted, respectively. The net amounts comparable to September 30, 2017 for the derivative asset and derivative liabilities as of December 31, 2016 were $104 and $(576), respectively.