Fair Value of Financial Instruments Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Sep. 30, 2016	Dec. 31, 2015
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	$ 3,110,467	$ 3,493,701
Derivative Instruments Not Designated as Hedging Instruments, Asset, at Fair Value	12,985	7,835
Derivative assets	12,985	7,835
Derivative liabilities	84,023	41,205
Fair Value, Measurements, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	3,110,467	3,493,701
Derivative Instruments Not Designated as Hedging Instruments, Asset, at Fair Value	12,985
Derivative assets		7,835
Total assets carried at fair value	3,123,452	3,501,536
Derivative liabilities	84,023	41,205
Total liabilities carried at fair value	84,023	41,205
Fair Value, Measurements, Recurring | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	0	0
Derivative assets	0	0
Total assets carried at fair value	0	0
Derivative liabilities	42,638	29,097
Total liabilities carried at fair value	42,638	29,097
Fair Value, Measurements, Recurring | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	3,097,055	3,477,266
Derivative assets	12,985	7,835
Total assets carried at fair value	3,110,040	3,485,101
Derivative liabilities	41,385	12,108
Total liabilities carried at fair value	41,385	12,108
Fair Value, Measurements, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	13,412	16,435
Derivative assets	0	0
Total assets carried at fair value	13,412	16,435
Derivative liabilities	0	0
Total liabilities carried at fair value	$ 0	$ 0