Fair Value of Financial Instruments Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	$ 3,208,735	$ 3,493,701
Derivative assets	18,421	7,835
Derivative liabilities	90,260	41,205
Fair Value, Measurements, Recurring
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	3,208,735	3,493,701
Derivative assets	18,421	7,835
Total assets carried at fair value	3,227,156	3,501,536
Derivative liabilities	90,260	41,205
Total liabilities carried at fair value	90,260	41,205
Fair Value, Measurements, Recurring | Level 1
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	0	0
Derivative assets	0	0
Total assets carried at fair value	0	0
Derivative liabilities	45,469	29,097
Total liabilities carried at fair value	45,469	29,097
Fair Value, Measurements, Recurring | Level 2
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	3,194,840	3,477,266
Derivative assets	18,421	7,835
Total assets carried at fair value	3,213,261	3,485,101
Derivative liabilities	44,791	12,108
Total liabilities carried at fair value	44,791	12,108
Fair Value, Measurements, Recurring | Level 3
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	13,895	16,435
Derivative assets	0	0
Total assets carried at fair value	13,895	16,435
Derivative liabilities	0	0
Total liabilities carried at fair value	$ 0	$ 0