Fair Value of Financial Instruments Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	$ 3,493,701	$ 3,516,239
Derivative assets	7,835	5,727
Derivative liabilities	41,205	35,898
Fair Value, Measurements, Recurring [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	3,493,701	3,516,239
Derivative assets	7,835	5,727
Total assets carried at fair value	3,501,536	3,521,966
Derivative liabilities	41,205	35,898
Total liabilities carried at fair value	41,205	35,898
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 1 [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	0	0
Derivative assets	0	0
Total assets carried at fair value	0	0
Derivative liabilities	29,097	32,896
Total liabilities carried at fair value	29,097	32,896
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 2 [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	3,477,266	3,472,282
Derivative assets	7,835	5,727
Total assets carried at fair value	3,485,101	3,478,009
Derivative liabilities	12,108	3,002
Total liabilities carried at fair value	12,108	3,002
Fair Value, Measurements, Recurring [Member] | Fair Value, Inputs, Level 3 [Member]
Fair Value, Assets and Liabilities Measured on Recurring and Nonrecurring Basis [Line Items]
Mortgage-backed securities	16,435	43,957
Derivative assets	0	0
Total assets carried at fair value	16,435	43,957
Derivative liabilities	0	0
Total liabilities carried at fair value	$ 0	$ 0