Derivative Financial Instruments - Aggregate Fair Value and Notional Amount of Derivative Financial Instruments (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Derivative [Line Items]
Fair Value	$ (29,734)	$ (46,401)
Notional Amount	4,980,343	6,483,976
Derivative Liabilities
Derivative [Line Items]
Fair Value	(66,329)	(62,794)
Notional Amount	2,423,362	3,083,315
Derivative Assets
Derivative [Line Items]
Fair Value	36,595	16,393
Notional Amount	2,556,981	3,400,661
Interest rate agreements | Cash Flow Hedging
Derivative [Line Items]
Notional Amount	140,000
Interest rate agreements | Derivative Liabilities
Derivative [Line Items]
Fair Value	(12,097)	(10,134)
Notional Amount	1,101,500	1,039,500
Interest rate agreements | Derivative Liabilities | Cash Flow Hedging
Derivative [Line Items]
Fair Value	(44,822)	(48,232)
Notional Amount	139,500	139,500
Interest rate agreements | Derivative Assets
Derivative [Line Items]
Fair Value	19,859	2,590
Notional Amount	1,009,000	658,000
TBAs
Derivative [Line Items]
Notional Amount	1,360,000	2,480,000
TBAs | Derivative Liabilities
Derivative [Line Items]
Fair Value	(4,681)	(2,519)
Notional Amount	510,000	1,450,500
TBAs | Derivative Assets
Derivative [Line Items]
Fair Value	8,300	2,734
Notional Amount	850,000	1,028,500
Futures
Derivative [Line Items]
Notional Amount	88,000	78,000
Futures | Derivative Liabilities
Derivative [Line Items]
Fair Value	(928)	(445)
Notional Amount	87,500	78,000
Swaptions | Derivative Assets
Derivative [Line Items]
Fair Value	5,121	5,191
Notional Amount	345,000	925,000
Credit default index swaps | Derivative Assets
Derivative [Line Items]
Fair Value	0	1,207
Notional Amount	0	25,000
Loan purchase commitments | Derivative Liabilities
Derivative [Line Items]
Fair Value	(3,801)	(1,464)
Notional Amount	584,862	375,815
Loan purchase commitments | Derivative Assets
Derivative [Line Items]
Fair Value	3,315	4,671
Notional Amount	$ 352,981	$ 764,161