Securities Available for Sale and Held to Maturity - Contractual Maturities of Available for Sale and Held to Maturity Securities (Details) - USD ($) $ in Thousands	Jun. 30, 2019	Dec. 31, 2018
Securities Available for Sale, Amortized Cost [Abstract]
Due in one year or less	$ 29,808
Due after one year through five years	8,541
Due after five years through ten years	2,680
Due after ten years	24,508
Total amortized cost, before mortgage-backed securities	65,537
Mortgage-backed securities	1,582,001
Securities Available for Sale, Amortized Cost	1,647,538	$ 1,854,529
Securities Available for Sale, Estimated Fair Value [Abstract]
Due in one year or less	29,944
Due after one year through five years	8,646
Due after five years through ten years	2,631
Due after ten years	24,123
Total fair value, before mortgage-backed securities	65,344
Mortgage-backed securities	1,578,381
Total	1,643,725	1,811,813
Securities Held to Maturity, Amortized Cost [Abstract]
Due in one year or less	180
Due after one year through five years	32,580
Due after five years through ten years	3,858
Due after ten years	0
Total amortized cost, before mortgage-backed securities	36,618
Mortgage-backed securities	788,918
Securities Held to Maturity, Amortized Cost	825,536	909,643
Securities Held to Maturity, Estimated Fair Value [Abstract]
Due in one year or less	181
Due after one year through five years	32,968
Due after five years through ten years	4,101
Due after ten years	0
Total fair value, before mortgage-backed securities	37,250
Mortgage-backed securities	795,111
Total	$ 832,361	$ 889,733