Securities Available for Sale and Held to Maturity - Contractual Maturities of Available for Sale and Held to Maturity Securities (Details) - USD ($) $ in Thousands	Sep. 30, 2019	Dec. 31, 2018
Securities Available for Sale, Amortized Cost [Abstract]
Due in one year or less	$ 29,699
Due after one year through five years	4,125
Due after five years through ten years	2,622
Due after ten years	23,376
Total amortized cost, before mortgage-backed securities	59,822
Mortgage-backed securities	1,493,542
Securities Available for Sale, Amortized Cost	1,553,364	$ 1,854,529
Securities Available for Sale, Estimated Fair Value [Abstract]
Due in one year or less	29,834
Due after one year through five years	4,164
Due after five years through ten years	2,571
Due after ten years	23,129
Total fair value, before mortgage-backed securities	59,698
Mortgage-backed securities	1,494,007
Total	1,553,705	1,811,813
Securities Held to Maturity, Amortized Cost [Abstract]
Due in one year or less	105
Due after one year through five years	31,701
Due after five years through ten years	3,770
Due after ten years	0
Total amortized cost, before mortgage-backed securities	35,576
Mortgage-backed securities	749,846
Securities Held to Maturity, Amortized Cost	785,422	909,643
Securities Held to Maturity, Estimated Fair Value [Abstract]
Due in one year or less	105
Due after one year through five years	32,081
Due after five years through ten years	4,002
Due after ten years	0
Total fair value, before mortgage-backed securities	36,188
Mortgage-backed securities	758,497
Total	$ 794,685	$ 889,733