Securities Available for Sale and Held to Maturity - Contractual Maturities of Available for Sale and Held to Maturity Securities (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Securities Available for Sale, Amortized Cost [Abstract]
Due in one year or less	$ 7,867
Due after one year through five years	1,559
Due after five years through ten years	2,161
Due after ten years	19,232
Total amortized cost, before mortgage-backed securities	30,819
Mortgage-backed securities	1,808,759
Securities Available for Sale, Amortized Cost	1,839,578	$ 1,600,984
Securities Available for Sale, Estimated Fair Value [Abstract]
Due in one year or less	7,945
Due after one year through five years	1,601
Due after five years through ten years	2,112
Due after ten years	19,416
Total fair value, before mortgage-backed securities	31,074
Mortgage-backed securities	1,853,079
Total	1,884,153	1,602,404
Securities Held to Maturity, Amortized Cost [Abstract]
Due in one year or less	18,599
Due after one year through five years	13,030
Due after five years through ten years	0
Due after ten years	0
Total amortized cost, before mortgage-backed securities	31,629
Mortgage-backed securities	628,419
Securities Held to Maturity, Amortized Cost	660,048	738,099
Securities Held to Maturity, Estimated Fair Value [Abstract]
Due in one year or less	18,715
Due after one year through five years	13,350
Due after five years through ten years	0
Due after ten years	0
Total fair value, before mortgage-backed securities	32,065
Mortgage-backed securities	658,496
Total	$ 690,561	$ 746,202