Securities Available for Sale and Held to Maturity - Contractual Maturities of Available for Sale and Held to Maturity Securities (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Securities Available for Sale, Amortized Cost [Abstract]
Due in one year or less	$ 4,667
Due after one year through five years	1,543
Due after five years through ten years	2,096
Due after ten years	18,726
Total amortized cost, before mortgage-backed securities	27,032
Mortgage-backed securities	1,858,828
Securities Available for Sale, Amortized Cost	1,885,860	$ 1,600,984
Securities Available for Sale, Estimated Fair Value [Abstract]
Due in one year or less	4,719
Due after one year through five years	1,589
Due after five years through ten years	2,051
Due after ten years	18,967
Total fair value, before mortgage-backed securities	27,326
Mortgage-backed securities	1,895,402
Total	1,922,728	1,602,404
Securities Held to Maturity, Amortized Cost [Abstract]
Due in one year or less	20,900
Due after one year through five years	10,705
Due after five years through ten years	0
Due after ten years	0
Total amortized cost, before mortgage-backed securities	31,605
Mortgage-backed securities	579,675
Securities Held to Maturity, Amortized Cost	611,280	738,099
Securities Held to Maturity, Estimated Fair Value [Abstract]
Due in one year or less	21,018
Due after one year through five years	10,950
Due after five years through ten years	0
Due after ten years	0
Total fair value, before mortgage-backed securities	31,968
Mortgage-backed securities	607,697
Total	$ 639,665	$ 746,202