Stockholders' Equity - Actual Capital Amounts and Ratios (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Parent Company
Compliance with Regulatory Capital Requirements under Banking Regulations [Line Items]
Capital conservation buffer, ratio	2.50%
Capital, amount	$ 1,061,555	$ 1,094,062
Capital to risk weighted assets, ratio	17.25%	23.45%
Capital required for capital adequacy, amount	$ 492,438	$ 373,276
Capital required for capital adequacy to risk weighted assets, ratio	8.00%	8.00%
Capital required for capital adequacy with capital buffer, amount	$ 646,324	$ 460,763
Capital required for capital adequacy with capital buffer to risk weighted assets, ratio	10.50%	9.88%
Tier one risk based capital, amount	$ 988,172	$ 1,035,477
Tier one risk based capital to risk weighted assets, ratio	16.05%	22.19%
Tier one risk based capital required for capital adequacy, amount	$ 369,328	$ 279,957
Tier one risk based capital required for capital adequacy to risk weighted assets, ratio	6.00%	6.00%
Tier one risk based capital required for capital adequacy with capital buffer, amount	$ 523,215	$ 367,444
Tier one risk based capital required for capital adequacy with capital buffer to risk weighted assets, ratio	8.50%	7.88%
Common equity tier one capital, amount	$ 980,995	$ 1,035,477
Common equity tier one capital ratio	15.94%	22.19%
Common equity tier one capital required for capital adequacy, amount	$ 276,996	$ 209,968
Common equity tier one capital required for capital adequacy to risk weighted assets, ratio	4.50%	4.50%
Common equity tier one risk based capital required for capital adequacy with capital buffer, amount	$ 430,883	$ 297,455
Common equity tier one risk based capital requirement for capital adequacy with capital buffer to risk weighted assets, ratio	7.00%	6.38%
Tier one leverage capital, amount	$ 988,172	$ 1,035,477
Tier one leverage capital to average assets, ratio	12.92%	15.75%
Tier one leverage capital required for capital adequacy, amount	$ 305,824	$ 263,037
Tier one leverage capital required for capital adequacy to average assets, ratio	4.00%	4.00%
Tier one leverage capital required for capital adequacy with capital buffer to average assets, amount	$ 305,824	$ 263,037
Tier one leverage capital requirement for capital adequacy with capital buffer to average assets, ratio	4.00%	4.00%
Subsidiaries
Compliance with Regulatory Capital Requirements under Banking Regulations [Line Items]
Capital, amount	$ 844,664	$ 886,728
Capital to risk weighted assets, ratio	14.25%	19.04%
Capital required for capital adequacy, amount	$ 474,125	$ 372,550
Capital required for capital adequacy to risk weighted assets, ratio	8.00%	8.00%
Capital required for capital adequacy with capital buffer, amount	$ 622,290	$ 459,866
Capital required for capital adequacy with capital buffer to risk weighted assets, ratio	10.50%	9.88%
Capital required to be well capitalized, amount	$ 592,657	$ 465,687
Capital required to be well capitalized to risk weighted assets, ratio	10.00%	10.00%
Tier one risk based capital, amount	$ 782,881	$ 828,257
Tier one risk based capital to risk weighted assets, ratio	13.21%	17.79%
Tier one risk based capital required for capital adequacy, amount	$ 355,594	$ 279,412
Tier one risk based capital required for capital adequacy to risk weighted assets, ratio	6.00%	6.00%
Tier one risk based capital required for capital adequacy with capital buffer, amount	$ 503,758	$ 366,729
Tier one risk based capital required for capital adequacy with capital buffer to risk weighted assets, ratio	8.50%	7.88%
Tier one risk based capital required to be well capitalized, amount	$ 474,125	$ 372,550
Tier one risk based capital required to be well capitalized to risk weighted assets, ratio	8.00%	8.00%
Common equity tier one capital, amount	$ 782,881	$ 828,257
Common equity tier one capital ratio	13.21%	17.79%
Common equity tier one capital required for capital adequacy, amount	$ 266,696	$ 209,559
Common equity tier one capital required for capital adequacy to risk weighted assets, ratio	4.50%	4.50%
Common equity tier one risk based capital required for capital adequacy with capital buffer, amount	$ 414,860	$ 296,875
Common equity tier one risk based capital requirement for capital adequacy with capital buffer to risk weighted assets, ratio	7.00%	6.38%
Common equity tier one capital required to be well-capitalized, amount	$ 385,227	$ 302,697
Common equity tier one capital required to be well-capitalized to risk weighted assets, ratio	6.50%	6.50%
Tier one leverage capital, amount	$ 782,881	$ 828,257
Tier one leverage capital to average assets, ratio	10.25%	12.60%
Tier one leverage capital required for capital adequacy, amount	$ 305,423	$ 263,025
Tier one leverage capital required for capital adequacy to average assets, ratio	4.00%	4.00%
Tier one leverage capital required for capital adequacy with capital buffer to average assets, amount	$ 305,423	$ 263,025
Tier one leverage capital requirement for capital adequacy with capital buffer to average assets, ratio	4.00%	4.00%
Tier one leverage capital required to be well capitalized, amount	$ 381,779	$ 382,006
Tier one leverage capital required to be well capitalized to average assets, ratio	5.00%	5.00%