Investment Securities - The amortized cost and estimated fair value of securities (Details) - USD ($)	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Investment securities available-for-sale:
Amortized Cost	$ 1,338,260,000	$ 1,111,168,000
Unrealized Gain	34,216,000	6,263,000
Unrealized Loss	(4,092,000)	(14,209,000)
Estimated Fair Value	1,368,384,000	1,103,222,000
Investment securities held-to-maturity:
Amortized Cost	37,838,000	45,210,000
Unrealized Gain	922,000	148,000
Unrealized Loss	0	(686,000)
Estimated Fair Value	38,760,000	44,672,000
Total investment securities
Amortized Cost	1,376,098,000	1,156,378,000
Unrealized Gain	35,138,000	6,411,000
Unrealized Loss	(4,092,000)	(14,895,000)
Estimated Fair Value	1,407,144,000	1,147,894,000
Additional disclosures
Accumulated other comprehensive income (loss)	30,100,000	(7,900,000)
Accumulated other comprehensive income (loss), net of tax	21,523,000	(5,601,000)
Average balance of repurchase agreement facilities		15,000,000.0
OTTI on investment securities		0	$ 0
Gross gains	10,300,000	1,600,000	3,100,000
Gross losses	1,800,000	208,000	386,000
Net proceeds from the sale of available-for-sale securities	551,784,000	407,004,000	$ 268,596,000
Investment securities pledged	125,700,000	215,300,000
Home Owners Association Loans
Additional disclosures
Value of inverse putable reverse repurchases secured by collateral		75,000
U.S. Treasury
Investment securities available-for-sale:
Amortized Cost	60,457,000	59,688,000
Unrealized Gain	3,137,000	1,224,000
Unrealized Loss	(39,000)	0
Estimated Fair Value	63,555,000	60,912,000
Agency
Investment securities available-for-sale:
Amortized Cost	240,348,000	128,958,000
Unrealized Gain	7,686,000	1,631,000
Unrealized Loss	(1,676,000)	(519,000)
Estimated Fair Value	246,358,000	130,070,000
Corporate debt
Investment securities available-for-sale:
Amortized Cost	149,150,000	104,158,000
Unrealized Gain	2,217,000	291,000
Unrealized Loss	(14,000)	(906,000)
Estimated Fair Value	151,353,000	103,543,000
Municipal bonds
Investment securities available-for-sale:
Amortized Cost	384,032,000	238,914,000
Unrealized Gain	13,450,000	1,941,000
Unrealized Loss	(184,000)	(2,225,000)
Estimated Fair Value	397,298,000	238,630,000
Collateralized mortgage obligation: residential
Investment securities available-for-sale:
Amortized Cost	9,869,000	24,699,000
Unrealized Gain	123,000	64,000
Unrealized Loss	(8,000)	(425,000)
Estimated Fair Value	9,984,000	24,338,000
Mortgage-backed securities: residential
Investment securities available-for-sale:
Amortized Cost	494,404,000	554,751,000
Unrealized Gain	7,603,000	1,112,000
Unrealized Loss	(2,171,000)	(10,134,000)
Estimated Fair Value	499,836,000	545,729,000
Investment securities held-to-maturity:
Amortized Cost	36,114,000	43,381,000
Unrealized Gain	922,000	148,000
Unrealized Loss	0	(686,000)
Estimated Fair Value	37,036,000	42,843,000
Additional disclosures
Estimated par value of securities pledged as collateral for the Bank's inverse putable reverse repurchases		20,300,000
Fair value of securities pledged as collateral for the Bank's inverse putable reverse repurchases		20,900,000
Other
Investment securities held-to-maturity:
Amortized Cost	1,724,000	1,829,000
Unrealized Gain	0	0
Unrealized Loss	0	0
Estimated Fair Value	$ 1,724,000	$ 1,829,000