Loans Receivable and Allowance for Loan Losses - Allowance for Credit Losses on Financing Receivables (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2019	Dec. 31, 2018
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	$ 92,557	$ 84,541	$ 79,343
Provision (credit) for loan losses	7,300	14,900	17,360
Loans charged off	(8,109)	(10,875)	(17,410)
Recoveries of charged offs	3,736	3,991	5,248
PCI allowance adjustment	(1,340)
Balance, end of period	94,144	92,557	84,541
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				$ 3,402	$ 4,791
Allowance for loan losses - Collectively evaluated for impairment				82,663	75,603
Allowance for loan losses - Total	94,144	84,541	84,541	94,144	92,557
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				90,494	104,007
Loans outstanding - Collectively evaluated for impairment				12,077,202	11,846,646
Total loans outstanding				12,273,265	12,097,906
Loans and Leases Receivable, Other Information
Commitment to lend				636	736
Credit for unfunded loan commitments	(100)	(100)	(2,358)
Real Estate
Loans outstsanding:
Total loans outstanding				8,664,965	8,721,600
Commercial Business
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	27,765
Balance, end of period	32,578	27,765
Allowance for loan losses:
Allowance for loan losses - Total	27,765	27,765		32,578	27,765
Loans outstsanding:
Total loans outstanding				2,558,351	2,127,630
Trade Finance
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	719
Balance, end of period	454	719
Allowance for loan losses:
Allowance for loan losses - Total	719	719		454	719
Loans outstsanding:
Total loans outstanding				160,859	197,190
Consumer and Other
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	7,306
Balance, end of period	7,519	7,306
Allowance for loan losses:
Allowance for loan losses - Total	7,306	7,306		7,519	7,306
Loans outstsanding:
Total loans outstanding				889,090	1,051,486
Receivables Acquired with Deteriorated Credit Quality
Allowance for Loan Losses and the Loans Receivables by Impairment Methodology
Purchased credit impaired loans with subsequent credit deterioration				18,800	57,900
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	12,163
Balance, end of period	8,079	12,163
Allowance for loan losses:
Allowance for loan losses - Total	12,163	12,163		8,079	12,163
Loans outstsanding:
Total loans outstanding				105,569	147,253
Legacy
Loans outstsanding:
Total loans outstanding					10,199,858
Legacy | Real Estate
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	49,446	45,360	38,956
Provision (credit) for loan losses	(2,993)	9,334	8,524
Loans charged off	(1,159)	(6,273)	(2,292)
Recoveries of charged offs	1,706	1,025	172
PCI allowance adjustment	0
Balance, end of period	47,000	49,446	45,360
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				202	176
Allowance for loan losses - Collectively evaluated for impairment				46,798	49,270
Allowance for loan losses - Total	49,446	45,360	45,360	47,000	49,446
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				37,218	39,976
Loans outstanding - Collectively evaluated for impairment				7,445,529	7,037,392
Total loans outstanding				7,482,747	7,077,368
Legacy | Commercial Business
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	21,826	17,228	23,430
Provision (credit) for loan losses	9,517	3,389	(1,036)
Loans charged off	(4,121)	(1,400)	(9,881)
Recoveries of charged offs	1,083	2,609	4,715
PCI allowance adjustment	0
Balance, end of period	28,305	21,826	17,228
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				2,198	4,221
Allowance for loan losses - Collectively evaluated for impairment				26,107	17,605
Allowance for loan losses - Total	21,826	21,826	17,228	28,305	21,826
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				19,044	29,624
Loans outstanding - Collectively evaluated for impairment				2,479,744	1,988,067
Total loans outstanding				2,498,788	2,017,691
Legacy | Trade Finance
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	719	1,674	1,897
Provision (credit) for loan losses	(481)	(588)	1,825
Loans charged off	0	(408)	(2,104)
Recoveries of charged offs	216	41	56
PCI allowance adjustment	0
Balance, end of period	454	719	1,674
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				0	0
Allowance for loan losses - Collectively evaluated for impairment				454	719
Allowance for loan losses - Total	454	1,674	1,674	454	719
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				103	5,887
Loans outstanding - Collectively evaluated for impairment				160,756	188,179
Total loans outstanding				160,859	194,066
Legacy | Consumer and Other
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	6,269	3,385	2,116
Provision (credit) for loan losses	1,978	4,098	2,207
Loans charged off	(1,144)	(1,245)	(943)
Recoveries of charged offs	34	31	5
PCI allowance adjustment	0
Balance, end of period	7,137	6,269	3,385
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				11	3
Allowance for loan losses - Collectively evaluated for impairment				7,126	6,266
Allowance for loan losses - Total	6,269	6,269	2,116	7,137	6,269
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				2,202	441
Loans outstanding - Collectively evaluated for impairment				843,061	910,292
Total loans outstanding				845,263	910,733
Legacy | Receivables Acquired with Deteriorated Credit Quality | Real Estate
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	0
Balance, end of period	0	0
Allowance for loan losses:
Allowance for loan losses - Total	0	0		0	0
Loans outstsanding:
Total loans outstanding				0	0
Legacy | Receivables Acquired with Deteriorated Credit Quality | Commercial Business
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	0
Balance, end of period	0	0
Allowance for loan losses:
Allowance for loan losses - Total	0	0		0	0
Loans outstsanding:
Total loans outstanding				0	0
Legacy | Receivables Acquired with Deteriorated Credit Quality | Trade Finance
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	0
Balance, end of period	0	0
Allowance for loan losses:
Allowance for loan losses - Total	0	0		0	0
Loans outstsanding:
Total loans outstanding				0	0
Legacy | Receivables Acquired with Deteriorated Credit Quality | Consumer and Other
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	0
Balance, end of period	0	0
Allowance for loan losses:
Allowance for loan losses - Total	0	0		0	0
Loans outstsanding:
Total loans outstanding				0	0
Acquired
Loans outstsanding:
Total loans outstanding					1,898,048
Acquired | Real Estate
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	7,321	13,322	12,791
Provision (credit) for loan losses	(482)	(5,551)	1,341
Loans charged off	(644)	(453)	(850)
Recoveries of charged offs	398	3	40
PCI allowance adjustment	0
Balance, end of period	6,593	7,321	13,322
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				110	261
Allowance for loan losses - Collectively evaluated for impairment				1,818	1,264
Allowance for loan losses - Total	6,593	13,322	12,791	6,593	7,321
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				27,380	18,080
Loans outstanding - Collectively evaluated for impairment				1,057,074	1,507,858
Total loans outstanding				1,182,218	1,644,232
Acquired | Commercial Business
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	5,939	3,527	117
Provision (credit) for loan losses	(120)	3,253	4,500
Loans charged off	(965)	(1,083)	(1,315)
Recoveries of charged offs	297	242	225
PCI allowance adjustment	(878)
Balance, end of period	4,273	5,939	3,527
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				875	130
Allowance for loan losses - Collectively evaluated for impairment				353	460
Allowance for loan losses - Total	4,273	3,527	3,527	4,273	5,939
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				3,695	5,734
Loans outstanding - Collectively evaluated for impairment				48,968	80,916
Total loans outstanding				59,563	109,939
Acquired | Trade Finance
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	0	42	0
Provision (credit) for loan losses	0	(42)	42
Loans charged off	0	0	0
Recoveries of charged offs	0	0	0
PCI allowance adjustment	0
Balance, end of period	0	0	42
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				0	0
Allowance for loan losses - Collectively evaluated for impairment				0	0
Allowance for loan losses - Total	0	0	42	0	0
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				0	3,124
Loans outstanding - Collectively evaluated for impairment				0	0
Total loans outstanding				0	3,124
Acquired | Consumer and Other
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	1,037	3	36
Provision (credit) for loan losses	(119)	1,007	(43)
Loans charged off	(76)	(13)	(25)
Recoveries of charged offs	2	40	35
PCI allowance adjustment	(462)
Balance, end of period	382	1,037	3
Allowance for loan losses:
Allowance for loan losses - Individually evaluated for impairment				6	0
Allowance for loan losses - Collectively evaluated for impairment				7	19
Allowance for loan losses - Total	1,037	1,037	$ 3	382	1,037
Loans outstsanding:
Loans outstanding - Individually evaluated for impairment				852	1,141
Loans outstanding - Collectively evaluated for impairment				42,070	133,942
Total loans outstanding				43,827	140,753
Acquired | Receivables Acquired with Deteriorated Credit Quality | Real Estate
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	5,796
Balance, end of period	4,665	5,796
Allowance for loan losses:
Allowance for loan losses - Total	5,796	5,796		4,665	5,796
Loans outstsanding:
Total loans outstanding				97,764	118,294
Acquired | Receivables Acquired with Deteriorated Credit Quality | Commercial Business
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	5,349
Balance, end of period	3,045	5,349
Allowance for loan losses:
Allowance for loan losses - Total	5,349	5,349		3,045	5,349
Loans outstsanding:
Total loans outstanding				6,900	23,289
Acquired | Receivables Acquired with Deteriorated Credit Quality | Trade Finance
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	0
Balance, end of period	0	0
Allowance for loan losses:
Allowance for loan losses - Total	0	0		0	0
Loans outstsanding:
Total loans outstanding				0	0
Acquired | Receivables Acquired with Deteriorated Credit Quality | Consumer and Other
Allowance for Loan Losses by Portfolio Segment
Balance, beginning of period	1,018
Balance, end of period	369	1,018
Allowance for loan losses:
Allowance for loan losses - Total	$ 1,018	$ 1,018		369	1,018
Loans outstsanding:
Total loans outstanding				$ 905	$ 5,670