Allowance for Loan Losses - Allowances by Loan Segment (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2021	Dec. 31, 2020	Dec. 31, 2019
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Balances at beginning of the period	$ 110,902	$ 52,223	$ 61,762
Reversal of (provision for) loan losses	(16,500)	88,620	(3,150)
Recoveries	3,277	920	2,450
Balances at end of the period	69,899	110,902	52,223
Allowance for loan losses by impairment methodology, Individually evaluated	11,791	29,948	4,274
Allowance for loan losses by impairment methodology, Collectively evaluated	58,108	80,954	47,949
Total Loans, Net of Unearned Income	5,409,440	5,842,337
Financing Receivable
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Investment in loans, net of unearned income, Individually evaluated	52,704	87,741	30,311
Investment in loans, net of unearned income, Collectively evaluated	5,356,736	5,754,596	5,714,028
Total Loans, Net of Unearned Income	5,409,440	5,842,337	5,744,339
Domestic
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	(27,780)	(30,558)	(3,744)
International
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	0	(303)	(5,095)
Total Loans, Net of Unearned Income	99,600	152,900
Real estate loans
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Balances at beginning of the period	50,227	25,040	22,778
Reversal of (provision for) loan losses	(21,338)	25,187	2,072
Recoveries	125	0	190
Balances at end of the period	17,952	50,227	25,040
Allowance for loan losses by impairment methodology, Individually evaluated	546	3,175	1,161
Allowance for loan losses by impairment methodology, Collectively evaluated	17,406	47,052	23,879
Total Loans, Net of Unearned Income	4,006,392	4,424,031
Real estate loans | Financing Receivable
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Investment in loans, net of unearned income, Individually evaluated	7,285	19,560	1,936
Investment in loans, net of unearned income, Collectively evaluated	2,346,923	2,796,092	2,968,589
Total Loans, Net of Unearned Income	2,354,208	2,815,652	2,970,525
Real estate loans | Domestic
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	(11,062)	0	0
Real estate loans | International
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	0	0	0
Commercial loans
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Balances at beginning of the period	48,130	22,482	30,018
Reversal of (provision for) loan losses	1,463	55,197	(6,165)
Recoveries	2,613	443	1,711
Balances at end of the period	38,979	48,130	22,482
Allowance for loan losses by impairment methodology, Individually evaluated	10,462	25,394	1,789
Allowance for loan losses by impairment methodology, Collectively evaluated	28,517	22,736	20,693
Total Loans, Net of Unearned Income	965,673	1,154,550
Commercial loans | Financing Receivable
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Investment in loans, net of unearned income, Individually evaluated	39,785	60,130	22,790
Investment in loans, net of unearned income, Collectively evaluated	2,075,338	2,210,601	2,206,566
Total Loans, Net of Unearned Income	2,115,123	2,270,731	2,229,356
Commercial loans | Domestic
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	(13,227)	(29,958)	(3,020)
Commercial loans | International
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	0	(34)	(62)
Loans to financial institutions and acceptances
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Balances at beginning of the period	1	42	445
Reversal of (provision for) loan losses	41	(41)	(403)
Recoveries	0	0	0
Balances at end of the period	42	1	42
Allowance for loan losses by impairment methodology, Individually evaluated	0	0	0
Allowance for loan losses by impairment methodology, Collectively evaluated	42	1	42
Total Loans, Net of Unearned Income	13,710	16,636
Loans to financial institutions and acceptances | Financing Receivable
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Investment in loans, net of unearned income, Individually evaluated	0	0	0
Investment in loans, net of unearned income, Collectively evaluated	14,127	17,574	16,552
Total Loans, Net of Unearned Income	14,127	17,574	16,552
Loans to financial institutions and acceptances | Domestic
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	0	0	0
Loans to financial institutions and acceptances | International
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	0	0	0
Consumer loans and overdrafts
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Balances at beginning of the period	12,544	4,659	8,521
Reversal of (provision for) loan losses	3,334	8,277	1,346
Recoveries	539	477	549
Balances at end of the period	12,926	12,544	4,659
Allowance for loan losses by impairment methodology, Individually evaluated	783	1,379	1,324
Allowance for loan losses by impairment methodology, Collectively evaluated	12,143	11,165	3,335
Total Loans, Net of Unearned Income	423,665	247,120	88,466
Consumer loans and overdrafts | Financing Receivable
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Investment in loans, net of unearned income, Individually evaluated	5,634	8,051	5,585
Investment in loans, net of unearned income, Collectively evaluated	920,348	730,329	522,321
Total Loans, Net of Unearned Income	925,982	738,380	527,906
Consumer loans and overdrafts | Domestic
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	(3,491)	(600)	(724)
Consumer loans and overdrafts | International
Financing Receivable, Allowance for Credit Loss [Roll Forward]
Loans charged-off	$ 0	$ (269)	$ (5,033)