Loans Receivable and Allowance for Loan Losses - Schedule of Accounts, Notes, Loans and Financing Receivable (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2019 USD ($) segment	Dec. 31, 2018 USD ($)	Dec. 31, 2017 USD ($)	Dec. 31, 2016 USD ($)
Real estate loans:
Loans	$ 12,273,265	$ 12,097,906
Less: deferred loan fees	2,742	209
Gross loans receivable	12,276,007	12,098,115
Less: allowance for loan losses	(94,144)	(92,557)	$ (84,541)	$ (79,343)
Loans receivable, net	$ 12,181,863	12,005,558
Number of portfolio segments | segment	4
Real Estate
Real estate loans:
Loans	$ 8,664,965	8,721,600
Real Estate | Real estate-Residential
Real estate loans:
Loans	52,437	51,197
Real Estate | Commercial & industrial
Real estate loans:
Loans	8,316,382	8,395,327
Real Estate | Real estate-Construction
Real estate loans:
Loans	296,146	275,076
Commercial Business
Real estate loans:
Loans	2,558,351	2,127,630
Less: allowance for loan losses	(32,578)	(27,765)
Trade Finance
Real estate loans:
Loans	160,859	197,190
Less: allowance for loan losses	(454)	(719)
Consumer and other
Real estate loans:
Loans	889,090	1,051,486
Less: allowance for loan losses	$ (7,519)	$ (7,306)