Accumulated Other Comprehensive Income (Loss) - Tax Effects (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2014
Accumulated Other Comprehensive Income (Loss)
Net unrealized gains (losses) on net investment hedges, before tax	$ (7,001)	$ 13,762
Net unrealized gains (losses) on net investment hedges, tax effect	1,815	(3,312)
Net unrealized gains (losses) on net investment hedges, net of tax	(5,186)	10,450	$ (2,746)
Total other comprehensive income (loss), before tax	111,512	(14,996)	24,897
Total other comprehensive income (loss), tax effect	(24,097)	375	(6,297)
Total other comprehensive income (loss), net of tax	87,415	(14,621)	18,600
Debt securities available for sale and interest-only strips
Net unrealized gains (losses) arising during the period, before tax	109,403	(16,373)	24,244
Net unrealized gains (losses) arising during the period, tax effect	(25,707)	4,002	(8,857)
Net unrealized gains (losses) arising during the period, net of tax	83,696	(12,371)	15,387
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), before tax	642	1,029	208
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), tax effect	(218)	(327)	859
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), net of tax	424	702	1,067
Net unrealized gains (losses) on debt securities available for sale and interest-only strips, before taxes	110,045	(15,344)	24,452
Net unrealized gains (losses) on debt securities available for sale and interest-only strips, tax effect	(25,925)	3,675	(7,998)
Net unrealized gains (losses) on debt securities available for sale and interest-only strips	84,120	(11,669)	16,454
Net investment hedges
Net unrealized gains (losses) on net investment hedges, before tax			(4,430)
Net unrealized gains (losses) on net investment hedges, tax effect			1,684
Net unrealized gains (losses) on net investment hedges, net of tax	87,415	10,450	(2,746)
Other Comprehensive Income (Loss), Foreign Currency Transaction and Translation Adjustment, Net of Tax [Abstract]
Foreign currency translation adjustment, before tax	8,514	(13,368)	4,921
Foreign currency translation adjustment, tax effect	0	0	0
Foreign currency translation adjustment, net of tax	8,514	(13,368)	4,921
Other Comprehensive (Income) Loss, Defined Benefit Plan, after Reclassification Adjustment, after Tax [Abstract]
Reclassification of amortization of prior service cost to Other non-interest expense, before tax	(46)	(46)	(46)
Reclassification of amortization of prior service cost to Other non-interest expense, tax effect	13	12	17
Reclassification of amortization of prior service cost to Other non-interest expense, net of tax	(33)	(34)	(29)
Available for sale securities transferred to held to maturity securities value				$ 191,700
Unrealized holding gain (loss) transfers from available for sale to held to maturity retained in accumulated other comprehensive income	8,400	11,000
Total interest income
Debt securities available for sale and interest-only strips
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), before tax	2,644	1,066	963
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), tax effect	(716)	(335)	572
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), net of tax	1,928	731	1,535
Gains (losses) on debt securities, net
Debt securities available for sale and interest-only strips
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), before tax	(1,517)	(127)
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), tax effect	369	31
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), net of tax	(1,148)	(96)
Other non-interest expense
Debt securities available for sale and interest-only strips
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), before tax	(485)	90	(755)
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), tax effect	129	(23)	287
Reclassification of net (gains) losses from accumulated other comprehensive income (loss), net of tax	$ (356)	$ 67	$ (468)