Loans, Allowance for Loan Losses and Allowance for Unfunded Credit Commitments - Recorded Investment in Loans Modified in TDRs (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	$ 55,669	$ 90,171	$ 41,074
Global fund banking
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	0	0	0
Investor dependent
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	31,905	29,209	28,368
Investor dependent | Early stage
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	6,112	9,471	660
Investor dependent | Mid stage
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	897	3,445	6,657
Investor dependent | Later stage
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	24,896	16,293	21,051
Cash flow dependent
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	22,766	48,153	12,386
Cash flow dependent | Sponsor led buyout
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	21,529	48,153	0
Cash flow dependent | Other
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	1,237	0	12,386
Private Bank
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	0	1,792	320
Balance sheet dependent
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	0	0	0
Premium wine
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	998	11,017	0
Other
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	0	0	0
SBA loans
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	0	0	0
Payment deferrals
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	54,800	86,900	$ 41,100
Partial forgiveness of principal
Financing Receivable, Troubled Debt Restructuring [Line Items]
Loans modified in TDRs during the period	$ 900	$ 3,300