Loans, Allowance for Loan Losses and Allowance for Unfunded Credit Commitments - Nonaccrual Loans with No Allowance for Credit Loss (Details) - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	$ 104,244	$ 102,669
Nonaccrual Loans with no Allowance for Credit Loss	7,850	7,230
Global fund banking
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	11	0
Nonaccrual Loans with no Allowance for Credit Loss	11	0
Investor dependent
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	51,053	34,719
Nonaccrual Loans with no Allowance for Credit Loss	3,280	734
Investor dependent | Early stage
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	18,340	11,093
Nonaccrual Loans with no Allowance for Credit Loss	3	460
Investor dependent | Mid stage
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	4,056	17,330
Nonaccrual Loans with no Allowance for Credit Loss	3,159	274
Investor dependent | Later stage
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	28,657	6,296
Nonaccrual Loans with no Allowance for Credit Loss	118	0
Cash flow dependent
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	46,000	62,266
Nonaccrual Loans with no Allowance for Credit Loss	1,138	2,782
Cash flow dependent | Sponsor led buyout
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	39,996	44,585
Nonaccrual Loans with no Allowance for Credit Loss	0	0
Cash flow dependent | Other
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	6,004	17,681
Nonaccrual Loans with no Allowance for Credit Loss	1,138	2,782
Private Bank
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	6,152	5,480
Nonaccrual Loans with no Allowance for Credit Loss	2,393	3,714
Balance sheet dependent
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	0	0
Nonaccrual Loans with no Allowance for Credit Loss	0	0
Premium wine
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	998	204
Nonaccrual Loans with no Allowance for Credit Loss	998	0
Other
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	30	0
Nonaccrual Loans with no Allowance for Credit Loss	30	0
SBA loans
Financing Receivable, Allowance for Credit Loss [Line Items]
Nonaccrual Loans	0	0
Nonaccrual Loans with no Allowance for Credit Loss	$ 0	$ 0