Loans Held for Investment - Delinquencies (Details) - USD ($) $ in Thousands	Jun. 30, 2024	Dec. 31, 2023
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	$ 12,518,152	$ 13,318,571
Basis adjustment associated with fair value hedge
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	12,518,152	13,318,571
Basis adjustment associated with fair value hedge	28,201	29,551
Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	12,500,229	13,308,517
30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,985	2,484
60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,289	1,294
90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	9,649	6,276
Investor loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	8,206,124	8,576,026
Investor loans secured by real estate | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,245,474	2,421,772
Investor loans secured by real estate | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,473,606	5,645,310
Investor loans secured by real estate | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	453,799	472,544
Investor loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	33,245	36,400
Investor loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	8,204,577	8,575,194
Investor loans secured by real estate | Current | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,244,424	2,421,360
Investor loans secured by real estate | Current | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	5,473,606	5,645,310
Investor loans secured by real estate | Current | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	453,799	472,544
Investor loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	32,748	35,980
Investor loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,547	832
Investor loans secured by real estate | 90+ | CRE non-owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,050	412
Investor loans secured by real estate | 90+ | Multifamily
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+ | Construction and land
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Investor loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	497	420
Business loans secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,417,673	2,546,589
Business loans secured by real estate | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	46,543	50,741
Business loans secured by real estate | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,096,485	2,191,334
Business loans secured by real estate | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	274,645	304,514
Business loans secured by real estate | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,408,942	2,541,505
Business loans secured by real estate | Current | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	46,543	50,604
Business loans secured by real estate | Current | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,088,046	2,186,679
Business loans secured by real estate | Current | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	274,353	304,222
Business loans secured by real estate | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,852	429
Business loans secured by real estate | 30-59 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	137
Business loans secured by real estate | 30-59 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,852	0
Business loans secured by real estate | 30-59 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	292
Business loans secured by real estate | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 60-89 | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 60-89 | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 60-89 | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,879	4,655
Business loans secured by real estate | 90+ | SBA secured by real estate
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Business loans secured by real estate | 90+ | CRE owner-occupied
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	4,587	4,655
Business loans secured by real estate | 90+ | Franchise real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	292	0
Commercial loans
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,822,597	2,121,255
Commercial loans | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,554,735	1,790,608
Commercial loans | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	257,516	319,721
Commercial loans | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	10,346	10,926
Commercial loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,814,952	2,117,136
Commercial loans | Current | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,552,024	1,788,855
Commercial loans | Current | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	253,117	318,162
Commercial loans | Current | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	9,811	10,119
Commercial loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,133	2,036
Commercial loans | 30-59 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,133	228
Commercial loans | 30-59 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	1,559
Commercial loans | 30-59 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	249
Commercial loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,289	1,294
Commercial loans | 60-89 | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	449	1,294
Commercial loans | 60-89 | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	2,840	0
Commercial loans | 60-89 | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Commercial loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	3,223	789
Commercial loans | 90+ | Commercial and industrial
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,129	231
Commercial loans | 90+ | Franchise non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,559	0
Commercial loans | 90+ | SBA non-real estate secured
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	535	558
Retail loans
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	71,758	74,701
Retail loans | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	70,380	72,752
Retail loans | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,378	1,949
Retail loans | Current
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	71,758	74,682
Retail loans | Current | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	70,380	72,733
Retail loans | Current | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	1,378	1,949
Retail loans | 30-59
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	19
Retail loans | 30-59 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	19
Retail loans | 30-59 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89 | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 60-89 | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+ | Single family residential
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	0	0
Retail loans | 90+ | Consumer
Financing Receivable, Past Due [Line Items]
Loans held for investment before basis adjustment	$ 0	$ 0