Loans Held for Investment - Aging (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	$ 40,798	$ 55,823
Current Loans	7,809,106	7,325,577
Total loans	7,849,904	7,381,400
Accruing Loans Past Due 90 Days or More	939	3
Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	17,336	31,935
Financing Receivables, 60 to 89 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	2,510	7,918
Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	20,952	15,970
Prime Lending | U.S. Government Agencies
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Accruing Loans Past Due 90 Days or More	123,700	102,700
Unpaid principal balance loans past due 90 days or more	125,200	104,000
Commercial Real Estate
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total loans	3,043,723	3,000,523
Commercial Real Estate | Non-owner occupied
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	2,975	6,852
Current Loans	1,729,473	1,702,500
Total loans	1,732,448	1,709,352
Commercial Real Estate | Non-owner occupied | Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	1,662	4,062
Commercial Real Estate | Non-owner occupied | Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	1,313	2,790
Commercial Real Estate | Owner occupied
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	9,165	5,958
Current Loans	1,302,110	1,285,213
Total loans	1,311,275	1,291,171
Commercial Real Estate | Owner occupied | Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	1,822	1,813
Commercial Real Estate | Owner occupied | Financing Receivables, 60 to 89 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	681	880
Commercial Real Estate | Owner occupied | Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	6,662	3,265
Commercial | Commercial And Industrial
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	8,234	11,097
Current Loans	2,727,855	2,014,623
Total loans	2,736,089	2,025,720
Accruing Loans Past Due 90 Days or More	939	3
Commercial | Commercial And Industrial | Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	1,325	5,967
Commercial | Commercial And Industrial | Financing Receivables, 60 to 89 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans		1,735
Commercial | Commercial And Industrial | Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	6,909	3,395
Construction and land development
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	3,784	9,407
Current Loans	928,115	931,157
Total loans	931,899	940,564
Construction and land development | Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	3,755	7,580
Construction and land development | Financing Receivables, 60 to 89 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans		1,827
Construction and land development | Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	29
1 - 4 family residential
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	16,262	22,020
Current Loans	656,581	769,000
Total loans	672,843	791,020
1 - 4 family residential | Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	8,700	12,058
1 - 4 family residential | Financing Receivables, 60 to 89 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	1,807	3,442
1 - 4 family residential | Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	5,755	6,520
Consumer
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	378	489
Current Loans	42,081	46,557
Total loans	42,459	47,046
Consumer | Financing Receivables, 30 to 59 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	72	455
Consumer | Financing Receivables, 60 to 89 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	22	34
Consumer | Financing Receivables, Equal to Greater than 90 Days Past Due
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Total Past Due Loans	284
Broker-dealer
Non-Covered Loans and Allowance for Non-Covered Loan Losses
Current Loans	422,891	576,527
Total loans	$ 422,891	$ 576,527