Loans and Asset Quality - Summary of Current, Past Due, and Nonaccrual Loans (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
Current	$ 1,644,564	$ 1,433,258
Nonaccrual	4,305	5,319
Loans held for investment	1,649,272	1,438,924
30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	321	347
90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	82	0
Real estate | Commercial real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Current	564,989	530,712
Nonaccrual	1,857	1,278
Loans held for investment	567,037	531,990
Real estate | Commercial real estate | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	191	0
Real estate | Commercial real estate | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | One-to-four family residential
Financing Receivable, Credit Quality Indicator [Line Items]
Current	425,959	419,229
Nonaccrual	729	607
Loans held for investment	426,758	420,020
Real estate | One-to-four family residential | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	70	184
Real estate | One-to-four family residential | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | Construction and development
Financing Receivable, Credit Quality Indicator [Line Items]
Current	129,879	132,423
Nonaccrual	0	38
Loans held for investment	129,879	132,461
Real estate | Construction and development | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | Construction and development | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Commercial and industrial
Financing Receivable, Credit Quality Indicator [Line Items]
Current	247,463	264,427
Nonaccrual	1,714	3,370
Loans held for investment	249,313	267,940
Commercial and industrial | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	56	143
Commercial and industrial | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	80	0
SBA PPP, net of deferred income
Financing Receivable, Credit Quality Indicator [Line Items]
Current	193,532	0
Nonaccrual	0	0
Loans held for investment	193,532	0
SBA PPP, net of deferred income | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
SBA PPP, net of deferred income | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Tax-exempt
Financing Receivable, Credit Quality Indicator [Line Items]
Current	59,418	56,494
Nonaccrual	0	0
Loans held for investment	59,418	56,494
Tax-exempt | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Tax-exempt | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Current	23,324	29,973
Nonaccrual	5	26
Loans held for investment	23,335	30,019
Consumer | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	4	20
Consumer | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	$ 2	$ 0