Loans and Asset Quality - Summary of Current, Past Due, and Nonaccrual Loans (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
Current	$ 1,610,333	$ 1,433,258
Nonaccrual	3,442	5,319
Loans held for investment	1,615,298	1,438,924
30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	1,523	347
90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | Commercial real estate
Financing Receivable, Credit Quality Indicator [Line Items]
Current	533,734	530,712
Nonaccrual	1,248	1,278
Loans held for investment	536,166	531,990
Real estate | Commercial real estate | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	1,184	0
Real estate | Commercial real estate | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | One-to-four family residential
Financing Receivable, Credit Quality Indicator [Line Items]
Current	419,808	419,229
Nonaccrual	613	607
Loans held for investment	420,545	420,020
Real estate | One-to-four family residential | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	124	184
Real estate | One-to-four family residential | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | Construction and development
Financing Receivable, Credit Quality Indicator [Line Items]
Current	140,907	132,423
Nonaccrual	0	38
Loans held for investment	140,907	132,461
Real estate | Construction and development | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Real estate | Construction and development | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Commercial and industrial
Financing Receivable, Credit Quality Indicator [Line Items]
Current	242,838	264,427
Nonaccrual	1,576	3,370
Loans held for investment	244,587	267,940
Commercial and industrial | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	173	143
Commercial and industrial | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
SBA PPP, net of deferred income
Financing Receivable, Credit Quality Indicator [Line Items]
Current	192,655	0
Nonaccrual	0	0
Loans held for investment	192,655	0
SBA PPP, net of deferred income | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
SBA PPP, net of deferred income | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Tax-exempt
Financing Receivable, Credit Quality Indicator [Line Items]
Current	56,065	56,494
Nonaccrual	0	0
Loans held for investment	56,065	56,494
Tax-exempt | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Tax-exempt | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	0	0
Consumer
Financing Receivable, Credit Quality Indicator [Line Items]
Current	24,326	29,973
Nonaccrual	5	26
Loans held for investment	24,373	30,019
Consumer | 30-89 Days Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	42	20
Consumer | 90 Days or More Past Due
Financing Receivable, Credit Quality Indicator [Line Items]
Past Due	$ 0	$ 0