Loans and Allowance for Loan Losses - Loans by Portfolio Segment (Details) - USD ($) $ in Thousands	Sep. 30, 2021	Jun. 30, 2021	Dec. 31, 2020	Sep. 30, 2020	Jun. 30, 2020	Dec. 31, 2019
Loans and Leases Receivable Disclosure [Abstract]
Loans	$ 3,647,585		$ 3,657,779
Less: Allowance for loan losses	38,022	$ 42,913	49,006	$ 55,110	$ 59,868	$ 24,797
Total loans	3,609,563		3,608,773
Construction Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	422,095		581,941
Less: Allowance for loan losses	4,983	8,018	6,490	5,626	8,500	3,539
1-4 Family Residential Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	660,689		719,952
Less: Allowance for loan losses	1,946	2,520	2,270	2,635	2,702	3,833
Commercial Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	1,605,132		1,295,746
Less: Allowance for loan losses	28,258	28,930	35,709	41,785	43,785	9,572
Commercial loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	443,708		557,122
Less: Allowance for loan losses	2,560	3,097	4,107	4,492	4,221	6,351
Municipal loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	427,259		409,028
Less: Allowance for loan losses	46	47	46	46	47	570
Loans to individuals
Loans and Leases Receivable Disclosure [Abstract]
Loans	88,702		93,990
Less: Allowance for loan losses	$ 229	$ 301	$ 384	$ 526	$ 613	$ 932