Loans and Allowance for Loan Losses - Loans by Portfolio Segment (Details) - USD ($) $ in Thousands	Jun. 30, 2021	Mar. 31, 2021	Dec. 31, 2020	Jun. 30, 2020	Mar. 31, 2020	Dec. 31, 2019
Loans and Leases Receivable Disclosure [Abstract]
Loans	$ 3,642,346		$ 3,657,779
Less: Allowance for loan losses	42,913	$ 41,454	49,006	$ 59,868	$ 53,638	$ 24,797
Total loans	3,599,433		3,608,773
Construction Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	528,157		581,941
Less: Allowance for loan losses	8,018	8,201	6,490	8,500	9,654	3,539
1-4 Family Residential Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	678,402		719,952
Less: Allowance for loan losses	2,520	2,508	2,270	2,702	2,640	3,833
Commercial Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	1,430,900		1,295,746
Less: Allowance for loan losses	28,930	27,236	35,709	43,785	36,120	9,572
Commercial loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	497,513		557,122
Less: Allowance for loan losses	3,097	3,108	4,107	4,221	4,519	6,351
Municipal loans
Loans and Leases Receivable Disclosure [Abstract]
Loans	417,398		409,028
Less: Allowance for loan losses	47	46	46	47	47	570
Loans to individuals
Loans and Leases Receivable Disclosure [Abstract]
Loans	89,976		93,990
Less: Allowance for loan losses	$ 301	$ 355	$ 384	$ 613	$ 658	$ 932