Loans and Allowance for Loan Losses - Loans by Portfolio Segment (Details) - USD ($)	Mar. 31, 2020	Jan. 01, 2020	Dec. 31, 2019	Mar. 31, 2019	Dec. 31, 2018
Loans and Leases Receivable Disclosure [Abstract]
Total loans	$ 3,601,002,000		$ 3,568,204,000
Less: Allowance for loan losses	53,638,000	$ 24,797,000	24,797,000	$ 24,155,000	$ 27,019,000
Total loans	3,547,364,000		3,543,407,000
Construction Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Total loans	603,952,000		644,948,000
Less: Allowance for loan losses	9,654,000	3,539,000	3,539,000	4,259,000	3,597,000
1-4 Family Residential Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Total loans	787,875,000		787,562,000
Less: Allowance for loan losses	2,640,000	3,833,000	3,833,000	3,382,000	3,844,000
Commercial Real Estate Loans
Loans and Leases Receivable Disclosure [Abstract]
Total loans	1,350,818,000		1,250,208,000
Less: Allowance for loan losses	36,120,000	9,572,000	9,572,000	10,660,000	13,968,000
Commercial loans
Loans and Leases Receivable Disclosure [Abstract]
Total loans	383,984,000		401,521,000
Less: Allowance for loan losses	4,519,000	6,351,000	6,351,000	4,287,000	3,974,000
Municipal loans
Loans and Leases Receivable Disclosure [Abstract]
Total loans	375,934,000		383,960,000
Less: Allowance for loan losses	47,000	570,000	570,000	508,000	525,000
Loans to individuals
Loans and Leases Receivable Disclosure [Abstract]
Total loans	98,439,000		100,005,000
Less: Allowance for loan losses	$ 658,000	$ 932,000	$ 932,000	$ 1,059,000	$ 1,111,000