Loans - Credit Quality Indicators by Class of Loans (Details) - USD ($) $ in Thousands	Jun. 30, 2019	Dec. 31, 2018	Jun. 30, 2018
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	$ 880,037	$ 899,785	$ 1,137,255
Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	226,871	267,340	401,048
Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	77,482	87,506	94,679
Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	409,694	368,449	444,277
Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	126,043	132,435	145,671
Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	39,476	43,506	50,888
Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	471	549	$ 692
Pass
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	789,043	822,634
Pass | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	194,672	240,232
Pass | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	70,963	83,240
Pass | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	366,939	329,213
Pass | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	116,599	125,984
Pass | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	39,399	43,416
Pass | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	471	549
Special Mention
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	35,325	49,604
Special Mention | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	10,727	20,259
Special Mention | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	1,463	3,910
Special Mention | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	21,554	23,475
Special Mention | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	1,581	1,955
Special Mention | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	5
Special Mention | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Substandard
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	54,960	27,547
Substandard | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	21,472	6,849
Substandard | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	5,047	356
Substandard | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	21,201	15,761
Substandard | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	7,176	4,496
Substandard | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	64	85
Substandard | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	709	0
Doubtful | Commercial, financial and agricultural
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Real estate – construction
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	9	0
Doubtful | Real estate – commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	0	0
Doubtful | Real estate – residential
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	687	0
Doubtful | Consumer and other
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	13	0
Doubtful | Lease financing receivable
Financing Receivable, Allowance for Credit Losses [Line Items]
Loans	$ 0	$ 0