Loans and Allowance for Loans Losses - Troubled debt restructurings (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due, Covered	$ 0	$ 0
60-89 Days Past Due, Covered	0	0
90 Days or More Past Due, Covered	0	0
Nonaccrual, Covered	8	17
Total Past Due and Nonaccrual, Covered	8	17
Not Past Due or Nonaccrual, Covered	0	0
Total covered loans	8	17
30-59 Days Past Due, Total Loans	3,925	5,109
60-89 Days Past Due, Total Loans	2,717	4,348
90 Days or More Past Due, Total Loans	874	1,722
Nonaccrual, Total Loans	76,913	80,408
Total Past Due and Nonaccrual, Total Loans	84,429	91,587
Not Past Due or Nonaccrual, Total Loans	143,396	151,804
Total Loans	227,825	243,391
Commercial, Financial and Agricultural
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	11
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	131	2,052
Total Past Due and Nonaccrual	131	2,063
Not Past Due or Nonaccrual	9,402	10,116
Total	9,533	12,179
Commercial real estate | Construction loans
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	495	200
Total Past Due and Nonaccrual	495	200
Not Past Due or Nonaccrual	2,247	2,112
Total	2,742	2,312
Commercial real estate | Mortgage receivable
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	371
60-89 Days Past Due	0	536
90 Days or More Past Due	0	0
Nonaccrual	7,205	7,068
Total Past Due and Nonaccrual	7,205	7,975
Not Past Due or Nonaccrual	33,904	38,934
Total	41,109	46,909
Residential real estate | Mortgage receivable
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	2,188	2,440
60-89 Days Past Due	1,935	2,688
90 Days or More Past Due	498	844
Nonaccrual	30,174	32,518
Total Past Due and Nonaccrual	34,795	38,490
Not Past Due or Nonaccrual	62,722	63,436
Total	97,517	101,926
Residential real estate | Equity line of credit
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	27,176	24,519
Total Past Due and Nonaccrual	27,176	24,519
Not Past Due or Nonaccrual	0	0
Total	27,176	24,519
Residential real estate | Equity loan
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	1,737	2,182
60-89 Days Past Due	782	1,124
90 Days or More Past Due	376	878
Nonaccrual	9,844	12,504
Total Past Due and Nonaccrual	12,739	16,688
Not Past Due or Nonaccrual	34,213	37,013
Total	46,952	53,701
Consumer loans | Credit card
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	0	0
Total Past Due and Nonaccrual	0	0
Not Past Due or Nonaccrual	0	0
Total	0	0
Consumer loans | Consumer direct
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	105
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	27	40
Total Past Due and Nonaccrual	27	145
Not Past Due or Nonaccrual	908	193
Total	935	338
Consumer loans | Consumer indirect
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	1,853	1,490
Total Past Due and Nonaccrual	1,853	1,490
Not Past Due or Nonaccrual	0	0
Total	$ 1,853	$ 1,490