Loans and Allowance for Loan Losses - Troubled debt restructurings (Details) - USD ($) $ in Thousands	Mar. 31, 2016	Dec. 31, 2015
Financing Receivable, Modifications [Line Items]
30 - 59 Days Past Due, Covered	$ 0	$ 0
60 - 89 Days Past Due, Covered	0	0
90 Days or More Past Due, Covered	0	0
Nonaccrual, Covered	0	8
Total Past Due and Nonaccrual, Covered	0	8
Not Past Due or Nonaccrual, Covered	0	0
Total, Covered	0	8
30 - 59 Days Past Due, Total Loans	5,299	3,925
60 - 89 Days Past Due, Total Loans	1,451	2,717
90 Days or More Past Due, Total Loans	491	874
Nonaccrual, Total Loans	77,354	76,913
Total Past Due and Nonaccrual, Total Loans	84,595	84,429
Not Past Due or Nonaccrual, Total Loans	113,566	143,396
Total Loans	198,161	227,825
Commercial Real Estate | Real estate – construction
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	3,855	495
Total Past Due and Nonaccrual	3,855	495
Not Past Due or Nonaccrual	2,664	2,247
Total	6,519	2,742
Commercial Real Estate | Mortgage
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	5,385	7,205
Total Past Due and Nonaccrual	5,385	7,205
Not Past Due or Nonaccrual	5,425	33,904
Total	10,810	41,109
Commercial, Financial and Agricultural
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	121	0
90 Days or More Past Due	0	0
Nonaccrual	15	131
Total Past Due and Nonaccrual	136	131
Not Past Due or Nonaccrual	9,424	9,402
Total	9,560	9,533
Residential | Mortgage
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	3,635	2,188
60-89 Days Past Due	594	1,935
90 Days or More Past Due	433	498
Nonaccrual	30,700	30,174
Total Past Due and Nonaccrual	35,362	34,795
Not Past Due or Nonaccrual	60,511	62,722
Total	95,873	97,517
Residential | Equity lines of credit
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	26,739	27,176
Total Past Due and Nonaccrual	26,739	27,176
Not Past Due or Nonaccrual	0	0
Total	26,739	27,176
Residential | Equity loans
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	1,664	1,737
60-89 Days Past Due	736	782
90 Days or More Past Due	58	376
Nonaccrual	8,648	9,844
Total Past Due and Nonaccrual	11,106	12,739
Not Past Due or Nonaccrual	34,674	34,213
Total	45,780	46,952
Consumer | Credit card
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	0	0
Total Past Due and Nonaccrual	0	0
Not Past Due or Nonaccrual	0	0
Total	0	0
Consumer | Consumer direct
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	22	27
Total Past Due and Nonaccrual	22	27
Not Past Due or Nonaccrual	868	908
Total	890	935
Consumer | Consumer indirect
Financing Receivable, Modifications [Line Items]
30-59 Days Past Due	0	0
60-89 Days Past Due	0	0
90 Days or More Past Due	0	0
Nonaccrual	1,990	1,853
Total Past Due and Nonaccrual	1,990	1,853
Not Past Due or Nonaccrual	0	0
Total	$ 1,990	$ 1,853