Loans and Allowance for Loan Losses (Details 4) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2014	Dec. 31, 2013	Sep. 30, 2013
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded	$ 20,939	$ 18,809
Unpaid Principal Balance, With no related allowance recorded	23,794	20,813
Recorded Investment, With an allowance recorded	14,687	16,266	24,000
Unpaid Principal Balance, With an allowance recorded	15,277	16,992
Specific Reserves, With an allowance recorded	4,661	4,796	4,600
Total impaired loans, Recorded Investment	35,626	35,075
Total impaired loans, Unpaid Principal Balance	39,071	37,805
Commercial, financial, and agricultural
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded	2,400	2,467
Unpaid Principal Balance, With no related allowance recorded	2,457	2,593
Recorded Investment, With an allowance recorded	2,027	1,548
Unpaid Principal Balance, With an allowance recorded	2,086	1,607
Specific Reserves, With an allowance recorded	663	721
Real estate - construction residential
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded	0	44
Unpaid Principal Balance, With no related allowance recorded	0	80
Recorded Investment, With an allowance recorded	2,040	2,160
Unpaid Principal Balance, With an allowance recorded	2,260	2,331
Specific Reserves, With an allowance recorded	273	392
Real estate - construction commercial
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded	5,679	6,101
Unpaid Principal Balance, With no related allowance recorded	6,403	7,148
Recorded Investment, With an allowance recorded	0	514
Unpaid Principal Balance, With an allowance recorded	0	514
Specific Reserves, With an allowance recorded	0	304
Real estate - residential
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded	3,116	2,121
Unpaid Principal Balance, With no related allowance recorded	3,568	2,654
Recorded Investment, With an allowance recorded	5,523	4,396
Unpaid Principal Balance, With an allowance recorded	5,620	4,570
Specific Reserves, With an allowance recorded	1,728	1,374
Real estate - commercial
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded	9,744	7,817
Unpaid Principal Balance, With no related allowance recorded	11,366	8,056
Recorded Investment, With an allowance recorded	4,829	7,605
Unpaid Principal Balance, With an allowance recorded	5,004	7,925
Specific Reserves, With an allowance recorded	1,942	1,989
Consumer
Financing Receivable, Impaired [Line Items]
Recorded Investment, With no related allowance recorded		259
Unpaid Principal Balance, With no related allowance recorded		282
Recorded Investment, With an allowance recorded	268	43
Unpaid Principal Balance, With an allowance recorded	307	45
Specific Reserves, With an allowance recorded	$ 55	$ 16