Loans and Allowance for Loan Losses - Aging (Details) - USD ($) $ in Thousands	Sep. 30, 2017	Dec. 31, 2016	Sep. 30, 2016
Aging categories of performing loans and nonaccrual loans
Current	$ 1,087,753	$ 1,045,208
Total Past Due	6,714	7,657
Total Loans	1,094,467	1,052,865	$ 1,076,073
Non-Accrual	6,559	6,229
90 Plus Days Still Accruing	0	0
Commercial
Aging categories of performing loans and nonaccrual loans
Current	777,072	753,561
Total Past Due	5,791	3,955
Total Loans	782,863	757,516	750,044
Non-Accrual	4,711	4,975
90 Plus Days Still Accruing	0	0
Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Current	362,942	387,208
Total Past Due	2,768	258
Total Loans	365,710	387,466
Non-Accrual	577	74
90 Plus Days Still Accruing	0	0
Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Current	297,132	270,339
Total Past Due	471	1,121
Total Loans	297,603	271,460
Non-Accrual	693	1,375
90 Plus Days Still Accruing	0	0
Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Current	116,998	96,014
Total Past Due	2,552	2,576
Total Loans	119,550	98,590
Non-Accrual	3,441	3,526
90 Plus Days Still Accruing	0	0
Residential
Aging categories of performing loans and nonaccrual loans
Current	233,554	212,502
Total Past Due	682	2,950
Total Loans	234,236	215,452	243,487
Non-Accrual	1,084	1,072
90 Plus Days Still Accruing	0	0
Home Equity
Aging categories of performing loans and nonaccrual loans
Current	64,184	64,791
Total Past Due	89	595
Total Loans	64,273	65,386	67,839
Non-Accrual	581	104
90 Plus Days Still Accruing	0	0
Consumer
Aging categories of performing loans and nonaccrual loans
Current	12,943	14,354
Total Past Due	152	157
Total Loans	13,095	14,511	$ 14,703
Non-Accrual	183	78
90 Plus Days Still Accruing	0	0
Financing Receivables, 30 to 59 Days Past Due
Aging categories of performing loans and nonaccrual loans
Total Past Due	1,536	2,891
Financing Receivables, 30 to 59 Days Past Due | Commercial
Aging categories of performing loans and nonaccrual loans
Total Past Due	1,468	244
Financing Receivables, 30 to 59 Days Past Due | Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Total Past Due	1,468	15
Financing Receivables, 30 to 59 Days Past Due | Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	229
Financing Receivables, 30 to 59 Days Past Due | Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	0
Financing Receivables, 30 to 59 Days Past Due | Residential
Aging categories of performing loans and nonaccrual loans
Total Past Due	41	2,067
Financing Receivables, 30 to 59 Days Past Due | Home Equity
Aging categories of performing loans and nonaccrual loans
Total Past Due	13	525
Financing Receivables, 30 to 59 Days Past Due | Consumer
Aging categories of performing loans and nonaccrual loans
Total Past Due	14	55
Financing Receivables, 60 to 89 Days Past Due
Aging categories of performing loans and nonaccrual loans
Total Past Due	1,237	622
Financing Receivables, 60 to 89 Days Past Due | Commercial
Aging categories of performing loans and nonaccrual loans
Total Past Due	990	169
Financing Receivables, 60 to 89 Days Past Due | Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Total Past Due	958	169
Financing Receivables, 60 to 89 Days Past Due | Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Total Past Due	32	0
Financing Receivables, 60 to 89 Days Past Due | Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	0
Financing Receivables, 60 to 89 Days Past Due | Residential
Aging categories of performing loans and nonaccrual loans
Total Past Due	247	419
Financing Receivables, 60 to 89 Days Past Due | Home Equity
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	0
Financing Receivables, 60 to 89 Days Past Due | Consumer
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	34
Financing Receivables, Equal to Greater than 90 Days Past Due
Aging categories of performing loans and nonaccrual loans
Total Past Due	3,941	4,144
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial
Aging categories of performing loans and nonaccrual loans
Total Past Due	3,333	3,542
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Total Past Due	342	74
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Total Past Due	439	892
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Total Past Due	2,552	2,576
Financing Receivables, Equal to Greater than 90 Days Past Due | Residential
Aging categories of performing loans and nonaccrual loans
Total Past Due	394	464
Financing Receivables, Equal to Greater than 90 Days Past Due | Home Equity
Aging categories of performing loans and nonaccrual loans
Total Past Due	76	70
Financing Receivables, Equal to Greater than 90 Days Past Due | Consumer
Aging categories of performing loans and nonaccrual loans
Total Past Due	$ 138	$ 68