Loans and Allowance for Loan Losses - Aging (Details) - USD ($) $ in Thousands	Mar. 31, 2017	Dec. 31, 2016	Mar. 31, 2016
Aging categories of performing loans and nonaccrual loans
Current	$ 1,069,271	$ 1,045,208
Total Past Due	7,511	7,657
Total Loans	1,076,782	1,052,865	$ 1,075,606
Non-Accrual	6,519	6,229
90 Plus Days Still Accruing	0	0
Commercial
Aging categories of performing loans and nonaccrual loans
Current	747,745	753,561
Total Past Due	5,515	3,955
Total Loans	753,260	757,516	753,798
Non-Accrual	4,508	4,975
90 Plus Days Still Accruing	0	0
Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Current	370,415	387,208
Total Past Due	1,823	258
Total Loans	372,238	387,466
Non-Accrual	74	74
90 Plus Days Still Accruing	0	0
Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Current	269,606	270,339
Total Past Due	941	1,121
Total Loans	270,547	271,460
Non-Accrual	932	1,375
90 Plus Days Still Accruing	0	0
Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Current	107,724	96,014
Total Past Due	2,751	2,576
Total Loans	110,475	98,590
Non-Accrual	3,502	3,526
90 Plus Days Still Accruing	0	0
Residential
Aging categories of performing loans and nonaccrual loans
Current	242,976	212,502
Total Past Due	1,615	2,950
Total Loans	244,591	215,452	236,643
Non-Accrual	1,269	1,072
90 Plus Days Still Accruing	0	0
Home Equity
Aging categories of performing loans and nonaccrual loans
Current	65,100	64,791
Total Past Due	70	595
Total Loans	65,170	65,386	68,318
Non-Accrual	620	104
90 Plus Days Still Accruing	0	0
Consumer
Aging categories of performing loans and nonaccrual loans
Current	13,450	14,354
Total Past Due	311	157
Total Loans	13,761	14,511	$ 16,847
Non-Accrual	122	78
90 Plus Days Still Accruing	0	0
Financing Receivables, 30 to 59 Days Past Due
Aging categories of performing loans and nonaccrual loans
Total Past Due	2,683	2,891
Financing Receivables, 30 to 59 Days Past Due | Commercial
Aging categories of performing loans and nonaccrual loans
Total Past Due	1,510	244
Financing Receivables, 30 to 59 Days Past Due | Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Total Past Due	1,326	15
Financing Receivables, 30 to 59 Days Past Due | Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Total Past Due	9	229
Financing Receivables, 30 to 59 Days Past Due | Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Total Past Due	175	0
Financing Receivables, 30 to 59 Days Past Due | Residential
Aging categories of performing loans and nonaccrual loans
Total Past Due	944	2,067
Financing Receivables, 30 to 59 Days Past Due | Home Equity
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	525
Financing Receivables, 30 to 59 Days Past Due | Consumer
Aging categories of performing loans and nonaccrual loans
Total Past Due	229	55
Financing Receivables, 60 to 89 Days Past Due
Aging categories of performing loans and nonaccrual loans
Total Past Due	508	622
Financing Receivables, 60 to 89 Days Past Due | Commercial
Aging categories of performing loans and nonaccrual loans
Total Past Due	423	169
Financing Receivables, 60 to 89 Days Past Due | Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Total Past Due	423	169
Financing Receivables, 60 to 89 Days Past Due | Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	0
Financing Receivables, 60 to 89 Days Past Due | Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	0
Financing Receivables, 60 to 89 Days Past Due | Residential
Aging categories of performing loans and nonaccrual loans
Total Past Due	72	419
Financing Receivables, 60 to 89 Days Past Due | Home Equity
Aging categories of performing loans and nonaccrual loans
Total Past Due	0	0
Financing Receivables, 60 to 89 Days Past Due | Consumer
Aging categories of performing loans and nonaccrual loans
Total Past Due	13	34
Financing Receivables, Equal to Greater than 90 Days Past Due
Aging categories of performing loans and nonaccrual loans
Total Past Due	4,320	4,144
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial
Aging categories of performing loans and nonaccrual loans
Total Past Due	3,582	3,542
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial | Commercial Business
Aging categories of performing loans and nonaccrual loans
Total Past Due	74	74
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial | Commercial Real Estate
Aging categories of performing loans and nonaccrual loans
Total Past Due	932	892
Financing Receivables, Equal to Greater than 90 Days Past Due | Commercial | Acquisition & Development
Aging categories of performing loans and nonaccrual loans
Total Past Due	2,576	2,576
Financing Receivables, Equal to Greater than 90 Days Past Due | Residential
Aging categories of performing loans and nonaccrual loans
Total Past Due	599	464
Financing Receivables, Equal to Greater than 90 Days Past Due | Home Equity
Aging categories of performing loans and nonaccrual loans
Total Past Due	70	70
Financing Receivables, Equal to Greater than 90 Days Past Due | Consumer
Aging categories of performing loans and nonaccrual loans
Total Past Due	$ 69	$ 68