Loans and Credit Quality - Loan Portfolio Segmented By Risk Class (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015
Financing Receivable, Recorded Investment [Line Items]
Subtotal	$ 979,449	$ 985,399
Less: Unearned origination fees, net of origination costs	(4,434)	(4,612)
Total loans	975,015	980,787
Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	939,573	951,901
Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	2,586	808
Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	37,290	32,690
Commercial
Financing Receivable, Recorded Investment [Line Items]
Subtotal	278,178	272,441
Commercial | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	257,639	257,293
Commercial | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	1,950	536
Commercial | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	18,589	14,612
Real estate construction one-to-four family
Financing Receivable, Recorded Investment [Line Items]
Subtotal	26,061	44,488
Real estate construction one-to-four family | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	26,061	44,488
Real estate construction one-to-four family | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	0	0
Real estate construction one-to-four family | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	0	0
Real estate construction other
Financing Receivable, Recorded Investment [Line Items]
Subtotal	72,159	74,956
Real estate construction other | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	72,159	74,956
Real estate construction other | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	0	0
Real estate construction other | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	0	0
Real estate term owner occupied
Financing Receivable, Recorded Investment [Line Items]
Subtotal	152,178	143,741
Real estate term owner occupied | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	135,359	127,226
Real estate term owner occupied | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	57	0
Real estate term owner occupied | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	16,762	16,515
Real estate term non-owner occupied
Financing Receivable, Recorded Investment [Line Items]
Subtotal	356,601	347,516
Real estate term non-owner occupied | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	355,903	347,157
Real estate term non-owner occupied | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	0	0
Real estate term non-owner occupied | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	698	359
Real estate term other
Financing Receivable, Recorded Investment [Line Items]
Subtotal	45,402	46,672
Real estate term other | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	44,733	45,884
Real estate term other | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	0	91
Real estate term other | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	669	697
Consumer secured by 1st deeds of trust
Financing Receivable, Recorded Investment [Line Items]
Subtotal	23,589	26,673
Consumer secured by 1st deeds of trust | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	22,568	26,015
Consumer secured by 1st deeds of trust | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	501	171
Consumer secured by 1st deeds of trust | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	520	487
Consumer other
Financing Receivable, Recorded Investment [Line Items]
Subtotal	25,281	28,912
Consumer other | Asset Quality Rating - Pass
Financing Receivable, Recorded Investment [Line Items]
Subtotal	25,151	28,882
Consumer other | Asset Quality Rating - Special Mention
Financing Receivable, Recorded Investment [Line Items]
Subtotal	78	10
Consumer other | Asset Quality Rating - Substandard
Financing Receivable, Recorded Investment [Line Items]
Subtotal	$ 52	$ 20