Loans and Credit Quality - Past Due Loans And Nonaccrual Loans (Details) - USD ($) $ in Thousands	Sep. 30, 2016	Dec. 31, 2015
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	$ 1,290	$ 1,150
Nonaccrual loans	10,885	3,686
Total current, subtotal	989,309	980,563
Total financing receivables	1,001,484	985,399
Less: Unearned origination fees, net of origination costs	(4,408)	(4,612)
Total loans	997,076	980,787
30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	672	1,129
60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	618	21
Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Commercial
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	640	263
Nonaccrual loans	4,384	3,013
Total current, subtotal	337,608	326,258
Total financing receivables	342,632	329,534
Commercial | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	332	242
Commercial | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	308	21
Commercial | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate construction one-to-four family
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	3,972	0
Total current, subtotal	33,030	44,488
Total financing receivables	37,002	44,488
Real estate construction one-to-four family | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate construction one-to-four family | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate construction one-to-four family | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate construction other
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	1,912	0
Total current, subtotal	70,179	74,931
Total financing receivables	72,091	74,931
Real estate construction other | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate construction other | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate construction other | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term owner occupied
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	32	38
Total current, subtotal	166,909	128,725
Total financing receivables	166,941	128,763
Real estate term owner occupied | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term owner occupied | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term owner occupied | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term non-owner occupied
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	209	359
Total current, subtotal	288,766	313,710
Total financing receivables	288,975	314,069
Real estate term non-owner occupied | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term non-owner occupied | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term non-owner occupied | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Real estate term other
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	440	289
Nonaccrual loans	0	0
Total current, subtotal	41,424	37,740
Total financing receivables	41,864	38,029
Real estate term other | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	141	289
Real estate term other | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	299	0
Real estate term other | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Consumer secured by 1st deeds of trust
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	148	568
Nonaccrual loans	376	256
Total current, subtotal	25,433	25,849
Total financing receivables	25,957	26,673
Consumer secured by 1st deeds of trust | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	138	568
Consumer secured by 1st deeds of trust | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	10	0
Consumer secured by 1st deeds of trust | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	0	0
Consumer other
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	62	30
Nonaccrual loans	0	20
Total current, subtotal	25,960	28,862
Total financing receivables	26,022	28,912
Consumer other | 30-59 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	61	30
Consumer other | 60-89 Days Past Due Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans	1	0
Consumer other | Greater Than 90 Days Still Accruing
Financing Receivable, Recorded Investment, Past Due [Line Items]
Past due loans		$ 0