Loans and Credit Quality - Loan Portfolio Segmented by Risk Class (Details) - USD ($) $ in Thousands	Jun. 30, 2018	Dec. 31, 2017
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	$ 971,909	$ 959,109
Less: Unearned origination fees, net of origination costs	(4,207)	(4,156)
Total loans	967,702	954,953
AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	908,698	905,022
AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	28,208	18,726
AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	35,003	35,361
Commercial
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	327,733	313,514
Commercial | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	293,585	277,371
Commercial | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	4,357	4,921
Commercial | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	29,791	31,222
Real estate construction one-to-four family
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	31,064	31,201
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	31,064	31,201
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	58,369	80,093
Real estate construction other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	58,369	80,093
Real estate construction other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate term owner occupied
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	127,384	132,042
Real estate term owner occupied | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	118,385	127,059
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	4,433	2,095
Real estate term owner occupied | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	4,566	2,888
Real estate term non-owner occupied
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	330,217	319,313
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	311,312	307,780
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	18,432	11,051
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	473	482
Real estate term other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	55,431	40,411
Real estate term other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	54,820	39,777
Real estate term other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	611	634
Real estate term other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Consumer secured by 1st deeds of trust
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20,820	22,616
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20,364	21,846
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	353	3
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	103	767
Consumer other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20,891	19,919
Consumer other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20,799	19,895
Consumer other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	22	22
Consumer other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	$ 70	$ 2