Loans and Credit Quality - Loan Portfolio Segmented by Risk Class (Details) - USD ($) $ in Thousands	Mar. 31, 2018	Dec. 31, 2017
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	$ 971,683	$ 959,109
Less: Unearned origination fees, net of origination costs	(4,108)	(4,156)
Total loans	967,575	954,953
AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	914,296	905,022
AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20,524	18,726
AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	36,794	35,361
AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	69
Commercial
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	316,081	313,514
Commercial | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	278,743	277,371
Commercial | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	6,101	4,921
Commercial | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	31,168	31,222
Commercial | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	69
Real estate construction one-to-four family
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	30,310	31,201
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	30,310	31,201
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction one-to-four family | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate construction other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	54,947	80,093
Real estate construction other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	54,947	80,093
Real estate construction other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate term owner occupied
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	132,589	132,042
Real estate term owner occupied | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	125,930	127,059
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	2,052	2,095
Real estate term owner occupied | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	4,607	2,888
Real estate term owner occupied | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate term non-owner occupied
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	339,621	319,313
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	328,144	307,780
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	11,000	11,051
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	477	482
Real estate term non-owner occupied | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate term other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	56,294	40,411
Real estate term other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	55,081	39,777
Real estate term other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	1,213	634
Real estate term other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate term other | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Consumer secured by 1st deeds of trust
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	21,896	22,616
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	21,227	21,846
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	138	3
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	531	767
Consumer secured by 1st deeds of trust | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Consumer other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	19,945	19,919
Consumer other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	19,914	19,895
Consumer other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20	22
Consumer other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	11	$ 2
Consumer other | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	$ 0