Loans and Credit Quality - Loan Portfolio Segmented by Risk Class (Details) - USD ($) $ in Thousands	Sep. 30, 2018	Dec. 31, 2017
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	$ 986,362	$ 959,109
Less: Unearned origination fees, net of origination costs	(4,355)	(4,156)
Total loans	982,007	954,953
AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	925,595	905,022
AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	29,278	18,726
AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	31,462	35,361
AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	27
Commercial
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	333,132	313,514
Commercial | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	301,757	277,371
Commercial | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	6,279	4,921
Commercial | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	25,096	31,222
Commercial | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate construction one-to-four family
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	36,119	31,201
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	36,119	31,201
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction one-to-four family | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate construction other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	61,857	80,093
Real estate construction other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	61,857	80,093
Real estate construction other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate term owner occupied
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	130,166	132,042
Real estate term owner occupied | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	120,902	127,059
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	4,484	2,095
Real estate term owner occupied | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	4,780	2,888
Real estate term owner occupied | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate term non-owner occupied
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	340,820	319,313
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	322,051	307,780
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	18,300	11,051
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	469	482
Real estate term non-owner occupied | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Real estate term other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	41,493	40,411
Real estate term other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	40,889	39,777
Real estate term other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0	634
Real estate term other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	577	0
Real estate term other | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	27
Consumer secured by 1st deeds of trust
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	20,202	22,616
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	19,525	21,846
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	199	3
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	478	767
Consumer secured by 1st deeds of trust | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	0
Consumer other
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	22,573	19,919
Consumer other | AQR Pass
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	22,495	19,895
Consumer other | AQR Special Mention
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	16	22
Consumer other | AQR Substandard
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	62	$ 2
Consumer other | AQR Doubtful
Financing Receivable, Recorded Investment [Line Items]
Total financing receivables	$ 0