Loans and Credit Quality - Loan Portfolio Segmented by Risk Class (Details) - USD ($) $ in Thousands	Sep. 30, 2019	Dec. 31, 2018
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	$ 1,041,171	$ 988,833
Less: Unearned origination fees, net of origination costs	(4,624)	(4,487)
Total loans	1,036,547	984,346
AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	990,129	928,552
AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	24,413	31,426
AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	26,629	28,854
AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		1
Commercial
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	398,231	342,420
Total loans	396,901	341,091
Commercial | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	377,537	315,112
Commercial | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	2,896	5,116
Commercial | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	17,798	22,192
Commercial | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate construction one-to-four family
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	37,270	37,111
Total loans	37,126	36,828
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	35,797	33,729
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	3,382
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	1,473	0
Real estate construction one-to-four family | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate construction other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	61,446	72,256
Total loans	60,697	71,658
Real estate construction other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	61,446	72,256
Real estate construction other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate term owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	127,045	126,414
Total loans	126,381	125,795
Real estate term owner occupied | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	117,505	117,174
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	3,694	3,987
Real estate term owner occupied | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	5,846	5,253
Real estate term owner occupied | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate term non-owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	332,169	325,720
Total loans	330,592	324,198
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	314,525	307,126
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	17,644	18,129
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	465
Real estate term non-owner occupied | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate term other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	45,142	42,039
Total loans	44,821	41,746
Real estate term other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	43,944	40,792
Real estate term other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	670
Real estate term other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	1,198	577
Real estate term other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Consumer secured by 1st deeds of trust
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	15,952	19,228
Total loans	15,943	19,234
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	15,607	18,768
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	179	140
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	166	320
Consumer secured by 1st deeds of trust | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Consumer other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	23,916	23,645
Total loans	24,086	23,796
Consumer other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	23,768	23,595
Consumer other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	2
Consumer other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	$ 148	47
Consumer other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		$ 1