Loans and Credit Quality - Loan Portfolio Segmented by Risk Class (Details) - USD ($) $ in Thousands	Jun. 30, 2019	Dec. 31, 2018
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	$ 1,020,022	$ 988,833
Less: Unearned origination fees, net of origination costs	(4,318)	(4,487)
Total loans	1,015,704	984,346
AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	968,019	928,552
AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	25,742	31,426
AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	26,261	28,854
AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	1
Commercial
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	387,257	342,420
Total loans	385,953	341,091
Commercial | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	365,705	315,112
Commercial | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	4,149	5,116
Commercial | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	17,403	22,192
Commercial | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction one-to-four family
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	39,013	37,111
Total loans	38,851	36,828
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	37,521	33,729
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	3,382
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	1,492	0
Real estate construction one-to-four family | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	58,824	72,256
Total loans	58,380	71,658
Real estate construction other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	58,824	72,256
Real estate construction other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate term owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	126,991	126,414
Total loans	126,360	125,795
Real estate term owner occupied | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	117,492	117,174
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	3,786	3,987
Real estate term owner occupied | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	5,713	5,253
Real estate term owner occupied | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate term non-owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	325,738	325,720
Total loans	324,104	324,198
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	307,931	307,126
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	17,807	18,129
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	465
Real estate term non-owner occupied | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate term other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	41,965	42,039
Total loans	41,658	41,746
Real estate term other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	40,767	40,792
Real estate term other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	670
Real estate term other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	1,198	577
Real estate term other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Consumer secured by 1st deeds of trust
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	16,848	19,228
Total loans	16,848	19,234
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	16,544	18,768
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	140
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	304	320
Consumer secured by 1st deeds of trust | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Consumer other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	23,386	23,645
Total loans	23,550	23,796
Consumer other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	23,235	23,595
Consumer other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	2
Consumer other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	151	47
Consumer other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	$ 0	$ 1