Loans and Credit Quality - Loan Portfolio Segmented By Risk Class (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	$ 1,048,456	$ 988,833
Less: Unearned origination fees, net of origination costs	(5,085)	(4,487)
Total loans	1,043,371	984,346
Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	994,583	928,552
Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	29,660	31,426
Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	24,213	28,854
Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		1
Commercial
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	412,690	342,420
Total loans	411,327	341,091
Commercial | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	394,107	315,112
Commercial | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	2,279	5,116
Commercial | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	16,304	22,192
Commercial | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate construction one-to-four family
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	38,818	37,111
Total loans	38,503	36,828
Real estate construction one-to-four family | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	34,132	33,729
Real estate construction one-to-four family | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	3,337	3,382
Real estate construction one-to-four family | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	1,349	0
Real estate construction one-to-four family | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate construction other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	61,808	72,256
Total loans	60,906	71,658
Real estate construction other | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	61,808	72,256
Real estate construction other | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	0
Real estate construction other | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate term owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	138,891	126,414
Total loans	138,181	125,795
Real estate term owner occupied | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	129,959	117,174
Real estate term owner occupied | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	3,828	3,987
Real estate term owner occupied | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	5,104	5,253
Real estate term owner occupied | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate term non-owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	312,960	325,720
Total loans	311,302	324,198
Real estate term non-owner occupied | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	295,482	307,126
Real estate term non-owner occupied | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	17,478	18,129
Real estate term non-owner occupied | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	465
Real estate term non-owner occupied | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Real estate term other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	42,506	42,039
Total loans	42,200	41,746
Real estate term other | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	38,771	40,792
Real estate term other | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	2,559	670
Real estate term other | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	1,176	577
Real estate term other | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Consumer secured by 1st deeds of trust
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	16,198	19,228
Total loans	16,191	19,234
Consumer secured by 1st deeds of trust | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	15,860	18,768
Consumer secured by 1st deeds of trust | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	179	140
Consumer secured by 1st deeds of trust | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	159	320
Consumer secured by 1st deeds of trust | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		0
Consumer other
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	24,585	23,645
Total loans	24,761	23,796
Consumer other | Asset Quality Rating - Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	24,464	23,595
Consumer other | Asset Quality Rating - Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	0	2
Consumer other | Asset Quality Rating - Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables	$ 121	47
Consumer other | Asset Quality Rating - Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Total financing receivables		$ 1