Loans and Credit Quality - Past Due Loans And Nonaccrual Loans (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Financing Receivable, Past Due [Line Items]
Past due loans	$ 1,534	$ 3,440
Nonaccrual loans	15,356	15,210
Current loans	1,031,566	970,183
Total financing receivables	1,048,456	988,833
Less: Unearned origination fees, net of origination costs	(5,085)	(4,487)
Total loans	1,043,371	984,346
30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	1,534	2,483
Nonaccrual loans	1,937	1,329
60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	957
Nonaccrual loans	474	324
Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	6,010	3,597
Commercial
Financing Receivable, Past Due [Line Items]
Past due loans	270	1,729
Nonaccrual loans	9,153	12,671
Current loans	403,267	328,020
Total financing receivables	412,690	342,420
Total loans	411,327	341,091
Commercial | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	270	872
Nonaccrual loans	270	1,329
Commercial | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	857
Nonaccrual loans	385	324
Commercial | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	2,862	1,287
Real estate construction one-to-four family
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	1,349	0
Current loans	37,469	37,111
Total financing receivables	38,818	37,111
Total loans	38,503	36,828
Real estate construction one-to-four family | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0
Real estate construction one-to-four family | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0
Real estate construction one-to-four family | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	1,349
Real estate construction other
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0	0
Current loans	61,808	72,256
Total financing receivables	61,808	72,256
Total loans	60,906	71,658
Real estate construction other | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Real estate construction other | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Real estate construction other | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Real estate term owner occupied
Financing Receivable, Past Due [Line Items]
Past due loans	338	1,197
Nonaccrual loans	3,489	1,694
Current loans	135,064	123,523
Total financing receivables	138,891	126,414
Total loans	138,181	125,795
Real estate term owner occupied | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	338	1,197
Nonaccrual loans	1,641	0
Real estate term owner occupied | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0	0
Real estate term owner occupied | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	623	1,694
Real estate term non-owner occupied
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0	0
Current loans	312,960	325,720
Total financing receivables	312,960	325,720
Total loans	311,302	324,198
Real estate term non-owner occupied | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Real estate term non-owner occupied | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Real estate term non-owner occupied | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Real estate term other
Financing Receivable, Past Due [Line Items]
Past due loans	26	0
Nonaccrual loans	1,176	577
Current loans	41,304	41,462
Total financing receivables	42,506	42,039
Total loans	42,200	41,746
Real estate term other | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	26	0
Nonaccrual loans	0	0
Real estate term other | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0	0
Real estate term other | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	1,176	577
Consumer secured by 1st deeds of trust
Financing Receivable, Past Due [Line Items]
Past due loans	750	324
Nonaccrual loans	68	220
Current loans	15,380	18,684
Total financing receivables	16,198	19,228
Total loans	16,191	19,234
Consumer secured by 1st deeds of trust | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	750	224
Nonaccrual loans	0	0
Consumer secured by 1st deeds of trust | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	100
Nonaccrual loans	0	0
Consumer secured by 1st deeds of trust | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	0	0
Consumer other
Financing Receivable, Past Due [Line Items]
Past due loans	150	190
Nonaccrual loans	121	48
Current loans	24,314	23,407
Total financing receivables	24,585	23,645
Total loans	24,761	23,796
Consumer other | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	150	190
Nonaccrual loans	26	0
Consumer other | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	89	0
Consumer other | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Past due loans	0	0
Nonaccrual loans	$ 0	$ 39