Loans and Credit Quality - Past Due Loans and Nonaccrual Loans (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Financing Receivable, Past Due [Line Items]
Total Past Due	$ 2,348	$ 1,534
Nonaccrual	12,647	15,356
Current	1,492,210	1,031,566
Loans, gross	1,507,205	1,048,456
Less: Unearned origination fees,  net of origination costs	(14,485)	(5,085)
Loans	1,492,720	1,043,371
30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	236	1,534
Nonaccrual	115	1,937
60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	2,112	0
Nonaccrual	226	474
Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	9,241	6,010
Commercial
Financing Receivable, Past Due [Line Items]
Total Past Due	0	270
Nonaccrual	6,831	9,153
Current	829,347	403,267
Loans, gross	836,178	412,690
Loans	825,702	411,327
Commercial | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	270
Nonaccrual	115	270
Commercial | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	385
Commercial | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	4,634	2,862
Real estate construction one-to-four family
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	702	1,349
Current	37,256	37,469
Loans, gross	37,958	38,818
Loans	37,751	38,503
Real estate construction one-to-four family | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	0
Real estate construction one-to-four family | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	0
Real estate construction one-to-four family | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	702	1,349
Real estate construction other
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	0
Current	82,661	61,808
Loans, gross	82,661	61,808
Loans	81,772	60,906
Real estate construction other | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Real estate construction other | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Real estate construction other | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Real estate term owner occupied
Financing Receivable, Past Due [Line Items]
Total Past Due	1,421	338
Nonaccrual	3,764	3,489
Current	143,808	135,064
Loans, gross	148,993	138,891
Loans	148,167	138,181
Real estate term owner occupied | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	338
Nonaccrual	0	1,641
Real estate term owner occupied | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	1,421	0
Nonaccrual	226	0
Real estate term owner occupied | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	2,729	623
Real estate term non-owner occupied
Financing Receivable, Past Due [Line Items]
Total Past Due	691	0
Nonaccrual	0	0
Current	320,002	312,960
Loans, gross	320,693	312,960
Loans	318,738	311,302
Real estate term non-owner occupied | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Real estate term non-owner occupied | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	691	0
Real estate term non-owner occupied | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Real estate term other
Financing Receivable, Past Due [Line Items]
Total Past Due	0	26
Nonaccrual	1,176	1,176
Current	42,363	41,304
Loans, gross	43,539	42,506
Loans	43,245	42,200
Real estate term other | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	26
Nonaccrual	0	0
Real estate term other | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	0
Real estate term other | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	1,176	1,176
Consumer secured by 1st deeds of trust
Financing Receivable, Past Due [Line Items]
Total Past Due	236	750
Nonaccrual	65	68
Current	13,749	15,380
Loans, gross	14,050	16,198
Loans	14,037	16,191
Consumer secured by 1st deeds of trust | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	236	750
Nonaccrual	0	0
Consumer secured by 1st deeds of trust | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	0
Consumer secured by 1st deeds of trust | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	0
Consumer other
Financing Receivable, Past Due [Line Items]
Total Past Due	0	150
Nonaccrual	109	121
Current	23,024	24,314
Loans, gross	23,133	24,585
Loans	23,308	24,761
Consumer other | 30-59 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	150
Nonaccrual	0	26
Consumer other | 60-89 Days Past Due Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	0	89
Consumer other | Greater Than 90 Days Still Accruing
Financing Receivable, Past Due [Line Items]
Total Past Due	0	0
Nonaccrual	$ 0	$ 0