Loans and Credit Quality - Loan Portfolio Segmented by Risk Class (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	$ 1,507,205	$ 1,048,456
Less: Unearned origination fees, net of origination costs	(14,485)	(5,085)
Loans	1,492,720	1,043,371
AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	1,459,762	994,583
AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	27,985	29,660
AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	19,150	24,213
AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	308
Commercial
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	836,178	412,690
Loans	825,702	411,327
Commercial | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	821,117	394,107
Commercial | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	5,782	2,279
Commercial | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	8,971	16,304
Commercial | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	308
Real estate construction one-to-four family
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	37,958	38,818
Loans	37,751	38,503
Real estate construction one-to-four family | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	37,256	34,132
Real estate construction one-to-four family | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0	3,337
Real estate construction one-to-four family | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	702	1,349
Real estate construction one-to-four family | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0
Real estate construction other
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	82,661	61,808
Loans	81,772	60,906
Real estate construction other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	82,661	61,808
Real estate construction other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0	0
Real estate construction other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0	0
Real estate construction other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0
Real estate term owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	148,993	138,891
Loans	148,167	138,181
Real estate term owner occupied | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	138,886	129,959
Real estate term owner occupied | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	2,677	3,828
Real estate term owner occupied | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	7,430	5,104
Real estate term owner occupied | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0
Real estate term non-owner occupied
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	320,693	312,960
Loans	318,738	311,302
Real estate term non-owner occupied | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	302,902	295,482
Real estate term non-owner occupied | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	17,178	17,478
Real estate term non-owner occupied | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	613	0
Real estate term non-owner occupied | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0
Real estate term other
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	43,539	42,506
Loans	43,245	42,200
Real estate term other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	40,191	38,771
Real estate term other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	2,172	2,559
Real estate term other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	1,176	1,176
Real estate term other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0
Consumer secured by 1st deeds of trust
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	14,050	16,198
Loans	14,037	16,191
Consumer secured by 1st deeds of trust | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	13,726	15,860
Consumer secured by 1st deeds of trust | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	176	179
Consumer secured by 1st deeds of trust | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	148	159
Consumer secured by 1st deeds of trust | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0
Consumer other
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	23,133	24,585
Loans	23,308	24,761
Consumer other | AQR Pass
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	23,023	24,464
Consumer other | AQR Special Mention
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	0	0
Consumer other | AQR Substandard
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	110	$ 121
Consumer other | AQR Doubtful
Financing Receivable, Credit Quality Indicator [Line Items]
Loans, gross	$ 0