Allowance for Loan Losses - Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2020	Sep. 30, 2019	Sep. 30, 2020	Sep. 30, 2019
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	$ 20,653	$ 20,518	$ 19,088	$ 19,519
Charge-Offs	(141)	(29)	(1,110)	(206)
Recoveries	604	723	674	849
Provision (benefit)	567	(2,075)	3,031	(1,025)
Balance, end of period	21,683	19,137	21,683	19,137
Balance, end of period: Individually evaluated for impairment	41	102	41	102
Balance, end of period: collectively evaluated for impairment	21,642	19,035	21,642	19,035
Commercial
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	7,366	7,123	6,604	5,660
Charge-Offs	(56)	(22)	(1,011)	(195)
Recoveries	600	709	656	801
Provision (benefit)	285	(1,340)	1,946	204
Balance, end of period	8,195	6,470	8,195	6,470
Balance, end of period: Individually evaluated for impairment	41	97	41	97
Balance, end of period: collectively evaluated for impairment	8,154	6,373	8,154	6,373
Real estate construction one-to-four family
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	690	739	643	675
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	10	(122)	57	(58)
Balance, end of period	700	617	700	617
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	700	617	700	617
Real estate construction other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	1,215	1,112	1,017	1,275
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	58	(101)	256	(264)
Balance, end of period	1,273	1,011	1,273	1,011
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	1,273	1,011	1,273	1,011
Real estate term owner occupied
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	2,533	2,281	2,188	2,027
Charge-Offs	(85)	0	(85)	0
Recoveries	0	0	0	0
Provision (benefit)	21	(299)	366	(45)
Balance, end of period	2,469	1,982	2,469	1,982
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	2,469	1,982	2,469	1,982
Real estate term non-owner occupied
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	5,421	6,231	5,180	5,799
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	61	(703)	302	(271)
Balance, end of period	5,482	5,528	5,482	5,528
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	5,482	5,528	5,482	5,528
Real estate term other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	702	761	671	716
Charge-Offs	0	0	0	0
Recoveries	1	1	2	28
Provision (benefit)	55	(48)	85	(30)
Balance, end of period	758	714	758	714
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	758	714	758	714
Consumer secured by 1st deeds of trust
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	258	322	270	306
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(2)	(49)	(14)	(33)
Balance, end of period	256	273	256	273
Balance, end of period: Individually evaluated for impairment	0	5	0	5
Balance, end of period: collectively evaluated for impairment	256	268	256	268
Consumer other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	447	486	436	426
Charge-Offs	0	(7)	(14)	(11)
Recoveries	3	13	16	20
Provision (benefit)	(7)	(62)	5	(5)
Balance, end of period	443	430	443	430
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	443	430	443	430
Unallocated
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	2,021	1,463	2,079	2,635
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	86	649	28	(523)
Balance, end of period	2,107	2,112	2,107	2,112
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	$ 2,107	$ 2,112	$ 2,107	$ 2,112