Allowance for Loan Losses - Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended		9 Months Ended
	Sep. 30, 2019	Sep. 30, 2018	Sep. 30, 2019	Sep. 30, 2018
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	$ 20,518	$ 20,108	$ 19,519	$ 21,461
Charge-Offs	(29)	(9)	(206)	(1,213)
Recoveries	723	61	849	212
Provision (benefit)	(2,075)	0	(1,025)	(300)
Balance, end of period	19,137	20,160	19,137	20,160
Balance, end of period: Individually evaluated for impairment	102	372	102	372
Balance, end of period: collectively evaluated for impairment	19,035	19,788	19,035	19,788
Commercial
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	7,123	5,626	5,660	6,172
Charge-Offs	(22)	0	(195)	(1,042)
Recoveries	709	57	801	200
Provision (benefit)	(1,340)	254	204	607
Balance, end of period	6,470	5,937	6,470	5,937
Balance, end of period: Individually evaluated for impairment	97	323	97	323
Balance, end of period: collectively evaluated for impairment	6,373	5,614	6,373	5,614
Real estate construction one-to-four family
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	739	596	675	629
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(122)	79	(58)	46
Balance, end of period	617	675	617	675
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	617	675	617	675
Real estate construction other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	1,112	1,086	1,275	1,566
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(101)	34	(264)	(446)
Balance, end of period	1,011	1,120	1,011	1,120
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	1,011	1,120	1,011	1,120
Real estate term owner occupied
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	2,281	2,170	2,027	2,194
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(299)	(43)	(45)	(67)
Balance, end of period	1,982	2,127	1,982	2,127
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	1,982	2,127	1,982	2,127
Real estate term non-owner occupied
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	6,231	6,219	5,799	6,043
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(703)	(11)	(271)	165
Balance, end of period	5,528	6,208	5,528	6,208
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	5,528	6,208	5,528	6,208
Real estate term other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	761	1,001	716	725
Charge-Offs	0	0	0	0
Recoveries	1	1	28	3
Provision (benefit)	(48)	(291)	(30)	(17)
Balance, end of period	714	711	714	711
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	714	711	714	711
Consumer secured by 1st deeds of trust
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	322	352	306	315
Charge-Offs	0	0	0	(91)
Recoveries	0	1	0	2
Provision (benefit)	(49)	21	(33)	148
Balance, end of period	273	374	273	374
Balance, end of period: Individually evaluated for impairment	5	49	5	49
Balance, end of period: collectively evaluated for impairment	268	325	268	325
Consumer other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	486	396	426	307
Charge-Offs	(7)	(9)	(11)	(80)
Recoveries	13	2	20	7
Provision (benefit)	(62)	27	(5)	182
Balance, end of period	430	416	430	416
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	430	416	430	416
Unallocated
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	1,463	2,662	2,635	3,510
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	649	(70)	(523)	(918)
Balance, end of period	2,112	2,592	2,112	2,592
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	$ 2,112	$ 2,592	$ 2,112	$ 2,592