Allowance for Loan Losses - Allowance for Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2019	Jun. 30, 2018	Jun. 30, 2019	Jun. 30, 2018
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	$ 20,209	$ 20,449	$ 19,519	$ 21,461
Charge-Offs	(68)	(100)	(177)	(1,204)
Recoveries	77	59	126	151
Provision (benefit)	300	(300)	1,050	(300)
Balance, end of period	20,518	20,108	20,518	20,108
Balance, end of period: Individually evaluated for impairment	6	73	6	73
Balance, end of period: collectively evaluated for impairment	20,512	20,035	20,512	20,035
Commercial
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	6,478	5,684	5,660	6,172
Charge-Offs	(64)	(77)	(173)	(1,042)
Recoveries	48	54	92	143
Provision (benefit)	661	(35)	1,544	353
Balance, end of period	7,123	5,626	7,123	5,626
Balance, end of period: Individually evaluated for impairment	6	73	6	73
Balance, end of period: collectively evaluated for impairment	7,117	5,553	7,117	5,553
Real estate construction one-to-four family
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	659	608	675	629
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	80	(12)	64	(33)
Balance, end of period	739	596	739	596
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	739	596	739	596
Real estate construction other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	1,367	1,069	1,275	1,566
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(255)	17	(163)	(480)
Balance, end of period	1,112	1,086	1,112	1,086
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	1,112	1,086	1,112	1,086
Real estate term owner occupied
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	2,320	2,174	2,027	2,194
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(39)	(4)	254	(24)
Balance, end of period	2,281	2,170	2,281	2,170
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	2,281	2,170	2,281	2,170
Real estate term non-owner occupied
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	6,122	6,379	5,799	6,043
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	109	(160)	432	176
Balance, end of period	6,231	6,219	6,231	6,219
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	6,231	6,219	6,231	6,219
Real estate term other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	844	1,025	716	725
Charge-Offs	0	0	0	0
Recoveries	25	1	27	2
Provision (benefit)	(108)	(25)	18	274
Balance, end of period	761	1,001	761	1,001
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	761	1,001	761	1,001
Consumer secured by 1st deeds of trust
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	396	305	306	315
Charge-Offs	0	(2)	0	(91)
Recoveries	0	0	0	1
Provision (benefit)	(74)	49	16	127
Balance, end of period	322	352	322	352
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	322	352	322	352
Consumer other
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	501	315	426	307
Charge-Offs	(4)	(21)	(4)	(71)
Recoveries	4	4	7	5
Provision (benefit)	(15)	98	57	155
Balance, end of period	486	396	486	396
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	486	396	486	396
Unallocated
Accounts Receivable, Allowance for Credit Loss [Roll Forward]
Balance, beginning of period	1,522	2,890	2,635	3,510
Charge-Offs	0	0	0	0
Recoveries	0	0	0	0
Provision (benefit)	(59)	(228)	(1,172)	(848)
Balance, end of period	1,463	2,662	1,463	2,662
Balance, end of period: Individually evaluated for impairment	0	0	0	0
Balance, end of period: collectively evaluated for impairment	$ 1,463	$ 2,662	$ 1,463	$ 2,662