Allowance for Loan Losses - Allowance For Loan Losses (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2018	Mar. 31, 2017
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	$ 21,461	$ 19,697
Charge-Offs	(1,104)	(279)
Recoveries	92	75
Provision (benefit)	0	400
Balance, end of period	20,449	19,893
Balance, end of period: Individually evaluated for impairment	372	994
Balance, end of period: collectively evaluated for impairment	20,077	18,899
Commercial
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	6,172	5,535
Charge-Offs	(965)	(262)
Recoveries	89	73
Provision (benefit)	388	355
Balance, end of period	5,684	5,701
Balance, end of period: Individually evaluated for impairment	370	994
Balance, end of period: collectively evaluated for impairment	5,314	4,707
Real estate construction one-to-four family
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	629	550
Charge-Offs	0	0
Recoveries	0	0
Provision (benefit)	(21)	(1)
Balance, end of period	608	549
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	608	549
Real estate construction other
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	1,566	1,465
Charge-Offs	0	0
Recoveries	0	0
Provision (benefit)	(497)	(82)
Balance, end of period	1,069	1,383
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	1,069	1,383
Real estate term owner occupied
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	2,194	2,358
Charge-Offs	0	0
Recoveries	0	0
Provision (benefit)	(20)	14
Balance, end of period	2,174	2,372
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	2,174	2,372
Real estate term non-owner occupied
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	6,043	6,853
Charge-Offs	0	0
Recoveries	0	0
Provision (benefit)	336	354
Balance, end of period	6,379	7,207
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	6,379	7,207
Real estate term other
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	725	819
Charge-Offs	0	0
Recoveries	1	0
Provision (benefit)	299	(45)
Balance, end of period	1,025	774
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	1,025	774
Consumer secured by 1st deeds of trust
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	315	313
Charge-Offs	(89)	0
Recoveries	1	0
Provision (benefit)	78	(2)
Balance, end of period	305	311
Balance, end of period: Individually evaluated for impairment	2	0
Balance, end of period: collectively evaluated for impairment	303	311
Consumer other
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	307	408
Charge-Offs	(50)	(17)
Recoveries	1	2
Provision (benefit)	57	(2)
Balance, end of period	315	391
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	315	391
Unallocated
Allowance for Doubtful Accounts Receivable [Roll Forward]
Balance, beginning of period	3,510	1,396
Charge-Offs	0	0
Recoveries	0	0
Provision (benefit)	(620)	(191)
Balance, end of period	2,890	1,205
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: collectively evaluated for impairment	$ 2,890	$ 1,205