Allowance for Loan Losses - Recorded Investment Segregated By Amounts Individually Or Collectively In Allowance For Loan Losses (Details) - USD ($) $ in Thousands	Mar. 31, 2017	Dec. 31, 2016
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	$ 965,225	$ 979,449
Balance, end of period: Individually evaluated for impairment	37,318	38,656
Balance, end of period: Collectively evaluated for impairment	927,907	940,793
Commercial
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	275,809	278,178
Balance, end of period: Individually evaluated for impairment	28,763	18,657
Balance, end of period: Collectively evaluated for impairment	247,046	259,521
Real estate construction one-to-four family
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	26,029	26,061
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: Collectively evaluated for impairment	26,029	26,061
Real estate construction other
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	67,975	72,159
Balance, end of period: Individually evaluated for impairment	0	0
Balance, end of period: Collectively evaluated for impairment	67,975	72,159
Real estate term owner occupied
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	140,102	152,178
Balance, end of period: Individually evaluated for impairment	5,560	16,946
Balance, end of period: Collectively evaluated for impairment	134,542	135,232
Real estate term non-owner occupied
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	366,254	356,601
Balance, end of period: Individually evaluated for impairment	1,075	1,084
Balance, end of period: Collectively evaluated for impairment	365,179	355,517
Real estate term other
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	42,218	45,402
Balance, end of period: Individually evaluated for impairment	1,272	1,302
Balance, end of period: Collectively evaluated for impairment	40,946	44,100
Consumer secured by 1st deeds of trust
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	22,762	23,589
Balance, end of period: Individually evaluated for impairment	596	615
Balance, end of period: Collectively evaluated for impairment	22,166	22,974
Consumer other
Financing Receivable, Allowance for Credit Losses [Line Items]
Total financing receivables	24,076	25,281
Balance, end of period: Individually evaluated for impairment	52	52
Balance, end of period: Collectively evaluated for impairment	$ 24,024	$ 25,229