Loans and The Allowance for Loan Losses - Summary of Loan Balances (Details) - USD ($) $ in Thousands	Mar. 31, 2018	Dec. 31, 2017	Mar. 31, 2017	Dec. 31, 2016
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	$ 2,201,336	$ 2,203,810
Allowance for loan losses	(20,185)	(20,877)	$ (18,875)	$ (18,750)
Discount and fair value adjustments on purchased loans	(1,314)	(1,477)
Deferred loan costs and fees, net	4,453	4,691
Loans, net	2,184,290	2,186,147
Consumer and Commercial Portfolio Segment | Construction
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	73,354	91,050
Allowance for loan losses	(1,281)	(1,708)	(1,457)	(1,353)
Consumer, Excluding Consumer Other Portfolio Segment
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	1,932,140	1,928,603
Consumer, Excluding Consumer Other Portfolio Segment | 1-4 Family Residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	934,595	922,627
Allowance for loan losses	(5,099)	(5,076)	(5,000)	(4,846)
Consumer, Excluding Consumer Other Portfolio Segment | Home Equity
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	75,013	80,662
Allowance for loan losses	(631)	(699)	(558)	(537)
Consumer, Excluding Consumer Other Portfolio Segment | Commercial Real Estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	849,178	834,264
Allowance for loan losses	(9,511)	(9,584)	(8,333)	(8,374)
Commercial business
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	249,285	253,509
Allowance for loan losses	(3,353)	(3,473)	(3,126)	(3,206)
Consumer
Accounts, Notes, Loans and Financing Receivable [Line Items]
Total loans	19,911	21,698
Allowance for loan losses	$ (310)	$ (337)	$ (401)	$ (434)