Loans and Allowance for Probable Loan Losses - Allowance for Loan Losses Activity by Portfolio Segment (Details)	3 Months Ended				9 Months Ended
	Sep. 30, 2016 USD ($)		Sep. 30, 2015 USD ($)		Sep. 30, 2016 USD ($) Loan		Sep. 30, 2015 USD ($)
Financing Receivable, Allowance for Credit Losses [Line Items]
Provision for loan losses	$ 1,631,000		$ 2,276,000		$ 7,715,000		$ 6,392,000
Provision for PCI Loans	0		446,000.000		1,400,000		446,000.000
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	14,908,000		16,822,000		19,736,000		13,292,000	[1]
Provision (reversal) for loan losses	1,631,000	[2]	2,276,000	[3]	7,715,000	[2]	6,392,000	[3]
Loans charged off	(1,257,000)		(1,301,000)		(13,701,000)		(3,280,000)
Recoveries of loans charged off	711,000		605,000		2,243,000		1,998,000
Balance at end of period	15,993,000		18,402,000		15,993,000		18,402,000
Partial Charge Offs on Commercial Relationships					$ 10,900,000
Number of loans partially charged-off | Loan					2
Construction Real Estate Loans
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	4,423,000		2,909,000		$ 4,350,000		2,456,000	[1]
Provision (reversal) for loan losses	734,000		630,000		538,000		991,000
Loans charged off	0		0		0		0
Recoveries of loans charged off	0		34,000		269,000		126,000
Balance at end of period	5,157,000		3,573,000		5,157,000		3,573,000
1-4 Family Residential Real Estate Loans
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	1,686,000		3,468,000		2,595,000		2,822,000	[1]
Provision (reversal) for loan losses	574,000		(634,000)		(449,000)		32,000
Loans charged off	(24,000)		(14,000)		(43,000)		(60,000)
Recoveries of loans charged off	7,000		83,000		140,000		109,000
Balance at end of period	2,243,000		2,903,000		2,243,000		2,903,000
Commercial Real Estate Loans
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	4,680,000		3,529,000		4,577,000		3,025,000	[1]
Provision (reversal) for loan losses	556,000		143,000		648,000		574,000
Loans charged off	0		0		0		0
Recoveries of loans charged off	6,000		5,000		17,000		78,000
Balance at end of period	5,242,000		3,677,000		5,242,000		3,677,000
Commercial Loans
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	2,474,000		5,013,000		6,596,000		3,279,000	[1]
Provision (reversal) for loan losses	(682,000)		1,514,000		6,032,000		3,271,000
Loans charged off	(452,000)		(78,000)		(11,375,000)	[4]	(185,000)
Recoveries of loans charged off	344,000		40,000		431,000		124,000
Balance at end of period	1,684,000		6,489,000		1,684,000		6,489,000
Municipal Loans
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	737,000		833,000		725,000		716,000	[1]
Provision (reversal) for loan losses	1,000		15,000		(236,000)		132,000
Loans charged off	0		0		0		0
Recoveries of loans charged off	0		0		249,000		0
Balance at end of period	738,000		848,000		738,000		848,000
Loans to Individuals
Change in Allowances for Loan Losses [Abstract]
Balance at beginning of period	908,000		1,070,000		893,000		994,000	[1]
Provision (reversal) for loan losses	448,000		608,000		1,182,000		1,392,000
Loans charged off	(781,000)		(1,209,000)		(2,283,000)		(3,035,000)
Recoveries of loans charged off	354,000		443,000		1,137,000		1,561,000
Balance at end of period	$ 929,000		$ 912,000		$ 929,000		$ 912,000

[1] Loans acquired with the Omni acquisition were measured at fair value on December 17, 2014 with no carryover of allowance for loan loss.

[2] Of the $1.6 million recorded in provision for loan losses for the three months ended September 30, 2016, none related to provision expense on PCI loans. Of the $7.7 million recorded in provision for loan losses for the nine months ended September 30, 2016, $1.4 million related to provision expense on PCI loans.

[3] Of the $2.3 million and $6.4 million recorded in provision for loan losses for the three and nine months ended September 30, 2015, $446,000 related to provision expense on PCI loans.

[4] Of the $11.4 million in commercial charge-offs recorded for the nine months ended September 30, 2016, $10.9 million relates to the charge-off of two large commercial borrowing relationships.