Discontinued Operations - Assets and Liabilities (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015	Jun. 30, 2015	Dec. 31, 2014
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Cash and Due from Banks	$ 10,564	$ 14,302
Total cash and cash equivalents	28,762	29,133	$ 14,014	$ 30,077
Premises and equipment	25,846	26,275
Interest Receivable and Other Assets	27,143	25,204
Total assets	1,484,962	1,384,476	1,253,066	1,110,459
Accrued interest payable and other liabilities	21,121	13,291
Total Liabilities	1,360,943	1,269,764
Retained earnings	27,526	20,054
Total Stockholders' Equity	124,019	114,712	$ 113,274	$ 109,438
Total liabilities and stockholders’ equity	1,484,962	1,384,476
USI Insurance Services [Member]
Income Statement, Balance Sheet and Additional Disclosures by Disposal Groups, Including Discontinued Operations [Line Items]
Cash and Due from Banks	1,684	2,245
Total cash and cash equivalents	1,684	2,245
Premises and equipment	42	618
Interest Receivable and Other Assets	11	2,154
Total assets	1,737	5,017
Accrued interest payable and other liabilities	424	2,834
Total Liabilities	424	2,834
Retained earnings	1,313	2,183
Total Stockholders' Equity	1,313	2,183
Total liabilities and stockholders’ equity	$ 1,737	$ 5,017