Investment Securities (Details) - Unrealized gains on investment securities - USD ($) $ in Thousands	Sep. 30, 2015	Dec. 31, 2014
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	$ 220,130	$ 281,461
Investment Securities Available-for-Sale, Gross Unrealized Gains	4,749	8,596
Investment Securities Available-for-Sale, Gross Unrealized Losses	(665)	(525)
Investment Securities Available-for-Sale, Fair Value	224,214	289,532
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	227,221	224,682
Investment Securities Held-to-Maturity, Gross Unrealized Gains	7,444	6,858
Investment Securities Held-to-Maturity, Gross Unrealized Losses	(172)	(95)
Investment Securities Held-to-Maturity, Fair Value	234,493	231,445
Total, Amortized Cost	447,351	506,143
Total, Gross Unrealized Gains	12,193	15,454
Total, Gross Unrealized Losses	(837)	(620)
Total, Fair Value	458,707	520,977
Federal Agency Obligations [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	30,474	32,650
Investment Securities Available-for-Sale, Gross Unrealized Gains	438	217
Investment Securities Available-for-Sale, Gross Unrealized Losses	(7)	(50)
Investment Securities Available-for-Sale, Fair Value	30,905	32,817
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	35,519	27,103
Investment Securities Held-to-Maturity, Gross Unrealized Gains	642	322
Investment Securities Held-to-Maturity, Gross Unrealized Losses	(47)	(28)
Investment Securities Held-to-Maturity, Fair Value	36,114	27,397
Residential Mortgage Backed Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	46,943	58,836
Investment Securities Available-for-Sale, Gross Unrealized Gains	1,162	1,531
Investment Securities Available-for-Sale, Gross Unrealized Losses	(14)	(11)
Investment Securities Available-for-Sale, Fair Value	48,091	60,356
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	4,240	5,955
Investment Securities Held-to-Maturity, Gross Unrealized Gains	19	28
Investment Securities Held-to-Maturity, Gross Unrealized Losses	(1)
Investment Securities Held-to-Maturity, Fair Value	4,258	5,983
Commercial Mortgage Backed Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	2,996	3,042
Investment Securities Available-for-Sale, Gross Unrealized Gains	59	4
Investment Securities Available-for-Sale, Fair Value	3,055	3,046
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	4,150	4,266
Investment Securities Held-to-Maturity, Gross Unrealized Gains	97	50
Investment Securities Held-to-Maturity, Fair Value	4,247	4,316
US States and Political Subdivisions Debt Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	8,191	8,201
Investment Securities Available-for-Sale, Gross Unrealized Gains	191	205
Investment Securities Available-for-Sale, Fair Value	8,382	8,406
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	118,877	120,144
Investment Securities Held-to-Maturity, Gross Unrealized Gains	4,470	4,512
Investment Securities Held-to-Maturity, Gross Unrealized Losses	(46)	(60)
Investment Securities Held-to-Maturity, Fair Value	123,301	124,596
Trust Preferred Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	16,087	16,086
Investment Securities Available-for-Sale, Gross Unrealized Gains	385	489
Investment Securities Available-for-Sale, Gross Unrealized Losses	(150)	(269)
Investment Securities Available-for-Sale, Fair Value	16,322	16,306
Corporate Bonds and Notes [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	76,049	119,838
Investment Securities Available-for-Sale, Gross Unrealized Gains	2,418	5,950
Investment Securities Available-for-Sale, Gross Unrealized Losses	(81)	(11)
Investment Securities Available-for-Sale, Fair Value	78,386	125,777
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	36,016	38,950
Investment Securities Held-to-Maturity, Gross Unrealized Gains	888	1,026
Investment Securities Held-to-Maturity, Gross Unrealized Losses	(78)	(7)
Investment Securities Held-to-Maturity, Fair Value	36,826	39,969
Asset-backed Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	21,403	27,393
Investment Securities Available-for-Sale, Gross Unrealized Gains	2	140
Investment Securities Available-for-Sale, Gross Unrealized Losses	(275)	(31)
Investment Securities Available-for-Sale, Fair Value	21,130	27,502
Certificates of Deposit [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	1,896	2,098
Investment Securities Available-for-Sale, Gross Unrealized Gains	27	27
Investment Securities Available-for-Sale, Gross Unrealized Losses	(3)	(2)
Investment Securities Available-for-Sale, Fair Value	1,920	2,123
Equity Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	376	376
Investment Securities Available-for-Sale, Gross Unrealized Losses	(38)	(69)
Investment Securities Available-for-Sale, Fair Value	338	307
Other Securities [Member]
Investment securities available-for-sale
Investment Securities Available-for-Sale, Amortized Cost	15,715	12,941
Investment Securities Available-for-Sale, Gross Unrealized Gains	67	33
Investment Securities Available-for-Sale, Gross Unrealized Losses	(97)	(82)
Investment Securities Available-for-Sale, Fair Value	15,685	12,892
US Treasury Securities [Member]
Investment securities held-to-maturity
Investment Securities Held-to-Maturity, Amortized Cost	28,419	28,264
Investment Securities Held-to-Maturity, Gross Unrealized Gains	1,328	920
Investment Securities Held-to-Maturity, Fair Value	$ 29,747	$ 29,184