Investment Securities - Amortized Cost and Fair Values of Investment Securities Available for Sale (Details) - USD ($) $ in Thousands	Sep. 30, 2016	Dec. 31, 2015
Available for sale (AFS)
Amortized Cost	$ 138,801	$ 175,766
Gross Unrealized Gains	3,702	3,436
Gross Unrealized Losses	(1,446)	(3,045)
Available for sale	141,057	176,157
Debt securities
Available for sale (AFS)
Amortized Cost	127,212	160,155
Gross Unrealized Gains	2,738	2,266
Gross Unrealized Losses	(1,298)	(2,596)
Available for sale	128,652	159,825
U.S. Government and agency securities
Available for sale (AFS)
Amortized Cost	0	3,586
Gross Unrealized Gains	0	0
Gross Unrealized Losses	0	(37)
Available for sale	0	3,549
Mortgage-backed securities
Available for sale (AFS)
Amortized Cost	10,079	9,785
Gross Unrealized Gains	242	284
Gross Unrealized Losses	(62)	(60)
Available for sale	10,259	10,009
Asset-backed securities
Available for sale (AFS)
Amortized Cost	1,543	1,960
Gross Unrealized Gains	0	0
Gross Unrealized Losses	(5)	(20)
Available for sale	1,538	1,940
State and political securities
Available for sale (AFS)
Amortized Cost	60,838	84,992
Gross Unrealized Gains	1,807	1,797
Gross Unrealized Losses	(3)	(234)
Available for sale	62,642	86,555
Other debt securities
Available for sale (AFS)
Amortized Cost	54,752	59,832
Gross Unrealized Gains	689	185
Gross Unrealized Losses	(1,228)	(2,245)
Available for sale	54,213	57,772
Other equity securities
Available for sale (AFS)
Amortized Cost	11,589	15,611
Gross Unrealized Gains	964	1,170
Gross Unrealized Losses	(148)	(449)
Available for sale	12,405	16,332
Financial institution equity securities
Available for sale (AFS)
Amortized Cost	9,822	10,397
Gross Unrealized Gains	951	1,100
Gross Unrealized Losses	0	(14)
Available for sale	10,773	11,483
Other equity securities
Available for sale (AFS)
Amortized Cost	1,767	5,214
Gross Unrealized Gains	13	70
Gross Unrealized Losses	(148)	(435)
Available for sale	$ 1,632	$ 4,849