Income Taxes - Components of Deferred Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2015	Dec. 31, 2014
Deferred tax assets
Allowance for loan losses	$ 3,342	$ 3,107
Deferred compensation, other	74	104
Stock based expense	87	130
Net operating loss carryforwards	34,180	36,955
Securities available-for-sale	405	0
Deferred tax credits	823	697
OREO valuation allowance	1,059	1,080
Donation carryforward	5	232
Capital loss carryforward	4	4
Other	246	45
Total deferred tax assets	40,225	42,354
Deferred tax liabilities
Depreciation	(58)	(41)
Adjustments arising from acquisitions	(146)	(502)
Mortgage servicing rights	(828)	(872)
Federal Home Loan Bank dividend received in stock	0	(166)
Federal Home Loan Bank dividend received in stock	(450)	(451)
Deferred loan fees & costs	(399)	(373)
Prepaid expenses	(164)	(190)
Total deferred tax liabilities	(2,045)	(2,595)
Valuation allowance	0	(39,759)
Net deferred tax assets	$ 38,180	$ 0