Income Taxes (Details 2) - USD ($)	Dec. 31, 2017	Dec. 31, 2016
Deferred tax assets
Allowance for loan losses	$ 13,966,000	$ 19,934,000
Loans	15,062,000	23,240,000
Deferred compensation	7,093,000	11,254,000
Net unrealized losses on securities	3,659,000	10,096,000
Impairment of assets	1,748,000	2,512,000
Net operating loss carryforwards	2,419,000	2,867,000
Other	4,722,000	7,149,000
Gross deferred tax assets	48,669,000	77,052,000
Valuation allowance on state net operating loss carryforwards	0	0
Total deferred tax assets	48,669,000	77,052,000
Deferred tax liabilities
Investment in partnerships	757,000	1,556,000
Depreciation	3,163,000	2,517,000
Mortgage servicing rights	10,139,000	3,360,000
Subordinated debt	2,394,000	4,111,000
Other	1,859,000	2,876,000
Total deferred tax liabilities	18,312,000	14,420,000
Net deferred tax assets	$ 30,357,000	$ 62,632,000