Other Real Estate Owned (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2016	Mar. 31, 2015	Dec. 31, 2015
Real Estate Properties [Line Items]
Mortgage Loans in Process of Foreclosure, Amount	$ 17,600		$ 15,600
Covered
Beginning balance	6,817	$ 22,982
Additions	3,936	4,244
Sales	(523)	(8,970)
Writedowns	(496)	(954)
Ending balance	9,734	17,302
Noncovered
Beginning balance	58,742	70,454
Additions	6,044	17,084
Sales	(7,547)	(13,573)
Writedowns	(2,235)	(1,275)
Ending balance	55,334	72,690
Total
Beginning balance	65,559	93,436
Additions	9,980	21,328
Sales	(8,070)	(22,543)
Writedowns	(2,731)	(2,229)
Ending balance	65,068	$ 89,992
Mortgage
Total
Beginning balance	16,100
Ending balance	14,900
North Milwaukee State Bank
Covered
Additions	0
Noncovered
Additions	330
Total
Additions	$ 330