Loans, Impaired Loans Recorded Investment, Unpaid Principal Balance, Related Allowance, Average Recorded Investment, and Interest Income Recognized (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	$ 76,102	$ 96,609
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	95,033	102,483
Related Allowance	16,165	15,304
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	77,235	110,079
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	3,012	3,699
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	33,941	37,081
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	52,987	62,283
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	36,462	35,627
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	616
Interest Income Recognized		(73)
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	110,043	133,690
Unpaid Principal Balance	148,020	164,766
Average Recorded Investment	113,697	145,707
Interest Income Recognized	$ 3,628	$ 3,626
Net Recorded Investment of the Impaired Loans	63.00%	72.00%
Bank Mutual [Member]
Total Impaired Loans Individually Evaluated [Line Items]
Loans and Leases Receivable, Net of Deferred Income, Purchased Credit Impaired Loans And Covered Loans	$ 2,000	$ 5,000
Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	16,165	15,304
Commercial and industrial
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	47,249	40,747
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	63,346	42,131
Related Allowance	12,010	5,721
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	45,290	52,461
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	1,832	1,167
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	14,787	22,406
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	33,438	45,024
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	20,502	21,352
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	63
Interest Income Recognized		(344)
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	62,035	63,153
Unpaid Principal Balance	96,784	87,155
Average Recorded Investment	65,792	73,813
Interest Income Recognized	1,895	823
Commercial and industrial | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	12,010	5,721
Commercial real estate - owner occupied
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	1,676	2,080
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	1,682	2,087
Related Allowance	19	24
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	1,774	2,179
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	88	104
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	0	3,772
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	0	4,823
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	0	3,975
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	0	0
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	1,676	5,852
Unpaid Principal Balance	1,682	6,910
Average Recorded Investment	1,774	6,154
Interest Income Recognized	88	104
Commercial real estate - owner occupied | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	19	24
Commercial and business lending
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	48,924	42,827
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	65,028	44,218
Related Allowance	12,029	5,745
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	47,064	54,640
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	1,919	1,271
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	14,787	26,178
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	33,438	49,847
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	20,502	25,327
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	63
Interest Income Recognized		(344)
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	63,711	69,005
Unpaid Principal Balance	98,466	94,065
Average Recorded Investment	67,566	79,967
Interest Income Recognized	1,982	927
Commercial and business lending | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	12,029	5,745
Commercial real estate - investor
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	928	799
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	2,104	805
Related Allowance	15	28
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	950	827
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	15	38
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	3,705	1,585
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	3,705	2,820
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	3,980	980
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	159	68
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	4,633	2,384
Unpaid Principal Balance	5,808	3,625
Average Recorded Investment	4,931	1,807
Interest Income Recognized	174	106
Commercial real estate - investor | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	15	28
Real estate construction
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	477	510
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	559	589
Related Allowance	67	75
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	494	533
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	30	32
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	0	0
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	0	0
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	0	0
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	0	0
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	477	510
Unpaid Principal Balance	559	589
Average Recorded Investment	494	533
Interest Income Recognized	30	32
Real estate construction | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	67	75
Commercial real estate lending
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	1,405	1,309
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	2,663	1,394
Related Allowance	82	103
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	1,445	1,360
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	45	70
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	3,705	1,585
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	3,705	2,820
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	3,980	980
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	159	68
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	5,110	2,894
Unpaid Principal Balance	6,367	4,214
Average Recorded Investment	5,425	2,340
Interest Income Recognized	204	138
Commercial real estate lending | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	82	103
Total commercial
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	50,329	44,136
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	67,691	45,612
Related Allowance	12,111	5,848
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	48,509	56,000
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	1,965	1,341
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	18,491	27,763
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	37,142	52,667
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	24,482	26,307
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	222
Interest Income Recognized		(276)
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	68,820	71,899
Unpaid Principal Balance	104,833	98,279
Average Recorded Investment	72,991	82,307
Interest Income Recognized	2,186	1,065
Total commercial | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	12,111	5,848
Residential mortgage
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	21,450	41,691
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	22,625	45,149
Related Allowance	2,740	6,023
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	23,721	42,687
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	856	1,789
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	14,104	8,795
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	14,461	9,074
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	10,962	8,790
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	373	203
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	35,554	50,486
Unpaid Principal Balance	37,087	54,223
Average Recorded Investment	34,683	51,477
Interest Income Recognized	1,229	1,992
Residential mortgage | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	2,740	6,023
Home equity
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	3,076	9,601
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	3,468	10,539
Related Allowance	1,190	3,312
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	3,756	10,209
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	191	566
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	1,346	523
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	1,383	542
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	1,017	530
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	21	0
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	4,422	10,124
Unpaid Principal Balance	4,851	11,081
Average Recorded Investment	4,773	10,739
Interest Income Recognized	211	566
Home equity | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	1,190	3,312
Other consumer
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	1,247	1,181
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	1,249	1,183
Related Allowance	125	121
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	1,250	1,184
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	1	3
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	0	0
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	0	0
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	0	0
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	0	0
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	1,247	1,181
Unpaid Principal Balance	1,249	1,183
Average Recorded Investment	1,250	1,184
Interest Income Recognized	1	3
Other consumer | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	125	121
Total consumer
Loans with a Related Allowance [Line Items]
Impaired Financing Receivable, with Related Allowance, Recorded Investment	25,773	52,473
Impaired Financing Receivable, with Related Allowance, Unpaid Principal Balance	27,342	56,871
Related Allowance	4,055	9,456
Impaired Financing Receivable, with Related Allowance, Average Recorded Investment	28,726	54,080
Impaired Financing Receivable, with Related Allowance, Interest Income, Accrual Method	1,047	2,358
Loans with no Related Allowance [Line Items]
Impaired Financing Receivable, with No Related Allowance, Recorded Investment	15,450	9,318
Impaired Financing Receivable, with No Related Allowance, Unpaid Principal Balance	15,845	9,616
Impaired Financing Receivable, with No Related Allowance, Average Recorded Investment	11,979	9,320
Impaired Financing Receivable, with No Related Allowance, Interest Income, Accrual Method	394	203
Total Impaired Loans Individually Evaluated [Line Items]
Recorded Investment	41,223	61,791
Unpaid Principal Balance	43,187	66,487
Average Recorded Investment	40,706	63,400
Interest Income Recognized	1,441	2,561
Total consumer | Loans with a related allowance
Loans with a Related Allowance [Line Items]
Related Allowance	$ 4,055	$ 9,456