Fair Value Measurements - By Level of Valuation Inputs (Details) - USD ($) $ in Thousands	Jun. 30, 2017	Dec. 31, 2016	Jun. 30, 2016	Dec. 31, 2015
Financial assets:
Cash and cash equivalents	$ 294,100	$ 166,706	$ 267,072	$ 319,280
Securities held to maturity	209,003	47,683
Portfolio loans, net	3,871,263	3,831,105
Financial liabilities:
Short-term borrowings	50,000	75,000
Junior subordinated debt owed to unconsolidated trusts	70,938	70,868
Level 1 | Carrying Amount
Financial assets:
Cash and cash equivalents	294,100	166,706
Level 1 | Fair Value
Financial assets:
Cash and cash equivalents	294,100	166,706
Level 2 | Carrying Amount
Financial assets:
Securities held to maturity	208,634	47,820
Accrued interest receivable	14,419	15,562
Financial liabilities:
Deposits	4,394,212	4,374,298
Securities sold under agreements to repurchase	178,597	189,157
Short-term borrowings	50,000	75,000
Long-term debt	80,000	80,000
Junior subordinated debt owed to unconsolidated trusts	70,938	70,868
Accrued interest payable	1,382	987
Level 2 | Fair Value
Financial assets:
Securities held to maturity	209,003	47,683
Accrued interest receivable	14,419	15,562
Financial liabilities:
Deposits	4,390,437	4,368,891
Securities sold under agreements to repurchase	178,597	189,157
Short-term borrowings	50,000	75,000
Long-term debt	80,000	80,000
Junior subordinated debt owed to unconsolidated trusts	70,938	70,868
Accrued interest payable	1,382	987
Level 3 | Carrying Amount
Financial assets:
Portfolio loans, net	3,871,263	3,831,105
Mortgage servicing rights	2,806	3,074
Other servicing rights	186
Level 3 | Fair Value
Financial assets:
Portfolio loans, net	3,865,697	3,841,760
Mortgage servicing rights	8,575	$ 7,803
Other servicing rights	683
Level 3 | Senior notes | Carrying Amount
Financial liabilities:
Long-term debt	39,351
Level 3 | Senior notes | Fair Value
Financial liabilities:
Long-term debt	39,360
Level 3 | Subordinated debt | Carrying Amount
Financial liabilities:
Long-term debt	59,022
Level 3 | Subordinated debt | Fair Value
Financial liabilities:
Long-term debt	$ 59,066