Fair Value Measurements - Segregated by Level of Valuation Inputs (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Financial assets:
Portfolio loans, net	$ 7,022,470	$ 6,633,501
Financial liabilities:
Short-term borrowings	4,651	8,551
Junior subordinated debt owed to unconsolidated trusts	71,427	71,308
Level 1 | Carrying Amount
Financial assets:
Cash and cash equivalents	479,721	529,288
Level 1 | Fair Value
Financial assets:
Cash and cash equivalents	479,721	529,288
Level 2 | Carrying Amount
Financial assets:
Accrued interest receivable	37,609	27,109
Financial liabilities:
Time deposits	1,227,767	1,534,850
Securities sold under agreements to repurchase	201,641	205,491
Short-term borrowings	4,651	8,551
Long-term debt	4,931	83,600
Junior subordinated debt owed to unconsolidated trusts	71,427	71,308
Accrued interest payable	6,648	5,000
Level 2 | Fair Value
Financial assets:
Accrued interest receivable	37,609	27,109
Financial liabilities:
Time deposits	1,243,305	1,538,597
Securities sold under agreements to repurchase	201,641	205,491
Short-term borrowings	4,651	8,552
Long-term debt	5,218	83,614
Junior subordinated debt owed to unconsolidated trusts	57,793	74,153
Accrued interest payable	6,648	5,000
Level 3 | Carrying Amount
Financial assets:
Portfolio loans, net	7,022,470	6,633,501
Mortgage servicing rights	11,982	12,326
Other servicing rights	1,199	1,071
Level 3 | Fair Value
Financial assets:
Portfolio loans, net	7,055,886	6,648,560
Mortgage servicing rights	12,570	18,193
Other servicing rights	1,696	1,740
Level 3 | Senior notes | Carrying Amount
Financial liabilities:
Long-term debt	39,775	39,674
Level 3 | Senior notes | Fair Value
Financial liabilities:
Long-term debt	40,099	40,099
Level 3 | Subordinated debt | Carrying Amount
Financial liabilities:
Long-term debt	182,095	59,248
Level 3 | Subordinated debt | Fair Value
Financial liabilities:
Long-term debt	$ 184,944	$ 61,514