Fair Value Measurements - Segregated by Level of Valuation Inputs (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Financial assets:
Portfolio loans, net	$ 6,661,115	$ 6,633,501
Financial liabilities:
Short-term borrowings	21,358	8,551
Junior subordinated debt owed to unconsolidated trusts	71,347	71,308
Level 1 | Carrying Amount
Financial assets:
Cash and cash equivalents	342,848	529,288
Level 1 | Fair Value
Financial assets:
Cash and cash equivalents	342,848	529,288
Level 2 | Carrying Amount
Financial assets:
Accrued interest receivable	27,310	27,109
Financial liabilities:
Time deposits	1,482,013	1,534,850
Securities sold under agreements to repurchase	167,250	205,491
Short-term borrowings	21,358	8,551
Long-term debt	35,595	83,600
Junior subordinated debt owed to unconsolidated trusts	71,347	71,308
Accrued interest payable	5,609	5,000
Level 2 | Fair Value
Financial assets:
Accrued interest receivable	27,310	27,109
Financial liabilities:
Time deposits	1,490,835	1,538,597
Securities sold under agreements to repurchase	167,250	205,491
Short-term borrowings	21,364	8,552
Long-term debt	35,811	83,614
Junior subordinated debt owed to unconsolidated trusts	72,749	74,153
Accrued interest payable	5,609	5,000
Level 3 | Carrying Amount
Financial assets:
Portfolio loans, net	6,661,115	6,633,501
Mortgage servicing rights	11,776	12,326
Other servicing rights	1,070	1,071
Level 3 | Fair Value
Financial assets:
Portfolio loans, net	6,687,117	6,648,560
Mortgage servicing rights	13,968	18,193
Other servicing rights	1,632	1,740
Level 3 | Senior notes | Carrying Amount
Financial liabilities:
Long-term debt	39,708	39,674
Level 3 | Senior notes | Fair Value
Financial liabilities:
Long-term debt	40,441	40,099
Level 3 | Subordinated debt | Carrying Amount
Financial liabilities:
Long-term debt	59,273	59,248
Level 3 | Subordinated debt | Fair Value
Financial liabilities:
Long-term debt	$ 60,950	$ 61,514