Fair Value Measurements - Segregated by Level of Valuation Inputs (Details) - USD ($) $ in Thousands	Mar. 31, 2021	Dec. 31, 2020
Financial assets:
Portfolio loans, net	$ 6,685,357	$ 6,713,129
Financial liabilities:
Short-term borrowings	4,663	4,658
Junior subordinated debt owed to unconsolidated trusts	71,509	71,468
Level 1 | Carrying Amount
Financial assets:
Cash and cash equivalents	404,802	688,537
Level 1 | Fair Value
Financial assets:
Cash and cash equivalents	404,802	688,537
Level 2 | Carrying Amount
Financial assets:
Accrued interest receivable	31,876	33,240
Financial liabilities:
Time deposits	1,022,468	1,119,348
Securities sold under agreements to repurchase	210,132	175,614
Short-term borrowings	4,663	4,658
Long-term debt	4,584	4,757
Junior subordinated debt owed to unconsolidated trusts	71,509	71,468
Accrued interest payable	5,671	3,401
Level 2 | Fair Value
Financial assets:
Accrued interest receivable	31,876	33,240
Financial liabilities:
Time deposits	1,031,778	1,132,107
Securities sold under agreements to repurchase	210,132	175,614
Short-term borrowings	4,669	4,661
Long-term debt	4,806	5,014
Junior subordinated debt owed to unconsolidated trusts	56,511	59,943
Accrued interest payable	5,671	3,401
Level 3 | Carrying Amount
Financial assets:
Portfolio loans, net	6,685,357	6,713,129
Mortgage servicing rights	10,409	10,912
Other servicing rights	1,528	1,434
Level 3 | Fair Value
Financial assets:
Portfolio loans, net	6,740,655	6,755,425
Mortgage servicing rights	12,461	11,107
Other servicing rights	2,055	1,966
Level 3 | Senior notes | Carrying Amount
Financial liabilities:
Long-term debt	39,843	39,809
Level 3 | Senior notes | Fair Value
Financial liabilities:
Long-term debt	40,066	40,104
Level 3 | Subordinated debt | Carrying Amount
Financial liabilities:
Long-term debt	182,370	182,226
Level 3 | Subordinated debt | Fair Value
Financial liabilities:
Long-term debt	$ 184,819	$ 187,697