Loans and Allowance for Credit Losses - Additional Information (Details)	3 Months Ended		9 Months Ended
	Sep. 30, 2020 USD ($)	Sep. 30, 2019 USD ($)	Sep. 30, 2020 USD ($) loan	Sep. 30, 2019 USD ($) loan	Jun. 30, 2020 USD ($)	Dec. 31, 2019 USD ($)	Jun. 30, 2019 USD ($)	Dec. 31, 2018 USD ($)
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans held for sale	$ 36,716,000		$ 36,716,000			$ 2,735,000
Premiums and discounts on acquired loans	20,121,000		20,121,000			13,573,000
Net deferred origination and factoring fees	5,237,000		5,237,000			1,392,000
Accrued interest on loans	18,933,000		18,933,000			18,553,000
Loans	4,852,911,000		4,852,911,000			4,194,512,000
Pledged loans	2,185,499,000		2,185,499,000			1,301,851,000
Loans transferred to loans held for sale	56,934,000	$ 21,180,000	172,565,000	$ 27,411,000
Proceeds from loans transferred to loans held for sale	58,313,000	19,322,000	145,513,000	25,653,000
Net gains (losses) on sale of loans			2,323,000	409,000
Allowance for credit loss on loans purchased with credit deterioration	37,415,000		37,415,000
Loans past due	116,596,000		116,596,000			72,859,000
Interest earned on nonaccrual loans	0	0	0	0
Recorded investments in troubled debt restructurings	18,326,000		18,326,000			5,221,000
Allowance for credit losses	90,995,000	31,895,000	$ 90,995,000	$ 31,895,000	$ 54,613,000	29,092,000	$ 29,416,000	$ 27,571,000
Number of defaults on modified loans | loan			2	2
Recorded investments in troubled debt restructurings			$ 18,000	$ 240,000
Financing receivable, modifications of loans cares act	12,625,000		617,976,000
Past Due 30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans past due	78,689,000		78,689,000			39,055,000
Troubled Debt Restructuring
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses	2,030,000		2,030,000			718,000
Changes in Allowance for Credit Losses
Accounts, Notes, Loans and Financing Receivable [Line Items]
Allowance for credit losses change due to loss drivers	600,000		23,300,000
Net charge-offs	700,000		3,300,000
Allowance for credit losses on charged off loans	800,000		1,600,000
Change in allowance for credit losses due to net new specific reserves	800,000		5,700,000
Other non interest income
Accounts, Notes, Loans and Financing Receivable [Line Items]
Net gains (losses) on sale of loans	(515,000)	129,000	(466,000)	229,000
Commercial real estate
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans held for sale	19,919,000		19,919,000			0
Loans	762,531,000		762,531,000			1,046,961,000
Allowance for credit loss on loans purchased with credit deterioration	0		0
Loans past due	11,373,000		11,373,000			4,839,000
Allowance for credit losses	13,152,000	5,798,000	13,152,000	5,798,000	15,539,000	5,353,000	5,677,000	4,493,000
Commercial real estate | Past Due 30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans past due	1,714,000		1,714,000			1,752,000
Factored receivables
Accounts, Notes, Loans and Financing Receivable [Line Items]
Customer reserves	110,510,000		110,510,000			66,754,000
Loans	1,016,337,000		1,016,337,000			619,986,000
Allowance for credit loss on loans purchased with credit deterioration	37,415,000		37,415,000
Loans past due	84,407,000		84,407,000			40,526,000
Allowance for credit losses	44,985,000	7,717,000	44,985,000	7,717,000	5,244,000	7,657,000	6,861,000	7,299,000
Factored receivables | Past Due 30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans past due	68,522,000		68,522,000			29,118,000
Factored receivables | Over-Formula Advances
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans	62,200,000		62,200,000
Allowance for credit loss on loans purchased with credit deterioration	37,400,000		37,400,000
Factored receivables | Over-Formula Advances | Past Due 30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans past due	38,500,000		38,500,000
1-4 family residential
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans held for sale	11,644,000		11,644,000			2,735,000
Loans	164,785,000		164,785,000			179,425,000
Allowance for credit loss on loans purchased with credit deterioration	0		0
Loans past due	2,959,000		2,959,000			2,673,000
Allowance for credit losses	1,972,000	$ 417,000	1,972,000	$ 417,000	$ 2,027,000	308,000	$ 409,000	$ 317,000
1-4 family residential | Past Due 30-59 Days
Accounts, Notes, Loans and Financing Receivable [Line Items]
Loans past due	1,049,000		1,049,000			1,396,000
1-4 family residential | Real Eatate Loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Residential real estate loans in process of foreclosure	$ 208,000		$ 208,000			$ 87,000