Loans and Allowance for Credit Losses - Loan Vintage (Details) - USD ($) $ in Thousands	Dec. 31, 2021	Dec. 31, 2020
Loan classification and delinquency by year of origination
UPB	$ 7,103,423	$ 4,102,035
Current fiscal year	3,702,631	522,961
Year before current fiscal year	605,289	1,229,069
Two years before current fiscal year	974,892	735,604
Three years before current fiscal year	523,579	243,042
Four years before current fiscal year	173,349	110,314
Five or more years before current fiscal year	1,045,667	1,211,980
Total	7,025,407	4,052,970
General allowance for loan losses	(28,113)	(29,539)
Total Loans, net of allowance for loan losses, including loans in consolidated VIEs	6,997,294	4,023,431
Specific allowance for loan losses	17,264	17,193
Current and less than 30 days past due
Loan classification and delinquency by year of origination
UPB	6,901,474	3,904,294
Current fiscal year	3,666,020	516,474
Year before current fiscal year	596,289	1,221,227
Two years before current fiscal year	953,269	707,068
Three years before current fiscal year	473,798	203,331
Four years before current fiscal year	167,629	100,003
Five or more years before current fiscal year	984,680	1,125,100
Total	6,841,685	3,873,203
30-59 Days Past Due
Loan classification and delinquency by year of origination
UPB	73,836	38,836
Current fiscal year	35,549	5,812
Year before current fiscal year	352	5,191
Two years before current fiscal year	18,393	15,097
Three years before current fiscal year	3,714	401
Four years before current fiscal year	228	2
Five or more years before current fiscal year	14,601	11,933
Total	72,837	38,436
60+ Days Past Due
Loan classification and delinquency by year of origination
UPB	128,113	158,905
Current fiscal year	1,062	675
Year before current fiscal year	8,648	2,651
Two years before current fiscal year	3,230	13,439
Three years before current fiscal year	46,067	39,310
Four years before current fiscal year	5,492	10,309
Five or more years before current fiscal year	46,386	74,947
Total	110,885	141,331
Originated Transitional loans
Loan classification and delinquency by year of origination
UPB	4,422,524	1,325,713
Current fiscal year	3,339,437	385,183
Year before current fiscal year	420,486	583,593
Two years before current fiscal year	424,118	306,971
Three years before current fiscal year	192,901	23,783
Four years before current fiscal year		16,794
Five or more years before current fiscal year	5,340
Total	4,382,282	1,316,324
General allowance for loan losses	(15,841)	(14,995)
Specific allowance for loan losses	4,375
Originated Transitional loans | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	4,289,571	1,278,829
Originated Transitional loans | 30-59 Days Past Due
Loan classification and delinquency by year of origination
Total	52,997	17,713
Originated Transitional loans | 60+ Days Past Due
Loan classification and delinquency by year of origination
Total	39,714	19,782
Originated SBC loans
Loan classification and delinquency by year of origination
UPB	975,628	1,055,387
Current fiscal year	142,947	66,715
Year before current fiscal year	58,651	486,033
Two years before current fiscal year	394,648	237,313
Three years before current fiscal year	163,912	110,354
Four years before current fiscal year	98,123	45,382
Five or more years before current fiscal year	121,752	113,508
Total	980,033	1,059,305
General allowance for loan losses	(2,666)	(2,640)
Specific allowance for loan losses	4,194	6,200
Originated SBC loans | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	944,803	1,003,628
Originated SBC loans | 30-59 Days Past Due
Loan classification and delinquency by year of origination
Total		6,591
Originated SBC loans | 60+ Days Past Due
Loan classification and delinquency by year of origination
Total	35,230	49,086
Acquired loans
Loan classification and delinquency by year of origination
UPB	1,073,737	1,053,679
Current fiscal year	126,803	21,414
Year before current fiscal year	80,705	40,572
Two years before current fiscal year	50,720	42,167
Three years before current fiscal year	44,530	38,649
Four years before current fiscal year	26,194	19,533
Five or more years before current fiscal year	733,996	883,774
Total	1,062,948	1,046,109
General allowance for loan losses	(2,953)	(5,457)
Specific allowance for loan losses	3,168	2,840
Acquired loans | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	1,018,805	978,346
Acquired loans | 30-59 Days Past Due
Loan classification and delinquency by year of origination
Total	13,099	7,729
Acquired loans | 60+ Days Past Due
Loan classification and delinquency by year of origination
Total	31,044	60,034
Originated SBA 7(a) loans
Loan classification and delinquency by year of origination
UPB	420,184	387,389
Current fiscal year	91,991	47,939
Year before current fiscal year	44,878	98,568
Two years before current fiscal year	91,208	133,812
Three years before current fiscal year	108,631	68,375
Four years before current fiscal year	48,771	22,056
Five or more years before current fiscal year	18,779	4,041
Total	404,258	374,791
General allowance for loan losses	(6,110)	(5,680)
Specific allowance for loan losses	2,986	4,371
Originated SBA 7(a) loans | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	397,554	369,416
Originated SBA 7(a) loans | 30-59 Days Past Due
Loan classification and delinquency by year of origination
Total	5,309	1,741
Originated SBA 7(a) loans | 60+ Days Past Due
Loan classification and delinquency by year of origination
Total	1,395	3,634
Acquired SBA 7(a) loans
Loan classification and delinquency by year of origination
UPB	197,828	262,571
Current fiscal year	39	139
Year before current fiscal year	77	19,658
Two years before current fiscal year	13,730	14,636
Three years before current fiscal year	13,605	283
Four years before current fiscal year	261	19
Five or more years before current fiscal year	154,842	204,703
Total	182,554	239,438
General allowance for loan losses	(543)	(767)
Specific allowance for loan losses	2,541	3,782
Acquired SBA 7(a) loans | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	179,039	228,651
Acquired SBA 7(a) loans | 30-59 Days Past Due
Loan classification and delinquency by year of origination
Total	1,432	4,008
Acquired SBA 7(a) loans | 60+ Days Past Due
Loan classification and delinquency by year of origination
Total	2,083	6,779
Originated SBC loans, at fair value
Loan classification and delinquency by year of origination
UPB	10,956	14,088
Three years before current fiscal year		1,598
Four years before current fiscal year		6,442
Five or more years before current fiscal year	10,766	5,755
Total	10,766	13,795
Originated SBC loans, at fair value | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	10,766	13,795
Originated Residential Agency loans
Loan classification and delinquency by year of origination
UPB	2,566	3,208
Current fiscal year	1,414	1,571
Year before current fiscal year	492	645
Two years before current fiscal year	468	705
Four years before current fiscal year		88
Five or more years before current fiscal year	192	199
Total	2,566	3,208
Originated Residential Agency loans | Current and less than 30 days past due
Loan classification and delinquency by year of origination
Total	1,147	538
Originated Residential Agency loans | 30-59 Days Past Due
Loan classification and delinquency by year of origination
Total		654
Originated Residential Agency loans | 60+ Days Past Due
Loan classification and delinquency by year of origination
Total	$ 1,419	$ 2,016