Loans Held for Investment and Loans Held for Investment at Fair Value - Schedule of Aging Status of Amortized Cost Basis in Loans Held for Investment Portfolio (Details) - COVID19 Pandemic - Aging Status and Accrual Status - USD ($) $ in Thousands	Sep. 30, 2022	Dec. 31, 2021
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	$ 3,450,893	$ 2,531,826
Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	256,257	276,418
Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	19,033	17,260
Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	81,204	85,935
Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	21,731	17,385
Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	91,368	107,552
Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	7,076	2,986
Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	35,845	45,300
Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	3,194,636	2,255,408
Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	703,835	492,161
Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	852,282	697,326
Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	608,338	391,385
Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	922,258	622,768
Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	65,929	26,003
Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	41,994	25,765
30-59 days
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	130,065	136,320
30-59 days | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	7,183	7,810
30-59 days | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	515	700
30-59 days | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	4,761	4,464
30-59 days | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	147
30-59 days | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	1,760	807
30-59 days | Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		1,224
30-59 days | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		615
30-59 days | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	122,882	128,510
30-59 days | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	19,008	17,319
30-59 days | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	39,899	31,769
30-59 days | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	14,802	14,905
30-59 days | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	46,982	39,045
30-59 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	598	21,412
30-59 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	1,593	4,060
60-89 days
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	67,839	43,877
60-89 days | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	11,461	11,912
60-89 days | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	679	2,314
60-89 days | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	8,203	6,818
60-89 days | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	324	682
60-89 days | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	2,255	1,088
60-89 days | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		1,010
60-89 days | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	56,378	31,965
60-89 days | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	7,126	4,034
60-89 days | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	12,110	7,025
60-89 days | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	9,081	5,580
60-89 days | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	23,643	9,548
60-89 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	500	217
60-89 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,918	5,561
90+days
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	237,613	256,531
90+days | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	237,613	256,531
90+days | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	17,839	14,246
90+days | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	68,240	74,488
90+days | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	21,260	16,703
90+days | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	87,353	105,657
90+days | Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	7,076	1,762
90+days | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	35,845	43,675
Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	435,517	436,728
Past Due | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	256,257	276,253
Past Due | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	19,033	17,260
Past Due | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	81,204	85,770
Past Due | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	21,731	17,385
Past Due | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	91,368	107,552
Past Due | Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	7,076	2,986
Past Due | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	35,845	45,300
Past Due | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	179,260	160,475
Past Due | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	26,134	21,353
Past Due | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	52,009	38,794
Past Due | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	23,883	20,485
Past Due | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	70,625	48,593
Past Due | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	1,098	21,629
Past Due | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	5,511	9,621
Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,015,376	2,095,098
Current | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		165
Current | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		165
Current | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,015,376	2,094,933
Current | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	677,701	470,808
Current | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	800,273	658,532
Current | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	584,455	370,900
Current | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	851,633	574,175
Current | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	64,831	4,374
Current | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	$ 36,483	$ 16,144