Loans Held for Investment and Loans Held for Investment at Fair Value - Schedule of Aging Status of Amortized Cost Basis in Loans Held for Investment Portfolio (Details) - COVID19 Pandemic - Aging Status and Accrual Status - USD ($) $ in Thousands	Jun. 30, 2022	Dec. 31, 2021
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	$ 3,123,704	$ 2,531,826
Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	255,202	276,418
Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	14,911	17,260
Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	80,442	85,935
Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	22,885	17,385
Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	96,649	107,552
Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	2,302	2,986
Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	38,013	45,300
Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	2,868,502	2,255,408
Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	662,580	492,161
Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	832,332	697,326
Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	506,911	391,385
Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	791,071	622,768
Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	55,671	26,003
Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Total loans	19,937	25,765
30-59 days
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	112,505	136,320
30-59 days | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	5,215	7,810
30-59 days | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	310	700
30-59 days | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,551	4,464
30-59 days | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	325
30-59 days | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	1,029	807
30-59 days | Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		1,224
30-59 days | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due		615
30-59 days | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	107,290	128,510
30-59 days | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	13,431	17,319
30-59 days | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	30,249	31,769
30-59 days | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	14,855	14,905
30-59 days | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	42,227	39,045
30-59 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,052	21,412
30-59 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,476	4,060
60-89 days
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	51,815	43,877
60-89 days | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	11,809	11,912
60-89 days | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	1,875	2,314
60-89 days | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	6,314	6,818
60-89 days | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	672	682
60-89 days | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	2,701	1,088
60-89 days | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	247	1,010
60-89 days | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	40,006	31,965
60-89 days | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	9,952	4,034
60-89 days | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	14,483	7,025
60-89 days | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,805	5,580
60-89 days | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	10,519	9,548
60-89 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	677	217
60-89 days | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	570	5,561
90+days
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	238,018	256,531
90+days | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	238,018	256,531
90+days | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	12,726	14,246
90+days | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	70,417	74,488
90+days | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	21,888	16,703
90+days | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	92,919	105,657
90+days | Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	2,302	1,762
90+days | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	37,766	43,675
Past Due
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	402,338	436,728
Past Due | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	255,042	276,253
Past Due | Loans Individually Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	14,911	17,260
Past Due | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	80,282	85,770
Past Due | Loans Individually Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	22,885	17,385
Past Due | Loans Individually Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	96,649	107,552
Past Due | Loans Individually Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	2,302	2,986
Past Due | Loans Individually Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	38,013	45,300
Past Due | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	147,296	160,475
Past Due | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	23,383	21,353
Past Due | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	44,732	38,794
Past Due | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	18,660	20,485
Past Due | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	52,746	48,593
Past Due | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	3,729	21,629
Past Due | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	4,046	9,621
Current
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	2,721,366	2,095,098
Current | Loans Individually Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	160	165
Current | Loans Individually Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	160	165
Current | Loans Collectively Evaluated
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	2,721,206	2,094,933
Current | Loans Collectively Evaluated | Commercial - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	639,197	470,808
Current | Loans Collectively Evaluated | Commercial - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	787,600	658,532
Current | Loans Collectively Evaluated | Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	488,251	370,900
Current | Loans Collectively Evaluated | Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	738,325	574,175
Current | Loans Collectively Evaluated | Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	51,942	4,374
Current | Loans Collectively Evaluated | Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment Past Due [Line Items]
Loans, past due	$ 15,891	$ 16,144