Loans Held for Investment and Loans Held for Investment at Fair Value - Schedule of Amortized Cost in Loans Held for Investment based on Accrual Status and by Loan Origination Year (Details) - Accrual Status - USD ($) $ in Thousands	Dec. 31, 2022	Dec. 31, 2021
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	$ 1,243,841	$ 1,143,849
Term Loans Amortized Cost Basis by Origination Year, 2021	1,033,695	214,490
Term Loans Amortized Cost Basis by Origination Year, 2020	168,977	500,689
Term Loans Amortized Cost Basis by Origination Year, 2019	358,100	326,082
Term Loans Amortized Cost Basis by Origination Year, 2018	231,024	171,370
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	241,647	175,345
Total	3,277,284	2,531,825
Commercial - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	275,224	277,906
Term Loans Amortized Cost Basis by Origination Year, 2021	256,059	47,617
Term Loans Amortized Cost Basis by Origination Year, 2020	37,643	87,082
Term Loans Amortized Cost Basis by Origination Year, 2019	62,131	50,817
Term Loans Amortized Cost Basis by Origination Year, 2018	36,398	27,703
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	33,953	18,296
Total	701,408	509,421
Commercial - Purchase | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	273,950	277,618
Term Loans Amortized Cost Basis by Origination Year, 2021	249,100	45,836
Term Loans Amortized Cost Basis by Origination Year, 2020	36,064	81,541
Term Loans Amortized Cost Basis by Origination Year, 2019	56,322	46,637
Term Loans Amortized Cost Basis by Origination Year, 2018	33,193	24,164
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	30,208	16,365
Total	678,837	492,161
Commercial - Purchase | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	1,274	288
Term Loans Amortized Cost Basis by Origination Year, 2021	6,959	1,781
Term Loans Amortized Cost Basis by Origination Year, 2020	1,579	5,541
Term Loans Amortized Cost Basis by Origination Year, 2019	5,809	4,180
Term Loans Amortized Cost Basis by Origination Year, 2018	3,205	3,539
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	3,745	1,931
Total	22,571	17,260
Commercial - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	272,766	242,170
Term Loans Amortized Cost Basis by Origination Year, 2021	222,699	68,915
Term Loans Amortized Cost Basis by Origination Year, 2020	59,650	170,029
Term Loans Amortized Cost Basis by Origination Year, 2019	127,056	145,604
Term Loans Amortized Cost Basis by Origination Year, 2018	113,569	78,866
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	111,357	77,676
Total	907,097	783,260
Commercial - Refinance | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	263,754	239,688
Term Loans Amortized Cost Basis by Origination Year, 2021	210,898	64,966
Term Loans Amortized Cost Basis by Origination Year, 2020	55,795	144,017
Term Loans Amortized Cost Basis by Origination Year, 2019	103,633	118,735
Term Loans Amortized Cost Basis by Origination Year, 2018	93,161	62,374
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	92,723	67,545
Total	819,964	697,325
Commercial - Refinance | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	9,012	2,482
Term Loans Amortized Cost Basis by Origination Year, 2021	11,801	3,949
Term Loans Amortized Cost Basis by Origination Year, 2020	3,855	26,012
Term Loans Amortized Cost Basis by Origination Year, 2019	23,423	26,869
Term Loans Amortized Cost Basis by Origination Year, 2018	20,408	16,492
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	18,634	10,131
Total	87,133	85,935
Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	256,906	264,552
Term Loans Amortized Cost Basis by Origination Year, 2021	237,342	15,627
Term Loans Amortized Cost Basis by Origination Year, 2020	12,875	52,826
Term Loans Amortized Cost Basis by Origination Year, 2019	33,998	33,280
Term Loans Amortized Cost Basis by Origination Year, 2018	20,444	16,350
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	26,868	26,135
Total	588,433	408,770
Residential 1-4 Unit - Purchase | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	249,625	263,180
Term Loans Amortized Cost Basis by Origination Year, 2021	227,235	12,878
Term Loans Amortized Cost Basis by Origination Year, 2020	10,710	48,930
Term Loans Amortized Cost Basis by Origination Year, 2019	31,685	29,544
Term Loans Amortized Cost Basis by Origination Year, 2018	18,891	12,863
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	22,303	23,990
Total	560,449	391,385
Residential 1-4 Unit - Purchase | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	7,281	1,372
Term Loans Amortized Cost Basis by Origination Year, 2021	10,107	2,749
Term Loans Amortized Cost Basis by Origination Year, 2020	2,165	3,896
Term Loans Amortized Cost Basis by Origination Year, 2019	2,313	3,736
Term Loans Amortized Cost Basis by Origination Year, 2018	1,553	3,487
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	4,565	2,145
Total	27,984	17,385
Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	360,350	354,845
Term Loans Amortized Cost Basis by Origination Year, 2021	310,218	37,374
Term Loans Amortized Cost Basis by Origination Year, 2020	31,610	145,961
Term Loans Amortized Cost Basis by Origination Year, 2019	111,954	90,451
Term Loans Amortized Cost Basis by Origination Year, 2018	55,704	48,451
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	69,469	53,238
Total	939,305	730,320
Residential 1-4 Unit - Refinance | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	338,959	343,199
Term Loans Amortized Cost Basis by Origination Year, 2021	285,195	31,334
Term Loans Amortized Cost Basis by Origination Year, 2020	24,703	114,145
Term Loans Amortized Cost Basis by Origination Year, 2019	84,208	59,825
Term Loans Amortized Cost Basis by Origination Year, 2018	39,870	31,774
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	52,461	42,491
Total	825,396	622,768
Residential 1-4 Unit - Refinance | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	21,391	11,646
Term Loans Amortized Cost Basis by Origination Year, 2021	25,023	6,040
Term Loans Amortized Cost Basis by Origination Year, 2020	6,907	31,816
Term Loans Amortized Cost Basis by Origination Year, 2019	27,746	30,626
Term Loans Amortized Cost Basis by Origination Year, 2018	15,834	16,677
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	17,008	10,747
Total	113,909	107,552
Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	42,254	1,890
Term Loans Amortized Cost Basis by Origination Year, 2021	6,156	17,147
Term Loans Amortized Cost Basis by Origination Year, 2020	16,654	9,847
Term Loans Amortized Cost Basis by Origination Year, 2019	4,716	105
Term Loans Amortized Cost Basis by Origination Year, 2018	104
Total	69,884	28,989
Short Term 1-4 Unit - Purchase | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	40,967	1,890
Term Loans Amortized Cost Basis by Origination Year, 2021	944	15,582
Term Loans Amortized Cost Basis by Origination Year, 2020	15,659	8,531
Term Loans Amortized Cost Basis by Origination Year, 2019	4,174
Total	61,744	26,003
Short Term 1-4 Unit - Purchase | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	1,287
Term Loans Amortized Cost Basis by Origination Year, 2021	5,212	1,565
Term Loans Amortized Cost Basis by Origination Year, 2020	995	1,316
Term Loans Amortized Cost Basis by Origination Year, 2019	542	105
Term Loans Amortized Cost Basis by Origination Year, 2018	104
Total	8,140	2,986
Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	36,341	2,486
Term Loans Amortized Cost Basis by Origination Year, 2021	1,221	27,810
Term Loans Amortized Cost Basis by Origination Year, 2020	10,545	34,944
Term Loans Amortized Cost Basis by Origination Year, 2019	18,245	5,825
Term Loans Amortized Cost Basis by Origination Year, 2018	4,805
Total	71,157	71,065
Short Term 1-4 Unit - Refinance | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	35,555	1,448
Term Loans Amortized Cost Basis by Origination Year, 2021		11,991
Term Loans Amortized Cost Basis by Origination Year, 2020		12,326
Total	35,555	25,765
Short Term 1-4 Unit - Refinance | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	786	1,038
Term Loans Amortized Cost Basis by Origination Year, 2021	1,221	15,819
Term Loans Amortized Cost Basis by Origination Year, 2020	10,545	22,618
Term Loans Amortized Cost Basis by Origination Year, 2019	18,245	5,825
Term Loans Amortized Cost Basis by Origination Year, 2018	4,805
Total	$ 35,602	$ 45,300