Loans Held for Investment and Loans Held for Investment at Fair Value - Schedule of Amortized Cost in Loans Held for Investment based on Accrual Status and by Loan Origination Year (Details) - Accrual Status - USD ($) $ in Thousands	Sep. 30, 2022	Dec. 31, 2021
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	$ 1,333,467	$ 1,143,849
Term Loans Amortized Cost Basis by Origination Year, 2021	1,061,536	214,490
Term Loans Amortized Cost Basis by Origination Year, 2020	173,933	500,689
Term Loans Amortized Cost Basis by Origination Year, 2019	382,198	326,082
Term Loans Amortized Cost Basis by Origination Year, 2018	244,011	171,370
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	255,748	175,346
Total	3,450,893	2,531,826
Commercial Purchase Portfolio Segment
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	276,449	277,906
Term Loans Amortized Cost Basis by Origination Year, 2021	264,923	47,617
Term Loans Amortized Cost Basis by Origination Year, 2020	40,062	87,082
Term Loans Amortized Cost Basis by Origination Year, 2019	66,231	50,817
Term Loans Amortized Cost Basis by Origination Year, 2018	39,500	27,703
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	35,703	18,296
Total	722,868	509,421
Commercial Purchase Portfolio Segment | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	275,946	277,618
Term Loans Amortized Cost Basis by Origination Year, 2021	259,673	45,836
Term Loans Amortized Cost Basis by Origination Year, 2020	38,948	81,541
Term Loans Amortized Cost Basis by Origination Year, 2019	60,485	46,637
Term Loans Amortized Cost Basis by Origination Year, 2018	36,425	24,164
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	32,358	16,365
Total	703,835	492,161
Commercial Purchase Portfolio Segment | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	503	288
Term Loans Amortized Cost Basis by Origination Year, 2021	5,250	1,781
Term Loans Amortized Cost Basis by Origination Year, 2020	1,114	5,541
Term Loans Amortized Cost Basis by Origination Year, 2019	5,746	4,180
Term Loans Amortized Cost Basis by Origination Year, 2018	3,075	3,539
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	3,345	1,931
Total	19,033	17,260
Commercial Refinance Portfolio Segment
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	276,390	242,170
Term Loans Amortized Cost Basis by Origination Year, 2021	226,172	68,915
Term Loans Amortized Cost Basis by Origination Year, 2020	60,385	170,029
Term Loans Amortized Cost Basis by Origination Year, 2019	135,636	145,604
Term Loans Amortized Cost Basis by Origination Year, 2018	116,734	78,866
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	118,169	77,677
Total	933,486	783,261
Commercial Refinance Portfolio Segment | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	274,678	239,688
Term Loans Amortized Cost Basis by Origination Year, 2021	215,490	64,966
Term Loans Amortized Cost Basis by Origination Year, 2020	56,081	144,017
Term Loans Amortized Cost Basis by Origination Year, 2019	113,984	118,735
Term Loans Amortized Cost Basis by Origination Year, 2018	96,016	62,374
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	96,033	67,546
Total	852,282	697,326
Commercial Refinance Portfolio Segment | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	1,712	2,482
Term Loans Amortized Cost Basis by Origination Year, 2021	10,682	3,949
Term Loans Amortized Cost Basis by Origination Year, 2020	4,304	26,012
Term Loans Amortized Cost Basis by Origination Year, 2019	21,652	26,869
Term Loans Amortized Cost Basis by Origination Year, 2018	20,718	16,492
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	22,136	10,131
Total	81,204	85,935
Residential 1-4 Unit - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	286,428	264,552
Term Loans Amortized Cost Basis by Origination Year, 2021	241,987	15,627
Term Loans Amortized Cost Basis by Origination Year, 2020	13,177	52,826
Term Loans Amortized Cost Basis by Origination Year, 2019	38,152	33,280
Term Loans Amortized Cost Basis by Origination Year, 2018	21,644	16,350
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	28,681	26,135
Total	630,069	408,770
Residential 1-4 Unit - Purchase | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	283,941	263,180
Term Loans Amortized Cost Basis by Origination Year, 2021	233,398	12,878
Term Loans Amortized Cost Basis by Origination Year, 2020	11,009	48,930
Term Loans Amortized Cost Basis by Origination Year, 2019	36,342	29,544
Term Loans Amortized Cost Basis by Origination Year, 2018	19,532	12,863
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	24,116	23,990
Total	608,338	391,385
Residential 1-4 Unit - Purchase | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	2,487	1,372
Term Loans Amortized Cost Basis by Origination Year, 2021	8,589	2,749
Term Loans Amortized Cost Basis by Origination Year, 2020	2,168	3,896
Term Loans Amortized Cost Basis by Origination Year, 2019	1,810	3,736
Term Loans Amortized Cost Basis by Origination Year, 2018	2,112	3,487
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	4,565	2,145
Total	21,731	17,385
Residential 1-4 Unit - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	408,459	354,845
Term Loans Amortized Cost Basis by Origination Year, 2021	320,978	37,374
Term Loans Amortized Cost Basis by Origination Year, 2020	31,910	145,961
Term Loans Amortized Cost Basis by Origination Year, 2019	117,973	90,451
Term Loans Amortized Cost Basis by Origination Year, 2018	61,111	48,451
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	73,195	53,238
Total	1,013,626	730,320
Residential 1-4 Unit - Refinance | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	401,616	343,199
Term Loans Amortized Cost Basis by Origination Year, 2021	304,684	31,334
Term Loans Amortized Cost Basis by Origination Year, 2020	25,145	114,145
Term Loans Amortized Cost Basis by Origination Year, 2019	92,280	59,825
Term Loans Amortized Cost Basis by Origination Year, 2018	42,422	31,774
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	56,111	42,491
Total	922,258	622,768
Residential 1-4 Unit - Refinance | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	6,843	11,646
Term Loans Amortized Cost Basis by Origination Year, 2021	16,294	6,040
Term Loans Amortized Cost Basis by Origination Year, 2020	6,765	31,816
Term Loans Amortized Cost Basis by Origination Year, 2019	25,693	30,626
Term Loans Amortized Cost Basis by Origination Year, 2018	18,689	16,677
Term Loans Amortized Cost Basis by Origination Year, Pre-2018	17,084	10,747
Total	91,368	107,552
Short Term 1-4 Unit - Purchase
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	45,397	1,890
Term Loans Amortized Cost Basis by Origination Year, 2021	6,255	17,147
Term Loans Amortized Cost Basis by Origination Year, 2020	16,201	9,847
Term Loans Amortized Cost Basis by Origination Year, 2019	5,047	105
Term Loans Amortized Cost Basis by Origination Year, 2018	105
Total	73,005	28,989
Short Term 1-4 Unit - Purchase | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	45,397	1,890
Term Loans Amortized Cost Basis by Origination Year, 2021	1,256	15,582
Term Loans Amortized Cost Basis by Origination Year, 2020	15,206	8,531
Term Loans Amortized Cost Basis by Origination Year, 2019	4,070
Total	65,929	26,003
Short Term 1-4 Unit - Purchase | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2021	4,999	1,565
Term Loans Amortized Cost Basis by Origination Year, 2020	995	1,316
Term Loans Amortized Cost Basis by Origination Year, 2019	977	105
Term Loans Amortized Cost Basis by Origination Year, 2018	105
Total	7,076	2,986
Short Term 1-4 Unit - Refinance
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	40,344	2,486
Term Loans Amortized Cost Basis by Origination Year, 2021	1,221	27,810
Term Loans Amortized Cost Basis by Origination Year, 2020	12,198	34,944
Term Loans Amortized Cost Basis by Origination Year, 2019	19,159	5,825
Term Loans Amortized Cost Basis by Origination Year, 2018	4,917
Total	77,839	71,065
Short Term 1-4 Unit - Refinance | Performing
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022	40,344	1,448
Term Loans Amortized Cost Basis by Origination Year, 2021	153	11,991
Term Loans Amortized Cost Basis by Origination Year, 2020	1,382	12,326
Term Loans Amortized Cost Basis by Origination Year, 2019	115
Total	41,994	25,765
Short Term 1-4 Unit - Refinance | Nonperforming
Financing Receivable Recorded Investment [Line Items]
Term Loans Amortized Cost Basis by Origination Year, 2022		1,038
Term Loans Amortized Cost Basis by Origination Year, 2021	1,068	15,819
Term Loans Amortized Cost Basis by Origination Year, 2020	10,816	22,618
Term Loans Amortized Cost Basis by Origination Year, 2019	19,044	5,825
Term Loans Amortized Cost Basis by Origination Year, 2018	4,917
Total	$ 35,845	$ 45,300