Fair Value Measurements - Carrying Amounts and Estimated Fair Values of Financial Instruments (Details) - USD ($) $ in Thousands	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021
Assets:
Restricted cash	$ 21,361	$ 16,808	$ 11,639
Loans held for sale, at fair value	17,590
Loans held for investment, net	2,828,123	3,272,390
Loans held for investment, at fair value	1,306,072	276,095
Mortgage servicing rights	8,578	9,238	$ 7,152
Liabilities:
Secured financing, net	211,083	209,846
Securitized debt at fair value	877,417
Derivative Liability	3,665
Carrying Value
Assets:
Cash	40,566	45,248
Restricted cash	21,361	16,808
Loans held for sale, at fair value	17,590
Loans held for investment, net	2,828,123	3,272,390
Loans held for investment, at fair value	1,306,072	276,095
Accrued interest receivables	27,028	20,463
Mortgage servicing rights	8,578	9,238
Liabilities:
Secured financing, net	211,083	209,846
Warehouse and repurchase facilities, net	334,755	330,814
Securitized debt, net	2,418,811	2,736,290
Securitized debt at fair value	877,417
Accrued interest payable	20,473	16,369
Derivative Liability	3,665
Estimate of Fair Value Measurement
Assets:
Cash	40,566	45,248
Restricted cash	21,361	16,808
Loans held for sale, at fair value	17,590
Loans held for investment, net	2,672,705	3,201,850
Loans held for investment, at fair value	1,306,072	276,095
Accrued interest receivables	27,028	20,463
Mortgage servicing rights	8,578	9,238
Liabilities:
Secured financing, net	212,625	211,854
Warehouse and repurchase facilities, net	334,755	330,814
Securitized debt, net	2,155,718	2,522,010
Securitized debt at fair value	877,417
Accrued interest payable	20,473	16,369
Derivative Liability	3,665
Level 1 | Estimate of Fair Value Measurement
Assets:
Cash	40,566	45,248
Restricted cash	21,361	16,808
Accrued interest receivables	27,028	20,463
Liabilities:
Accrued interest payable	20,473	16,369
Derivative Liability	3,665
Level 2
Liabilities:
Securitized debt, net	2,155,718
Securitized debt at fair value	877,417
Level 2 | Estimate of Fair Value Measurement
Assets:
Loans held for sale, at fair value	17,590
Liabilities:
Warehouse and repurchase facilities, net	334,755	330,814
Level 3 | Estimate of Fair Value Measurement
Assets:
Loans held for investment, net	2,672,705	3,201,850
Loans held for investment, at fair value	1,306,072	276,095
Mortgage servicing rights	8,578	9,238
Liabilities:
Secured financing, net	212,625	211,854
Securitized debt, net	$ 0	$ 2,522,010