Basis of Presentation and Summary of Significant Accounting Policies - Summary of Assets and Liabilities of Trusts (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Variable Interest Entity [Line Items]
Loans held for investment, net	$ 1,861,819	$ 1,863,360
Real estate owned, net	15,648	13,068
Other assets	9,042	12,667
Total assets	2,187,051	2,214,766
Accounts payable and accrued expenses	55,938	56,146
Total liabilities	1,891,024	2,061,922
Variable Interest Entity, Primary Beneficiary
Variable Interest Entity [Line Items]
Restricted cash	1,501
Loans held for investment, net	1,813,658	1,571,100
Accrued interest and other receivables	48,012	56,675
Real estate owned, net	8,440	6,091
Other assets	5	11
Total assets	1,871,616	1,633,877
Accounts payable and accrued expenses	29,888	21,265
Securities issued	1,599,719	1,438,629
Total liabilities	$ 1,629,607	$ 1,459,894