Real Estate Securities, Available for Sale - Assets and Liabilities Related to Securitized Trust (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019	Jul. 31, 2019
Assets
Mortgage loans held in securitization trusts, at fair value	$ 1,839,953	$ 1,872,970
Receivables, net	46,325	46,456
Total assets	6,850,267	7,414,306
Liabilities
Mortgage obligations issued by securitization trusts, at fair value	1,758,325	1,762,914
Accrued and other liabilities	101,945	140,424
Total liabilities	4,849,430	5,212,956
Primary beneficiary, securitization trust
Assets
Mortgage loans held in securitization trusts, at fair value	1,839,953	1,872,970
Receivables, net	7,170	7,020
Total assets	1,847,123	1,879,990	$ 1,200,000
Liabilities
Mortgage obligations issued by securitization trusts, at fair value	1,758,325	1,762,914
Accrued and other liabilities	6,254	6,267
Total liabilities	$ 1,764,579	$ 1,769,181	$ 1,200,000