Financings - Narrative (Details)	1 Months Ended	3 Months Ended
	May 31, 2019 USD ($)	Mar. 31, 2020 USD ($) agreement item	Apr. 21, 2020 USD ($) Counterparty	Dec. 31, 2019 USD ($)
Short-term Debt [Line Items]
Number of repurchase agreements | agreement		7
Securities loaned, collateral, right to reclaim cash		$ 90,500,000		$ 43,500,000
Cash posted by counterparties		8,600,000		709,000
Securities purchased under agreements to resell, fair value of collateral		1,800,000		0
Securitized debt, at fair value		1,139,121,000		1,477,454,000
Repurchase agreements, net		$ 1,553,715,000		$ 2,824,801,000
Number of counterparties from whom entity borrowed under repurchase agreement | item		19
Subsequent Event
Short-term Debt [Line Items]
Number of counterparties from whom entity borrowed under repurchase agreement | Counterparty			3
RETL Trust
Short-term Debt [Line Items]
Trust certificates issued		$ 443,900,000
RETL Trust | Affiliated entities
Short-term Debt [Line Items]
Trust certificates issued		49,600,000
RETL Trust | Third parties
Short-term Debt [Line Items]
Trust certificates issued		394,200,000
CMSC Trust
Short-term Debt [Line Items]
Commercial pass-through certificate, amount		25,000,000.0
Trust certificates, fair value		$ 23,800,000
Debt instrument interest rate, effective percentage		8.90%
Securitized commercial loans | CMSC Trust
Short-term Debt [Line Items]
Amount acquired, eliminated in consolidation		$ 13,400,000
Securitized debt, at fair value		10,500,000
Residential  Whole Loans | Subsequent Event
Short-term Debt [Line Items]
Repurchase agreements, net			$ 385,000,000
Residential  Whole Loans | Arroyo Trust
Short-term Debt [Line Items]
Repurchase agreements, net		919,000,000.0
Securitized debt | Securitized commercial loans | CMSC Trust
Short-term Debt [Line Items]
Securitized debt, at fair value		10,500,000
Secured Debt | RETL Trust
Short-term Debt [Line Items]
Securitized debt, at fair value		453,336,000
Secured Debt | Arroyo Trust
Short-term Debt [Line Items]
Fair value		742,297,000
Secured Debt | Residential  Whole Loans | Arroyo Trust
Short-term Debt [Line Items]
Fair value		$ 756,700,000
Maximum aggregate collateral balance over original principal balance percentage		20.00%
Class C | Secured Debt | RETL Trust
Short-term Debt [Line Items]
Securitized debt, at fair value		$ 265,490,000
Estimated Fair Value | Class C | Secured Debt | RETL Trust
Short-term Debt [Line Items]
Commercial pass-through certificate, amount		26,300,000
Securitized debt, at fair value		$ 386,400,000
Arroyo Trust | Residential  Whole Loans | Arroyo Trust
Short-term Debt [Line Items]
Transfered of non-qm residential whole loans	$ 945,500,000