Commercial Loans - Narrative (Details) $ in Thousands	1 Months Ended			3 Months Ended								12 Months Ended
	Mar. 31, 2019 USD ($)	Mar. 31, 2018 USD ($)	Nov. 30, 2015 USD ($)	Dec. 31, 2019 USD ($)	Sep. 30, 2019 USD ($)	Jun. 30, 2019 USD ($)	Mar. 31, 2019 USD ($)	Dec. 31, 2018 USD ($)	Sep. 30, 2018 USD ($)	Jun. 30, 2018 USD ($)	Mar. 31, 2018 USD ($)	Dec. 31, 2019 USD ($) entity	Dec. 31, 2018 USD ($)	Dec. 31, 2017 USD ($)
Variable Interest Entity [Line Items]
Number of VIEs, as primary beneficiary | entity												3
Securitized commercial loans, at fair value				$ 909,040				$ 1,013,511				$ 909,040	$ 1,013,511
Income tax provision (benefit)				622	$ (55)	$ 478	$ 12	154	$ 206	$ 36	$ 313	1,057	709	$ 3,487
Debt instrument, repurchase amount				62,700								$ 62,700
VIE, consolidated, number of commercial loan trusts | entity												4
Commercial Loans
Variable Interest Entity [Line Items]
VIE, consolidated, number of commercial loan trusts | entity												7
CMSC Trust
Variable Interest Entity [Line Items]
Trust certificates issued				24,000								$ 24,000
Principal balance				24,048				24,456				24,048	24,456
CMSC Trust | Securitized Commercial Loans
Variable Interest Entity [Line Items]
Amount acquired, eliminated in consolidation			$ 14,000									13,500
VIE, consolidated, carrying amount, assets				13,500								13,500
Variable interest entity, fair value				13,500								13,500
RETL Trust
Variable Interest Entity [Line Items]
Trust certificates issued				598,500								598,500
Principal balance				674,331				988,609				674,331	988,609
RETL Trust | Securitized Commercial Loans
Variable Interest Entity [Line Items]
Amount acquired, eliminated in consolidation	$ 65,300	$ 67,800
Variable interest entity, amount acquired, eligible risk retention	$ 45,300
MRCD Trust
Variable Interest Entity [Line Items]
Trust certificates issued				83,600								83,600
Principal balance				245,000				$ 0				245,000	$ 0
MRCD Trust | Securitized Commercial Loans
Variable Interest Entity [Line Items]
Amount acquired, eliminated in consolidation												$ 161,400
VIE, consolidated, number of commercial loan trusts												2
MRCD Trust | Commercial Loans
Variable Interest Entity [Line Items]
Debt instrument, basis spread on variable rate		4.27%
MRCD Trust | Class HRR
Variable Interest Entity [Line Items]
Debt instrument, basis spread on variable rate												12.02%
CMSC Trust
Variable Interest Entity [Line Items]
Securitized commercial loans, at fair value				24,000								$ 24,000
Loans receivable, basis spread on variable rate												9.00%
Trust Certificate | RETL Trust | Securitized Commercial Loans
Variable Interest Entity [Line Items]
Weighted average expected remaining term to the expected maturity of investment portfolio		5 years
Class HRR | MRCD Trust | Securitized Commercial Loans
Variable Interest Entity [Line Items]
Amount acquired, eliminated in consolidation												$ 10,500
London Interbank Offered Rate (LIBOR) | RETL Trust
Variable Interest Entity [Line Items]
Debt instrument, basis spread on variable rate		2.30%
London Interbank Offered Rate (LIBOR) | Trust Certificate | RETL Trust | Securitized Commercial Loans
Variable Interest Entity [Line Items]
Loans receivable, basis spread on variable rate		9.50%
Mortgage Receivable | RETL Trust | Commercial Loans
Variable Interest Entity [Line Items]
Principal balance				674,300								674,300
Mortgage Receivable | MRCD Trust | Commercial Loans
Variable Interest Entity [Line Items]
Principal balance				234,500								234,500
Mortgage Receivable | MRCD Trust | Class HRR
Variable Interest Entity [Line Items]
Principal balance				$ 10,500								$ 10,500