Equity and Debt Investments - Debt Securities (Details)	3 Months Ended		6 Months Ended		12 Months Ended
	Jun. 30, 2020 USD ($) security	Jun. 30, 2019 USD ($)	Jun. 30, 2020 USD ($) security	Jun. 30, 2019 USD ($)	Dec. 31, 2019 USD ($) security
Debt Securities, Available-for-sale [Line Items]
Allowance for credit loss recorded	$ 21,400,000		$ 22,200,000
Number of AFS debt securities with unrealized loss without allowance for credit loss | security	0		0
OTTI loss		$ 700,000		$ 700,000
Number of AFS debt securities with unrealized loss in AOCI | security					0
N-Star CDO bonds
Debt Securities, Available-for-sale [Line Items]
Amortized Cost	$ 46,922,000		$ 46,922,000		$ 46,002,000
Allowance for Credit Loss	(22,229,000)		(22,229,000)
Gross cumulative unrealized gains	7,578,000		7,578,000		8,857,000
Gross cumulative unrealized loss	0		0		0
Fair Value	$ 32,271,000		32,271,000		54,859,000
Colony | CMBS of consolidated fund
Debt Securities, Available-for-sale [Line Items]
Proceeds from sale			$ 0		$ 0
Minimum
Debt Securities, Available-for-sale [Line Items]
Contractual maturity period			17 years
Available For Sale Securities Expected Maturity Period			7 months
Maximum
Debt Securities, Available-for-sale [Line Items]
Contractual maturity period			21 years
Available For Sale Securities Expected Maturity Period			3 years 6 months