Investments (Details) - USD ($)		3 Months Ended
		Mar. 31, 2020	Mar. 31, 2019	Dec. 31, 2018	Dec. 31, 2019
Debt Securities, Available-for-sale [Line Items]
Interest and Dividend Income, Securities, Operating, Available-for-sale		$ 300,000
Proceeds from Sale of Debt Securities, Available-for-sale		0	$ 39,635,000
Debt and Equity Securities, Gain (Loss)			8,000
Investments in beneficial interests	[1]	64,703,000			$ 57,954,000
Investment In Securities		61,300,000			187,800,000
Investments at fair value	[2]	247,372,000			231,685,000
Provision for credit losses		(10,156,000)	0
Provision for credit losses on beneficial interests		(2,987,000)	0
Debt Securities, Available-for-sale [Abstract]
Investment in debt securities - Basis		274,539,000			230,408,000
Investment in debt securities - Gross unrealized gains		0			1,643,000
Investment in debt securities - Gross unrealized loss		(27,167,000)			(366,000)
Investment in debt securities		247,372,000
Oileus Residential Loan Trust
Debt Securities, Available-for-sale [Line Items]
Available-for-sale Securities, Gross Realized Gains				$ 0
Beneficial interests
Debt Securities, Available-for-sale [Line Items]
Provision for credit losses		(4,200,000)
Provision for credit losses on beneficial interests		(2,987,000)	0
Senior Notes
Debt Securities, Available-for-sale [Line Items]
Investments at fair value		49,600,000	50,000,000.0
Subordinated Debt
Debt Securities, Available-for-sale [Line Items]
Investments at fair value		4,600,000	$ 4,600,000
Accumulated Net Investment Gain (Loss) Attributable to Noncontrolling Interest
Debt Securities, Available-for-sale [Abstract]
Investment in beneficial interests - Basis		339,242,000			288,362,000
Investment in beneficial interests - Gross unrealized gains					1,643,000
Investment in beneficial interests - Gross unrealized loss					(366,000)
Investment in beneficial interests		312,075,000			289,639,000
Beneficial interests
Debt Securities, Available-for-sale [Abstract]
Investment in beneficial interests - Basis		64,703,000			57,954,000
Investment in beneficial interests - Gross unrealized gains		0			0
Investment in beneficial interests - Gross unrealized loss		0			0
Investment in beneficial interests		64,703,000			57,954,000
Securities with unrealized loss longer than 12 months
Debt Securities, Available-for-sale [Abstract]
Investment in debt securities - Gross unrealized loss		(5,200,000)			(400,000)
Beneficial interests
Debt Securities, Available-for-sale [Line Items]
Investments in beneficial interests		$ 7,100,000			$ 9,500,000

[1] Investments in beneficial interests includes allowance for credit losses of $7.2 million at March 31, 2020. No allowance for credit losses were recorded as of December 31, 2019.

[2] As of March 31, 2020 and December 31, 2019 Investments at fair value include amortized cost basis of $274.5 million and $230.4 million, respectively, and unrealized losses of $27.2 million and unrealized gains of $1.3 million, respectively.