Commercial Loans - Consolidated commercial loan trusts included in the Consolidated Balance Sheets (Details) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2020 USD ($) loan	Dec. 31, 2019 USD ($) loan	Mar. 31, 2019 USD ($)
Noncontrolling Interest [Line Items]
Number of commercial loans trusts | loan	3	3
Cash and cash equivalents	$ 10,342	$ 31,331	$ 19,558
Restricted cash	33,229	52,948	$ 71,178
Securitized commercial loans, at fair value	477,131	909,040
Commercial Loans, at fair value	320,308	350,213
Interest receivable	14,805	19,413
Total assets	3,203,583	5,160,971
Interest payable	6,429	15,001
Accounts payable and accrued expenses	6,307	3,188
Other liabilities	45,779	52,948
Total liabilities	3,021,350	4,596,510
VIE
Noncontrolling Interest [Line Items]
Cash and cash equivalents	4,542	7,589
Restricted cash	33,229	52,948
Securitized commercial loans, at fair value	477,131	909,040
Commercial Loans, at fair value	71,684	90,788
Interest receivable	10,226	10,829
Total assets	1,959,017	2,501,179
Interest payable	3,215	3,886
Accounts payable and accrued expenses	128	185
Other liabilities	33,229	52,948
Total liabilities	1,175,693	1,534,473
Securitized commercial loans | VIE
Noncontrolling Interest [Line Items]
Cash and cash equivalents	0	5,778
Restricted cash	33,229	52,948
Securitized commercial loans, at fair value	477,131	909,040
Commercial Loans, at fair value	71,684	90,788
Interest receivable	1,709	2,989
Total assets	583,753	1,061,543
Securitized debt, at fair value	396,824	681,643
Interest payable	1,005	1,519
Accounts payable and accrued expenses	6	12
Other liabilities	33,229	52,948
Total liabilities	$ 431,064	$ 736,122
Commercial Loans
Noncontrolling Interest [Line Items]
Number of commercial loans trusts | loan	5