Commercial Loans - Consolidated trusts included in the Consolidated Balance Sheets (Details) - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Noncontrolling Interest [Line Items]
Cash	$ 31,331	$ 21,987	$ 48,024
Restricted cash	52,948	55,808	$ 0
Securitized commercial loans, at fair value	909,040	1,013,511
Commercial Loans, at fair value	350,213	196,123
Interest receivable	19,413	21,959
Total assets	5,160,971	4,497,395
Securitized debt	1,057
Interest payable	15,001	8,532
Accounts payable and accrued expenses	3,188	3,858
Other liabilities	52,948	56,031
Total liabilities	4,596,510	3,994,386
VIE
Noncontrolling Interest [Line Items]
Cash	7,589	674
Restricted cash	52,948	55,808
Securitized commercial loans, at fair value	909,040	1,013,511
Commercial Loans, at fair value	90,788	196,123
Interest receivable	10,829	15,540
Total assets	2,501,179	2,588,150
Interest payable	3,886	2,419
Accounts payable and accrued expenses	185	708
Other liabilities	52,948	56,033
Total liabilities	1,534,473	1,008,786
VIE | Securitized Commercial Loans
Noncontrolling Interest [Line Items]
Cash	5,778	0
Restricted cash	52,948	55,808
Commercial Loans, at fair value	90,788	166,123
Interest receivable	2,989	3,733
Total assets	1,061,543	1,239,175
Securitized debt	681,643	949,626
Interest payable	1,519	2,419
Accounts payable and accrued expenses	12	31
Other liabilities	52,948	55,808
Total liabilities	$ 736,122	$ 1,007,884