Real Estate Securities, Available for Sale - Narrative (Details)	1 Months Ended	3 Months Ended			9 Months Ended
	Jul. 31, 2019 USD ($) securitization_trust	Sep. 30, 2020 USD ($) securitization_trust security loan	Jun. 30, 2020 USD ($)	Sep. 30, 2019 USD ($)	Sep. 30, 2020 USD ($) security securitization_trust loan	Sep. 30, 2019 USD ($)	Dec. 31, 2019 USD ($) security
Debt Securities, Available-for-sale [Line Items]
Gross sales price of securities sold					$ 118,586,000	$ 0
Loss from sale of securities					51,889,000	0
Unrealized gains (losses) in OCI		$ 4,300,000		$ 5,100,000	(12,200,000)	22,700,000
Realized loss on securities from write-down to fair value		3,400,000			32,606,000	0
Securities in unrealized loss position		$ 0			$ 0
Interest receivable							$ 700,000
Number of securitization trusts held | securitization_trust		2			2
Proceeds from sale of mortgage loans held in securitization trusts					$ 0	$ 39,848,000
Gross assets		$ 6,511,657,000			6,511,657,000		7,414,306,000
Gross liabilities		4,494,765,000			4,494,765,000		5,212,956,000
Primary beneficiary, securitization trust
Debt Securities, Available-for-sale [Line Items]
Number of securitization trusts sold | securitization_trust	1
Proceeds from sale of mortgage loans held in securitization trusts	$ 33,400,000
Gross assets	1,200,000,000	1,846,662,000			1,846,662,000		1,879,990,000
Gross liabilities	$ 1,200,000,000	1,777,231,000			1,777,231,000		1,769,181,000
Primary beneficiary
Debt Securities, Available-for-sale [Line Items]
Gross assets		3,797,304,000			3,797,304,000		3,502,341,000
Gross liabilities		3,249,193,000			3,249,193,000		3,117,364,000
Mortgage loans held in trust, unpaid principal balance		1,800,000,000			1,800,000,000		1,800,000,000
Mortgage obligations held in trust, unpaid principal balance		$ 1,600,000,000			$ 1,600,000,000		1,600,000,000
Number of underlying mortgage loans | loan		115			115
Weighted average coupon (in percentage)		4.50%			4.50%
Weighted average loan to value ratio (in percentage)		56.80%			56.80%
Difference between held and issued mortgage loans in securitization trusts		$ 68,500,000			$ 68,500,000		$ 110,100,000
Credit facilities | CMBS credit facilities
Debt Securities, Available-for-sale [Line Items]
Debt repaid			$ 73,900,000		$ 79,200,000
Real estate securities, available for sale
Debt Securities, Available-for-sale [Line Items]
Number of securities sold | security		5			32
Gross sales price of securities sold		$ 28,800,000			$ 118,500,000
Gain from sale of securities		$ 5,200,000
Loss from sale of securities					$ 51,800,000
Number of securities held | security		11			11		43
Carrying value of securities		$ 31,959,000			$ 31,959,000		$ 236,303,000
Weighted average contractual maturity					28 years 7 months 6 days
Expected maturity					5 years 6 months