Restricted Cash, Other Assets and Other Liabilities - Restricted Cash (Details) - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019	Sep. 30, 2019	Dec. 31, 2018
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	$ 167,109	$ 91,063	$ 113,897	$ 135,650
Capital expenditures reserves
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	13,802	18,314
Real estate escrow reserves
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	21,247	15,455
Borrower escrow deposits
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	6,707	8,079
Lender restricted cash
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	87,155	27,409
Other (4)
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	$ 38,198	$ 21,806