Restricted Cash, Other Assets and Other Liabilities - Restricted Cash (Details) - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019	Jun. 30, 2019	Dec. 31, 2018
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	$ 145,229	$ 203,923	$ 336,491	$ 364,605
Capital expenditures reserves
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	36,779	89,901
Real estate escrow reserves
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	33,811	38,326
Borrower escrow deposits
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	7,327	8,079
Lender restricted cash
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	46,722	41,591
Other
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	$ 20,590	$ 26,026