Restricted Cash, Other Assets and Other Liabilities - Restricted Cash (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019	Mar. 31, 2019	Dec. 31, 2018
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	$ 166,568	$ 203,923	$ 323,503	$ 364,605
Capital expenditures reserves
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	61,524	89,901
Real estate escrow reserves
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	34,061	38,326
Borrower escrow deposits
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	8,234	8,079
Lender restricted cash
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	39,019	41,591
Other
Restricted Cash and Cash Equivalents Items [Line Items]
Restricted cash	$ 23,730	$ 26,026