Loans - Ratings (Details)	9 Months Ended
	Sep. 30, 2017 USD ($) loan	Dec. 31, 2016 USD ($)
Investments in loans
Total gross loans	$ 7,071,952,000	$ 5,956,062,000
Total gross loans (as a percent)	100.00%	100.00%
Number of impaired individual mortgage loans held-for-investment | loan	0
Amount of loan impairment charges on individual loans held-for-investment	$ 0
Rating 1
Investments in loans
Total gross loans	$ 22,651,000	$ 921,000
Total gross loans (as a percent)	0.30%
Rating 1 | Maximum
Investments in loans
LTV (as a percent)	65.00%
Rating 2
Investments in loans
Total gross loans	$ 2,246,678,000	$ 1,349,603,000
Total gross loans (as a percent)	31.80%	22.60%
Rating 2 | Maximum
Investments in loans
LTV (as a percent)	70.00%
Rating 3
Investments in loans
Total gross loans	$ 3,883,953,000	$ 4,025,809,000
Total gross loans (as a percent)	54.90%	67.60%
Rating 3 | Maximum
Investments in loans
LTV (as a percent)	80.00%
Rating 4
Investments in loans
Total gross loans	$ 253,856,000	$ 458,133,000
Total gross loans (as a percent)	3.60%	7.70%
Allowance for loan losses as a percent of carrying amount	1.50%
Rating 4 | Minimum
Investments in loans
LTV (as a percent)	80.00%
Rating 4 | Maximum
Investments in loans
LTV (as a percent)	90.00%
Rating 5
Investments in loans
Total gross loans	$ 56,190,000	$ 58,317,000
Total gross loans (as a percent)	0.80%	1.00%
Allowance for loan losses as a percent of carrying amount	5.00%
Rating 5 | Minimum
Investments in loans
LTV (as a percent)	90.00%
N/A
Investments in loans
Total gross loans	$ 608,624,000	$ 63,279,000
Total gross loans (as a percent)	8.60%	1.10%
Total loans held-for-investment
Investments in loans
Total gross loans	$ 6,388,989,000	$ 5,857,783,000
Total loans held-for-investment | First Mortgages, excluding Cost Recovery Loans
Investments in loans
Total gross loans	5,527,219,000	4,865,994,000
Total loans held-for-investment | First Mortgages, excluding Cost Recovery Loans | Rating 1
Investments in loans
Total gross loans	2,205,000	921,000
Total loans held-for-investment | First Mortgages, excluding Cost Recovery Loans | Rating 2
Investments in loans
Total gross loans	2,109,411,000	1,092,731,000
Total loans held-for-investment | First Mortgages, excluding Cost Recovery Loans | Rating 3
Investments in loans
Total gross loans	3,164,497,000	3,348,874,000
Total loans held-for-investment | First Mortgages, excluding Cost Recovery Loans | Rating 4
Investments in loans
Total gross loans	194,916,000	365,151,000
Total loans held-for-investment | First Mortgages, excluding Cost Recovery Loans | Rating 5
Investments in loans
Total gross loans	56,190,000	58,317,000
Total loans held-for-investment | Subordinated mortgages
Investments in loans
Total gross loans	222,990,000	278,032,000
Total loans held-for-investment | Subordinated mortgages | Rating 2
Investments in loans
Total gross loans	4,424,000	27,069,000
Total loans held-for-investment | Subordinated mortgages | Rating 3
Investments in loans
Total gross loans	218,566,000	250,963,000
Total loans held-for-investment | Mezzanine Loans
Investments in loans
Total gross loans	615,562,000	713,757,000
Total loans held-for-investment | Mezzanine Loans | Rating 2
Investments in loans
Total gross loans	132,843,000	194,803,000
Total loans held-for-investment | Mezzanine Loans | Rating 3
Investments in loans
Total gross loans	423,779,000	425,972,000
Total loans held-for-investment | Mezzanine Loans | Rating 4
Investments in loans
Total gross loans	58,940,000	92,982,000
Total loans held-for-investment | Other
Investments in loans
Total gross loans	23,218,000
Total loans held-for-investment | Other | Rating 1
Investments in loans
Total gross loans	20,446,000
Total loans held-for-investment | Other | Rating 3
Investments in loans
Total gross loans	2,772,000
Loans held-for-sale
Investments in loans
Total gross loans	608,624,000	63,279,000
Loans held-for-sale | N/A
Investments in loans
Total gross loans	608,624,000	63,279,000
Loans transferred as secured borrowings
Investments in loans
Total gross loans	74,339,000	35,000,000
Loans transferred as secured borrowings | Rating 2
Investments in loans
Total gross loans		$ 35,000,000
Loans transferred as secured borrowings | Rating 3
Investments in loans
Total gross loans	$ 74,339,000