Consolidated Statements of Cash Flows - USD ($) $ in Thousands	3 Months Ended		9 Months Ended		12 Months Ended
	Sep. 30, 2017	Sep. 30, 2016	Sep. 30, 2017	Sep. 30, 2016	Dec. 31, 2016	Sep. 30, 2017	Dec. 31, 2016	Sep. 30, 2016
Cash flows from operating activities:
Net income (loss)	$ 30,453	$ 49,598	$ 63,515	$ 42,099	$ 113,720
Adjustments to reconcile net income (loss) to net cash provided by (used in) operating activities:
(Gain) loss on extinguishment of debt	0	0	54	(5,382)
Depreciation and amortization	10,606	9,733	29,323	28,789
Unrealized (gain) loss on derivative instruments			3,509	6,273
Unrealized (gain) loss on Agency interest-only securities	(577)	47	(1,034)	(29)
Unrealized (gain) loss on investment in mutual fund			(57)	0
Provision for loan losses	0	0	0	300
Amortization of equity based compensation			10,481	12,694
Amortization of deferred financing costs included in interest expense			5,673	5,935
Amortization of premium on mortgage loan financing			(681)	(660)
Amortization of above- and below-market lease intangibles			(451)	(115)
Accretion of premium on liability for transfers not considered sales			188	0
Amortization of premium/(accretion) of discount and other fees on loans			(7,905)	(6,515)
Amortization of premium/(accretion) of discount and other fees on securities			48,315	56,151
Realized (gain) loss on sale of mortgage loan receivables held for sale	775	(19,640)	1,774	(30,265)
Realized (gain) loss on real estate securities	(6,688)	(7,126)	(19,182)	(9,524)
Realized gain on sale of real estate, net	(3,228)	(4,649)	(7,790)	(15,616)
Realized gain on sale of derivative instruments			(1,623)	(24)
Origination of mortgage loan receivables held for sale			(887,978)	(865,497)
Purchases of mortgage loan receivables held for sale			0	(21,667)
Repayment of mortgage loan receivables held for sale			1,655	1,161
Proceeds from sales of mortgage loan receivables held for sale			5	703,846
(Income) loss from investments in unconsolidated joint ventures in excess of distributions received	(127)	141	(64)	(485)
Distributions from operations of investment in unconsolidated joint ventures			0	1,017
Deferred tax asset			(6,556)	(6,263)
Payments pursuant to tax receivable agreement			(230)	0
Changes in operating assets and liabilities:
Accrued interest receivable			(1,987)	(1,018)
Other assets			(3,320)	(13,472)
Accrued expenses and other liabilities			(10,947)	(21,718)
Net cash provided by (used in) operating activities			(785,313)	(139,985)
Cash flows from investing activities:
Purchase of derivative instruments			(199)	(73)
Sale of derivative instruments			0	49
Purchases of real estate securities			(109,198)	(837,190)
Repayment of real estate securities			93,233	307,847
Proceeds from sales of real estate securities			983,386	308,429
Origination of mortgage loan receivables held for investment			(869,981)	(480,622)
Purchases of mortgage loan receivables held for investment			(94,079)	0
Repayment of mortgage loan receivables held for investment			266,359	582,447
Distributions received from investments in unconsolidated joint ventures in excess of income			0	49
Capitalization of interest on investment in unconsolidated joint ventures			(918)	(644)
Purchases of real estate			(230,677)	(50,252)
Capital improvements of real estate			(3,943)	(6,813)
Proceeds from sale of real estate			20,522	60,516
Net cash provided by (used in) investing activities			54,505	(116,257)
Cash flows from financing activities:
Deferred financing costs paid			(14,752)	(2,136)
Proceeds from borrowings under debt obligations			8,248,079	9,290,374
Repayment of borrowings under debt obligations			(7,352,225)	(8,960,397)
Cash dividends paid to Class A common shareholders			(99,452)	(67,166)
Payment of liability assumed in exchange for shares for the minimum withholding taxes on vesting restricted stock			(13,258)	(786)
Purchase of treasury stock			0	(4,652)
Net cash provided by (used in) financing activities			738,757	216,138
Net increase (decrease) in cash, cash equivalents and restricted cash			7,949	(40,104)
Cash, cash equivalents and restricted cash at beginning of period			89,428	162,794	162,794
Cash, cash equivalents and restricted cash at end of period	97,377	122,690	97,377	122,690	89,428
Supplemental information:
Cash paid for interest, net of amounts capitalized			112,838	93,732
Cash paid (received) for income taxes			1,670	14,127
Non-cash investing and financing activities:
Securities and derivatives purchased, not settled			(37)	(16,151)
Securities and derivatives sold, not settled			12,517	0
Origination of mortgage loans receivable held for investment			0	50,378
Repayment of mortgage loans receivable held for investment			0	(50,378)
Transfer from mortgage loans receivable held for sale to mortgage loans receivable held for investment, at amortized cost			719,465	0
Exchange of noncontrolling interest for common stock			188,520	56,461
Change in deferred tax asset related to exchanges of noncontrolling interest for common stock			1,935	(1,413)
Dividends declared, not paid			1,988	1,801
Stock dividends			17,319	64,100
Cash, Cash Equivalents, Restricted Cash and Restricted Cash Equivalents [Abstract]
Cash and cash equivalents						$ 48,894	$ 44,615	$ 59,693
Restricted cash						48,483	44,813	62,997
Total cash, cash equivalents and restricted cash shown in the consolidated statement of cash flows	$ 97,377	$ 122,690	97,377	162,794	$ 162,794	$ 97,377	$ 89,428	$ 122,690
Operating Partnership
Cash flows from financing activities:
Capital contributed by noncontrolling interests in operating partnership			0	250
Capital distributed to noncontrolling interests			(36,372)	(39,040)
Consolidated Joint Venture
Cash flows from financing activities:
Capital contributed by noncontrolling interests in operating partnership			6,935	0
Capital distributed to noncontrolling interests			$ (198)	$ (309)