Consolidated Statements of Cash Flows - USD ($) $ in Thousands	3 Months Ended		12 Months Ended
	Mar. 31, 2017	Mar. 31, 2016	Dec. 31, 2016	Mar. 31, 2017	Dec. 31, 2016	Mar. 31, 2016
Cash flows from operating activities:
Net income (loss)	$ 19,630	$ (11,444)
Adjustments to reconcile net income (loss) to net cash provided by (used in) operating activities:
(Gain) loss on extinguishment of debt	54	(5,382)
Depreciation and amortization	8,592	9,802
Unrealized (gain) loss on derivative instruments	5,649	9,630
Unrealized (gain) loss on Agency interest-only securities	(159)	(660)
Unrealized (gain) loss on investment in mutual fund	(40)	0
Provision for loan losses	0	150
Amortization of equity based compensation	7,254	3,464
Amortization of deferred financing costs included in interest expense	2,436	2,688
Amortization of premium on mortgage loan financing	(226)	(216)
Amortization of above- and below-market lease intangibles	(25)	(27)
Amortization of premium/(accretion) of discount and other fees on loans	(2,468)	(3,013)
Amortization of premium/(accretion) of discount and other fees on securities	19,357	18,958
Realized (gain) loss on sale of mortgage loan receivables held for sale	999	(7,830)
Realized (gain) loss on real estate securities	(5,361)	573
Realized gain on sale of real estate, net	(2,331)	(6,095)
Origination of mortgage loan receivables held for sale	(279,898)	(69,360)
Purchases of mortgage loan receivables held for sale	0	(21,667)
Repayment of mortgage loan receivables held for sale	247	524
Proceeds from sales of mortgage loan receivables held for sale	0	316,766
(Income) loss from investments in unconsolidated joint ventures in excess of distributions received	74	(794)
Deferred tax asset	(2,728)	(1,991)
Changes in operating assets and liabilities:
Accrued interest receivable	(2,826)	637
Other assets	29,476	(6,502)
Amount payable pursuant to tax receivable agreement	(230)	0
Accrued expenses and other liabilities	(22,781)	(46,056)
Net cash provided by (used in) operating activities	(225,305)	182,155
Cash flows from investing activities:
Purchase of derivative instruments	(199)	0
Purchases of real estate securities	(43,572)	(218,837)
Repayment of real estate securities	74,285	36,136
Proceeds from sales of real estate securities	361,323	15,477
Origination of mortgage loan receivables held for investment	(249,829)	(49,735)
Repayment of mortgage loan receivables held for investment	68,251	218,410
Capital contributions to investment in unconsolidated joint ventures	0	(59)
Capitalization of interest on investment in unconsolidated joint ventures	(234)	(204)
Purchases of real estate	(3,892)	0
Capital improvements of real estate	(752)	(2,042)
Proceeds from sale of real estate	6,325	23,515
Net cash provided by (used in) investing activities	211,706	22,661
Cash flows from financing activities:
Deferred financing costs paid	(8,143)	(416)
Proceeds from borrowings under debt obligations	2,666,161	3,235,468
Repayment of borrowings under debt obligations	(2,231,899)	(3,414,355)
Cash dividends paid to Class A common shareholders	(48,367)	(33,020)
(Increase) decrease in amount due from trustee included in other assets	(302,281)	0
Payment of liability assumed in exchange for shares for the minimum withholding taxes on vesting restricted stock	(13,258)	(786)
Purchase of treasury stock	0	(4,652)
Net cash provided by (used in) financing activities	41,179	(230,552)
Net increase (decrease) in cash, cash equivalents and restricted cash	27,580	(25,736)
Cash, cash equivalents and restricted cash at beginning of period	89,428	162,794	$ 162,794
Cash, cash equivalents and restricted cash at end of period	117,008	137,058	89,428
Supplemental information:
Cash paid for interest, net of amounts capitalized	26,773	37,683
Cash paid (received) for income taxes	(1,147)	9,807
Non-cash investing and financing activities:
Securities and derivatives purchased, not settled	(2,167)	(8,922)
Transfer from mortgage loans receivable held for sale to mortgage loans receivable held for investment, at amortized cost	119,952	0
Exchange of noncontrolling interest for common stock	93,691	28,328
Change in deferred tax asset related to exchanges of noncontrolling interest for common stock	845	(772)
Dividends declared, not paid	1,017	826
Stock dividends	17,319	64,100
Cash, Cash Equivalents, Restricted Cash And Restricted Cash Equivalents [Abstract]
Cash and cash equivalents				$ 62,568	$ 44,615	$ 82,678
Restricted cash				54,440		54,380
Total cash, cash equivalents and restricted cash shown in the consolidated statement of cash flows	89,428	162,794	$ 89,428	$ 117,008	$ 89,428	$ 137,058
Operating Partnership
Cash flows from financing activities:
Capital contributed by noncontrolling interests in operating partnership	0	250
Capital distributed to noncontrolling interests	(21,034)	(12,937)
Consolidated Joint Venture
Cash flows from financing activities:
Capital distributed to noncontrolling interests	$ 0	$ (104)