INCOME TAXES - Schedule of Deferred Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2018	Dec. 31, 2017	Dec. 31, 2016
Deferred tax assets:
Net operating losses and tax credit carryforwards	$ 41,713	$ 20,682
Interest accruals not currently deductible for tax purposes	0	2,628
Basis differences for REO and other assets	8,453	8,034
Unrealized mark to market	36,758	0
Other	3,087	2,279
Total deferred tax assets	90,011	33,623
Less valuation allowance	0	(12,404)	$ (10,054)
Net deferred tax assets	90,011	21,219
Deferred tax liabilities:
Basis difference for partnership and other investments	(24,179)	(3,873)
Interest accruals not currently includible in income for tax purposes	0	(6,979)
Unrealized gains on servicer advances	0	(29,585)
Total deferred tax liability	(24,179)	(40,437)
Net deferred tax assets	65,832
Net deferred tax liability		$ (19,218)
Taxable REIT Subsidiaries
Operating Loss Carryforwards [Line Items]
Net operating loss carryforwards	$ 131,300