Income Taxes - Schedule of Deferred Income Tax Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Deferred tax assets
Net operating losses	$ 112,553	$ 36,014
Goodwill	111,865	0
Reverse loans	64,899	46,906
Servicer and protective advances	49,301	37,751
Curtailment liability	44,461	36,086
Intangible assets	32,750	37,013
Accrued expenses	25,022	23,824
Mandatory call obligation	19,695	19,768
Accrued legal contingencies and settlements	13,184	10,284
Servicing rights related liabilities	0	45,063
Other	65,175	48,668
Total deferred tax assets	538,905	341,377
Valuation allowance	(41,499)	(2,999)	$ (3,096)
Total deferred tax assets, net of valuation allowance	497,406	338,378
Deferred tax liabilities
Servicing rights	(135,125)	(151,163)
Net investment in residential loans	(33,126)	(34,967)
Discount on Convertible Notes	(12,515)	(19,743)
Intangible assets	(3,797)	(9,723)
Goodwill	0	(1,498)
Other	(12,917)	(13,234)
Total deferred tax liabilities	(197,480)	(230,328)
Deferred tax assets, net	$ 299,926	$ 108,050