Separate Financial Information of Subsidiary Guarantors of Indebtedness - Condensed Consolidated Statements of Cash Flows (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Condensed Cash Flow Statements, Captions [Line Items]
Cash flows provided by (used in) operating activities	$ 451,950	$ (48,093)	$ (204,270)
Investing activities
Purchases and originations of reverse loans held for investment	(896,879)	(1,471,275)	(1,505,215)
Principal payments received on reverse loans held for investment	1,122,267	871,832	548,660
Principal payments received on mortgage loans held for investment	92,619	114,906	162,257
Payments received on charged-off loans held for investment	23,060	26,385	14,929
Payments received on receivables related to Non-Residual Trusts	8,110	7,481	9,471
Proceeds from sales of real estate owned, net	111,091	76,703	55,306
Purchases of premises and equipment	(32,866)	(27,761)	(21,573)
Decrease in restricted cash and cash equivalents	8,946	9,219	11,333
Payments for acquisitions of businesses, net of cash acquired	(3,066)	(5,095)	(197,061)
Acquisitions of servicing rights, net	(9,794)	(264,743)	(268,618)
Proceeds from sales of servicing rights, net	280,970	0	0
Acquisitions of charged-off loans held for investment	0	0	(64,548)
Proceeds from sale of residual interests in Residual Trusts	0	189,513	0
Proceeds from sale of investment	0	14,376	0
Capital contributions to subsidiaries and VIEs	0	0	0
Returns of capital from subsidiaries and VIEs	0	0	0
Change in due from affiliates	0	0	0
Other	(4,649)	3,511	10,948
Cash flows provided by (used in) investing activities	699,809	(454,948)	(1,244,111)
Financing activities
Payments on corporate debt	(480)	(12,901)	(17,220)
Extinguishments and settlement of debt	(31,037)	(79,877)	0
Proceeds from securitizations of reverse loans	960,157	1,622,481	1,617,399
Payments on HMBS related obligations	(1,371,375)	(1,025,458)	(637,272)
Issuances of servicing advance liabilities	2,179,488	2,073,227	2,299,930
Payments on servicing advance liabilities	(2,625,476)	(2,206,965)	(1,905,331)
Net change in warehouse borrowings related to mortgage loans	(151,172)	207,305	75,726
Net change in warehouse borrowings related to reverse loans	14,139	(43,873)	15,667
Proceeds from sales of excess servicing spreads and servicing rights	34,307	55,698	75,426
Payments on servicing rights related liabilities	(22,092)	(12,317)	(6,822)
Payments on mortgage-backed debt	(107,598)	(136,493)	(181,155)
Other debt issuance costs paid	(11,039)	(13,949)	(17,281)
Repurchase of shares under stock repurchase plan	0	(28,065)	0
Capital contributions	0	0	0
Capital distributions	0	0	0
Change in due to affiliates	0	0	0
Other	2,189	(13,119)	(42,396)
Cash flows provided by (used in) financing activities	(1,129,989)	385,694	1,276,671
Net increase (decrease) in cash and cash equivalents	21,770	(117,347)	(171,710)
Cash and cash equivalents at the beginning of the year	202,828	320,175	491,885
Cash and cash equivalents at the end of the year	224,598	202,828	320,175
Reportable Legal Entities [Member] | Parent Company [Member]
Condensed Cash Flow Statements, Captions [Line Items]
Cash flows provided by (used in) operating activities	(204,359)	(107,969)	(74,452)
Investing activities
Purchases and originations of reverse loans held for investment	0	0	0
Principal payments received on reverse loans held for investment	0	0	0
Principal payments received on mortgage loans held for investment	940	828	535
Payments received on charged-off loans held for investment	0	0	0
Payments received on receivables related to Non-Residual Trusts	0	0	0
Proceeds from sales of real estate owned, net	30	118	227
Purchases of premises and equipment	(595)	(175)	0
Decrease in restricted cash and cash equivalents	9,011	(6)	4,246
Payments for acquisitions of businesses, net of cash acquired	0	0	0
Acquisitions of servicing rights, net	0	0	0
Proceeds from sales of servicing rights, net	0
Acquisitions of charged-off loans held for investment			0
Proceeds from sale of residual interests in Residual Trusts	0	189,513	0
Proceeds from sale of investment	0	14,376	0
Capital contributions to subsidiaries and VIEs	0	(9,072)	(83,544)
Returns of capital from subsidiaries and VIEs	10,991	27,309	76,214
Change in due from affiliates	126,883	(8,331)	88,360
Other	309	2,656	(2,283)
Cash flows provided by (used in) investing activities	147,569	217,216	83,755
Financing activities
Payments on corporate debt	0	(11,250)	(15,000)
Extinguishments and settlement of debt	(31,037)	(79,877)	0
Proceeds from securitizations of reverse loans	0	0	0
Payments on HMBS related obligations	0	0	0
Issuances of servicing advance liabilities	0	0	0
Payments on servicing advance liabilities	0	0	0
Net change in warehouse borrowings related to mortgage loans	0	0	0
Net change in warehouse borrowings related to reverse loans	0	0	0
Proceeds from sales of excess servicing spreads and servicing rights	0	0	0
Payments on servicing rights related liabilities	0	0	0
Payments on mortgage-backed debt	0	0	0
Other debt issuance costs paid	(528)	0	0
Repurchase of shares under stock repurchase plan	0	(28,065)	0
Capital contributions	0	0	0
Capital distributions	0	0	0
Change in due to affiliates	85,801	11,553	(98,071)
Other	(689)	(754)	6,921
Cash flows provided by (used in) financing activities	53,547	(108,393)	(106,150)
Net increase (decrease) in cash and cash equivalents	(3,243)	854	(96,847)
Cash and cash equivalents at the beginning of the year	4,016	3,162	100,009
Cash and cash equivalents at the end of the year	773	4,016	3,162
Reportable Legal Entities [Member] | Guarantor Subsidiaries [Member]
Condensed Cash Flow Statements, Captions [Line Items]
Cash flows provided by (used in) operating activities	203,585	(129,281)	1,005,449
Investing activities
Purchases and originations of reverse loans held for investment	(896,879)	(1,471,275)	(1,505,215)
Principal payments received on reverse loans held for investment	1,122,267	871,832	548,660
Principal payments received on mortgage loans held for investment	0	0	399
Payments received on charged-off loans held for investment	23,060	26,385	14,929
Payments received on receivables related to Non-Residual Trusts	0	0	0
Proceeds from sales of real estate owned, net	107,347	69,307	39,452
Purchases of premises and equipment	(32,271)	(27,586)	(21,573)
Decrease in restricted cash and cash equivalents	(114)	824	3,723
Payments for acquisitions of businesses, net of cash acquired	(3,066)	(5,095)	(197,061)
Acquisitions of servicing rights, net	(9,794)	(264,743)	(268,618)
Proceeds from sales of servicing rights, net	280,970
Acquisitions of charged-off loans held for investment			(64,548)
Proceeds from sale of residual interests in Residual Trusts		0
Proceeds from sale of investment		0
Capital contributions to subsidiaries and VIEs	(26,440)	(37,285)	(131)
Returns of capital from subsidiaries and VIEs	33,233	24,107	28,085
Change in due from affiliates	2,372	11,809	214,613
Other	(4,958)	855	13,231
Cash flows provided by (used in) investing activities	595,727	(800,865)	(1,194,054)
Financing activities
Payments on corporate debt	(480)	(1,651)	(2,220)
Extinguishments and settlement of debt	0	0
Proceeds from securitizations of reverse loans	960,157	1,622,481	1,617,399
Payments on HMBS related obligations	(1,371,375)	(1,025,458)	(637,272)
Issuances of servicing advance liabilities	228,059	283,953	874,729
Payments on servicing advance liabilities	(331,376)	(253,070)	(1,572,482)
Net change in warehouse borrowings related to mortgage loans	(151,172)	207,305	75,726
Net change in warehouse borrowings related to reverse loans	14,139	(43,873)	15,667
Proceeds from sales of excess servicing spreads and servicing rights	34,307	55,698	75,426
Payments on servicing rights related liabilities	(22,092)	(12,317)	(6,822)
Payments on mortgage-backed debt	0	0	0
Other debt issuance costs paid	(7,206)	(8,320)	(17,264)
Repurchase of shares under stock repurchase plan		0
Capital contributions	0	9,072	83,544
Capital distributions	(5,430)	(13,006)	(24,874)
Change in due to affiliates	(121,164)	5,893	(329,277)
Other	(666)	(11,559)	(40,509)
Cash flows provided by (used in) financing activities	(774,299)	815,148	111,771
Net increase (decrease) in cash and cash equivalents	25,013	(114,998)	(76,834)
Cash and cash equivalents at the beginning of the year	196,812	311,810	388,644
Cash and cash equivalents at the end of the year	221,825	196,812	311,810
Reportable Legal Entities [Member] | Non-Guarantor Subsidiaries and VIEs [Member]
Condensed Cash Flow Statements, Captions [Line Items]
Cash flows provided by (used in) operating activities	452,724	189,157	(1,137,754)
Investing activities
Purchases and originations of reverse loans held for investment	0	0	0
Principal payments received on reverse loans held for investment	0	0	0
Principal payments received on mortgage loans held for investment	91,679	114,078	161,323
Payments received on charged-off loans held for investment	0	0	0
Payments received on receivables related to Non-Residual Trusts	8,110	7,481	9,471
Proceeds from sales of real estate owned, net	3,714	7,278	15,627
Purchases of premises and equipment	0	0	0
Decrease in restricted cash and cash equivalents	49	8,401	3,364
Payments for acquisitions of businesses, net of cash acquired	0	0	0
Acquisitions of servicing rights, net	0	0	0
Proceeds from sales of servicing rights, net	0
Acquisitions of charged-off loans held for investment			0
Proceeds from sale of residual interests in Residual Trusts		0
Proceeds from sale of investment		0
Capital contributions to subsidiaries and VIEs	0	0	0
Returns of capital from subsidiaries and VIEs	0	0	0
Change in due from affiliates	(5,899)	1,388	250,108
Other	0	0	0
Cash flows provided by (used in) investing activities	97,653	138,626	439,893
Financing activities
Payments on corporate debt	0	0	0
Extinguishments and settlement of debt	0	0
Proceeds from securitizations of reverse loans	0	0	0
Payments on HMBS related obligations	0	0	0
Issuances of servicing advance liabilities	1,951,429	1,789,274	1,425,201
Payments on servicing advance liabilities	(2,294,100)	(1,953,895)	(332,849)
Net change in warehouse borrowings related to mortgage loans	0	0	0
Net change in warehouse borrowings related to reverse loans	0	0	0
Proceeds from sales of excess servicing spreads and servicing rights	0	0	0
Payments on servicing rights related liabilities	0	0	0
Payments on mortgage-backed debt	(107,598)	(136,493)	(181,155)
Other debt issuance costs paid	(3,305)	(5,629)	(17)
Repurchase of shares under stock repurchase plan		0
Capital contributions	26,440	37,285	131
Capital distributions	(38,794)	(38,410)	(79,425)
Change in due to affiliates	(87,993)	(22,312)	(123,246)
Other	3,544	(806)	(8,808)
Cash flows provided by (used in) financing activities	(550,377)	(330,986)	699,832
Net increase (decrease) in cash and cash equivalents	0	(3,203)	1,971
Cash and cash equivalents at the beginning of the year	2,000	5,203	3,232
Cash and cash equivalents at the end of the year	2,000	2,000	5,203
Eliminations and Reclassifications [Member]
Condensed Cash Flow Statements, Captions [Line Items]
Cash flows provided by (used in) operating activities	0	0	2,487
Investing activities
Purchases and originations of reverse loans held for investment	0	0	0
Principal payments received on reverse loans held for investment	0	0	0
Principal payments received on mortgage loans held for investment	0	0	0
Payments received on charged-off loans held for investment	0	0	0
Payments received on receivables related to Non-Residual Trusts	0	0	0
Proceeds from sales of real estate owned, net	0	0	0
Purchases of premises and equipment	0	0	0
Decrease in restricted cash and cash equivalents	0	0	0
Payments for acquisitions of businesses, net of cash acquired	0	0	0
Acquisitions of servicing rights, net	0	0	0
Proceeds from sales of servicing rights, net	0
Acquisitions of charged-off loans held for investment			0
Proceeds from sale of residual interests in Residual Trusts		0
Proceeds from sale of investment		0
Capital contributions to subsidiaries and VIEs	26,440	46,357	83,675
Returns of capital from subsidiaries and VIEs	(44,224)	(51,416)	(104,299)
Change in due from affiliates	(123,356)	(4,866)	(553,081)
Other	0	0	0
Cash flows provided by (used in) investing activities	(141,140)	(9,925)	(573,705)
Financing activities
Payments on corporate debt	0	0	0
Extinguishments and settlement of debt	0	0
Proceeds from securitizations of reverse loans	0	0	0
Payments on HMBS related obligations	0	0	0
Issuances of servicing advance liabilities	0	0	0
Payments on servicing advance liabilities	0	0	0
Net change in warehouse borrowings related to mortgage loans	0	0	0
Net change in warehouse borrowings related to reverse loans	0	0	0
Proceeds from sales of excess servicing spreads and servicing rights	0	0	0
Payments on servicing rights related liabilities	0	0	0
Payments on mortgage-backed debt	0	0	0
Other debt issuance costs paid	0	0	0
Repurchase of shares under stock repurchase plan		0
Capital contributions	(26,440)	(46,357)	(83,675)
Capital distributions	44,224	51,416	104,299
Change in due to affiliates	123,356	4,866	550,594
Other	0	0	0
Cash flows provided by (used in) financing activities	141,140	9,925	571,218
Net increase (decrease) in cash and cash equivalents	0	0	0
Cash and cash equivalents at the beginning of the year	0	0	0
Cash and cash equivalents at the end of the year	$ 0	$ 0	$ 0