Business Segment Reporting - Schedule of Segment Reporting Information (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018	Sep. 30, 2018	Jun. 30, 2018	Mar. 31, 2018	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Segment Reporting Information [Line Items]
Loss on write-off of fixed assets, net									$ 0	$ 0	$ 8,502
Results of Operations
Revenue	$ 261,171	$ 283,660	$ 274,493	$ 304,051	$ 310,929	$ 238,278	$ 253,581	$ 260,257	1,123,375	1,063,045	1,194,576
MSR valuation adjustments, net	829	134,561	(147,268)	(108,998)	(61,762)	(41,448)	(33,118)	(17,129)	(120,876)	(153,457)	(52,962)
Operating expenses	138,858	179,430	184,381	171,270	241,057	176,078	172,532	189,372	673,939	779,039	945,683
Other income (expense):
Interest income									17,104	14,026	15,965
Interest expense									(114,129)	(103,371)	(126,927)
Pledged MSR liability expense									(372,089)	(171,670)	(236,311)
Gain on repurchase of senior secured notes									5,099	0	0
Bargain purchase gain									(381)	64,036	0
Gain on sale of MSRs, net									453	1,325	10,537
Other, net									8,892	(6,371)	(3,168)
Total other expense, net	(85,899)	(277,108)	(27,177)	(64,867)	(15,873)	(61,025)	(76,336)	(48,791)	(455,051)	(202,025)	(339,904)
Loss from continuing operations before income taxes	37,243	$ (38,317)	$ (84,333)	(41,084)	(7,763)	$ (40,273)	$ (28,405)	$ 4,965	(126,491)	(71,476)	(143,973)
Total Assets
Balance	10,406,199				9,394,216				10,406,199	9,394,216	8,403,164
Forward Wholesale Lending Segment
Segment Reporting Information [Line Items]
Loss on write-off of fixed assets, net									6,800
Corporate Items and Other
Other income (expense):
Severance costs									20,300	11,900
Recovery of direct costs				$ 30,700
Operating Segments | Servicing
Results of Operations
Revenue									985,102	951,224	1,041,290
MSR valuation adjustments, net									(120,646)	(152,983)	(52,689)
Operating expenses									536,153	619,484	663,695
Other income (expense):
Interest income									8,051	5,383	783
Interest expense									(47,347)	(41,830)	(57,284)
Pledged MSR liability expense									(372,172)	(172,342)	(236,311)
Gain on repurchase of senior secured notes									0
Bargain purchase gain									0	0
Gain on sale of MSRs, net									453	1,325	10,537
Other, net									11,942	(3,241)	4,049
Total other expense, net									(399,073)	(210,705)	(278,226)
Loss from continuing operations before income taxes									(70,770)	(31,948)	46,680
Total Assets
Balance	3,378,515				3,306,208				3,378,515	3,306,208	3,033,243
Operating Segments | Lending
Results of Operations
Revenue									125,086	93,672	127,475
MSR valuation adjustments, net									(230)	(474)	(273)
Operating expenses									84,280	82,432	127,785
Other income (expense):
Interest income									7,277	6,061	10,914
Interest expense									(7,911)	(7,311)	(13,893)
Pledged MSR liability expense									0	672	0
Gain on repurchase of senior secured notes									0
Bargain purchase gain									0	0
Gain on sale of MSRs, net									0	0	0
Other, net									791	966	(869)
Total other expense, net									157	388	(3,848)
Loss from continuing operations before income taxes									40,733	11,154	(4,431)
Total Assets
Balance	6,459,367				5,603,481				6,459,367	5,603,481	4,945,456
Operating Segments | Corporate Items and Other
Results of Operations
Revenue									13,187	18,149	25,811
MSR valuation adjustments, net									0	0	0
Operating expenses									53,506	77,123	154,203
Other income (expense):
Interest income									1,776	2,582	4,268
Interest expense									(58,871)	(54,230)	(55,750)
Pledged MSR liability expense									83	0	0
Gain on repurchase of senior secured notes									5,099
Bargain purchase gain									(381)	64,036
Gain on sale of MSRs, net									0	0	0
Other, net									(3,841)	(4,096)	(6,348)
Total other expense, net									(56,135)	8,292	(57,830)
Loss from continuing operations before income taxes									(96,454)	(50,682)	(186,222)
Total Assets
Balance	568,317				484,527				568,317	484,527	424,465
Intersegment Eliminations
Results of Operations
Revenue									0	0	0
MSR valuation adjustments, net									0	0	0
Operating expenses									0	0	0
Other income (expense):
Interest income									0	0	0
Interest expense									0	0	0
Pledged MSR liability expense									0	0	0
Gain on repurchase of senior secured notes									0
Bargain purchase gain										0
Gain on sale of MSRs, net									0	0	0
Other, net									0	0	0
Total other expense, net									0	0	0
Loss from continuing operations before income taxes									0	0	0
Total Assets
Balance	$ 0				$ 0				$ 0	$ 0	$ 0