Rights to MSRs - Financing Liability Rollforward (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018
Servicing Liability at Fair Value, Amount [Roll Forward]
Beginning balance	$ 1,032,856	$ 508,291
Additions	1,276	667
Sales	44
Receipt of lump-sum cash payments		279,586
Ending balance	950,593	1,032,856
Original Rights to MSRs Agreements
Servicing Liability at Fair Value, Amount [Roll Forward]
Beginning balance	436,511	499,042
Changes in fair value:	229,198	(171)
Runoff and settlement:	(48,730)	(58,837)
Calls	(13,933)	(3,523)
Ending balance	603,046	436,511
2017 Agreements and New RMSR Agreements
Servicing Liability at Fair Value, Amount [Roll Forward]
Beginning balance	138,854	9,249
Receipt of lump-sum cash payments		279,586
Changes in fair value:	5,866	(14,369)
Runoff and settlement:	(101,003)	(134,509)
Calls	(8,272)	(1,103)
Ending balance	35,445	138,854
PMC MSR Agreements
Servicing Liability at Fair Value, Amount [Roll Forward]
Beginning balance	457,491	0
Additions	1,276	667
Sales	44
Changes in fair value:	(82,078)	(4,729)
Runoff and settlement:	(64,631)	(18,420)
Calls	0	(1,047)
Ending balance	$ 312,102	457,491
PHH Corporation
Servicing Liability at Fair Value, Amount [Roll Forward]
Additions		481,020
PHH Corporation | PMC MSR Agreements
Servicing Liability at Fair Value, Amount [Roll Forward]
Additions		$ 481,020