Loans Held for Sale - Summary of Activity in Balance of Loans Held for Sale, at Fair Value (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Movement In Loans Held For Sale At Fair Value [Roll Forward]
Beginning balance	$ 176,525	$ 214,262	$ 284,632
Originations and purchases	1,168,885	944,627	2,678,372
Proceeds from sales	(1,124,247)	(1,019,211)	(2,785,422)
Principal collections	(23,116)	(20,774)	(4,867)
Acquired in connection with the acquisition of PHH	0	42,324	0
Loans held for investment, at fair value	1,892	1,038	3,803
Receivables	(2,480)	(1,132)	0
Real estate owned (Other assets)	(2,520)	(1,886)	0
Gain on sale of loans	25,253	34,724	35,429
(Decrease) increase in fair value of loans	(589)	(13,435)	151
Other	(10,851)	(4,012)	2,164
Ending balance	$ 208,752	$ 176,525	$ 214,262