Related Party Transactions - Amounts due from Affiliate (Details) - PMT - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Amounts due from affiliate
Margin settlement relating to loan sales	$ 48,243
Allocated expenses	30,207	$ 3,724
Fulfillment fees	18,060	18,285
Correspondent production fees	8,509	10,579
Management fees	11,285	10,606
Servicing fees	6,121	4,659
Interest on assets purchased under agreements to resell	43	85
Conditional Reimbursement	10	221
Total due from affiliate	122,478	48,159
Payable to affiliate
Amounts advanced by PMT	58,264	70,520
MSR Recapture Payable to PMT	197	149
Other expenses	18,675	2,611
Payable to affiliates	$ 77,136	$ 73,280