Transactions with Affiliates - Amounts due from Affiliate (Details) - PMT - USD ($) $ in Thousands	Dec. 31, 2019	Dec. 31, 2018
Amounts due from affiliate
Fulfillment fees	$ 18,285	$ 10,006
Management fees	10,579	6,559
Correspondent production fees	10,606	2,071
Servicing fees	4,659	4,841
Allocated expenses	3,724	9,066
Conditional Reimbursement	221	801
Interest on assets purchased under agreements to resell	85	120
Total due from affiliate	48,159	33,464
Payable to affiliate
Amounts advanced by PMT	70,520	100,554
MSR Recapture Payable to PMT	149	179
Other expenses	2,611	3,898
Payable to affiliates	$ 73,280	$ 104,631