Transactions with Affiliates - Mortgage Loan Servicing (Details)		3 Months Ended		9 Months Ended
	Sep. 12, 2016 USD ($) item	Sep. 30, 2018 USD ($)	Sep. 30, 2017 USD ($)	Sep. 30, 2018 USD ($)	Sep. 30, 2017 USD ($)
Mortgage Loan Servicing Agreement
Transactions with Affiliates
Base servicing fees per month for REO	$ 75
Rental fee per month per REO	30
Renewal fee, per lease renewal, on REO property	$ 100
Property management fees on REOs, as a percent of gross rental income	9.00%
Base servicing fees per month for fixed-rate non-distressed loans subserviced	$ 7.50
Base servicing fees per month for adjustable rate non-distressed loans subserviced	8.50
Supplemental fee per month for each distressed whole loan	25
Activity-based fee, percent, due to a streamline modification	750
Activity-based fee, percent, due to a liquidation	1,750
Activity-based fee due to a deed-in-lieu of foreclosure	$ 500
Maximum number of liquidation, reperformance, or modification fees that can be earned during earnable period | item	1
Liquidation, reperformance, or modification fees earnable period	18 months
Related party transaction, automatic renewal period	18 months
Minimum | Mortgage Loan Servicing Agreement
Transactions with Affiliates
Servicing fees amount per month for current loans	$ 30
Maximum | Mortgage Loan Servicing Agreement
Transactions with Affiliates
Servicing fees amount per month for severely delinquent loans	$ 85
PMT
Summary of mortgage loan servicing fees earned
Loan servicing fees		$ 10,071,000	$ 11,402,000	$ 30,521,000	$ 31,987,000
PMT | Mortgage loans acquired for sale at fair value
Summary of mortgage loan servicing fees earned
Base and supplemental		98,000	88,000	250,000	235,000
Activity-based		218,000	188,000	489,000	507,000
Loan servicing fees		316,000	276,000	739,000	742,000
PMT | Mortgage loans at fair value
Summary of mortgage loan servicing fees earned
Base and supplemental		614,000	1,571,000	2,328,000	5,284,000
Activity-based		657,000	2,702,000	3,200,000	6,859,000
Loan servicing fees		1,271,000	4,273,000	5,528,000	12,143,000
PMT | Mortgage servicing rights
Summary of mortgage loan servicing fees earned
Base and supplemental		8,326,000	6,702,000	23,875,000	18,727,000
Activity-based		158,000	151,000	379,000	375,000
Loan servicing fees		8,484,000	6,853,000	24,254,000	19,102,000
PMT | Other income
Summary of mortgage loan servicing fees earned
Property management fees		$ 122,000	$ 95,000	$ 333,000	$ 261,000