Transactions with Affiliates - Mortgage Loan Servicing (Details)		3 Months Ended
	Sep. 12, 2016 USD ($) item	Mar. 31, 2020 USD ($)	Mar. 31, 2019 USD ($)
Summary of mortgage loan servicing fees earned
Loan servicing fees		$ 198,653,000	$ 166,790,000
Loan Servicing Agreement
Transactions with Affiliates
Base servicing fees per month for REO	$ 75
Rental fee per month per REO	30
Renewal fee, per lease renewal, on REO property	$ 100
Property management fees on REOs, as a percent of gross rental income	9.00%
Base servicing fees per month for fixed-rate non-distressed loans subserviced	$ 7.50
Base servicing fees per month for adjustable rate non-distressed loans subserviced	8.50
Supplemental fee per month for each distressed whole loan	25
Activity-based fee, percent, due to a streamline modification	750
Activity-based fee, percent, due to a liquidation	1,750
Activity-based fee due to a deed-in-lieu of foreclosure	$ 500
Maximum number of liquidation, reperformance, or modification fees that can be earned during earnable period | item	1
Liquidation, reperformance, or modification fees earnable period	18 months
Related party transaction, automatic renewal period	18 months
Minimum | Loan Servicing Agreement
Transactions with Affiliates
Servicing fees amount per month for current loans	$ 30
Additional servicing fee amount per month for delinquent loans	10
Maximum | Loan Servicing Agreement
Transactions with Affiliates
Servicing fees amount per month for severely delinquent loans	85
Additional servicing fee amount per month for delinquent loans	$ 55
PMT
Summary of mortgage loan servicing fees earned
Loan servicing fees		14,521,000	10,570,000
PMT | Loans acquired for sale at fair value
Summary of mortgage loan servicing fees earned
Loan servicing fees		536,000	239,000
PMT | Loans at fair value
Summary of mortgage loan servicing fees earned
Loan servicing fees		300,000	463,000
PMT | Mortgage servicing rights
Summary of mortgage loan servicing fees earned
Loan servicing fees		$ 13,685,000	9,868,000
PMT | Other income.
Summary of mortgage loan servicing fees earned
Property management fees			$ 123,000