Concentration of Credit and Other Risks - Seller Concentration (Details)	9 Months Ended
	Sep. 30, 2020	Sep. 30, 2019
Single-family loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	44.00%	56.00%
JPMorgan Chase Bank, National Association | Single-family loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	5.00%	16.00%
Other top 10 sellers | Single-family loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	39.00%	40.00%
Top ten multifamily sellers | Multifamily loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	77.00%	79.00%
CBRE Capital Markets, Inc. | Multifamily loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	15.00%	16.00%
Berkadia Commercial Mortgage LLC | Multifamily loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	14.00%	15.00%
Other top 10 sellers | Multifamily loan purchase volume
Concentration Risk [Line Items]
Concentration risk percentage	48.00%	48.00%