Fair Value of Financial Instruments - Fair Values Measured on Recurring Basis (Detail) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed and credit risk transfer securities	$ 21,771,786	$ 17,396,642
Derivative assets, at fair value		15,089
Derivative liabilities, at fair value	$ 352	$ 10,239
Weighted average remaining term of investments in unconsolidated ventures	2 years 2 months 12 days	2 years 7 months 6 days
Loan participation interest	$ 44,654	$ 54,981
Level 3
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Fair value amount	10,281	22,771
Recurring
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed and credit risk transfer securities	21,771,786	17,396,642
Derivative assets, at fair value	18,533	15,089
Other assets, at fair value	66,652	78,993
Total assets	21,856,971	17,490,724
Derivative liabilities, at fair value	352	23,390
Total liabilities	352	23,390
Recurring | Level 1
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed and credit risk transfer securities	0	0
Derivative assets, at fair value	0	0
Other assets, at fair value	0	0
Total assets	0	0
Derivative liabilities, at fair value	0	7,836
Total liabilities	0	7,836
Recurring | Level 2
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed and credit risk transfer securities	21,761,505	17,373,871
Derivative assets, at fair value	18,533	15,089
Other assets, at fair value	0	0
Total assets	21,780,038	17,388,960
Derivative liabilities, at fair value	352	15,554
Total liabilities	352	15,554
Recurring | Level 3
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed and credit risk transfer securities	10,281	22,771
Derivative assets, at fair value	0	0
Other assets, at fair value	44,654	54,981
Total assets	54,935	77,752
Derivative liabilities, at fair value	0	0
Total liabilities	0	0
Recurring | NAV as a practical expedient
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
NAV as a practical expedient	21,998	24,012
Mortgage-Backed and Credit Risk Transfer Securities, at Fair Value | Recurring | Level 3
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Embedded derivatives at fair value	10,300	22,800
Fair value amount	19,500	28,800
Embedded derivative liability	$ 9,200	$ 6,000