Summary of the Significant Accounting Policies (Details) - USD ($)	3 Months Ended								12 Months Ended
	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018	Sep. 30, 2018	Jun. 30, 2018	Mar. 31, 2018	Dec. 31, 2019		Dec. 31, 2018		Dec. 31, 2017	[1]
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Non-Agency RMBS, at fair value	$ 2,614,408,000				$ 2,486,130,000				$ 2,614,408,000		$ 2,486,130,000
Agency RMBS, at fair value	6,490,293,000				9,240,057,000				6,490,293,000		9,240,057,000
Agency CMBS, at fair value	2,850,717,000				2,948,893,000				2,850,717,000		2,948,893,000
Interest income	340,662,000	$ 330,144,000	$ 339,914,000	$ 350,389,000	348,033,000	$ 321,715,000	$ 306,436,000	$ 297,132,000	1,361,110,000	[1]	1,273,316,000	[1]	$ 1,138,758,000
Accrual for penalties and interest	0				0				0		0
Seasoned Subprime Residential Mortgage Loans | Loans Held for Investment at Fair Value
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Real estate acquired through foreclosure	25,000,000				17,000,000				25,000,000		17,000,000
Non Agency Residential Mortgage Backed Securities Fair Value Option
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Non-Agency RMBS, at fair value	423,000,000				12,000,000				423,000,000		12,000,000
Residential Mortgage-Backed Securities | Interest-Only-Strip
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Interest income									27,000,000		25,000,000
Residential Mortgage-Backed Securities | Non Agency Senior And Subordinated Mortgage Backed Securities Interest Only Available For Sale | Interest-Only-Strip
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Non-Agency RMBS, at fair value	289,000,000				267,000,000				289,000,000		267,000,000
Residential Mortgage-Backed Securities | Agency Senior And Subordinated Mortgage Backed Securities Interest Only Available For Sale | Interest-Only-Strip
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Agency RMBS, at fair value	128,000,000				66,000,000				128,000,000		66,000,000
Commercial Mortgage Backed Securities | Agency CMBS | Interest-Only-Strip
New Accounting Pronouncements or Change in Accounting Principle [Line Items]
Agency CMBS, at fair value	$ 49,000,000				$ 68,000,000				$ 49,000,000		$ 68,000,000

[1] Includes interest income of consolidated VIEs of $780,746, $904,830 and $914,022 for the years ended December 31, 2019, 2018 and 2017, respectively. See Note 8 to consolidated financial statements for further discussion.