Income Taxes - Reconciliation of GAAP Net Income to Estimated REIT Taxable Income (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Dec. 31, 2019	Sep. 30, 2019	Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018	Sep. 30, 2018	Jun. 30, 2018	Mar. 31, 2018	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Income Tax Disclosure [Abstract]
GAAP net income (loss)	$ 108,681	$ (60,955)	$ (183,250)	$ (114,381)	$ (211,979)	$ 47,704	$ 13,562	$ 44,747	$ (249,905)	$ (105,966)	$ 181,154
Book to tax differences:
TRS income									(147)	(265)	0
Premium amortization expense									0	(1,132)	(468)
Credit Risk and Non-Agency Securities									24,459	26,509	(68,505)
Interest-Only Securities									85	318	1,216
U.S. Treasury Securities									(2,024)	6,365	0
Changes in interest rate contracts									375,493	(1,346)	(49,930)
(Gain) Loss on sales of Agency Securities									(9,611)	152,950	8,486
Other than temporary loss on Agency Securities									0	12,090	13,707
Amortization of deferred hedging costs									(69,302)	(56,378)	(59,930)
Series A Cumulative Preferred Stock dividend- Called for redemption									375	0	0
Other									18	16	15
Estimated taxable income									$ 69,441	$ 33,161	$ 25,745