Derivative Instruments - Gains and Losses (Details) - USD ($) $ in Thousands	3 Months Ended								12 Months Ended
	Dec. 31, 2017	Sep. 30, 2017	Jun. 30, 2017	Mar. 31, 2017	Dec. 31, 2016	Sep. 30, 2016	Jun. 30, 2016	Mar. 31, 2016	Dec. 31, 2017	Dec. 31, 2016	Dec. 31, 2015
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations											$ 18,418
Variation Margin Settlement											0
Return (Recovery) of Basis											(16,015)
Mark-to-market adjustments									$ 148,308	$ (5,135)	(73,599)
Contractual interest income (expense), net(1)											2,301
Total	$ 19,327	$ 7,217	$ (18,555)	$ (4,697)	$ 32,479	$ 6,121	$ (14,165)	$ (45,170)	3,292	(20,735)	(68,895)
Collateral already posted, aggregate fair value	63,300				206,600				63,300	206,600
Derivative collateral offset against derivatives									(157,913)	0	0
Cash pledged as collateral	$ 0				$ 470				0	470
Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(157,261)	7,517
Variation Margin Settlement									20,258	0
Return (Recovery) of Basis									(5,471)	(10,766)
Mark-to-market adjustments									152,465	(5,135)
Contractual interest income (expense), net(1)									(6,699)	(12,351)
Total									3,292	(20,735)
Interest rate swaps | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(150,607)	(33,999)	23,680
Variation Margin Settlement									20,258	0	0
Return (Recovery) of Basis									524	672	1,250
Mark-to-market adjustments									148,947	11,013	(68,843)
Contractual interest income (expense), net(1)									(14,606)	(27,903)	(20,366)
Total									4,516	(50,217)	(64,279)
Interest rate swaption | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(115)	(1,035)	(5,242)
Variation Margin Settlement									0	0	0
Return (Recovery) of Basis									0	0	0
Mark-to-market adjustments									0	1,631	(1,486)
Contractual interest income (expense), net(1)									0	0	0
Total									(115)	596	(6,728)
Agency RMBS Interest-Only Strips accounted for as derivatives, included in MBS | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									526	1,317	595
Variation Margin Settlement									0	0	0
Return (Recovery) of Basis									(5,995)	(11,438)	(17,265)
Mark-to-market adjustments									(902)	(4,726)	(4,283)
Contractual interest income (expense), net(1)									7,438	14,148	21,872
Total									1,067	(699)	919
Option | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(1,453)	4,756	(711)
Variation Margin Settlement									0	0	0
Return (Recovery) of Basis									0	0	0
Mark-to-market adjustments									(300)	0	0
Contractual interest income (expense), net(1)									0	0	0
Total									(1,753)	4,756	(711)
Futures contracts | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(9,130)	23,609	(527)
Variation Margin Settlement									0	0	0
Return (Recovery) of Basis									0	0	0
Mark-to-market adjustments									3,044	(1,781)	105
Contractual interest income (expense), net(1)									0	0	0
Total									(6,086)	21,828	(422)
Foreign currency forwards | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									32	(66)	(901)
Variation Margin Settlement									0	0	0
Return (Recovery) of Basis									0	0	0
Mark-to-market adjustments									35	(56)	323
Contractual interest income (expense), net(1)									0	0	0
Total									67	(122)	(578)
Foreign currency swaps | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations										6,771	0
Variation Margin Settlement										0	0
Return (Recovery) of Basis										0	0
Mark-to-market adjustments										(7,168)	3,311
Contractual interest income (expense), net(1)										283	795
Total										(114)	4,106
Total Return Swap | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(552)	38
Variation Margin Settlement									0	0
Return (Recovery) of Basis									0	0
Mark-to-market adjustments									1,673	(1,673)
Contractual interest income (expense), net(1)									469	1,121
Total									1,590	(514)
Credit Default Swap | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									(11)
Variation Margin Settlement									0
Return (Recovery) of Basis									0
Mark-to-market adjustments									(22)
Contractual interest income (expense), net(1)									0
Total									(33)
TBAs | Derivative instruments not accounted as hedges under GAAP
Amounts recognized on the statements of operations related to the Company's derivatives
Other Settlements / Expirations									4,049	6,126	1,524
Variation Margin Settlement									0	0	0
Return (Recovery) of Basis									0	0	0
Mark-to-market adjustments									(10)	(2,375)	(2,726)
Contractual interest income (expense), net(1)									0	0	0
Total									$ 4,039	$ 3,751	$ (1,202)