Carrying Values and Fair Values of All Financial Instruments (Details) (USD $) In Thousands, unless otherwise specified	Jun. 30, 2014		Dec. 31, 2013		Jun. 30, 2013	Dec. 31, 2012
Fair Value Balance Sheet Grouping Financial Statement Captions [Line Items]
Mortgage-backed securities	$ 16,651,220		$ 17,642,532
Cash and cash equivalents	377,350		763,326		88,987	168,424
Restricted cash	305,082		225,379
Unsettled purchased mortgage-backed securities, at fair value	60,393
Receivable for securities sold	152,177		231,214
Accrued interest receivable	52,578		55,156
Principal payments receivable	120,026		95,021
Debt security, held to maturity	15,000		15,000
Short term investment	20,203		19,910
Repurchase agreements	15,019,880		16,129,683
Dollar roll liabilities			351,826
Payable for unsettled securities	59,811
Accrued interest payable	3,148		8,279
Carrying (Reported) Amount, Fair Value Disclosure
Fair Value Balance Sheet Grouping Financial Statement Captions [Line Items]
Mortgage-backed securities	16,651,220		17,642,532
Cash and cash equivalents	377,350		763,326
Restricted cash	305,082		225,379
Unsettled purchased mortgage-backed securities, at fair value	60,393
Receivable for securities sold	152,177		231,214
Accrued interest receivable	52,578		55,156
Principal payments receivable	120,026		95,021
Debt security, held to maturity	15,000	[1]	15,000	[1]
Interest rate swap and swaption assets	11,642	[2]	15,841	[2]
Futures Contracts			11,148	[2]
Short term investment	20,203	[3]	19,910	[3]
Forward purchase commitments	16,939	[2]
Repurchase agreements	15,019,880		16,129,683
Dollar roll liabilities			351,826
Payable for unsettled securities	59,811
Accrued interest payable	3,148		8,279
Interest rate swap liability	89,225	[3]	125,133	[3]
Futures Contracts	132,972	[3]	36,733	[3]
Forward purchase commitments			5,741	[3]
Estimate of Fair Value, Fair Value Disclosure
Fair Value Balance Sheet Grouping Financial Statement Captions [Line Items]
Mortgage-backed securities	16,651,220		17,642,532
Cash and cash equivalents	377,350		763,326
Restricted cash	305,082		225,379
Unsettled purchased mortgage-backed securities, at fair value	60,393
Receivable for securities sold	152,177		231,214
Accrued interest receivable	52,578		55,156
Principal payments receivable	120,026		95,021
Debt security, held to maturity	15,007	[1]	14,307	[1]
Interest rate swap and swaption assets	11,642	[2]	15,841	[2]
Futures Contracts			11,148	[2]
Short term investment	20,203	[3]	19,910	[3]
Forward purchase commitments	16,939	[2]
Repurchase agreements	15,019,880		16,129,683
Dollar roll liabilities			351,826
Payable for unsettled securities	59,811
Accrued interest payable	3,148		8,279
Interest rate swap liability	89,225	[3]	125,133	[3]
Futures Contracts	132,972	[3]	36,733	[3]
Forward purchase commitments			$ 5,741	[3]

[1] These lines are included in other investments on the consolidated balance sheets.

[2] These lines are included in derivative assets on the consolidated balance sheets.

[3] These lines are included in derivative liabilities on the consolidated balance sheets.