Carrying Values and Fair Values of All Financial Instruments (Details) (USD $) In Thousands, unless otherwise specified	Sep. 30, 2014		Dec. 31, 2013		Sep. 30, 2013	Dec. 31, 2012
Fair Value Balance Sheet Grouping Financial Statement Captions [Line Items]
Mortgage-backed securities	$ 16,890,424		$ 17,642,532
Cash and cash equivalents	215,073		763,326		547,536	168,424
Restricted cash	226,764		225,379
Unsettled purchased mortgage-backed securities, at fair value	45,691
Receivable for securities sold			231,214
Accrued interest receivable	52,757		55,156
Principal payments receivable	128,041		95,021
Debt security, held-to-maturity	15,000		15,000
Short term investments	20,163		19,910
Repurchase agreements	14,920,959		16,129,683
Dollar roll liability			351,826
Payable for unsettled securities	45,571
Accrued interest payable	2,841		8,279
Carrying (Reported) Amount, Fair Value Disclosure
Fair Value Balance Sheet Grouping Financial Statement Captions [Line Items]
Mortgage-backed securities	16,890,424		17,642,532
Cash and cash equivalents	215,073		763,326
Restricted cash	226,764		225,379
Unsettled purchased mortgage-backed securities, at fair value	45,691
Receivable for securities sold			231,214
Accrued interest receivable	52,757		55,156
Principal payments receivable	128,041		95,021
Debt security, held-to-maturity	15,000	[1]	15,000	[1]
Interest rate swaps and swaptions	27,815	[2]	15,841	[2]
Futures Contracts asset	1,299	[2]	11,148	[2]
Short term investments	20,163	[1]	19,910	[1]
Forward purchase commitments	1,888	[2]
Repurchase agreements	14,920,959		16,129,683
Dollar roll liability			351,826
Payable for unsettled securities	45,571
Accrued interest payable	2,841		8,279
Interest rate swap liability	58,484	[3]	125,133	[3]
Futures Contracts liability	96,867	[3]	36,733	[3]
Forward purchase commitments	19,971	[3]	5,741	[3]
Estimate of Fair Value, Fair Value Disclosure
Fair Value Balance Sheet Grouping Financial Statement Captions [Line Items]
Mortgage-backed securities	16,890,424		17,642,532
Cash and cash equivalents	215,073		763,326
Restricted cash	226,764		225,379
Unsettled purchased mortgage-backed securities, at fair value	45,691
Receivable for securities sold			231,214
Accrued interest receivable	52,757		55,156
Principal payments receivable	128,041		95,021
Debt security, held-to-maturity	14,905	[1]	14,307	[1]
Interest rate swaps and swaptions	27,815	[2]	15,841	[2]
Futures Contracts asset	1,299	[2]	11,148	[2]
Short term investments	20,163	[1]	19,910	[1]
Forward purchase commitments	1,888	[2]
Repurchase agreements	14,920,959		16,129,683
Dollar roll liability			351,826
Payable for unsettled securities	45,571
Accrued interest payable	2,841		8,279
Interest rate swap liability	58,484	[3]	125,133	[3]
Futures Contracts liability	96,867	[3]	36,733	[3]
Forward purchase commitments	$ 19,971	[3]	$ 5,741	[3]

[1] Included in other investments on the consolidated balance sheets.

[2] Included in derivative assets on the consolidated balance sheets.

[3] Included in derivative liabilities on the consolidated balance sheets.