Carrying Values and Fair Values of All Financial Instruments (Details) (USD $) In Thousands, unless otherwise specified	Dec. 31, 2013		Dec. 31, 2012		Dec. 31, 2011	Dec. 31, 2010
Fair Value Of Financial Instruments [Line Items]
Mortgage-backed securities	$ 17,642,532		$ 23,919,251
Unsettled purchased mortgage backed securities			138,338
Cash and cash equivalents	763,326		168,424		347,045	112,626
Restricted cash	225,379		281,021
Accrued interest receivable	55,156		77,113
Receivable for securities sold	231,214		1,587,535
Principal payments receivable	95,021		190,832
Debt security, held to maturity	15,000		15,000
Interest rate swap asset	15,841
Futures Contracts	11,148
Forward purchase commitments			5,452
Repurchase agreements	16,129,683		22,866,429
Dollar roll liabilities	351,826
Payable for unsettled securities			137,121
Accrued interest payable	8,279		7,592
Interest rate swap liability	125,133		243,945
Futures contract liability	36,733
Forward purchase commitments	5,741
Carrying (Reported) Amount, Fair Value Disclosure
Fair Value Of Financial Instruments [Line Items]
Mortgage-backed securities	17,642,532		23,919,251
Unsettled purchased mortgage backed securities			138,338
Cash and cash equivalents	763,326		168,424
Restricted cash	225,379		281,021
Accrued interest receivable	55,156		77,113
Receivable for securities sold	231,214		1,587,535
Principal payments receivable	95,021		190,832
Debt security, held to maturity	15,000		15,000
Interest rate swap asset	15,841
Futures Contracts	11,148	[1]
Short term investment	19,910	[1]	20,187	[1]
Forward purchase commitments			5,452	[1]
Repurchase agreements	16,129,683		22,866,429
Dollar roll liabilities	351,826
Payable for unsettled securities			137,121
Accrued interest payable	8,279		7,592
Interest rate swap liability	125,133		243,945
Futures contract liability	36,733
Forward purchase commitments	5,741	[2]
Estimate of Fair Value, Fair Value Disclosure
Fair Value Of Financial Instruments [Line Items]
Mortgage-backed securities	17,642,532		23,919,251
Unsettled purchased mortgage backed securities			138,338
Cash and cash equivalents	763,326		168,424
Restricted cash	225,379		281,021
Accrued interest receivable	55,156		77,113
Receivable for securities sold	231,214		1,587,535
Principal payments receivable	95,021		190,832
Debt security, held to maturity	14,307		13,981
Interest rate swap asset	15,841
Futures Contracts	11,148	[1]
Short term investment	19,910	[1]	20,187	[1]
Forward purchase commitments			5,452	[1]
Repurchase agreements	16,129,683		22,866,429
Dollar roll liabilities	351,826
Payable for unsettled securities			137,121
Accrued interest payable	8,279		7,592
Interest rate swap liability	125,133		243,945
Futures contract liability	36,733
Forward purchase commitments	$ 5,741	[2]

[1] These lines are included in other assets on the consolidated balance sheets

[2] This line included in accounts payable and other liabilities on the consolidated balance sheets