Repurchase Agreements (Details 2) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2015 counterparty	Dec. 31, 2014 counterparty
Information about MBS [Line Items]
Repurchase agreements	$ 9,809,586	$ 8,267,388
Gross amount of recognized liabilities for repurchase agreements	9,809,586	8,267,388
Amounts related to repurchase agreements not included in offsetting disclosure	0
Number of counterparties	26	25
Threshold of stockholders' equity at risk with single counterparty to repurchase agreements or linked transactions (percent)	5.00%
Repurchase Agreement Borrowings
Information about MBS [Line Items]
Repurchase agreements	9,809,586	8,267,388
Weighted Average Interest Rate (as a percent)	0.94%	0.79%
Repurchase Agreement Borrowings | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Weighted Average Interest Rate (as a percent)	0.00%
Repurchase Agreement Borrowings | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	6,945,604
Weighted Average Interest Rate (as a percent)	0.70%
Repurchase Agreement Borrowings | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	1,468,203
Weighted Average Interest Rate (as a percent)	1.15%
Repurchase Agreement Borrowings | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	1,395,779
Weighted Average Interest Rate (as a percent)	1.91%
Repurchase Agreement Borrowings | Agency Mortgage Backed Securities
Information about MBS [Line Items]
Repurchase agreements	4,982,381	5,177,835
Repurchase Agreement Borrowings | Agency Mortgage Backed Securities | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Agency Mortgage Backed Securities | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	4,982,381
Repurchase Agreement Borrowings | Agency Mortgage Backed Securities | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Agency Mortgage Backed Securities | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Legacy Non-Agency MBS
Information about MBS [Line Items]
Repurchase agreements	2,228,517	2,233,236
Repurchase Agreement Borrowings | Legacy Non-Agency MBS | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Legacy Non-Agency MBS | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	934,794
Repurchase Agreement Borrowings | Legacy Non-Agency MBS | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	256,595
Repurchase Agreement Borrowings | Legacy Non-Agency MBS | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	1,037,128
Repurchase Agreement Borrowings | RPL/NPL MBS
Information about MBS [Line Items]
Repurchase agreements	1,870,625	130,919
Repurchase Agreement Borrowings | RPL/NPL MBS | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | RPL/NPL MBS | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	956,381
Repurchase Agreement Borrowings | RPL/NPL MBS | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	694,498
Repurchase Agreement Borrowings | RPL/NPL MBS | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	219,746
Repurchase Agreement Borrowings | U.S. Treasuries
Information about MBS [Line Items]
Repurchase agreements	497,469	507,114
Repurchase Agreement Borrowings | U.S. Treasuries | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | U.S. Treasuries | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | U.S. Treasuries | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	497,469
Repurchase Agreement Borrowings | U.S. Treasuries | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | CRT securities
Information about MBS [Line Items]
Repurchase agreements	91,689	75,960
Repurchase Agreement Borrowings | CRT securities | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | CRT securities | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	72,048
Repurchase Agreement Borrowings | CRT securities | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	19,641
Repurchase Agreement Borrowings | CRT securities | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Residential whole loans
Information about MBS [Line Items]
Repurchase agreements	138,905	142,324
Repurchase Agreement Borrowings | Residential whole loans | Overnight
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Residential whole loans | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Residential whole loans | Over 30 days to 90 days
Information about MBS [Line Items]
Repurchase agreements	0
Repurchase Agreement Borrowings | Residential whole loans | Over 90 days to 12 months
Information about MBS [Line Items]
Repurchase agreements	138,905
Repurchase Agreement Borrowings | Within 30 days
Information about MBS [Line Items]
Repurchase agreements	7,949,936	7,144,737
Weighted Average Interest Rate (as a percent)	0.85%	0.72%
Repurchase Agreement Borrowings | Over 30 days to 3 months
Information about MBS [Line Items]
Repurchase agreements	1,502,606	1,000,313
Weighted Average Interest Rate (as a percent)	1.16%	1.12%
Repurchase Agreement Borrowings | Over 3 months to 12 months
Information about MBS [Line Items]
Repurchase agreements	357,044	122,338
Weighted Average Interest Rate (as a percent)	1.95%	1.98%
Linked Transactions | Legacy Non-Agency MBS
Information about MBS [Line Items]
Linked repurchase agreements		$ 1,520,000