Repurchase Agreements (Details) (USD $)	3 Months Ended	12 Months Ended
	Mar. 31, 2015	Dec. 31, 2014
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	$ 9,809,586,000	$ 8,267,388,000
Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Weighted average remaining term-to-interest rate reset of borrowings under repurchase agreements	29 days	25 days
Effective repricing period	17 months	21 months
Balance of repurchase agreements	9,809,586,000	8,267,388,000
Agency Mortgage Backed Securities | Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	4,982,381,000	5,177,835,000
Fair Value of securities pledged as collateral under repurchase agreements	5,242,368,000	5,462,566,000
Weighted average haircut (as a percent)	4.66%	4.79%
Legacy Non-Agency MBS | Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	2,228,517,000	2,233,236,000
Fair Value of securities pledged as collateral under repurchase agreements	3,242,911,000	3,491,312,000
Weighted average haircut (as a percent)	25.80%	28.88%
Non-Agency MBS acquired from consolidated VIEs	589,300,000	1,275,000,000
RPL/NPL MBS | Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	1,870,625,000	130,919,000
Fair Value of securities pledged as collateral under repurchase agreements	2,317,938,000	160,688,000
Weighted average haircut (as a percent)	20.99%	20.00%
U.S. Treasuries | Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	497,469,000	507,114,000
Fair Value of securities pledged as collateral under repurchase agreements	506,861,000	512,105,000
Weighted average haircut (as a percent)	1.64%	1.62%
CRT securities | Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	91,689,000	75,960,000
Fair Value of securities pledged as collateral under repurchase agreements	118,384,000	94,610,000
Weighted average haircut (as a percent)	25.18%	25.00%
Residential whole loans | Securities Sold under Agreements to Repurchase
Assets Sold under Agreements to Repurchase [Line Items]
Balance of repurchase agreements	138,905,000	142,324,000
Fair Value of securities pledged as collateral under repurchase agreements	$ 209,468,000	$ 212,986,000
Weighted average haircut (as a percent)	33.52%	33.43%