Mortgage Backed Securities Mortgage backed securities designated as AFS (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended	12 Months Ended	3 Months Ended			3 Months Ended
	Mar. 31, 2014	Dec. 31, 2013	Mar. 31, 2014 Agency MBS [Member]	Dec. 31, 2013 Agency MBS [Member]	Mar. 31, 2014 Agency MBS [Member] Residential Mortgage Backed Securities [Member]	Dec. 31, 2013 Agency MBS [Member] Residential Mortgage Backed Securities [Member]	Mar. 31, 2014 Agency MBS [Member] Commercial Mortgage Backed Securities [Member]	Dec. 31, 2013 Agency MBS [Member] Commercial Mortgage Backed Securities [Member]	Mar. 31, 2014 Agency MBS [Member] CMBS IO [Member]	Dec. 31, 2013 Agency MBS [Member] CMBS IO [Member]	Mar. 31, 2014 Non-Agency MBS [Member]	Dec. 31, 2013 Non-Agency MBS [Member]	Mar. 31, 2014 Non-Agency MBS [Member] Residential Mortgage Backed Securities [Member]	Dec. 31, 2013 Non-Agency MBS [Member] Residential Mortgage Backed Securities [Member]	Mar. 31, 2014 Non-Agency MBS [Member] Commercial Mortgage Backed Securities [Member]	Dec. 31, 2013 Non-Agency MBS [Member] Commercial Mortgage Backed Securities [Member]	Mar. 31, 2014 Non-Agency MBS [Member] CMBS IO [Member]	Dec. 31, 2013 Non-Agency MBS [Member] CMBS IO [Member]	Mar. 31, 2014 Agency MBS [Member]	Mar. 31, 2013 Agency MBS [Member]	Dec. 31, 2013 Agency MBS [Member]	Mar. 31, 2014 Non-Agency MBS [Member]	Mar. 31, 2013 Non-Agency MBS [Member]	Dec. 31, 2013 Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	$ 3,115,727	$ 3,254,946	$ 2,725,064	$ 2,865,398	$ 2,438,133	$ 2,591,568	$ 286,931	$ 273,830	$ 0	$ 0	$ 390,663	$ 389,548	$ 15,454	$ 13,845	$ 375,209	$ 375,703	$ 0	$ 0
Investments, Unamortized Premium (Discount)	812,735	758,950	621,260	627,047	139,899	154,220	19,838	19,061	461,523	453,766	191,475	131,903	(7)	(338)	(17,819)	(18,277)	209,301	150,518
Notional balance for interest only securities									10,582,484	10,160,502							5,053,884	4,274,957
Available-for-sale MBS, Amortized Cost	3,928,462	4,013,896	3,346,324	3,492,445	2,578,032	2,745,788	306,769	292,891	461,523	453,766	582,138	521,451	15,447	13,507	357,390	357,426	209,301	150,518
Available-for-sale MBS, Gross Unrealized Gain	52,704	45,114	29,673	26,792	6,708	6,104	11,384	10,793	11,581	9,895	23,031	18,322	209	338	19,448	15,366	3,374	2,618
Available-for-sale MBS, Gross Unrealized Loss	49,882	69,566	47,806	63,976	45,438	59,742	509	900	1,859	3,334	2,076	5,590	(86)	80	842	3,511	(1,148)	1,999
Available-for-sale MBS, Fair Value	3,931,284	3,989,444	3,328,191	3,455,261	2,539,302	2,692,150	317,644	302,784	471,245	460,327	603,093	534,183	15,570	13,765	375,996	369,281	211,527	151,137
Available-for-sale MBS, Weighted Average Coupon					3.20%	3.22%	5.23%	5.07%	0.94%	0.83%			4.42%	4.61%	5.05%	5.10%	0.80%	0.66%
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value																			1,563,078		1,912,937	126,578		162,558
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses																			(29,207)	(43,543)		(1,990)	(5,435)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year																			120		150	23		39
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value																			767,744		670,402	1,676		6,310
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses																			$ (18,600)	$ (20,433)		$ (86)	$ (155)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year																			80		67	5		6