Mortgage Backed Securities Mortgage backed securities designated as AFS (Details) (USD $) In Thousands, unless otherwise specified	6 Months Ended	12 Months Ended
	Jun. 30, 2014	Dec. 31, 2013
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	$ 3,074,848	$ 3,254,946
Available-for-sale MBS, Unamortized Premium (Discount)	836,884	758,950
Available-for-sale MBS, Amortized Cost	3,911,732	4,013,896
Available-for-sale MBS, Gross Unrealized Gain	72,329	45,114
Available-for-sale MBS, Gross Unrealized Loss	(34,096)	(69,566)
Available-for-sale MBS, Fair Value	3,949,965	3,989,444
Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	2,670,034	2,865,398
Available-for-sale MBS, Unamortized Premium (Discount)	585,108	627,047
Available-for-sale MBS, Amortized Cost	3,255,142	3,492,445
Available-for-sale MBS, Gross Unrealized Gain	38,604	26,792
Available-for-sale MBS, Gross Unrealized Loss	(33,509)	(63,976)
Available-for-sale MBS, Fair Value	3,260,237	3,455,261
Agency MBS [Member] | Residential Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	2,360,799	2,591,568
Available-for-sale MBS, Unamortized Premium (Discount)	132,342	154,220
Available-for-sale MBS, Amortized Cost	2,493,141	2,745,788
Available-for-sale MBS, Gross Unrealized Gain	7,657	6,104
Available-for-sale MBS, Gross Unrealized Loss	(33,257)	(59,742)
Available-for-sale MBS, Fair Value	2,467,541	2,692,150
Available-for-sale MBS, Weighted Average Coupon	3.16%	3.22%
Agency MBS [Member] | Commercial Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	309,235	273,830
Available-for-sale MBS, Unamortized Premium (Discount)	20,178	19,061
Available-for-sale MBS, Amortized Cost	329,413	292,891
Available-for-sale MBS, Gross Unrealized Gain	15,892	10,793
Available-for-sale MBS, Gross Unrealized Loss	(109)	(900)
Available-for-sale MBS, Fair Value	345,196	302,784
Available-for-sale MBS, Weighted Average Coupon	5.23%	5.07%
Agency MBS [Member] | CMBS IO [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	0	0
Available-for-sale MBS, Unamortized Premium (Discount)	432,588	453,766
Available-for-sale MBS, Amortized Cost	432,588	453,766
Available-for-sale MBS, Gross Unrealized Gain	15,055	9,895
Available-for-sale MBS, Gross Unrealized Loss	(143)	(3,334)
Available-for-sale MBS, Fair Value	447,500	460,327
Available-for-sale MBS, Weighted Average Coupon	0.88%	0.83%
Notional balance for interest only securities	9,703,887	10,160,502
Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	404,814	389,548
Available-for-sale MBS, Unamortized Premium (Discount)	251,776	131,903
Available-for-sale MBS, Amortized Cost	656,590	521,451
Available-for-sale MBS, Gross Unrealized Gain	33,725	18,322
Available-for-sale MBS, Gross Unrealized Loss	(587)	(5,590)
Available-for-sale MBS, Fair Value	689,728	534,183
Non-Agency MBS [Member] | Residential Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	13,482	13,845
Available-for-sale MBS, Unamortized Premium (Discount)	(6)	(338)
Available-for-sale MBS, Amortized Cost	13,476	13,507
Available-for-sale MBS, Gross Unrealized Gain	198	338
Available-for-sale MBS, Gross Unrealized Loss	81	(80)
Available-for-sale MBS, Fair Value	13,593	13,765
Available-for-sale MBS, Weighted Average Coupon	4.48%	4.61%
Non-Agency MBS [Member] | Commercial Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	391,332	375,703
Available-for-sale MBS, Unamortized Premium (Discount)	(17,190)	(18,277)
Available-for-sale MBS, Amortized Cost	374,142	357,426
Available-for-sale MBS, Gross Unrealized Gain	27,166	15,366
Available-for-sale MBS, Gross Unrealized Loss	0	(3,511)
Available-for-sale MBS, Fair Value	401,308	369,281
Available-for-sale MBS, Weighted Average Coupon	4.96%	5.10%
Non-Agency MBS [Member] | CMBS IO [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	0	0
Available-for-sale MBS, Unamortized Premium (Discount)	268,972	150,518
Available-for-sale MBS, Amortized Cost	268,972	150,518
Available-for-sale MBS, Gross Unrealized Gain	6,361	2,618
Available-for-sale MBS, Gross Unrealized Loss	506	(1,999)
Available-for-sale MBS, Fair Value	274,827	151,137
Available-for-sale MBS, Weighted Average Coupon	0.70%	0.66%
Notional balance for interest only securities	6,958,238	4,274,957
Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	95,177	1,912,937
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses	(741)	(43,543)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year	18	150
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	1,905,306	670,402
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses	(32,768)	(20,433)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year	151	67
Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	11,711	162,558
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses	(73)	(5,435)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year	2	39
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	27,008	6,310
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses	$ (514)	$ (155)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year	8	6