Mortgage Backed Securities Mortgage backed securities designated as AFS (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2014	Mar. 31, 2013	Dec. 31, 2013
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	$ 3,115,727		$ 3,254,946
Available-for-sale MBS, Net Premium (Discount)	812,735		758,950
Available-for-sale MBS, Amortized Cost	3,928,462		4,013,896
Available-for-sale MBS, Gross Unrealized Gain	52,704	45,114
Available-for-sale MBS, Gross Unrealized Loss	(49,882)	(69,566)
Available-for-sale MBS, Fair Value	3,931,284		3,989,444
Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	2,725,064		2,865,398
Available-for-sale MBS, Net Premium (Discount)	621,260		627,047
Available-for-sale MBS, Amortized Cost	3,346,324		3,492,445
Available-for-sale MBS, Gross Unrealized Gain	29,673	26,792
Available-for-sale MBS, Gross Unrealized Loss	(47,806)	(63,976)
Available-for-sale MBS, Fair Value	3,328,191		3,455,261
Agency MBS [Member] | Residential Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	2,438,133		2,591,568
Available-for-sale MBS, Net Premium (Discount)	139,899		154,220
Available-for-sale MBS, Amortized Cost	2,578,032		2,745,788
Available-for-sale MBS, Gross Unrealized Gain	6,708	6,104
Available-for-sale MBS, Gross Unrealized Loss	(45,438)	(59,742)
Available-for-sale MBS, Fair Value	2,539,302		2,692,150
Available-for-sale MBS, Weighted Average Coupon	3.20%		3.22%
Agency MBS [Member] | Commercial Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	286,931		273,830
Available-for-sale MBS, Net Premium (Discount)	19,838		19,061
Available-for-sale MBS, Amortized Cost	306,769		292,891
Available-for-sale MBS, Gross Unrealized Gain	11,384	10,793
Available-for-sale MBS, Gross Unrealized Loss	(509)	(900)
Available-for-sale MBS, Fair Value	317,644		302,784
Available-for-sale MBS, Weighted Average Coupon	5.23%		5.07%
Agency MBS [Member] | CMBS IO [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	0		0
Available-for-sale MBS, notional balance for CMBS IO	10,582,484		4,274,957
Available-for-sale MBS, Net Premium (Discount)	461,523		453,766
Available-for-sale MBS, Amortized Cost	461,523		453,766
Available-for-sale MBS, Gross Unrealized Gain	11,581	9,895
Available-for-sale MBS, Gross Unrealized Loss	(1,859)	(3,334)
Available-for-sale MBS, Fair Value	471,245		460,327
Available-for-sale MBS, Weighted Average Coupon	0.94%		0.83%
Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	390,663		389,548
Available-for-sale MBS, Net Premium (Discount)	191,475		131,903
Available-for-sale MBS, Amortized Cost	582,138		521,451
Available-for-sale MBS, Gross Unrealized Gain	23,031	18,322
Available-for-sale MBS, Gross Unrealized Loss	(2,076)	(5,590)
Available-for-sale MBS, Fair Value	603,093		534,183
Non-Agency MBS [Member] | Residential Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	15,454		13,845
Available-for-sale MBS, Net Premium (Discount)	(7)		(338)
Available-for-sale MBS, Amortized Cost	15,447		13,507
Available-for-sale MBS, Gross Unrealized Gain	209	338
Available-for-sale MBS, Gross Unrealized Loss	(86)	(80)
Available-for-sale MBS, Fair Value	15,570		13,765
Available-for-sale MBS, Weighted Average Coupon	4.42%		4.61%
Non-Agency MBS [Member] | Commercial Mortgage Backed Securities [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	375,209		375,703
Available-for-sale MBS, Net Premium (Discount)	(17,819)		(18,277)
Available-for-sale MBS, Amortized Cost	357,390		357,426
Available-for-sale MBS, Gross Unrealized Gain	19,448	15,366
Available-for-sale MBS, Gross Unrealized Loss	(842)	(3,511)
Available-for-sale MBS, Fair Value	375,996		369,281
Available-for-sale MBS, Weighted Average Coupon	5.05%		5.10%
Non-Agency MBS [Member] | CMBS IO [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Par Balance	0		0
Available-for-sale MBS, notional balance for CMBS IO	5,053,884		10,160,502
Available-for-sale MBS, Net Premium (Discount)	209,301		150,518
Available-for-sale MBS, Amortized Cost	209,301		150,518
Available-for-sale MBS, Gross Unrealized Gain	3,374	2,618
Available-for-sale MBS, Gross Unrealized Loss	(1,148)	(1,999)
Available-for-sale MBS, Fair Value	211,527		151,137
Available-for-sale MBS, Weighted Average Coupon	0.80%		0.66%
Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	1,563,078		1,912,937
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses	(29,207)	(43,543)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year	120		150
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	767,744		670,402
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses	(18,600)	(20,433)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year	80		67
Non-Agency MBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than Twelve Months, Fair Value	126,578		162,558
Available-for-sale MBS, Continuous Unrealized Loss Position, Less than 12 Months, Gross Unrealized Losses	(1,990)	(5,435)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Less than One Year	23		39
Available-for-sale MBS, Continuous Unrealized Loss Position, Twelve Months or Longer, Fair Value	1,676		6,310
Available-for-sale MBS, Continuous Unrealized Loss Position, 12 Months or Longer, Gross Unrealized Losses	$ (86)	$ (155)
Available-for-sale MBS, Number of Securities in Continuous Unrealized Loss Position for Greater than or Equal to One Year	5		6