Accumulated Other Comprehensive Income (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended		9 Months Ended
	Sep. 30, 2014	Sep. 30, 2013	Sep. 30, 2014	Sep. 30, 2013
Accumulated Other Comprehensive Income Loss [Line Items]
Amounts reclassified to net gain/loss on the sale of MBS	$ (4,337)	$ 46,500	$ (5,535)	$ 32,862
Amounts reclassified for termination of all- in-one cash flow hedge accounting on dollar roll TBAs			5,083
Amounts reclassified for impairment loss on MBS		8,102		8,102
Amounts reclassified to interest expense	19,806	30,954	67,413	91,470
Amounts reclassified to net gain/loss on the sale of MBS	(237)	225,635	(3,089)	214,333
Net current period other comprehensive income (loss)	17,923	45,227	75,026	107,952
Beginning balance accumulated other comprehensive income (loss)	164,223	(320,833)	(8,052)	262,854
Other comprehensive income (loss) before reclassification	(28,691)	(74,737)	94,944	(694,000)
Other comprehensive income (loss)	(13,459)	236,454	158,816	(347,233)
Ending balance accumulated other comprehensive income (loss)	150,764	(84,379)	150,764	(84,379)
Derivative Instruments
Accumulated Other Comprehensive Income Loss [Line Items]
Amounts reclassified for termination of all- in-one cash flow hedge accounting on dollar roll TBAs			5,083
Amounts reclassified to interest expense	19,806	30,954	67,413	91,470
Beginning balance	(59,812)	(174,874)	(116,915)	(237,599)
Other comprehensive income (loss) before reclassification	1,872	(29,451)	7,669	(15,313)
Amount reclassified to MBS available for sale	(3,755)	43,724	(5,139)	31,795
Net current period other comprehensive income (loss)	17,923	45,227	75,026	107,952
Ending balance	(41,889)	(129,647)	(41,889)	(129,647)
Other Investments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	(495)	(505)	(627)	(107)
Other comprehensive income (loss) before reclassification	(111)	82	21	(316)
Net current period other comprehensive income (loss)	(111)	82	21	(316)
Ending balance accumulated other comprehensive income (loss)	(606)	(423)	(606)	(423)
Available-for-sale MBS
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	223,948	(142,678)	109,490	499,343
Other comprehensive income (loss) before reclassification	(30,572)	(48,294)	86,738	(679,013)
Amounts reclassified to net gain/loss on the sale of MBS	(237)	225,635	(3,089)	214,333
Amounts reclassified for impairment loss on MBS		8,102		8,102
Net current period other comprehensive income (loss)	(30,809)	185,443	83,649	(456,578)
Ending balance accumulated other comprehensive income (loss)	193,139	42,765	193,139	42,765
Unsettled MBS
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	582	(2,776)		1,217
Other comprehensive income (loss) before reclassification	120	2,926	516	642
Amount reclassified to MBS available for sale	(582)	2,776	(396)	1,067
Net current period other comprehensive income (loss)	(462)	5,702	120	1,709
Ending balance accumulated other comprehensive income (loss)	$ 120	$ 2,926	$ 120	$ 2,926