Accumulated Other Comprehensive Income (Details) (USD $) In Thousands, unless otherwise specified	12 Months Ended
	Dec. 31, 2014	Dec. 31, 2013	Dec. 31, 2012
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	$ (8,052)	$ 262,854
Other comprehensive income (loss) before reclassification	140,325	(701,899)
Amount reclassified to MBS available for sale	(7,527)	23,032
Amounts reclassified to net gain/loss on the sale of MBS	(5,196)	283,012
Amounts reclassified for termination of all- in-one cash flow hedge accounting on dollar roll TBAs	5,083
Amounts reclassified for impairment loss on mortgage-backed securities		8,102
Amounts reclassified to interest expense	81,132	116,847
Net current period other comprehensive income (loss)	90,993	120,684	(25,476)
Other comprehensive income (loss)	213,817	(270,906)	89,532
Ending balance accumulated other comprehensive income (loss)	205,765	(8,052)	262,854
Derivative Instruments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance	(116,915)	(237,599)
Other comprehensive income (loss) before reclassification	11,789	(21,054)
Amount reclassified to MBS available for sale	(7,011)	24,891
Amounts reclassified for termination of all- in-one cash flow hedge accounting on dollar roll TBAs	5,083
Amounts reclassified to interest expense	81,132	116,847
Net current period other comprehensive income (loss)	90,993	120,684
Ending balance	(25,922)	(116,915)
Other Investments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	(627)	(107)
Other comprehensive income (loss) before reclassification	32	(520)
Net current period other comprehensive income (loss)	32	(520)
Ending balance accumulated other comprehensive income (loss)	(595)	(627)
Available-for-sale MBS
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	109,490	499,343
Other comprehensive income (loss) before reclassification	127,946	(680,967)
Amounts reclassified to net gain/loss on the sale of MBS	(5,196)	283,012
Amounts reclassified for impairment loss on mortgage-backed securities		8,102
Net current period other comprehensive income (loss)	122,750	(389,853)
Ending balance accumulated other comprehensive income (loss)	232,240	109,490
Unsettled MBS
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)		1,217
Other comprehensive income (loss) before reclassification	558	642
Amount reclassified to MBS available for sale	(516)	(1,859)
Net current period other comprehensive income (loss)	42	(1,217)
Ending balance accumulated other comprehensive income (loss)	$ 42