Accumulated Other Comprehensive Income (Details) (USD $) In Thousands, unless otherwise specified	3 Months Ended
	Mar. 31, 2015	Mar. 31, 2014
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	$ 205,765	$ (8,052)
Other comprehensive income before reclassification	95,890	43,616
Amount reclassified to MBS available for sale	(4,162)	658
Amounts reclassified to net gain/loss on the sale of MBS	(16,453)	(7,436)
Amounts reclassified for termination of all- in-one cash flow hedge accounting on dollar roll TBAs		5,083
Amounts reclassified to interest expense	13,438	24,684
Net current period other comprehensive income (loss)	14,167	32,467
Other comprehensive income	88,713	66,605
Ending balance accumulated other comprehensive income (loss)	294,478	58,553
Available for sale MBS
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	232,240	109,490
Other comprehensive income before reclassification	90,119	41,702
Amounts reclassified to net gain/loss on the sale of MBS	(16,453)	(7,436)
Net current period other comprehensive income (loss)	73,666	34,266
Ending balance accumulated other comprehensive income (loss)	305,906	143,756
Unsettled MBS
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	42
Other comprehensive income before reclassification	794	(186)
Amount reclassified to MBS available for sale	(42)
Net current period other comprehensive income (loss)	752	(186)
Ending balance accumulated other comprehensive income (loss)	794	(186)
Other Investments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	(595)	(627)
Other comprehensive income before reclassification	129	58
Net current period other comprehensive income (loss)	129	58
Ending balance accumulated other comprehensive income (loss)	(466)	(569)
Derivative Instruments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance	(25,922)	(116,915)
Other comprehensive income before reclassification	4,848	2,042
Amounts reclassified for termination of all- in-one cash flow hedge accounting on dollar roll TBAs		5,083
Amounts reclassified to interest expense	13,438	24,684
Amount reclassified to MBS available for sale	(4,120)	658
Net current period other comprehensive income (loss)	14,166	32,467
Ending balance	$ (11,756)	$ (84,448)