Accumulated Other Comprehensive Income (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2016	Mar. 31, 2015
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance accumulated other comprehensive income (loss)	$ 98,658	$ 205,765
Other comprehensive income (loss) before reclassification	37,755	95,890
Amount reclassified to MBS available for sale	(164)	(4,162)
Amounts reclassified to net gain/loss on the sale of MBS	(4,287)	(16,453)
Amounts reclassified to interest expense	1,376	13,438
Net current period other comprehensive income (loss), unrealized gain/(loss) on derivative instruments	2,793	14,167
Other comprehensive income	34,680	88,713
Ending balance, total accumulated other comprehensive income (loss)	133,338	294,478
Derivative Instruments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance	838	(25,922)
Other comprehensive income (loss) before reclassification	1,582	4,848
Amount reclassified to MBS available for sale	(165)	(4,120)
Amounts reclassified to interest expense	1,376	13,438
Net current period other comprehensive income (loss), unrealized gain/(loss) on derivative instruments	2,793	14,166
Ending balance	3,631	(11,756)
Other Investments | Unrealized Gain (Loss) on Investments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance, accumulated other comprehensive income (loss), unrealized gain/(loss) on investments	(547)	(595)
Other comprehensive income (loss) before reclassification, unrealized gain/(loss) on derivative instruments	94	129
Net current period other comprehensive income (loss), unrealized gain/(loss) on investments	94	129
Ending balance, accumulated other comprehensive income (loss), unrealized gain/(loss) on investments	(453)	(466)
Unsettled MBS | Unrealized Gain (Loss) on Investments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance, accumulated other comprehensive income (loss), unrealized gain/(loss) on investments	(1)	42
Other comprehensive income (loss) before reclassification, unrealized gain/(loss) on derivative instruments	54	794
Amount reclassified to MBS available for sale	1	(42)
Net current period other comprehensive income (loss), unrealized gain/(loss) on investments	55	752
Ending balance, accumulated other comprehensive income (loss), unrealized gain/(loss) on investments	54	794
MSB | Unrealized Gain (Loss) on Investments
Accumulated Other Comprehensive Income Loss [Line Items]
Beginning balance, accumulated other comprehensive income (loss), unrealized gain/(loss) on investments	98,368	232,240
Other comprehensive income (loss) before reclassification, unrealized gain/(loss) on derivative instruments	36,025	90,119
Amounts reclassified to net gain/loss on the sale of MBS	(4,287)	(16,453)
Net current period other comprehensive income (loss), unrealized gain/(loss) on investments	31,738	73,666
Ending balance, accumulated other comprehensive income (loss), unrealized gain/(loss) on investments	$ 130,106	$ 305,906