Schedule of Location of Derivatives on Balance Sheet (Details) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2014	Dec. 31, 2013
Derivative [Line Items]
Interest rate swap asset	$ 12,703	$ 15,841
Future contracts - Asset Derivatives found in Other Assets	5,617	11,148
Forward purchase commitments - Asset Derivatives found in Other Assets	2,042	0
Interest rate swap liability	106,132	125,133
Futures contracts - Asset Derivatives found in Futures contract liability	48,584	36,733
Forward purchase commitments including TBA dollar rolls - Accounts payable and other liabilities	$ 13,634	$ 5,741