Fair Value of Financial Instruments Recorded basis and Fair Value (Details) - USD ($) $ in Thousands		Sep. 30, 2016	Dec. 31, 2015
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Derivative assets		$ 12,985	$ 7,835
Derivative liabilities		84,023	41,205
Carrying Value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed Securities Available-for-sale, Fair Value Disclosure		3,110,467	3,493,701
Mortgage loans held for investment, net, Fair Value Disclosure		20,366	24,145
Investment in FHLB stock, Fair Value Disclosure		5,260	11,475
Derivative assets		12,985	7,835
Repurchase agreements, Fair Value Disclosure		2,478,278	2,589,420
FHLB advances, Fair Value Disclosure		263,000	520,000
Non-recourse collateralized financing, Fair Value		7,025	8,442
Derivative liabilities		84,023	41,205
Fair Value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed Securities Available-for-sale, Fair Value Disclosure		3,110,467	3,493,701
Mortgage loans held for investment, net, Fair Value Disclosure	[1]	16,996	20,849
Investment in FHLB stock, Fair Value Disclosure	[2]	5,260	11,475
Derivative assets		12,985	7,835
Repurchase agreements, Fair Value Disclosure	[2]	2,478,278	2,589,420
FHLB advances, Fair Value Disclosure	[2]	263,000	520,000
Non-recourse collateralized financing, Fair Value	[1]	6,802	8,102
Derivative liabilities		$ 84,023	$ 41,205

[1] The Company determines the fair value of its mortgage loans held for investment, net and its non-recourse collateralized financing using internally developed cash flow models with inputs similar to those used to estimate the fair value of the Company's Level 3 non-Agency MBS.

[2] The carrying value of repurchase agreements and FHLB advances generally approximates fair value due to their short term maturities