Fair Value of Financial Instruments Recorded basis and Fair Value (Details) - USD ($) $ in Thousands		Mar. 31, 2017	Dec. 31, 2016
Carrying Value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed Securities Available-for-sale, Fair Value Disclosure		$ 3,186,749	$ 3,212,084
Mortgage loans held for investment, net, Fair Value Disclosure		18,183	19,036
Derivative assets, Fair Value Disclosure		272	28,534
Repurchase agreements, Fair Value Disclosure		2,825,945	2,898,952
Non-recourse collateralized financing, Fair Value		6,075	6,440
Derivative liabilities, Fair Value Disclosure		58	6,922
Fair Value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed Securities Available-for-sale, Fair Value Disclosure		3,186,749	3,212,084
Mortgage loans held for investment, net, Fair Value Disclosure	[1]	15,120	15,971
Derivative assets, Fair Value Disclosure		272	28,534
Repurchase agreements, Fair Value Disclosure	[2]	2,825,945	2,898,952
Non-recourse collateralized financing, Fair Value	[1]	6,024	6,357
Derivative liabilities, Fair Value Disclosure		$ 58	$ 6,922

[1] The Company determines the fair value of its mortgage loans held for investment, net and its non-recourse collateralized financing using internally developed cash flow models with inputs similar to those used to estimate the fair value of the Company's Level 3 non-Agency MBS.

[2] The carrying value of repurchase agreements generally approximates fair value due to their short term maturities