Fair Value of Financial Instruments Recorded basis and Fair Value (Details) - USD ($) $ in Thousands		Dec. 31, 2017	Dec. 31, 2016
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed securities		$ 3,026,989	$ 3,212,084
U.S. Treasuries		146,530	0
Derivative assets		2,940	28,534
Derivative liabilities		269	6,922
Carrying Value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed securities		3,026,989	3,212,084
U.S. Treasuries		146,530	0
Mortgage loans held for investment, net (1)		15,738	19,036
Derivative assets		2,940	28,534
Repurchase agreements (2)		2,565,902	2,898,952
Non-recourse collateralized financing (1)		5,520	6,440
Derivative liabilities		269	6,922
Fair Value
Fair Value, Balance Sheet Grouping, Financial Statement Captions [Line Items]
Mortgage-backed securities		3,026,989	3,212,084
U.S. Treasuries		146,530	0
Mortgage loans held for investment, net (1)	[1]	12,973	15,971
Derivative assets		2,940	28,534
Repurchase agreements (2)	[2]	2,565,902	2,898,952
Non-recourse collateralized financing (1)	[1]	5,554	6,357
Derivative liabilities		$ 269	$ 6,922

[1] The Company determines the fair value of its mortgage loans held for investment, net and its non-recourse collateralized financing using internally developed cash flow models with inputs similar to those used to estimate the fair value of the Company’s Level 3 non-Agency MBS.

[2] The carrying value of repurchase agreements generally approximates fair value due to their short term maturities