Investments in RMBS, Assets by Maturity (Details) $ in Thousands		3 Months Ended	12 Months Ended
		Mar. 31, 2017 USD ($) Security	Dec. 31, 2016 USD ($) Security
Schedule of Available-for-sale Securities [Line Items]
Carrying value		$ 1,139,056	$ 671,904
RMBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		1,212,770	750,710
Book value		1,143,777	678,305
Gross unrealized gains		4,326	2,961
Gross unrealized losses		(9,047)	(9,362)
Carrying value	[1]	$ 1,139,056	$ 671,904
Number of securities | Security		158	104
Weighted average coupon		3.80%	3.74%
Weighted average yield	[2]	3.62%	3.53%
Weighted average maturity	[3]	25 years	24 years
RMBS [Member] | 5-10 Years [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		$ 16,069	$ 16,069
Book value		16,892	17,110
Gross unrealized gains		214	185
Gross unrealized losses		(323)	(454)
Carrying value	[1]	$ 16,783	$ 16,841
Number of securities | Security		3	3
Weighted average rating	[4]
Weighted average coupon		4.18%	4.18%
Weighted average yield	[2]	3.90%	3.94%
Weighted average maturity	[3]	8 years	8 years
RMBS [Member] | Over 10 Years [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		$ 1,196,701	$ 734,641
Book value		1,126,885	661,195
Gross unrealized gains		4,112	2,776
Gross unrealized losses		(8,724)	(8,908)
Carrying value	[1]	$ 1,122,273	$ 655,063
Number of securities | Security		155	101
Weighted average rating	[4]
Weighted average coupon		3.79%	3.73%
Weighted average yield	[2]	3.61%	3.52%
Weighted average maturity	[3]	25 years	24 years

[1] See Note 9 regarding the estimation of fair value, which approximates carrying value for all securities.

[2] The weighted average yield is based on the most recent annualized monthly interest income, divided by the book value of settled securities. Prior period amounts have been reclassified to conform to current period presentation.

[3] The weighted average stated maturity.

[4] The Company used an implied AAA rating for the Fannie Mae and Freddie Mac securities, other than collateralized mortgage obligations, which are unrated.