Investments in RMBS (Details) $ in Thousands		3 Months Ended	12 Months Ended
		Mar. 31, 2017 USD ($) Security	Dec. 31, 2016 USD ($) Security
Schedule of Available-for-sale Securities [Line Items]
Carrying value		$ 1,139,056	$ 671,904
OTTI charges recognized in earnings		0	173
RMBS [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		1,212,770	750,710
Book value		1,143,777	678,305
Gross unrealized gains		4,326	2,961
Gross unrealized losses		(9,047)	(9,362)
Carrying value	[1]	$ 1,139,056	$ 671,904
Number of securities | Security		158	104
Weighted average coupon		3.80%	3.74%
Weighted average yield	[2]	3.62%	3.53%
Weighted average maturity	[3]	25 years	24 years
Carrying value of collateral for repurchase agreements		$ 812,300	$ 608,600
RMBS [Member] | Fannie Mae [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		772,819	493,645
Book value		732,675	454,012
Gross unrealized gains		1,766	1,517
Gross unrealized losses		(6,174)	(6,592)
Carrying value	[1]	$ 728,267	$ 448,937
Number of securities | Security		101	68
Weighted average rating	[4]
Weighted average coupon		3.78%	3.74%
Weighted average yield	[2]	3.56%	3.52%
Weighted average maturity	[3]	25 years	24 years
RMBS [Member] | Freddie Mac [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		$ 389,576	$ 222,469
Book value		371,377	200,207
Gross unrealized gains		1,063	587
Gross unrealized losses		(2,789)	(2,691)
Carrying value	[1]	$ 369,651	$ 198,103
Number of securities | Security		45	27
Weighted average rating	[4]
Weighted average coupon		3.77%	3.62%
Weighted average yield	[2]	3.52%	3.44%
Weighted average maturity	[3]	26 years	26 years
RMBS [Member] | CMOs [Member]
Schedule of Available-for-sale Securities [Line Items]
Original face value		$ 50,375	$ 34,596
Book value		39,725	24,086
Gross unrealized gains		1,497	857
Gross unrealized losses		(84)	(79)
Carrying value	[1]	$ 41,138	$ 24,864
Number of securities | Security		12	9
Weighted average rating		Unrated	Unrated
Weighted average coupon		4.47%	4.78%
Weighted average yield	[2]	5.10%	4.24%
Weighted average maturity	[3]	12 years	12 years

[1] See Note 9 regarding the estimation of fair value, which approximates carrying value for all securities.

[2] The weighted average yield is based on the most recent annualized monthly interest income, divided by the book value of settled securities. Prior period amounts have been reclassified to conform to current period presentation.

[3] The weighted average stated maturity.

[4] The Company used an implied AAA rating for the Fannie Mae and Freddie Mac securities, other than collateralized mortgage obligations, which are unrated.