Interest Receivable (Details) (USD $) In Thousands, unless otherwise specified	Mar. 31, 2015	Dec. 31, 2014
Interest receivable by investment category
Total interest receivable	$ 32,885	$ 31,257
Total MBS
Interest receivable by investment category
Total interest receivable	31,000	31,170
Fannie Mae
Interest receivable by investment category
Total interest receivable	10,975	11,761
Freddie Mac
Interest receivable by investment category
Total interest receivable	2,486	2,598
Ginnie Mae
Interest receivable by investment category
Total interest receivable	16	17
Non-Agency MBS
Interest receivable by investment category
Total interest receivable	17,523	16,794
Residential whole loans
Interest receivable by investment category
Total interest receivable	1,763	0
CRT securities
Interest receivable by investment category
Total interest receivable	101	66
Money market and other investments
Interest receivable by investment category
Total interest receivable	$ 21	$ 21