CONSOLIDATED STATEMENTS OF INCOME - USD ($)	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Interest income
Residential mortgage investments	$ 212,694,000	$ 215,989,000	$ 226,749,000
Other	637,000	341,000	315,000
Interest income	213,331,000	216,330,000	227,064,000
Interest expense
Secured borrowings	(107,653,000)	(85,521,000)	(65,155,000)
Unsecured borrowings	(7,833,000)	(8,454,000)	(8,488,000)
Interest expense	(115,486,000)	(93,975,000)	(73,643,000)
Net interest income (expense)	97,845,000	122,355,000	153,421,000
Other revenue (expense)
Compensation-related expense	(8,784,000)	(10,200,000)	(8,302,000)
Separation of service charge	(2,965,000)
Other general and administrative expense	(4,682,000)	(4,798,000)	(4,157,000)
Miscellaneous other revenue (expense)	1,459,000	968,000	(142,000)
Operating expenses	(14,972,000)	(14,030,000)	(12,601,000)
Net income	82,873,000	108,325,000	140,820,000
Net income available to common stockholders
Net income	82,873,000	108,325,000	140,820,000
Less preferred stock dividends	(15,372,000)	(15,160,000)	(13,781,000)
Net income available to common stockholders	$ 67,501,000	$ 93,165,000	$ 127,039,000
Basic and diluted net income per common share	$ 0.70	$ 0.97	$ 1.33