Real Estate Securities - Activity of Credit Component of Other-than-Temporary Impairments (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2016	Dec. 31, 2015	Dec. 31, 2014
Other than Temporary Impairment, Credit Losses Recognized in Earnings [Roll Forward]
Balance at beginning of period	$ 28,277	$ 33,849	$ 37,149
Initial credit impairments	346	246	261
Subsequent credit impairments	8	0	70
Securities sold, or expected to sell	(261)	(4,567)	(922)
Securities with no outstanding principal at period end	(109)	(1,251)	(2,709)
Balance at End of Period	$ 28,261	$ 28,277	$ 33,849