Residential Mortgage Loans Held in Securitization Trusts (Net) and Real Estate Owned - Real Estate Owned (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Reconciliation of Carrying Amount of Real Estate Investments [Roll Forward]
Balance at beginning of period	$ 965	$ 1,108	$ 732
Write downs	0	(103)	(433)
Transfer from mortgage loans held in securitization trusts	0	537	1,184
Disposal	(554)	(577)	(375)
Balance at the end of period	$ 411	$ 965	$ 1,108