Residential Whole Loans at Carrying Value-Provision for Loan Losses (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2015	Dec. 31, 2014	Dec. 31, 2013
Allowance for Loan and Lease Losses [Roll Forward]
Balance at the beginning of period	$ 1,165	$ 137	$ 0
Provisions for loan losses	1,028	137
Balance at the end of period	$ 1,165	$ 137