Residential Whole Loans (Provision for Loan Losses) (Details) - Purchased credit impaired loans - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2019	Jun. 30, 2018	Jun. 30, 2019	Jun. 30, 2018
Allowance for Loan and Lease Losses [Roll Forward]
Balance at the beginning of period	$ 1,151	$ 380	$ 968	$ 330
Provisions/(reversal of provisions) for loan losses	385		568
Provisions/(reversal of provisions) for loan losses		(83)		(33)
Balance at the end of period	$ 1,536	$ 297	$ 1,536	$ 297