Use of Special Purpose Entities and Variable Interest Entities (Narrative) (Details) - USD ($) $ in Thousands		Jun. 30, 2019	Mar. 31, 2019	Dec. 31, 2018	Jun. 30, 2018	Mar. 31, 2018	Dec. 31, 2017
Variable Interest Entity [Line Items]
Residential whole loans, at carrying value	[1]	$ 4,391,983		$ 3,016,715
Residential whole loans, at fair value	[1]	1,525,814		1,665,978
Securitized debt		627,487		684,420
Total residential whole loans		5,900,000		4,700,000
Real estate owned		334,069	$ 290,587	249,413	$ 192,162	$ 182,940	$ 152,356
Other real estate, fair value		369,700		273,400
Asset-backed Securities, Securitized Loans and Receivables
Variable Interest Entity [Line Items]
Residential whole loans, at carrying value		197,000		209,400
Residential whole loans, at fair value		616,000		694,700
Securitized debt		627,487		684,420
Other Assets | Asset-backed Securities, Securitized Loans and Receivables
Variable Interest Entity [Line Items]
Real estate owned at fair value		$ 123,000		$ 79,000

[1] Includes approximately $197.0 million and $209.4 million of Residential whole loans, at carrying value and $616.0 million and $694.7 million of Residential whole loans, at fair value transferred to consolidated variable interest entities (“VIEs”) at June 30, 2019 and December 31, 2018, respectively. Such assets can be used only to settle the obligations of each respective VIE.