Related Party Transactions - Schedule of Balance Sheet of Related Party Transaction (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Prepaid expenses and ather assets
Related Party Transaction [Line Items]
Amount of transaction	$ 377	$ 241
Accrued expenses and other liabilities
Related Party Transaction [Line Items]
Amount of transaction	7	10
Gregory | Prepaid expenses and ather assets
Related Party Transaction [Line Items]
Amount of transaction	865	687
Gregory | Prepaid expenses and ather assets | Receivable from Servicer for REO acquisitions
Related Party Transaction [Line Items]
Amount of transaction	757	585
Gregory | Receivable from Servicer
Related Party Transaction [Line Items]
Amount of transaction	17,322	17,013
Thetis | Prepaid expenses and ather assets
Related Party Transaction [Line Items]
Amount of transaction	87	87
Thetis | Management Fee Payable
Related Party Transaction [Line Items]
Amount of transaction	1,795	1,634
Gaea | Prepaid expenses and ather assets
Related Party Transaction [Line Items]
Amount of transaction	$ 98
Gaea | Accrued expenses and other liabilities
Related Party Transaction [Line Items]
Amount of transaction		$ 68