Related Party Transactions - Schedule Statement of Income of Related Party Transactions (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Related Party Transaction [Line Items]
Related Party Loan Servicing Fee Expense	$ 9,133	$ 10,148	$ 8,245
Related Party Management Fee Expense	7,356	6,025	5,340
Interest Income
Related Party Transaction [Line Items]
Related Party Transaction, Amounts of Transaction	13,081	1,967
Gregory | Loan servicing fees
Related Party Transaction [Line Items]
Related Party Loan Servicing Fee Expense	9,133	10,148	8,245
Gregory | Loan transaction expense
Related Party Transaction [Line Items]
Related Party Transaction, Amounts of Transaction		99	101
Gregory | Other fees and expenses
Related Party Transaction [Line Items]
Related Party Transaction, Amounts of Transaction		40	80
Thetis | Management fee
Related Party Transaction [Line Items]
Related Party Management Fee Expense	7,356	6,025	5,340
Thetis | Other fees and expenses
Related Party Transaction [Line Items]
Related Party Transaction, Amounts of Transaction			$ 4
Thetis | Income from investments in affiliates
Related Party Transaction [Line Items]
Related Party Transaction, Amounts of Transaction	928	436
Great Ajax FS LLC | Income from investments in affiliates
Related Party Transaction [Line Items]
Related Party Transaction, Amounts of Transaction	$ 179	$ 90