Match Funded Assets - Schedule of Activity in Match Funded Advances (Details) - USD ($) $ in Thousands	12 Months Ended
	Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Match Funded Advances [Roll Forward]
Decrease (increase) in allowance for losses	$ 13,334	$ (6,794)	$ 21,487
Residential Mortgage
Match Funded Advances [Roll Forward]
Beginning balance	937,294	1,144,600	1,451,964
Transfers from advances		71,623
Sales			(691)
New advances (collections), net	(135,304)	(278,929)	(306,673)
Ending balance	801,990	937,294	1,144,600
Automotive Dealer Financing Notes
Match Funded Advances [Roll Forward]
Beginning balance	$ 0	32,757	0
Transfer (to) from Other assets		(36,896)	25,180
New advances (collections), net		1,504	10,212
Decrease (increase) in allowance for losses		2,635	(2,635)
Ending balance		$ 0	$ 32,757