OTHER INCOME (LOSS), GENERAL AND ADMINISTRATIVE, OTHER ASSETS AND LIABILITIES - Schedule of Accretion and Other Amortization (Details) - USD ($) $ in Thousands	6 Months Ended
	Jun. 30, 2020	Jun. 30, 2019
Accretion and other amortization:
Accretion of net discount on securities and loans	$ 62,791	$ 235,003
Accretion of servicer advances receivable discount and servicer advance investments	13,464	12,957
Accretion of excess mortgage servicing rights income	12,582	11,830
Amortization of deferred financing costs	(5,992)	(1,925)
Accretion of loan discount and premium amortization, net	80,990	256,955
Notes Payable
Accretion and other amortization:
Amortization of discount on notes and bonds payable	(246)	(910)
Term Loan
Accretion and other amortization:
Amortization of discount on notes and bonds payable	$ (1,609)	$ 0