DEBT OBLIGATIONS - Schedule of Borrowing Capacity (Details) $ in Thousands	12 Months Ended
	Dec. 31, 2018 USD ($)
Debt Instrument [Line Items]
Face Amount of Notes and Bonds Payable	$ 22,673,065
Non-Agency Bonds
Debt Instrument [Line Items]
Face Amount of Notes and Bonds Payable	86,300
Residential mortgage loans and REO
Debt Instrument [Line Items]
Borrowing Capacity	5,575,197
Balance Outstanding	3,774,136
Available Financing	1,801,061
Non-Agency RMBS
Debt Instrument [Line Items]
Borrowing Capacity	250,000
Balance Outstanding	241,535
Available Financing	8,465
Excess MSRs
Debt Instrument [Line Items]
Borrowing Capacity	150,000
Balance Outstanding	100,000
Available Financing	50,000
MSRs
Debt Instrument [Line Items]
Borrowing Capacity	990,000
Balance Outstanding	645,319
Available Financing	344,681
Servicer Advances
Debt Instrument [Line Items]
Borrowing Capacity	1,678,541
Balance Outstanding	1,372,576
Available Financing	$ 305,965
Unused borrowing capacity fee	0.10%
Consumer loan bonds
Debt Instrument [Line Items]
Borrowing Capacity	$ 150,000
Balance Outstanding	21,303
Available Financing	128,697
Debt Borrowing Capacity
Debt Instrument [Line Items]
Borrowing Capacity	8,793,738
Balance Outstanding	6,154,869
Available Financing	$ 2,638,869