INVESTMENTS IN RESIDENTIAL MORTGAGE LOANS - Residential Mortgage Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	3 Months Ended
	Mar. 31, 2018 USD ($) loan	Dec. 31, 2017 USD ($)
Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year 4 months 30 days
Threshold period past due (in days)	60 days
Performing Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 534,823
Carrying Value	$ 489,493	$ 507,615
Loan Count | loan	8,619
Weighted Average Yield	8.00%
Weighted Average Life (Years)	5 years 3 months 22 days
Floating Rate Loans as a % of Face Amount	22.30%
Loan to Value Ratio (LTV)	79.30%
Weighted Average Delinquency	5.90%
Weighted Average FICO	649
Performing Loans | Ginnie Mae
Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 32,500
Purchased Credit Deteriorated Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	216,332
Carrying Value	$ 158,467	183,540
Loan Count | loan	1,936
Weighted Average Yield	7.30%
Weighted Average Life (Years)	3 years 1 month 4 days
Floating Rate Loans as a % of Face Amount	16.10%
Loan to Value Ratio (LTV)	86.40%
Weighted Average Delinquency	76.20%
Weighted Average FICO	597
Residential Mortgage Loans, held-for-investment
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 751,155
Carrying Value	$ 647,960	691,155
Loan Count | loan	10,555
Weighted Average Yield	7.70%
Weighted Average Life (Years)	4 years 9 months 11 days
Floating Rate Loans as a % of Face Amount	17.30%
Loan to Value Ratio (LTV)	79.60%
Weighted Average Delinquency	30.70%
Weighted Average FICO	634
Reverse Mortgage Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 17,181
Carrying Value	$ 7,635	6,870
Loan Count | loan	47
Weighted Average Yield	7.50%
Weighted Average Life (Years)	5 years 1 day
Floating Rate Loans as a % of Face Amount	14.30%
Loan to Value Ratio (LTV)	130.20%
Weighted Average Delinquency	68.10%
Performing Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 728,083
Carrying Value	$ 741,903	1,071,371
Loan Count | loan	10,813
Weighted Average Yield	4.10%
Weighted Average Life (Years)	4 years 8 months 15 days
Floating Rate Loans as a % of Face Amount	8.50%
Loan to Value Ratio (LTV)	57.80%
Weighted Average Delinquency	4.40%
Weighted Average FICO	661
Non-Performing Loans
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 895,724
Carrying Value	$ 692,417	647,293
Loan Count | loan	5,905
Weighted Average Yield	6.20%
Weighted Average Life (Years)	5 years 3 months 21 days
Floating Rate Loans as a % of Face Amount	19.80%
Loan to Value Ratio (LTV)	91.40%
Weighted Average Delinquency	53.00%
Weighted Average FICO	581
Residential Mortgage Loans, held-for-sale
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 1,640,988
Carrying Value	$ 1,441,955	$ 1,725,534
Loan Count | loan	16,765
Weighted Average Yield	4.80%
Weighted Average Life (Years)	4 years 6 months 19 days
Floating Rate Loans as a % of Face Amount	18.90%
Loan to Value Ratio (LTV)	82.40%
Weighted Average Delinquency	33.00%
Weighted Average FICO	622
Reverse Mortgage Loans
Mortgage Loans on Real Estate [Line Items]
Interest in reverse mortgage loans	70.00%
Unpaid principal balance	$ 400
Percentage of loans that have reached a termination event	44.00%
Reverse Mortgage Loans | Nationstar
Mortgage Loans on Real Estate [Line Items]
Interest in reverse mortgage loans	30.00%
Non-Performing Loans | Ginnie Mae
Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 64,500