INVESTMENTS IN RESIDENTIAL MORTGAGE LOANS - Residential Mortgage Loans Outstanding by Loan Type, Excluding REO (Details) $ in Thousands	9 Months Ended
	Sep. 30, 2017 USD ($) loan	Dec. 31, 2016 USD ($)
Mortgage Loans on Real Estate [Line Items]
Weighted Average Life (Years)	1 year 3 months 1 day
Threshold period past due (in days)	60 days
Reverse Mortgage Loans, held-for-investment
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 0
Carrying Value	$ 0	$ 0
Loan Count | loan	0
Weighted Average Yield	0.00%
Weighted Average Life (Years)	0 years
Floating Rate Loans as a % of Face Amount	0.00%
Loan to Value Ratio (LTV)	0.00%
Weighted Average Delinquency	0.00%
Performing Loans, held-for-investment
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 556,361
Carrying Value	$ 507,300	0
Loan Count | loan	8,079
Weighted Average Yield	8.00%
Weighted Average Life (Years)	5 years 7 months 6 days
Floating Rate Loans as a % of Face Amount	18.70%
Loan to Value Ratio (LTV)	78.80%
Weighted Average Delinquency	7.60%
Weighted Average FICO	651
Performing Loans, held-for-investment | Ginnie Mae
Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 34,900
Purchased Credit Deteriorated Loans, held-for-investment
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	264,183
Carrying Value	$ 194,927	190,761
Loan Count | loan	2,258
Weighted Average Yield	7.10%
Weighted Average Life (Years)	3 years
Floating Rate Loans as a % of Face Amount	14.40%
Loan to Value Ratio (LTV)	81.50%
Weighted Average Delinquency	79.30%
Weighted Average FICO	597
Residential Mortgage Loans, held-for-investment
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 820,544
Carrying Value	$ 702,227	190,761
Loan Count | loan	10,337
Weighted Average Yield	7.70%
Weighted Average Life (Years)	4 years 9 months 18 days
Floating Rate Loans as a % of Face Amount	17.30%
Loan to Value Ratio (LTV)	79.60%
Weighted Average Delinquency	30.70%
Weighted Average FICO	634
Reverse Mortgage Loans, held-for-sale
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 19,207
Carrying Value	$ 9,342	11,468
Loan Count | loan	51
Weighted Average Yield	7.20%
Weighted Average Life (Years)	4 years 3 months 18 days
Floating Rate Loans as a % of Face Amount	13.60%
Loan to Value Ratio (LTV)	135.10%
Weighted Average Delinquency	81.60%
Performing Loans, held-for-sale
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 779,618
Carrying Value	$ 791,134	175,194
Loan Count | loan	11,139
Weighted Average Yield	4.00%
Weighted Average Life (Years)	4 years 10 months 24 days
Floating Rate Loans as a % of Face Amount	16.40%
Loan to Value Ratio (LTV)	65.80%
Weighted Average Delinquency	3.40%
Weighted Average FICO	663
Non-Performing Loans, held-for-sale
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 820,912
Carrying Value	$ 626,275	510,003
Loan Count | loan	5,228
Weighted Average Yield	5.50%
Weighted Average Life (Years)	4 years 3 months 18 days
Floating Rate Loans as a % of Face Amount	21.40%
Loan to Value Ratio (LTV)	97.00%
Weighted Average Delinquency	60.00%
Weighted Average FICO	583
Residential Mortgage Loans, held-for-sale
Mortgage Loans on Real Estate [Line Items]
Outstanding Face Amount	$ 1,619,737
Carrying Value	$ 1,426,751	$ 696,665
Loan Count | loan	16,418
Weighted Average Yield	4.80%
Weighted Average Life (Years)	4 years 7 months 6 days
Floating Rate Loans as a % of Face Amount	18.90%
Loan to Value Ratio (LTV)	82.40%
Weighted Average Delinquency	33.00%
Weighted Average FICO	622
Reverse Mortgage Loans
Mortgage Loans on Real Estate [Line Items]
Interest in reverse mortgage loans	70.00%
Unpaid principal balance	$ 500
Percentage of loans that have reached a termination event	59.00%
Non-Performing Loans | Ginnie Mae
Mortgage Loans on Real Estate [Line Items]
Unpaid Principal Balance	$ 70,600