VARIABLE INTEREST ENTITIES - Geographic Concentrations of Credit Risk Exceeding 5% of Total Loan Unpaid Principal Balances (Details) - Consolidated VIEs $ in Thousands	3 Months Ended
	Mar. 31, 2018 USD ($)
FREMF Trusts
Concentration Risk [Line Items]
Principal Balance	$ 2,665,427
FREMF Trusts | Maryland
Concentration Risk [Line Items]
Principal Balance	494,604
FREMF Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	361,641
FREMF Trusts | Virginia
Concentration Risk [Line Items]
Principal Balance	329,250
FREMF Trusts | New York
Concentration Risk [Line Items]
Principal Balance	280,925
FREMF Trusts | North Carolina
Concentration Risk [Line Items]
Principal Balance	232,271
FREMF Trusts | Pennsylvania
Concentration Risk [Line Items]
Principal Balance	225,810
FREMF Trusts | Massachusetts
Concentration Risk [Line Items]
Principal Balance	179,440
FREMF Trusts | Ohio
Concentration Risk [Line Items]
Principal Balance	156,834
FREMF Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 404,652
FREMF Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
Percentage of balance	100.00%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Maryland
Concentration Risk [Line Items]
Percentage of balance	18.50%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
Percentage of balance	13.60%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Virginia
Concentration Risk [Line Items]
Percentage of balance	12.40%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | New York
Concentration Risk [Line Items]
Percentage of balance	10.50%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | North Carolina
Concentration Risk [Line Items]
Percentage of balance	8.70%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Pennsylvania
Concentration Risk [Line Items]
Percentage of balance	8.50%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Massachusetts
Concentration Risk [Line Items]
Percentage of balance	6.70%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Ohio
Concentration Risk [Line Items]
Percentage of balance	5.90%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
Percentage of balance	15.20%
Residential Mortgage Loan Trusts
Concentration Risk [Line Items]
Principal Balance	$ 231,536
Residential Mortgage Loan Trusts | California
Concentration Risk [Line Items]
Principal Balance	76,367
Residential Mortgage Loan Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	18,357
Residential Mortgage Loan Trusts | Florida
Concentration Risk [Line Items]
Principal Balance	21,340
Residential Mortgage Loan Trusts | Illinois
Concentration Risk [Line Items]
Principal Balance	15,752
Residential Mortgage Loan Trusts | NEW JERSEY
Concentration Risk [Line Items]
Principal Balance	13,290
Residential Mortgage Loan Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 86,430
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
Percentage of balance	100.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | California
Concentration Risk [Line Items]
Percentage of balance	33.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
Percentage of balance	7.90%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Florida
Concentration Risk [Line Items]
Percentage of balance	9.20%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Illinois
Concentration Risk [Line Items]
Percentage of balance	6.80%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | NEW JERSEY
Concentration Risk [Line Items]
Percentage of balance	5.70%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
Percentage of balance	37.40%