VARIABLE INTEREST ENTITIES - Geographic Concentrations of Credit Risk Exceeding 5% of Total Loan Unpaid Principal Balances (Details) - Consolidated VIEs $ in Thousands	9 Months Ended
	Sep. 30, 2018 USD ($)
FREMF Trusts
Concentration Risk [Line Items]
Principal Balance	$ 3,495,230
FREMF Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	478,593
FREMF Trusts | Maryland
Concentration Risk [Line Items]
Principal Balance	448,646
FREMF Trusts | California
Concentration Risk [Line Items]
Principal Balance	360,279
FREMF Trusts | Virginia
Concentration Risk [Line Items]
Principal Balance	347,002
FREMF Trusts | Pennsylvania
Concentration Risk [Line Items]
Principal Balance	281,384
FREMF Trusts | New York
Concentration Risk [Line Items]
Principal Balance	280,925
FREMF Trusts | North Carolina
Concentration Risk [Line Items]
Principal Balance	251,187
FREMF Trusts | Massachusetts
Concentration Risk [Line Items]
Principal Balance	179,440
FREMF Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 867,774
FREMF Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
Percentage of balance	100.00%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
Percentage of balance	13.70%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Maryland
Concentration Risk [Line Items]
Percentage of balance	12.80%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | California
Concentration Risk [Line Items]
Percentage of balance	10.30%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Virginia
Concentration Risk [Line Items]
Percentage of balance	9.90%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Pennsylvania
Concentration Risk [Line Items]
Percentage of balance	8.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | New York
Concentration Risk [Line Items]
Percentage of balance	8.00%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | North Carolina
Concentration Risk [Line Items]
Percentage of balance	7.20%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Massachusetts
Concentration Risk [Line Items]
Percentage of balance	5.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
Percentage of balance	24.90%
Residential Mortgage Loan Trusts
Concentration Risk [Line Items]
Principal Balance	$ 108,480
Residential Mortgage Loan Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	13,981
Residential Mortgage Loan Trusts | California
Concentration Risk [Line Items]
Principal Balance	48,718
Residential Mortgage Loan Trusts | Illinois
Concentration Risk [Line Items]
Principal Balance	8,297
Residential Mortgage Loan Trusts | Washington
Concentration Risk [Line Items]
Principal Balance	7,559
Residential Mortgage Loan Trusts | Florida
Concentration Risk [Line Items]
Principal Balance	5,424
Residential Mortgage Loan Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 24,501
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
Percentage of balance	12.90%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | California
Concentration Risk [Line Items]
Percentage of balance	44.90%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Illinois
Concentration Risk [Line Items]
Percentage of balance	7.60%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Washington
Concentration Risk [Line Items]
Percentage of balance	7.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Florida
Concentration Risk [Line Items]
Percentage of balance	5.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
Percentage of balance	22.60%