VARIABLE INTEREST ENTITIES - Geographic Concentrations of Credit Risk Exceeding 5% of Total Loan Unpaid Principal Balances (Details) - Consolidated VIEs $ in Thousands	12 Months Ended
	Dec. 31, 2017 USD ($)
FREMF Trusts
Concentration Risk [Line Items]
Principal Balance	$ 2,770,824
FREMF Trusts | Maryland
Concentration Risk [Line Items]
Principal Balance	494,885
FREMF Trusts | Texas
Concentration Risk [Line Items]
Principal Balance	362,898
FREMF Trusts | Virginia
Concentration Risk [Line Items]
Principal Balance	329,250
FREMF Trusts | New York
Concentration Risk [Line Items]
Principal Balance	280,925
FREMF Trusts | North Carolina
Concentration Risk [Line Items]
Principal Balance	242,707
FREMF Trusts | Pennsylvania
Concentration Risk [Line Items]
Principal Balance	225,810
FREMF Trusts | Massachusetts
Concentration Risk [Line Items]
Principal Balance	179,440
FREMF Trusts | Ohio
Concentration Risk [Line Items]
Principal Balance	168,746
FREMF Trusts | Florida
Concentration Risk [Line Items]
Principal Balance	146,960
FREMF Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 339,203
FREMF Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
% of Balance	100.00%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Maryland
Concentration Risk [Line Items]
% of Balance	17.90%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Texas
Concentration Risk [Line Items]
% of Balance	13.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Virginia
Concentration Risk [Line Items]
% of Balance	11.90%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | New York
Concentration Risk [Line Items]
% of Balance	10.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | North Carolina
Concentration Risk [Line Items]
% of Balance	8.80%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Pennsylvania
Concentration Risk [Line Items]
% of Balance	8.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Massachusetts
Concentration Risk [Line Items]
% of Balance	6.50%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Ohio
Concentration Risk [Line Items]
% of Balance	6.10%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Florida
Concentration Risk [Line Items]
% of Balance	5.30%
FREMF Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
% of Balance	12.20%
Residential Mortgage Loan Trusts
Concentration Risk [Line Items]
Principal Balance	$ 476,362
Residential Mortgage Loan Trusts | Virginia
Concentration Risk [Line Items]
Principal Balance	24,437
Residential Mortgage Loan Trusts | California
Concentration Risk [Line Items]
Principal Balance	191,804
Residential Mortgage Loan Trusts | Florida
Concentration Risk [Line Items]
Principal Balance	36,159
Residential Mortgage Loan Trusts | Illinois
Concentration Risk [Line Items]
Principal Balance	24,446
Residential Mortgage Loan Trusts | Other
Concentration Risk [Line Items]
Principal Balance	$ 199,516
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk
Concentration Risk [Line Items]
% of Balance	100.00%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Virginia
Concentration Risk [Line Items]
% of Balance	5.10%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | California
Concentration Risk [Line Items]
% of Balance	40.30%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Florida
Concentration Risk [Line Items]
% of Balance	7.60%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Illinois
Concentration Risk [Line Items]
% of Balance	5.10%
Residential Mortgage Loan Trusts | Securitized Loans | Geographic Concentration Risk | Other
Concentration Risk [Line Items]
% of Balance	41.90%