FAIR VALUE OF FINANCIAL INSTRUMENTS - Summary of Financial Assets and Liabilities (Details) - USD ($) $ in Thousands		Sep. 30, 2019	Dec. 31, 2018
Assets:
Fair value of assets		$ 3,487,684	$ 3,564,681
Provision for loan losses	[1]	(18,500)	(17,900)
Liabilities:
Fair value of liabilities		4,939,146	4,401,295
Repurchase agreements - short-term
Liabilities:
Outstanding Face Amount		1,486,049	436,957
Fair value of liabilities		1,486,049	436,957
Repurchase agreements - long-term
Liabilities:
Outstanding Face Amount		299,999	226,728
Fair value of liabilities		299,999	226,728
Mortgage loan financing
Liabilities:
Outstanding Face Amount		718,351	738,825
Fair value of liabilities		748,489	735,662
CLO debt
Liabilities:
Outstanding Face Amount		117,760	601,543
Fair value of liabilities		117,760	601,543
Participation Financing - Mortgage Loan Receivable
Liabilities:
Outstanding Face Amount			2,453
Fair value of liabilities			2,453
Borrowings from the FHLB
Liabilities:
Outstanding Face Amount		1,076,449	1,286,000
Fair value of liabilities		1,084,876	1,286,664
Senior unsecured notes
Liabilities:
Outstanding Face Amount		1,166,201	1,166,201
Fair value of liabilities		1,201,973	1,111,288
CMBS
Assets:
Outstanding Face Amount		12,144	12,210
Fair value of assets		11,627	11,306
CMBS interest-only
Assets:
Outstanding Face Amount		11,123	11,189
Fair value of assets		839	902
Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Outstanding Face Amount		3,249,711	3,340,381
Fair value of assets		3,249,383	3,324,588
Mortgage loan receivables held for sale
Assets:
Outstanding Face Amount		173,957	181,905
Fair value of assets		182,716	187,870
FHLB stock
Assets:
Outstanding Face Amount		61,619	57,915
Fair value of assets		61,619	57,915
Level 1
Assets:
Fair value of assets		0	0
Provision for loan losses		0	0
Liabilities:
Fair value of liabilities		0	0
Level 1 | Repurchase agreements - short-term
Liabilities:
Fair value of liabilities		0	0
Level 1 | Repurchase agreements - long-term
Liabilities:
Fair value of liabilities		0	0
Level 1 | Mortgage loan financing
Liabilities:
Fair value of liabilities		0	0
Level 1 | CLO debt
Liabilities:
Fair value of liabilities		0	0
Level 1 | Participation Financing - Mortgage Loan Receivable
Liabilities:
Fair value of liabilities			0
Level 1 | Borrowings from the FHLB
Liabilities:
Fair value of liabilities		0	0
Level 1 | Senior unsecured notes
Liabilities:
Fair value of liabilities		0	0
Level 1 | CMBS
Assets:
Fair value of assets		0	0
Level 1 | CMBS interest-only
Assets:
Fair value of assets		0	0
Level 1 | Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Fair value of assets		0	0
Level 1 | Mortgage loan receivables held for sale
Assets:
Fair value of assets		0	0
Level 1 | FHLB stock
Assets:
Fair value of assets		0	0
Level 2
Assets:
Fair value of assets		0	0
Provision for loan losses		0	0
Liabilities:
Fair value of liabilities		0	0
Level 2 | Repurchase agreements - short-term
Liabilities:
Fair value of liabilities		0	0
Level 2 | Repurchase agreements - long-term
Liabilities:
Fair value of liabilities		0	0
Level 2 | Mortgage loan financing
Liabilities:
Fair value of liabilities		0	0
Level 2 | CLO debt
Liabilities:
Fair value of liabilities		0	0
Level 2 | Participation Financing - Mortgage Loan Receivable
Liabilities:
Fair value of liabilities			0
Level 2 | Borrowings from the FHLB
Liabilities:
Fair value of liabilities		0	0
Level 2 | Senior unsecured notes
Liabilities:
Fair value of liabilities		0	0
Level 2 | CMBS
Assets:
Fair value of assets		0	0
Level 2 | CMBS interest-only
Assets:
Fair value of assets		0	0
Level 2 | Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Fair value of assets		0	0
Level 2 | Mortgage loan receivables held for sale
Assets:
Fair value of assets		0	0
Level 2 | FHLB stock
Assets:
Fair value of assets		0	0
Level 3
Assets:
Fair value of assets		3,487,684	3,564,681
Provision for loan losses		(18,500)	(17,900)
Liabilities:
Fair value of liabilities		4,939,146	4,401,295
Level 3 | Repurchase agreements - short-term
Liabilities:
Fair value of liabilities		1,486,049	436,957
Level 3 | Repurchase agreements - long-term
Liabilities:
Fair value of liabilities		299,999	226,728
Level 3 | Mortgage loan financing
Liabilities:
Fair value of liabilities		748,489	735,662
Level 3 | CLO debt
Liabilities:
Fair value of liabilities		117,760	601,543
Level 3 | Participation Financing - Mortgage Loan Receivable
Liabilities:
Fair value of liabilities			2,453
Level 3 | Borrowings from the FHLB
Liabilities:
Fair value of liabilities		1,084,876	1,286,664
Level 3 | Senior unsecured notes
Liabilities:
Fair value of liabilities		1,201,973	1,111,288
Level 3 | CMBS
Assets:
Fair value of assets		11,627	11,306
Level 3 | CMBS interest-only
Assets:
Fair value of assets		839	902
Level 3 | Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Fair value of assets		3,249,383	3,324,588
Level 3 | Mortgage loan receivables held for sale
Assets:
Fair value of assets		182,716	187,870
Level 3 | FHLB stock
Assets:
Fair value of assets		61,619	57,915
Recurring
Assets:
Fair value of assets		1,899,012	1,397,918
Recurring | Nonhedge derivatives
Assets:
Fair value of assets		82
Liabilities:
Nonhedge derivative liabilities		270,471	605,871
Fair value of liabilities			975
Recurring | CMBS
Assets:
Outstanding Face Amount		1,767,314	1,246,609
Fair value of assets		1,777,085	1,241,334
Recurring | CMBS interest-only
Assets:
Outstanding Face Amount		2,128,234	2,362,747
Fair value of assets		40,601	54,789
Recurring | GNMA interest-only
Assets:
Outstanding Face Amount		113,096	135,932
Fair value of assets		2,026	2,648
Recurring | Agency securities
Assets:
Outstanding Face Amount		641	668
Fair value of assets		654	662
Recurring | GNMA permanent securities
Assets:
Outstanding Face Amount		31,760	32,633
Fair value of assets		32,795	33,064
Recurring | Corporate bonds
Assets:
Outstanding Face Amount		32,088	55,305
Fair value of assets		32,372	53,871
Recurring | Equity securities
Assets:
Fair value of assets		13,457	11,550
Recurring | Nonhedge derivatives
Assets:
Fair value of assets		22
Nonhedge derivative assets		6,000
Recurring | Level 1
Assets:
Fair value of assets		13,457	11,550
Recurring | Level 1 | Nonhedge derivatives
Assets:
Fair value of assets		0
Liabilities:
Fair value of liabilities			0
Recurring | Level 1 | CMBS
Assets:
Fair value of assets		0	0
Recurring | Level 1 | CMBS interest-only
Assets:
Fair value of assets		0	0
Recurring | Level 1 | GNMA interest-only
Assets:
Fair value of assets		0	0
Recurring | Level 1 | Agency securities
Assets:
Fair value of assets		0	0
Recurring | Level 1 | GNMA permanent securities
Assets:
Fair value of assets		0	0
Recurring | Level 1 | Corporate bonds
Assets:
Fair value of assets		0	0
Recurring | Level 1 | Equity securities
Assets:
Fair value of assets		13,457	11,550
Recurring | Level 1 | Nonhedge derivatives
Assets:
Fair value of assets		0
Recurring | Level 2
Assets:
Fair value of assets		22	0
Recurring | Level 2 | Nonhedge derivatives
Liabilities:
Fair value of liabilities		82	975
Recurring | Level 2 | CMBS
Assets:
Fair value of assets		0	0
Recurring | Level 2 | CMBS interest-only
Assets:
Fair value of assets		0	0
Recurring | Level 2 | GNMA interest-only
Assets:
Fair value of assets		0	0
Recurring | Level 2 | Agency securities
Assets:
Fair value of assets		0	0
Recurring | Level 2 | GNMA permanent securities
Assets:
Fair value of assets		0	0
Recurring | Level 2 | Corporate bonds
Assets:
Fair value of assets		0	0
Recurring | Level 2 | Nonhedge derivatives
Assets:
Fair value of assets		22
Recurring | Level 3
Assets:
Fair value of assets		1,885,533	1,386,368
Recurring | Level 3 | Nonhedge derivatives
Liabilities:
Fair value of liabilities		0	0
Recurring | Level 3 | CMBS
Assets:
Fair value of assets		1,777,085	1,241,334
Recurring | Level 3 | CMBS interest-only
Assets:
Fair value of assets		40,601	54,789
Recurring | Level 3 | GNMA interest-only
Assets:
Fair value of assets		2,026	2,648
Recurring | Level 3 | Agency securities
Assets:
Fair value of assets		654	662
Recurring | Level 3 | GNMA permanent securities
Assets:
Fair value of assets		32,795	33,064
Recurring | Level 3 | Corporate bonds
Assets:
Fair value of assets		32,372	$ 53,871
Recurring | Level 3 | Nonhedge derivatives
Assets:
Fair value of assets		$ 0

[1] Includes amounts relating to consolidated variable interest entities. See Note 10.