FAIR VALUE OF FINANCIAL INSTRUMENTS - Estimated Fair Values of Financial Instruments (Details) $ in Thousands		9 Months Ended		12 Months Ended
		Sep. 30, 2019 USD ($)	Sep. 30, 2018	Dec. 31, 2018 USD ($)
Assets:
Fair Value		$ 3,487,684		$ 3,564,681
Provision for loan losses	[1]	(18,500)		(17,900)
Liabilities:
Fair Value		$ 4,939,146		$ 4,401,295
Total mortgage loan receivables held for investment, net, at amortized cost
Liabilities:
Period of short interest rate reset risk		30 days		30 days
CLO debt
Liabilities:
Period of short interest rate reset risk		30 days	30 days
Recurring
Assets:
Fair Value		$ 1,899,012		$ 1,397,918
Recurring | CMBS | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		1,779,458		1,258,819
Amortized Cost Basis/Purchase Price		1,780,233		1,257,801
Fair Value		$ 1,788,712		$ 1,252,640
Liabilities:
Financial instruments, measurement input		0.0314		0.0314
Weighted average remaining maturity/duration		2 years 4 months 17 days		2 years 3 months 29 days
Recurring | CMBS interest-only | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 2,139,357		$ 2,373,936
Amortized Cost Basis/Purchase Price		39,961		55,534
Fair Value		$ 41,440		$ 55,691
Liabilities:
Financial instruments, measurement input		0.0365		0.0280
Weighted average remaining maturity/duration		2 years 7 months 9 days		2 years 8 months 8 days
Recurring | GNMA interest-only | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 113,096		$ 135,932
Amortized Cost Basis/Purchase Price		2,202		2,862
Fair Value		$ 2,026		$ 2,648
Liabilities:
Financial instruments, measurement input		0.0965		0.0630
Weighted average remaining maturity/duration		2 years 8 months 23 days		4 years 1 month 9 days
Recurring | Agency securities | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 641		$ 668
Amortized Cost Basis/Purchase Price		652		682
Fair Value		$ 654		$ 662
Liabilities:
Financial instruments, measurement input		0.0174		0.0183
Weighted average remaining maturity/duration		1 year 11 months 19 days		2 years 4 months 9 days
Recurring | GNMA permanent securities | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 31,760		$ 32,633
Amortized Cost Basis/Purchase Price		31,984		32,889
Fair Value		$ 32,795		$ 33,064
Liabilities:
Financial instruments, measurement input		0.0327		0.0376
Weighted average remaining maturity/duration		4 years 6 months 18 days		5 years 10 days
Recurring | Corporate bonds | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 32,088		$ 55,305
Amortized Cost Basis/Purchase Price		31,604		54,257
Fair Value		$ 32,372		$ 53,871
Liabilities:
Financial instruments, measurement input		0.0481		0.0504
Weighted average remaining maturity/duration		1 year 3 months 21 days		2 years 6 months 3 days
Recurring | Equity securities | Internal Model Third Party Inputs Valuation Technique
Assets:
Amortized Cost Basis/Purchase Price		$ 13,720		$ 13,154
Fair Value		13,457		11,550
Recurring | Total mortgage loan receivables held for investment, net, at amortized cost | Discounted Cash Flow
Assets:
Outstanding Face Amount		3,249,711		3,340,381
Amortized Cost Basis/Purchase Price		3,231,443		3,318,390
Fair Value		$ 3,249,383		$ 3,324,588
Liabilities:
Financial instruments, measurement input		0.0729		0.0784
Weighted average remaining maturity/duration		1 year 4 months 13 days		1 year 3 months 25 days
Recurring | Provisions For Loan Losses
Assets:
Provision for loan losses		$ (18,500)		$ (17,900)
Recurring | First mortgage loans | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		173,957		181,905
Amortized Cost Basis/Purchase Price		174,214		182,439
Fair Value		$ 182,716		$ 187,870
Liabilities:
Financial instruments, measurement input		0.0459		0.0546
Weighted average remaining maturity/duration		9 years 8 months 4 days		9 years 9 months
Recurring | FHLB stock | FHLB stock
Assets:
Outstanding Face Amount		$ 61,619		$ 57,915
Amortized Cost Basis/Purchase Price		61,619		57,915
Fair Value		$ 61,619		$ 57,915
Liabilities:
Financial instruments, measurement input		0.0550		0.0450
Recurring | Nonhedge derivatives | Counterparty Quotations Valuation Technique
Assets:
Nonhedge derivative assets		$ 6,000		$ 0
Fair Value		$ 22		$ 0
Liabilities:
Weighted average remaining maturity/duration		9 days		0 years
Recurring | Repurchase agreements - short-term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 1,486,049		$ 436,957
Amortized Cost Basis/Purchase Price		1,486,049		436,957
Fair Value		$ 1,486,049		$ 436,957
Financial instruments, measurement input		0.0265		0.0342
Weighted average remaining maturity/duration		5 days		2 months 23 days
Recurring | Repurchase agreements - long-term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 299,999		$ 226,728
Amortized Cost Basis/Purchase Price		299,999		226,728
Fair Value		$ 299,999		$ 226,728
Financial instruments, measurement input		0.0320		0.0347
Weighted average remaining maturity/duration		1 year 3 months 10 days		1 year 8 months 23 days
Recurring | Mortgage loan financing | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 718,351		$ 738,825
Amortized Cost Basis/Purchase Price		723,313		743,902
Fair Value		$ 748,489		$ 735,662
Financial instruments, measurement input		0.0496		0.0509
Weighted average remaining maturity/duration		1 year 6 months 3 days		2 years 7 months 9 days
Recurring | CLO debt | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 117,760		$ 601,543
Amortized Cost Basis/Purchase Price		117,760		601,543
Fair Value		$ 117,760		$ 601,543
Financial instruments, measurement input		0.0497		0.0441
Weighted average remaining maturity/duration		5 years 10 months 28 days		9 years 4 months 24 days
Recurring | Participation Financing - Mortgage Loan Receivable | Discounted Cash Flow
Liabilities:
Outstanding Face Amount				$ 2,453
Amortized Cost Basis/Purchase Price				2,453
Fair Value				$ 2,453
Financial instruments, measurement input				0.1700
Weighted average remaining maturity/duration				5 months 4 days
Recurring | Borrowings from the FHLB | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 1,076,449		$ 1,286,000
Amortized Cost Basis/Purchase Price		1,076,449		1,286,000
Fair Value		$ 1,084,876		$ 1,286,664
Financial instruments, measurement input		0.0250		0.0255
Weighted average remaining maturity/duration		2 years 3 months 25 days		2 years 5 months 15 days
Recurring | Senior unsecured notes | Broker Quotations Pricing Services Valuation Technique
Liabilities:
Outstanding Face Amount		$ 1,166,201		$ 1,166,201
Amortized Cost Basis/Purchase Price		1,157,117		1,154,991
Fair Value		$ 1,201,973		$ 1,111,288
Financial instruments, measurement input		0.0539		0.0539
Weighted average remaining maturity/duration		3 years 6 months 10 days		4 years 3 months 10 days
Recurring | Nonhedge derivatives | Counterparty Quotations Valuation Technique
Liabilities:
Nonhedge derivative liabilities		$ 270,471		$ 578,971
Fair Value		$ 82		$ 975
Weighted average remaining maturity/duration		3 months		3 months

[1] Includes amounts relating to consolidated variable interest entities. See Note 10.