FAIR VALUE OF FINANCIAL INSTRUMENTS - Estimated Fair Values of Financial Instruments (Details) $ in Thousands		6 Months Ended		12 Months Ended
		Jun. 30, 2019 USD ($)	Jun. 30, 2018	Dec. 31, 2018 USD ($)
Assets:
Fair Value		$ 3,305,747		$ 3,564,681
Provision for loan losses	[1]	(18,500)		(17,900)
Liabilities:
Fair Value		4,665,834		4,401,295
Level 3
Assets:
Fair Value		3,305,747		3,564,681
Provision for loan losses		(18,500)		(17,900)
Liabilities:
Fair Value		$ 4,665,834		$ 4,401,295
Mortgage loan receivables held for investment, net, at amortized cost
Liabilities:
Period of short interest rate reset risk		30 days		30 days
CLO debt
Liabilities:
Period of short interest rate reset risk		30 days	30 days
Recurring
Assets:
Fair Value		$ 1,776,433		$ 1,397,918
Recurring | Level 3
Assets:
Fair Value		1,762,693		1,386,368
Recurring | CMBS | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		1,642,160		1,258,819
Amortized Cost Basis/Purchase Price		1,643,149		1,257,801
Fair Value		$ 1,653,703		$ 1,252,640
Liabilities:
Financial instruments, measurement input		0.0331		0.0314
Weighted average remaining maturity/duration		2 years 3 months 3 days		2 years 3 months 29 days
Recurring | CMBS interest-only | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 2,166,640		$ 2,373,936
Amortized Cost Basis/Purchase Price		42,689		55,534
Fair Value		$ 44,052		$ 55,691
Liabilities:
Financial instruments, measurement input		0.0363		0.0280
Weighted average remaining maturity/duration		2 years 8 months 1 day		2 years 8 months 8 days
Recurring | GNMA interest-only | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 118,858		$ 135,932
Amortized Cost Basis/Purchase Price		2,483		2,862
Fair Value		$ 2,291		$ 2,648
Liabilities:
Financial instruments, measurement input		0.0913		0.0630
Weighted average remaining maturity/duration		2 years 8 months 19 days		4 years 1 month 9 days
Recurring | Agency securities | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 652		$ 668
Amortized Cost Basis/Purchase Price		664		682
Fair Value		$ 663		$ 662
Liabilities:
Financial instruments, measurement input		0.0178		0.0183
Weighted average remaining maturity/duration		2 years 1 month 9 days		2 years 4 months 9 days
Recurring | GNMA permanent securities | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 32,055		$ 32,633
Amortized Cost Basis/Purchase Price		32,291		32,889
Fair Value		$ 33,084		$ 33,064
Liabilities:
Financial instruments, measurement input		0.0374		0.0376
Weighted average remaining maturity/duration		4 years 8 months 19 days		5 years 10 days
Recurring | Corporate bonds | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		$ 41,205		$ 55,305
Amortized Cost Basis/Purchase Price		40,581		54,257
Fair Value		$ 41,418		$ 53,871
Liabilities:
Financial instruments, measurement input		0.0482		0.0504
Weighted average remaining maturity/duration		1 year 7 months 20 days		2 years 6 months 3 days
Recurring | Equity securities | Internal Model Third Party Inputs Valuation Technique
Assets:
Amortized Cost Basis/Purchase Price		$ 13,720		$ 13,154
Fair Value		13,204		11,550
Recurring | Mortgage loan receivables held for investment, net, at amortized cost | Discounted Cash Flow
Assets:
Outstanding Face Amount		3,136,334		3,340,381
Amortized Cost Basis/Purchase Price		3,119,857		3,318,390
Fair Value		$ 3,133,579		$ 3,324,588
Liabilities:
Financial instruments, measurement input		0.0801		0.0784
Weighted average remaining maturity/duration		1 year 2 months 8 days		1 year 3 months 25 days
Recurring | Provisions For Loan Losses
Assets:
Provision for loan losses		$ (18,500)		$ (17,900)
Recurring | Mortgage loan receivables held for sale | Internal Model Third Party Inputs Valuation Technique
Assets:
Outstanding Face Amount		112,099		181,905
Amortized Cost Basis/Purchase Price		111,977		182,439
Fair Value		$ 116,531		$ 187,870
Liabilities:
Financial instruments, measurement input		0.0505		0.0546
Weighted average remaining maturity/duration		9 years 4 months 28 days		9 years 9 months
Recurring | FHLB stock | FHLB stock
Assets:
Outstanding Face Amount		$ 61,619		$ 57,915
Amortized Cost Basis/Purchase Price		61,619		57,915
Fair Value		$ 61,619		$ 57,915
Liabilities:
Financial instruments, measurement input		0.0550		0.0450
Recurring | Nonhedge derivatives | Counterparty Quotations Valuation Technique
Assets:
Nonhedge derivative assets		$ 344,171		$ 0
Fair Value		$ 536		$ 0
Liabilities:
Weighted average remaining maturity/duration		3 months		0 years
Recurring | Repurchase agreements - short-term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 957,902		$ 436,957
Amortized Cost Basis/Purchase Price		957,902		436,957
Fair Value		$ 957,902		$ 436,957
Financial instruments, measurement input		0.0307		0.0342
Weighted average remaining maturity/duration		3 months 14 days		2 months 23 days
Recurring | Repurchase agreements - long-term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 309,469		$ 226,728
Amortized Cost Basis/Purchase Price		309,469		226,728
Fair Value		$ 309,469		$ 226,728
Financial instruments, measurement input		0.0336		0.0347
Weighted average remaining maturity/duration		1 year 6 months 7 days		1 year 8 months 23 days
Recurring | Mortgage loan financing | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 729,685		$ 738,825
Amortized Cost Basis/Purchase Price		734,652		743,902
Fair Value		$ 750,161		$ 735,662
Financial instruments, measurement input		0.0509		0.0509
Weighted average remaining maturity/duration		1 year 9 months 25 days		2 years 7 months 9 days
Recurring | CLO debt | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 263,216		$ 601,543
Amortized Cost Basis/Purchase Price		263,216		601,543
Fair Value		$ 263,216		$ 601,543
Financial instruments, measurement input		0.0510		0.0441
Weighted average remaining maturity/duration		4 years 10 months 20 days		9 years 4 months 24 days
Recurring | Participation Financing - Mortgage Loan Receivable | Discounted Cash Flow
Liabilities:
Outstanding Face Amount				$ 2,453
Amortized Cost Basis/Purchase Price				2,453
Fair Value				$ 2,453
Financial instruments, measurement input				0.1700
Weighted average remaining maturity/duration				5 months 4 days
Recurring | Borrowings from the FHLB | Discounted Cash Flow
Liabilities:
Outstanding Face Amount		$ 1,191,449		$ 1,286,000
Amortized Cost Basis/Purchase Price		1,191,449		1,286,000
Fair Value		$ 1,199,510		$ 1,286,664
Financial instruments, measurement input		0.0258		0.0255
Weighted average remaining maturity/duration		2 years 4 months 2 days		2 years 5 months 15 days
Recurring | Senior unsecured notes | Broker Quotations Pricing Services Valuation Technique
Liabilities:
Outstanding Face Amount		$ 1,166,201		$ 1,166,201
Amortized Cost Basis/Purchase Price		1,156,400		1,154,991
Fair Value		$ 1,185,576		$ 1,111,288
Financial instruments, measurement input		0.0539		0.0539
Weighted average remaining maturity/duration		3 years 9 months 10 days		4 years 3 months 10 days
Recurring | Nonhedge derivatives | Counterparty Quotations Valuation Technique
Liabilities:
Nonhedge derivative liabilities				$ 578,971
Fair Value				$ 975
Weighted average remaining maturity/duration				3 months

[1] Includes amounts relating to consolidated variable interest entities. See Note 3.