FAIR VALUE OF FINANCIAL INSTRUMENTS - Summary of Financial Assets and Liabilities (Details) - USD ($) $ in Thousands	Dec. 31, 2023	Dec. 31, 2022	Dec. 31, 2021	Dec. 31, 2020
Assets:
Fair value of assets	$ 3,192,150	$ 3,922,131
Liabilities:
Fair value of liabilities	3,683,621	3,994,458
Allowance for credit losses	(43,165)	(20,755)	$ (31,752)	$ (41,507)
Repurchase agreements - short-term
Liabilities:
Principal Amount	337,631	481,465
Fair value of liabilities	337,631	481,465
Repurchase agreements - long-term
Liabilities:
Principal Amount	268,976	366,398
Fair value of liabilities	268,976	366,398
Mortgage loan financing
Liabilities:
Principal Amount	437,384	497,454
Fair value of liabilities	425,992	477,101
CLO debt
Liabilities:
Principal Amount	1,062,777	1,064,365
Fair value of liabilities	1,060,719	1,058,462
Borrowings from the FHLB
Liabilities:
Principal Amount	115,000	213,000
Fair value of liabilities	115,000	213,055
Senior unsecured notes
Liabilities:
Principal Amount	1,575,614	1,643,794
Fair value of liabilities	1,475,303	1,397,977
CMBS
Assets:
Principal Amount	9,281	9,415
Fair value of assets	8,955	9,030
CMBS interest-only
Assets:
Principal Amount	8,327	8,460
Fair value of assets	309	417
Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Principal Amount	3,164,226	3,907,295
Fair value of assets	3,150,843	3,875,708
Mortgage loan  receivables held for sale
Assets:
Principal Amount	31,350	31,350
Fair value of assets	26,868	27,391
FHLB stock
Assets:
Principal Amount	5,175	9,585
Fair value of assets	5,175	9,585
Level 1
Assets:
Fair value of assets	0	0
Liabilities:
Fair value of liabilities	0	0
Level 1 | Repurchase agreements - short-term
Liabilities:
Fair value of liabilities	0	0
Level 1 | Repurchase agreements - long-term
Liabilities:
Fair value of liabilities	0	0
Level 1 | Mortgage loan financing
Liabilities:
Fair value of liabilities	0	0
Level 1 | CLO debt
Liabilities:
Fair value of liabilities	0	0
Level 1 | Borrowings from the FHLB
Liabilities:
Fair value of liabilities	0	0
Level 1 | Senior unsecured notes
Liabilities:
Fair value of liabilities	0	0
Level 1 | CMBS
Assets:
Fair value of assets	0	0
Level 1 | CMBS interest-only
Assets:
Fair value of assets	0	0
Level 1 | Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Fair value of assets	0	0
Level 1 | Mortgage loan  receivables held for sale
Assets:
Fair value of assets	0	0
Level 1 | FHLB stock
Assets:
Fair value of assets	0	0
Level 2
Assets:
Fair value of assets	9,264	0
Liabilities:
Fair value of liabilities	1,667,326	0
Level 2 | Repurchase agreements - short-term
Liabilities:
Fair value of liabilities	337,631	0
Level 2 | Repurchase agreements - long-term
Liabilities:
Fair value of liabilities	268,976	0
Level 2 | Mortgage loan financing
Liabilities:
Fair value of liabilities	0	0
Level 2 | CLO debt
Liabilities:
Fair value of liabilities	1,060,719	0
Level 2 | Borrowings from the FHLB
Liabilities:
Fair value of liabilities	0	0
Level 2 | Senior unsecured notes
Liabilities:
Fair value of liabilities	0	0
Level 2 | CMBS
Assets:
Fair value of assets	8,955	0
Level 2 | CMBS interest-only
Assets:
Fair value of assets	309	0
Level 2 | Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Fair value of assets	0	0
Level 2 | Mortgage loan  receivables held for sale
Assets:
Fair value of assets	0	0
Level 2 | FHLB stock
Assets:
Fair value of assets	0	0
Level 3
Assets:
Fair value of assets	3,182,886	3,922,131
Liabilities:
Fair value of liabilities	2,016,295	3,994,458
Level 3 | Repurchase agreements - short-term
Liabilities:
Fair value of liabilities	0	481,465
Level 3 | Repurchase agreements - long-term
Liabilities:
Fair value of liabilities	0	366,398
Level 3 | Mortgage loan financing
Liabilities:
Fair value of liabilities	425,992	477,101
Level 3 | CLO debt
Liabilities:
Fair value of liabilities	0	1,058,462
Level 3 | Borrowings from the FHLB
Liabilities:
Fair value of liabilities	115,000	213,055
Level 3 | Senior unsecured notes
Liabilities:
Fair value of liabilities	1,475,303	1,397,977
Level 3 | CMBS
Assets:
Fair value of assets	0	9,030
Level 3 | CMBS interest-only
Assets:
Fair value of assets	0	417
Level 3 | Total mortgage loan receivables held for investment, net, at amortized cost
Assets:
Fair value of assets	3,150,843	3,875,708
Level 3 | Mortgage loan  receivables held for sale
Assets:
Fair value of assets	26,868	27,391
Level 3 | FHLB stock
Assets:
Fair value of assets	5,175	9,585
Recurring
Assets:
Fair value of assets	477,743	580,130
Recurring | Total mortgage loan receivables held for investment, net, at amortized cost | Discounted Cash Flow
Assets:
Principal Amount	3,164,226	3,907,295
Fair value of assets	3,150,843	3,875,708
Liabilities:
Allowance for credit losses	(43,200)	(20,800)
Recurring | CMBS
Assets:
Principal Amount	430,398	553,424
Fair value of assets	415,935	532,304
Recurring | CMBS interest-only
Assets:
Principal Amount	868,228	1,017,735
Fair value of assets	6,260	10,026
Recurring | GNMA interest-only
Assets:
Principal Amount	37,053	45,369
Fair value of assets	213	281
Recurring | Agency securities
Assets:
Principal Amount	22	36
Fair value of assets	21	35
Recurring | U.S. Treasury securities
Assets:
Principal Amount	54,031	36,000
Fair value of assets	53,716	35,328
Recurring | Equity securities
Assets:
Fair value of assets	144	118
Recurring | Nonhedge derivatives
Assets:
Principal Amount	194,900	204,700
Fair value of assets	1,454	2,038
Recurring | Level 1
Assets:
Fair value of assets	53,860	35,446
Recurring | Level 1 | CMBS
Assets:
Fair value of assets	0	0
Recurring | Level 1 | CMBS interest-only
Assets:
Fair value of assets	0	0
Recurring | Level 1 | GNMA interest-only
Assets:
Fair value of assets	0	0
Recurring | Level 1 | Agency securities
Assets:
Fair value of assets	0	0
Recurring | Level 1 | U.S. Treasury securities
Assets:
Fair value of assets	53,716	35,328
Recurring | Level 1 | Equity securities
Assets:
Fair value of assets	144	118
Recurring | Level 1 | Nonhedge derivatives
Assets:
Fair value of assets	0	0
Recurring | Level 2
Assets:
Fair value of assets	423,883	2,038
Recurring | Level 2 | CMBS
Assets:
Fair value of assets	415,935	0
Recurring | Level 2 | CMBS interest-only
Assets:
Fair value of assets	6,260	0
Recurring | Level 2 | GNMA interest-only
Assets:
Fair value of assets	213	0
Recurring | Level 2 | Agency securities
Assets:
Fair value of assets	21	0
Recurring | Level 2 | U.S. Treasury securities
Assets:
Fair value of assets	0	0
Recurring | Level 2 | Equity securities
Assets:
Fair value of assets	0	0
Recurring | Level 2 | Nonhedge derivatives
Assets:
Fair value of assets	1,454	2,038
Recurring | Level 3
Assets:
Fair value of assets	0	542,646
Recurring | Level 3 | CMBS
Assets:
Fair value of assets	0	532,304
Recurring | Level 3 | CMBS interest-only
Assets:
Fair value of assets	0	10,026
Recurring | Level 3 | GNMA interest-only
Assets:
Fair value of assets	0	281
Recurring | Level 3 | Agency securities
Assets:
Fair value of assets	0	35
Recurring | Level 3 | U.S. Treasury securities
Assets:
Fair value of assets	0	0
Recurring | Level 3 | Equity securities
Assets:
Fair value of assets	0	0
Recurring | Level 3 | Nonhedge derivatives
Assets:
Fair value of assets	$ 0	$ 0