FAIR VALUE OF FINANCIAL INSTRUMENTS - Summary of Financial Assets and Liabilities (Details) - USD ($) $ in Thousands	Jun. 30, 2016	Dec. 31, 2015
Assets:
Fair Value	$ 2,249,583	$ 2,416,966
Liabilities:
Fair Value	4,430,837	4,271,537
Repurchase agreements - short-term
Liabilities:
Outstanding Face Amount	1,139,615	1,224,942
Fair Value	1,139,615	1,224,942
Repurchase agreements - long-term
Liabilities:
Outstanding Face Amount	0	35,813
Fair Value	0	35,813
Revolving credit facility
Liabilities:
Outstanding Face Amount	100,000
Fair Value	100,000
Mortgage loan financing
Liabilities:
Outstanding Face Amount	544,527	540,764
Fair Value	587,333	557,841
Borrowings from the FHLB
Liabilities:
Outstanding Face Amount	2,049,700	1,856,700
Fair Value	2,064,247	1,861,584
Senior unsecured notes
Liabilities:
Outstanding Face Amount	563,872	619,555
Fair Value	539,642	591,357
Mortgage loan receivables held for investment
Assets:
Outstanding Face Amount	1,553,562	1,749,556
Fair Value	1,565,966	1,756,774
Mortgage loan receivables held for sale
Assets:
Outstanding Face Amount	583,089	571,638
Fair Value	605,702	582,277
FHLB stock
Assets:
Outstanding Face Amount	77,915	77,915
Fair Value	77,915	77,915
Level 3
Assets:
Fair Value	2,249,583	2,416,966
Liabilities:
Fair Value	4,430,837	4,271,537
Level 3 | Repurchase agreements - short-term
Liabilities:
Fair Value	1,139,615	1,224,942
Level 3 | Repurchase agreements - long-term
Liabilities:
Fair Value	0	35,813
Level 3 | Revolving credit facility
Liabilities:
Fair Value	100,000
Level 3 | Mortgage loan financing
Liabilities:
Fair Value	587,333	557,841
Level 3 | Borrowings from the FHLB
Liabilities:
Fair Value	2,064,247	1,861,584
Level 3 | Senior unsecured notes
Liabilities:
Fair Value	539,642	591,357
Level 3 | Mortgage loan receivables held for investment
Assets:
Fair Value	1,565,966	1,756,774
Level 3 | Mortgage loan receivables held for sale
Assets:
Fair Value	605,702	582,277
Level 3 | FHLB stock
Assets:
Fair Value	77,915	77,915
Recurring
Assets:
Fair Value	2,700,428	2,410,038
Recurring | Nonhedge derivatives
Liabilities:
Derivative liability face amount	1,181,500	374,200
Fair Value	26,494	5,504
Recurring | CMBS
Assets:
Outstanding Face Amount	2,221,936	1,972,492
Fair Value	2,302,280	1,991,506
Recurring | CMBS interest-only
Assets:
Outstanding Face Amount	7,676,007	7,436,379
Fair Value	334,613	344,423
Recurring | GNMA interest-only
Assets:
Outstanding Face Amount	538,877	632,175
Fair Value	20,833	26,194
Recurring | GNMA construction securities
Assets:
Outstanding Face Amount		27,091
Fair Value		28,639
Recurring | GNMA permanent securities
Assets:
Outstanding Face Amount	40,787	16,249
Fair Value	42,484	16,455
Recurring | Nonhedge derivatives
Assets:
Derivative asset face amount	15,011	868,700
Fair Value	218	2,821
Recurring | Level 2
Assets:
Fair Value	218	2,821
Recurring | Level 2 | Nonhedge derivatives
Liabilities:
Fair Value	26,494	5,504
Recurring | Level 2 | Nonhedge derivatives
Assets:
Fair Value	218	2,821
Recurring | Level 3
Assets:
Fair Value	2,700,210	2,407,217
Recurring | Level 3 | CMBS
Assets:
Fair Value	2,302,280	1,991,506
Recurring | Level 3 | CMBS interest-only
Assets:
Fair Value	334,613	344,423
Recurring | Level 3 | GNMA interest-only
Assets:
Fair Value	20,833	26,194
Recurring | Level 3 | GNMA construction securities
Assets:
Fair Value		28,639
Recurring | Level 3 | GNMA permanent securities
Assets:
Fair Value	$ 42,484	$ 16,455