FAIR VALUE OF FINANCIAL INSTRUMENTS - Summary of Financial Assets and Liabilities (Details) - USD ($) $ in Thousands	Mar. 31, 2016	Dec. 31, 2015
Assets:
Fair Value	$ 2,033,506	$ 2,416,966
Liabilities:
Fair Value	4,105,052	4,271,537
Repurchase agreements - short-term
Liabilities:
Outstanding Face Amount	1,082,656	1,224,942
Fair Value	1,082,656	1,224,942
Repurchase agreements - long-term
Liabilities:
Outstanding Face Amount	21,683	35,813
Fair Value	21,683	35,813
Mortgage loan financing
Liabilities:
Outstanding Face Amount	544,527	540,764
Fair Value	579,054	557,841
Borrowings from the FHLB
Liabilities:
Outstanding Face Amount	1,881,200	1,856,700
Fair Value	1,892,923	1,861,584
Senior unsecured notes
Liabilities:
Outstanding Face Amount	563,872	619,555
Fair Value	528,736	591,357
Mortgage loan receivables held for investment
Assets:
Outstanding Face Amount	1,582,358	1,749,556
Fair Value	1,593,676	1,756,774
Mortgage loan receivables held for sale
Assets:
Outstanding Face Amount	353,273	571,638
Fair Value	361,915	582,277
FHLB stock
Assets:
Outstanding Face Amount	77,915	77,915
Fair Value	77,915	77,915
Level 3
Assets:
Fair Value	2,033,506	2,416,966
Liabilities:
Fair Value	4,105,052	4,271,537
Level 3 | Repurchase agreements - short-term
Liabilities:
Fair Value	1,082,656	1,224,942
Level 3 | Repurchase agreements - long-term
Liabilities:
Fair Value	21,683	35,813
Level 3 | Mortgage loan financing
Liabilities:
Fair Value	579,054	557,841
Level 3 | Borrowings from the FHLB
Liabilities:
Fair Value	1,892,923	1,861,584
Level 3 | Senior unsecured notes
Liabilities:
Fair Value	528,736	591,357
Level 3 | Mortgage loan receivables held for investment
Assets:
Fair Value	1,593,676	1,756,774
Level 3 | Mortgage loan receivables held for sale
Assets:
Fair Value	361,915	582,277
Level 3 | FHLB stock
Assets:
Fair Value	77,915	77,915
Recurring
Assets:
Fair Value	2,599,089	2,410,038
Recurring | Nonhedge derivatives
Liabilities:
Derivative liability face amount	1,026,800	374,200
Fair Value	12,743	5,504
Recurring | CMBS
Assets:
Outstanding Face Amount	2,139,819	1,972,492
Fair Value	2,192,154	1,991,506
Recurring | CMBS interest-only
Assets:
Outstanding Face Amount	7,216,114	7,436,379
Fair Value	340,127	344,423
Recurring | GNMA interest-only
Assets:
Outstanding Face Amount	600,054	632,175
Fair Value	23,738	26,194
Recurring | GN construction securities
Assets:
Outstanding Face Amount		27,091
Fair Value		28,639
Recurring | GN permanent securities
Assets:
Outstanding Face Amount	41,148	16,249
Fair Value	42,855	16,455
Recurring | Nonhedge derivatives
Assets:
Derivative asset face amount	11,100	868,700
Fair Value	215	2,821
Recurring | Level 2
Assets:
Fair Value	215	2,821
Recurring | Level 2 | Nonhedge derivatives
Liabilities:
Fair Value	12,743	5,504
Recurring | Level 2 | Nonhedge derivatives
Assets:
Fair Value	215	2,821
Recurring | Level 3
Assets:
Fair Value	2,598,874	2,407,217
Recurring | Level 3 | CMBS
Assets:
Fair Value	2,192,154	1,991,506
Recurring | Level 3 | CMBS interest-only
Assets:
Fair Value	340,127	344,423
Recurring | Level 3 | GNMA interest-only
Assets:
Fair Value	23,738	26,194
Recurring | Level 3 | GN construction securities
Assets:
Fair Value		28,639
Recurring | Level 3 | GN permanent securities
Assets:
Fair Value	$ 42,855	$ 16,455