FAIR VALUE OF FINANCIAL INSTRUMENTS - Estimated Fair Values of Financial Instruments (Details) - USD ($) $ in Thousands	3 Months Ended	12 Months Ended
	Mar. 31, 2016	Dec. 31, 2015
Assets:
Fair Value	$ 2,033,506	$ 2,416,966
Liabilities:
Fair Value	4,105,052	4,271,537
Repurchase agreements - short term
Liabilities:
Outstanding Face Amount	1,082,656	1,224,942
Fair Value	1,082,656	1,224,942
Repurchase agreements - long term
Liabilities:
Outstanding Face Amount	21,683	35,813
Fair Value	21,683	35,813
Borrowings under credit agreement | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	0
Amortized Cost Basis	0
Fair Value	$ 0
Fair Value Assumptions:
Weighted average yield %
Weighted Average Remaining Maturity/Duration
Revolving credit facility | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 0
Amortized Cost Basis	0
Fair Value	$ 0
Fair Value Assumptions:
Weighted average yield %
Weighted Average Remaining Maturity/Duration
Mortgage loan financing
Liabilities:
Outstanding Face Amount	$ 544,527	540,764
Fair Value	579,054	557,841
Borrowings from the FHLB
Liabilities:
Outstanding Face Amount	1,881,200	1,856,700
Fair Value	1,892,923	1,861,584
Senior unsecured notes
Liabilities:
Outstanding Face Amount	563,872	619,555
Fair Value	528,736	591,357
Mortgage loan receivables held for investment, at amortized cost
Assets:
Outstanding Face Amount	1,582,358	1,749,556
Fair Value	1,593,676	1,756,774
Mortgage loan receivables held for sale
Assets:
Outstanding Face Amount	353,273	571,638
Fair Value	361,915	582,277
FHLB stock
Assets:
Outstanding Face Amount	77,915	77,915
Fair Value	$ 77,915	$ 77,915
Mortgage loan receivables
Fair Value Assumptions:
Period of short interest rate reset risk	30 days	30 days
Recurring
Assets:
Fair Value	$ 2,599,089	$ 2,410,038
Recurring | Repurchase agreements - short term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	1,082,656	1,224,942
Amortized Cost Basis	1,082,656	1,224,942
Fair Value	$ 1,082,656	$ 1,224,942
Fair Value Assumptions:
Weighted average yield %	1.73%	1.67%
Weighted Average Remaining Maturity/Duration	4 months 2 days	5 months 4 days
Recurring | Repurchase agreements - long term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 21,683	$ 35,813
Amortized Cost Basis	21,683	35,813
Fair Value	$ 21,683	$ 35,813
Fair Value Assumptions:
Weighted average yield %	2.59%	1.87%
Weighted Average Remaining Maturity/Duration	1 year 1 month 24 days	1 year 4 months 24 days
Recurring | Mortgage loan financing | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 544,527	$ 540,764
Amortized Cost Basis	547,776	544,663
Fair Value	$ 579,054	$ 557,841
Fair Value Assumptions:
Weighted average yield %	4.86%	4.86%
Weighted Average Remaining Maturity/Duration	7 years 8 months 8 days	7 years 11 months 4 days
Recurring | Borrowings from the FHLB | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 1,881,200	$ 1,856,700
Amortized Cost Basis	1,881,200	1,856,700
Fair Value	$ 1,892,923	$ 1,861,584
Fair Value Assumptions:
Weighted average yield %	0.97%	0.84%
Weighted Average Remaining Maturity/Duration	2 years 6 months 10 days	1 year 5 months 1 day
Recurring | Senior unsecured notes | Broker quotations, pricing services
Liabilities:
Outstanding Face Amount	$ 563,872	$ 619,555
Fair Value	$ 528,736	$ 591,357
Fair Value Assumptions:
Weighted average yield %	6.67%	6.65%
Weighted Average Remaining Maturity/Duration	3 years 3 months 21 days	3 years 7 months 9 days
Recurring | Senior unsecured notes | Broker quotations, pricing services | Various Dates | Senior Unsecured Notes
Liabilities:
Amortized Cost Basis	$ 558,134	$ 612,605
Recurring | Nonhedge derivatives
Liabilities:
Derivative liability face amount	1,026,800	374,200
Fair Value	12,743	5,504
Recurring | Nonhedge derivatives | Counterparty quotations
Liabilities:
Derivative liability face amount	1,026,800	374,200
Fair Value	$ 12,743	$ 5,504
Fair Value Assumptions:
Weighted Average Remaining Maturity/Duration	1 year 11 months 15 days	3 years 5 months 1 day
Recurring | CMBS
Assets:
Outstanding Face Amount	$ 2,139,819	$ 1,972,492
Fair Value	2,192,154	1,991,506
Recurring | CMBS | Internal model, third-party inputs
Assets:
Outstanding Face Amount	2,139,819	1,972,492
Amortized Cost Basis	2,160,286	1,994,928
Fair Value	$ 2,192,154	$ 1,991,506
Fair Value Assumptions:
Weighted average yield %	2.86%	2.59%
Weighted Average Remaining Maturity/Duration	3 years 1 month 24 days	3 years 1 month 24 days
Recurring | CMBS interest-only
Assets:
Outstanding Face Amount	$ 7,216,114	$ 7,436,379
Fair Value	340,127	344,423
Recurring | CMBS interest-only | Internal model, third-party inputs
Assets:
Outstanding Face Amount	7,216,114	7,436,379
Amortized Cost Basis	344,809	348,222
Fair Value	$ 340,127	$ 344,423
Fair Value Assumptions:
Weighted average yield %	3.76%	3.81%
Weighted Average Remaining Maturity/Duration	3 years 3 months 10 days	3 years 4 months 2 days
Recurring | GNMA interest-only
Assets:
Outstanding Face Amount	$ 600,054	$ 632,175
Fair Value	23,738	26,194
Recurring | GNMA interest-only | Internal model, third-party inputs
Assets:
Outstanding Face Amount	600,054	632,175
Amortized Cost Basis	25,194	28,311
Fair Value	$ 23,738	$ 26,194
Fair Value Assumptions:
Weighted average yield %	4.28%	4.26%
Weighted Average Remaining Maturity/Duration	5 years 1 month 17 days	5 years 2 months 19 days
Recurring | GN construction securities
Assets:
Outstanding Face Amount		$ 27,091
Fair Value		28,639
Recurring | GN construction securities | Internal model, third-party inputs
Assets:
Outstanding Face Amount		27,091
Amortized Cost Basis		27,581
Fair Value		$ 28,639
Fair Value Assumptions:
Weighted average yield %		3.86%
Weighted Average Remaining Maturity/Duration		9 years 3 months 29 days
Recurring | GN permanent securities
Assets:
Outstanding Face Amount	$ 41,148	$ 16,249
Fair Value	42,855	16,455
Recurring | GN permanent securities | Internal model, third-party inputs
Assets:
Outstanding Face Amount	41,148	16,249
Amortized Cost Basis	42,051	16,685
Fair Value	$ 42,855	$ 16,455
Fair Value Assumptions:
Weighted average yield %	3.49%	3.94%
Weighted Average Remaining Maturity/Duration	7 years 7 days	5 years 5 months 4 days
Recurring | Mortgage loan receivables held for investment, at amortized cost | Discounted Cash Flow
Assets:
Outstanding Face Amount	$ 1,582,358	$ 1,749,556
Amortized Cost Basis	1,572,833	1,738,645
Fair Value	$ 1,593,676	$ 1,756,774
Fair Value Assumptions:
Weighted average yield %	7.61%	7.56%
Weighted Average Remaining Maturity/Duration	1 year 5 months 19 days	1 year 4 months 17 days
Recurring | Mortgage loan receivables held for sale | Discounted Cash Flow
Assets:
Outstanding Face Amount	$ 353,273	$ 571,638
Amortized Cost Basis	353,331	571,764
Fair Value	$ 361,915	$ 582,277
Fair Value Assumptions:
Weighted average yield %	4.33%	4.56%
Weighted Average Remaining Maturity/Duration	5 years 8 months 15 days	6 years 2 months 12 days
Recurring | FHLB stock | Put Value
Assets:
Outstanding Face Amount	$ 77,915	$ 77,915
Amortized Cost Basis	77,915	77,915
Fair Value	$ 77,915	$ 77,915
Fair Value Assumptions:
Weighted average yield %	3.50%	3.50%
Recurring | Nonhedge derivatives
Assets:
Derivative asset face amount	$ 11,100	$ 868,700
Fair Value	215	2,821
Recurring | Nonhedge derivatives | Counterparty quotations
Assets:
Derivative asset face amount	11,100	868,700
Fair Value	$ 215	$ 2,821
Fair Value Assumptions:
Weighted Average Remaining Maturity/Duration	5 years 2 months 1 day	8 months 8 days