FAIR VALUE OF FINANCIAL INSTRUMENTS - Estimated Fair Values of Financial Instruments (Details) - USD ($) $ in Thousands	9 Months Ended	12 Months Ended
	Sep. 30, 2016	Dec. 31, 2015
Assets:
Fair Value	$ 2,555,231	$ 2,416,966
Liabilities:
Fair Value	4,642,867	4,271,537
Repurchase agreements - short term
Liabilities:
Outstanding Face Amount	1,320,470	1,224,942
Fair Value	1,320,470	1,224,942
Repurchase agreements - long term
Liabilities:
Outstanding Face Amount	137,857	35,813
Fair Value	137,857	35,813
Revolving credit facility
Liabilities:
Outstanding Face Amount	100,000
Fair Value	$ 100,000
Fair Value Assumptions:
Period of short interest rate reset risk	30 days
Revolving credit facility | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 100,000
Amortized Cost Basis	100,000
Fair Value	$ 100,000
Fair Value Assumptions:
Weighted average yield %	3.69%
Weighted Average Remaining Maturity/Duration	4 months 13 days
Mortgage loan financing
Liabilities:
Outstanding Face Amount	$ 573,435	540,764
Fair Value	610,789	557,841
Mortgage loan receivable financing
Liabilities:
Outstanding Face Amount	63,177
Fair Value	63,177
Borrowings from the FHLB
Liabilities:
Outstanding Face Amount	1,844,700	1,856,700
Fair Value	1,857,780	1,861,584
Senior unsecured notes
Liabilities:
Outstanding Face Amount	563,872	619,555
Fair Value	552,794	591,357
Mortgage loan receivables held for investment, at amortized cost
Assets:
Outstanding Face Amount	1,654,554	1,749,556
Fair Value	$ 1,664,661	1,756,774
Fair Value Assumptions:
Period of short interest rate reset risk	30 days
Mortgage loan receivables held for sale
Assets:
Outstanding Face Amount	$ 783,441	571,638
Fair Value	812,655	582,277
FHLB stock
Assets:
Outstanding Face Amount	77,915	77,915
Fair Value	77,915	$ 77,915
Mortgage loan receivables
Fair Value Assumptions:
Period of short interest rate reset risk		30 days
Recurring
Assets:
Fair Value	2,651,213	$ 2,410,038
Recurring | Repurchase agreements - short term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	1,320,470	1,224,942
Amortized Cost Basis	1,320,470	1,224,942
Fair Value	$ 1,320,470	$ 1,224,942
Fair Value Assumptions:
Weighted average yield %	1.18%	1.67%
Weighted Average Remaining Maturity/Duration	2 months 4 days	5 months 4 days
Recurring | Repurchase agreements - long term | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 137,857	$ 35,813
Amortized Cost Basis	137,857	35,813
Fair Value	$ 137,857	$ 35,813
Fair Value Assumptions:
Weighted average yield %	1.73%	1.87%
Weighted Average Remaining Maturity/Duration	1 year 11 months 15 days	1 year 4 months 24 days
Recurring | Mortgage loan financing | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	$ 573,435	$ 540,764
Amortized Cost Basis	575,533	544,663
Fair Value	$ 610,789	$ 557,841
Fair Value Assumptions:
Weighted average yield %	4.84%	4.86%
Weighted Average Remaining Maturity/Duration	7 years 3 months 29 days	7 years 11 months 4 days
Recurring | Mortgage loan receivable financing | Sales Proceeds
Liabilities:
Outstanding Face Amount	$ 63,177
Amortized Cost Basis	63,177
Fair Value	63,177
Recurring | Borrowings from the FHLB | Discounted Cash Flow
Liabilities:
Outstanding Face Amount	1,844,700	$ 1,856,700
Amortized Cost Basis	1,844,700	1,856,700
Fair Value	$ 1,857,780	$ 1,861,584
Fair Value Assumptions:
Weighted average yield %	1.05%	0.84%
Weighted Average Remaining Maturity/Duration	2 years 4 months 20 days	1 year 5 months 1 day
Recurring | Senior unsecured notes | Broker quotations, pricing services
Liabilities:
Outstanding Face Amount	$ 563,872	$ 619,555
Amortized Cost Basis	559,274	612,605
Fair Value	$ 552,794	$ 591,357
Fair Value Assumptions:
Weighted average yield %	6.67%	6.65%
Weighted Average Remaining Maturity/Duration	2 years 9 months 21 days	3 years 7 months 9 days
Recurring | Nonhedge derivatives
Liabilities:
Derivative liability face amount	$ 1,035,500	$ 374,200
Fair Value	9,368	5,504
Recurring | Nonhedge derivatives | Counterparty quotations
Liabilities:
Derivative liability face amount	1,035,500	374,200
Fair Value	$ 9,368	$ 5,504
Fair Value Assumptions:
Weighted Average Remaining Maturity/Duration	2 years 1 month 13 days	3 years 5 months 1 day
Recurring | CMBS
Assets:
Outstanding Face Amount	$ 2,169,241	$ 1,972,492
Fair Value	2,241,393	1,991,506
Recurring | CMBS | Internal model, third-party inputs
Assets:
Outstanding Face Amount	2,169,241	1,972,492
Amortized Cost Basis	2,199,698	1,994,928
Fair Value	$ 2,241,393	$ 1,991,506
Fair Value Assumptions:
Weighted average yield %	2.79%	2.59%
Weighted Average Remaining Maturity/Duration	3 years 21 days	3 years 1 month 24 days
Recurring | CMBS interest-only
Assets:
Outstanding Face Amount	$ 8,626,247	$ 7,436,379
Fair Value	347,367	344,423
Recurring | CMBS interest-only | Internal model, third-party inputs
Assets:
Outstanding Face Amount	8,626,247	7,436,379
Amortized Cost Basis	343,218	348,222
Fair Value	$ 347,367	$ 344,423
Fair Value Assumptions:
Weighted average yield %	3.52%	3.81%
Weighted Average Remaining Maturity/Duration	3 years 1 month 20 days	3 years 4 months 2 days
Recurring | GNMA interest-only
Assets:
Outstanding Face Amount	$ 519,846	$ 632,175
Fair Value	19,356	26,194
Recurring | GNMA interest-only | Internal model, third-party inputs
Assets:
Outstanding Face Amount	519,846	632,175
Amortized Cost Basis	21,444	28,311
Fair Value	$ 19,356	$ 26,194
Fair Value Assumptions:
Weighted average yield %	4.02%	4.26%
Weighted Average Remaining Maturity/Duration	4 years 7 months 27 days	5 years 2 months 19 days
Recurring | GNMA construction securities
Assets:
Outstanding Face Amount		$ 27,091
Fair Value		28,639
Recurring | GNMA construction securities | Internal model, third-party inputs
Assets:
Outstanding Face Amount		27,091
Amortized Cost Basis		27,581
Fair Value		$ 28,639
Fair Value Assumptions:
Weighted average yield %		3.86%
Weighted Average Remaining Maturity/Duration		9 years 3 months 29 days
Recurring | Agency securities
Assets:
Outstanding Face Amount	$ 798
Fair Value	830
Recurring | Agency securities | Internal model, third-party inputs
Assets:
Outstanding Face Amount	798
Amortized Cost Basis	828
Fair Value	$ 830
Fair Value Assumptions:
Weighted average yield %	1.29%
Weighted Average Remaining Maturity/Duration	3 years 5 months 4 days
Recurring | GNMA permanent securities
Assets:
Outstanding Face Amount	$ 40,120	$ 16,249
Fair Value	42,013	16,455
Recurring | GNMA permanent securities | Internal model, third-party inputs
Assets:
Outstanding Face Amount	40,120	16,249
Amortized Cost Basis	40,977	16,685
Fair Value	$ 42,013	$ 16,455
Fair Value Assumptions:
Weighted average yield %	3.85%	3.94%
Weighted Average Remaining Maturity/Duration	10 years 1 month 9 days	5 years 5 months 4 days
Recurring | Mortgage loan receivables held for investment, at amortized cost | Discounted Cash Flow
Assets:
Outstanding Face Amount	$ 1,654,554	$ 1,749,556
Amortized Cost Basis	1,643,035	1,738,645
Fair Value	$ 1,664,661	$ 1,756,774
Fair Value Assumptions:
Weighted average yield %	7.09%	7.56%
Weighted Average Remaining Maturity/Duration	1 year 8 months 12 days	1 year 4 months 17 days
Recurring | Mortgage loan receivables held for sale | Internal model, third-party inputs
Assets:
Outstanding Face Amount	$ 783,441	$ 571,638
Amortized Cost Basis	784,186	571,764
Fair Value	$ 812,655	$ 582,277
Fair Value Assumptions:
Weighted average yield %	4.40%	4.56%
Weighted Average Remaining Maturity/Duration	7 years 6 months 14 days	6 years 2 months 12 days
Recurring | FHLB stock | Put Value
Assets:
Outstanding Face Amount	$ 77,915	$ 77,915
Amortized Cost Basis	77,915	77,915
Fair Value	$ 77,915	$ 77,915
Fair Value Assumptions:
Weighted average yield %	4.25%	3.50%
Recurring | Nonhedge derivatives
Assets:
Derivative asset face amount	$ 241,011	$ 868,700
Fair Value	254	2,821
Recurring | Nonhedge derivatives | Counterparty quotations
Assets:
Derivative asset face amount	241,011	868,700
Fair Value	$ 254	$ 2,821
Fair Value Assumptions:
Weighted Average Remaining Maturity/Duration	1 year 9 months 21 days	8 months 8 days