RESIDENTIAL MORTGAGE INVESTMENTS - Schedule of Residential Mortgage Investments (Details) - USD ($) $ in Thousands		3 Months Ended	12 Months Ended
		Mar. 31, 2020	Dec. 31, 2019
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 8,222,071	$ 10,844,719
Investment Premiums		235,744	332,196
Amortized Cost Basis		8,457,815	11,176,915
Carrying Amount	[1]	$ 8,503,171	$ 11,222,182
Net WAC	[2]	3.33%	3.46%
Average Yield	[3]	2.49%	2.67%
Agency Securities [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 8,220,605	$ 10,843,186
Investment Premiums		235,727	332,177
Amortized Cost Basis		8,456,332	11,175,363
Carrying Amount	[1]	$ 8,501,688	$ 11,220,630
Net WAC	[2]	3.33%	3.46%
Average Yield	[3]	2.49%	2.67%
Agency Securities [Member] | Fannie Mae/Freddie Mac [Member] | Fixed-Rate [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 74	$ 83
Amortized Cost Basis		74	83
Carrying Amount	[1]	$ 74	$ 83
Net WAC	[2]	6.50%	6.50%
Average Yield	[3]	6.32%	6.34%
Agency Securities [Member] | Fannie Mae/Freddie Mac [Member] | ARMs [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 7,345,360	$ 8,628,656
Investment Premiums		214,992	262,293
Amortized Cost Basis		7,560,352	8,890,949
Carrying Amount	[1]	$ 7,588,267	$ 8,931,789
Net WAC	[2]	3.30%	3.45%
Average Yield	[3]	2.77%	2.66%
Agency Securities [Member] | Ginnie Mae [Member] | ARMs [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 875,171	$ 2,214,447
Investment Premiums		20,735	69,884
Amortized Cost Basis		895,906	2,284,331
Carrying Amount	[1]	$ 913,347	$ 2,288,758
Net WAC	[2]	3.64%	3.53%
Average Yield	[3]	1.30%	2.71%
Residential Mortgage Loans [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 588	$ 633
Investment Premiums		3	4
Amortized Cost Basis		591	637
Carrying Amount	[1]	$ 591	$ 637
Net WAC	[2]	4.27%	4.26%
Average Yield	[3]	2.24%	2.94%
Residential Mortgage Loans [Member] | Fixed-Rate [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 101	$ 106
Investment Premiums			1
Amortized Cost Basis		101	107
Carrying Amount	[1]	$ 101	$ 107
Net WAC	[2]	4.84%	4.84%
Average Yield	[3]	2.01%	2.07%
Residential Mortgage Loans [Member] | ARMs [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 487	$ 527
Investment Premiums		3	3
Amortized Cost Basis		490	530
Carrying Amount	[1]	$ 490	$ 530
Net WAC	[2]	4.15%	4.15%
Average Yield	[3]	2.29%	3.11%
Collateral for Structured Financings [Member]
Financing Receivable, Recorded Investment [Line Items]
Unpaid Principal Balance		$ 878	$ 900
Investment Premiums		14	15
Amortized Cost Basis		892	915
Carrying Amount	[1]	$ 892	$ 915
Net WAC	[2]	7.86%	7.99%
Average Yield	[3]	7.46%	7.51%

[1] Includes unrealized gains and losses for residential mortgage investments classified as available-for-sale.

[2] Net WAC, or weighted average coupon, is the weighted average interest rate of the mortgage loans underlying the indicated investments net of servicing and other fees as of the indicated balance sheet date.  Net WAC is expressed as a percentage calculated on an annualized basis on the unpaid principal balances of the mortgage loans underlying these investments.

[3] Average yield is presented for the quarter then ended and is based on the cash component of interest income expressed as a percentage calculated on an annualized basis on average amortized cost basis (the “cash yield”) less the effects of amortizing investment premiums.  Investment premium amortization is determined using the interest method and incorporates actual and anticipated future mortgage prepayments.