Consolidated Balance Sheets - USD ($) $ in Thousands	Dec. 31, 2020	Dec. 31, 2019
Assets [Abstract]
Cash and cash equivalents	$ 295,602	$ 62,582
Restricted cash	7,077	71,648
Mortgage-backed securities (including pledged of $2,467,859 and $5,024,625, respectively), at fair value	2,596,255	5,188,163
Mortgage loans held for investment (includes $6,264 and $8,857 at fair value, respectively); see Note 3	6,264	0
Other Investments and Securities, at Cost	0	9,405
Receivable for securities sold	150,432	0
Derivative assets	11,342	4,290
Accrued interest receivable	14,388	26,209
Other assets, net	6,394	8,307
Total assets	3,087,754	5,370,604
Liabilities:
Repurchase agreements	2,437,163	4,752,348
Payable for unsettled securities	5	6,180
Non-recourse collateralized financing	111	2,733
Derivative liabilities	1,634	974
Accrued interest payable	1,410	15,585
Accrued dividends payable	5,814	6,280
Other liabilities	8,164	3,516
Total liabilities	2,454,301	4,787,616
Stockholders' Equity Attributable to Parent [Abstract]
Preferred stock, par value $0.01 per share; 50,000,000 shares authorized; 7,248,330 and 6,788,330 shares issued and outstanding, respectively ($181,208 and $169,708 aggregate liquidation preference, respectively)	174,564	162,807
Common stock, par value $0.01 per share, 90,000,000 shares authorized; 23,697,970 and 22,945,993 shares issued and outstanding, respectively	237	229
Additional paid-in capital	869,495	858,347
Accumulated other comprehensive income	80,261	173,806
Accumulated deficit	(491,104)	(612,201)
Total shareholders’ equity	633,453	582,988
Total liabilities and shareholders’ equity	$ 3,087,754	$ 5,370,604