Consolidated Balance Sheets - USD ($) $ in Thousands	Sep. 30, 2020	Dec. 31, 2019
Assets [Abstract]
Cash and cash equivalents	$ 158,897	$ 62,582
Restricted cash	26,006	71,648
Mortgage-backed securities (including pledged of $2,766,132 and $5,024,625, respectively), at fair value	2,995,660	5,188,163
Mortgage loans held for investment (includes 6,920,907 and $8,857 at fair value, respectively); see Note 1	6,921	0
Other Investments and Securities, at Cost	0	9,405
Receivable for securities sold	1,145	0
Derivative assets	4,266	4,290
Accrued interest receivable	15,340	26,209
Other assets, net	6,804	8,307
Total assets	3,215,039	5,370,604
Liabilities:
Repurchase agreements	2,594,683	4,752,348
Payable for unsettled securities	190	6,180
Non-recourse collateralized financing	645	2,733
Derivative liabilities	5,164	974
Accrued interest payable	1,059	15,585
Accrued dividends payable	5,755	6,280
Other liabilities	3,990	3,516
Total liabilities	2,611,486	4,787,616
Stockholders' Equity Attributable to Parent [Abstract]
Preferred stock, par value $0.01 per share; 50,000,000 shares authorized; 7,248,330 and 6,788,330 shares issued and outstanding, respectively ($181,208 and $169,708 aggregate liquidation preference, respectively)	174,709	162,807
Common stock, par value $0.01 per share, 90,000,000 shares authorized; 23,145,238 and 22,945,993 shares issued and outstanding, respectively	231	229
Additional paid-in capital	859,089	858,347
Accumulated other comprehensive income	88,729	173,806
Accumulated deficit	(519,205)	(612,201)
Total shareholders’ equity	603,553	582,988
Total liabilities and shareholders’ equity	$ 3,215,039	$ 5,370,604