Consolidated Balance Sheets - USD ($) $ in Thousands	Jun. 30, 2020	Dec. 31, 2019
Assets [Abstract]
Cash and cash equivalents	$ 147,243	$ 62,582
Restricted cash	59,150	71,648
Receivables from Brokers-Dealers and Clearing Organizations	156,047	0
Mortgage-backed securities (including pledged of $5,024,625 and $3,511,604, respectively)	3,496,925	5,188,163
Other Investments and Securities, at Cost		9,405
Derivative assets	7,926	4,290
Accrued interest receivable	16,620	26,209
Other assets, net	6,351	8,307
Total assets	3,897,628	5,370,604
Liabilities:
Repurchase agreements	3,314,991	4,752,348
Payable for unsettled securities	0	6,180
Non-Recourse Debt	1,282	2,733
Derivative liabilities	4,208	974
Accrued interest payable	907	15,585
Accrued dividends payable	5,735	6,280
Other liabilities	3,036	3,516
Total liabilities	3,330,159	4,787,616
Stockholders' Equity Attributable to Parent [Abstract]
Preferred stock, par value $0.01 per share; 50,000,000 shares authorized; 7,248,330 and 6,788,330 shares issued and outstanding, respectively ($181,208 and $169,708 aggregate liquidation preference, respectively)	174,709	162,807
Common stock, par value $0.01 per share, 90,000,000 shares authorized; 23,140,617 and 22,945,993 shares issued and outstanding, respectively	231	229
Additional paid-in capital	858,451	858,347
Accumulated other comprehensive income	81,731	173,806
Accumulated deficit	(547,653)	(612,201)
Total shareholders’ equity	567,469	582,988
Total liabilities and shareholders’ equity	3,897,628	5,370,604
Mortgage Loans on Real Estate, Commercial and Consumer, Net	$ 7,366	$ 8,857