Consolidated Balance Sheets - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019
Assets [Abstract]
Cash and cash equivalents	$ 51,411	$ 62,582
Restricted cash	59,204	71,648
Receivables from Brokers-Dealers and Clearing Organizations	1,503,571	0
Mortgage-backed securities (including pledged of $5,024,625 and $3,511,604, respectively)	3,719,706	5,188,163
Other Investments and Securities, at Cost	7,922	9,405
Derivative assets	3,031	4,290
Accrued interest receivable	23,104	26,209
Other assets, net	6,269	8,307
Total assets	5,374,218	5,370,604
Liabilities:
Repurchase agreements	4,408,106	4,752,348
Payable for unsettled securities	382,371	6,180
Non-Recourse Debt	2,171	2,733
Derivative liabilities	13,640	974
Accrued interest payable	9,606	15,585
Accrued dividends payable	5,424	6,280
Other liabilities	2,293	3,516
Total liabilities	4,823,611	4,787,616
Stockholders' Equity Attributable to Parent [Abstract]
Preferred stock, par value $0.01 per share; 50,000,000 shares authorized; 7,248,330 and 6,788,330 shares issued and outstanding, respectively ($181,208 and $169,708 aggregate liquidation preference, respectively)	174,709	162,807
Common stock, par value $0.01 per share, 90,000,000 shares authorized; 22,981,978 and 22,945,993 shares issued and outstanding, respectively	230	229
Additional paid-in capital	858,203	858,347
Accumulated other comprehensive income	246,778	173,806
Accumulated deficit	(729,313)	(612,201)
Total shareholders’ equity	550,607	582,988
Total liabilities and shareholders’ equity	$ 5,374,218	$ 5,370,604