Mortgage-Backed Securities and other securities - Gross gains and losses (Details) - USD ($) $ in Thousands	3 Months Ended
	Mar. 31, 2020	Mar. 31, 2019
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Net Gain  (Loss)	$ 89,186	$ (5,105)
Agency CMBS
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Proceeds	1,259,032	206,710
Gross Gains	94,307	0
Gross Losses	(6,454)	(4,189)
Net Gain  (Loss)	87,853	(4,189)
Subtotal Agency
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Proceeds	391,436	0
Gross Gains	10,420	0
Gross Losses	(38)	0
Net Gain  (Loss)	10,382	0
Non-Agency RMBS
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Proceeds	51,940	9,000
Gross Gains	1	0
Gross Losses	(8,802)	(829)
Net Gain  (Loss)	(8,801)	(829)
Non-Agency CMBS
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Proceeds	12,702	0
Gross Gains	0	0
Gross Losses	(16)	0
Net Gain  (Loss)	(16)	0
Other securities
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Proceeds	17,746	0
Gross Gains	113	0
Gross Losses	0	0
Net Gain  (Loss)	113	0
Total
Debt Securities, Available-for-sale, Gain (Loss) [Abstract]
Proceeds	1,732,856	215,710
Gross Gains	104,841	0
Gross Losses	(15,310)	(5,018)
Net Gain  (Loss)	$ 89,531	$ (5,018)