Mortgage-Backed and Credit Risk Transfer Securities - OTTI included in earnings (Details) - USD ($) $ in Thousands	12 Months Ended
Dec. 31, 2019	Dec. 31, 2018	Dec. 31, 2017
Debt Securities, Available-for-sale [Line Items]
Other-than-temporary credit impairment losses	$ 7,731	$ 7,846	$ 11,962
RMBS interest-only securities
Debt Securities, Available-for-sale [Line Items]
Other-than-temporary credit impairment losses	6,707	7,761	11,208
Non-Agency RMBS
Debt Securities, Available-for-sale [Line Items]
Other-than-temporary credit impairment losses	$ 1,024	$ 85	$ 754

The amount by which the fair value of an investment is less than the amortized cost basis or carrying amount of that investment at the balance sheet date and the decline in fair value is deemed to be other than temporary, before considering whether or not such amount is recognized in earnings or other comprehensive income.

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 320
-SubTopic 10
-Section 45
-Paragraph 8A
-URI http://asc.fasb.org/extlink&oid=118954083&loc=SL6284422-111562

Name:	us-gaap_ImpairmentOfInvestments
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	duration