Fair Value of Financial Instruments - Fair value of Financial Instruments Included in the Consolidated Financial Statements (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Assets
Mortgage servicing rights	$ 24,328	$ 41,470	$ 64,728
Carrying Amount
Assets
Cash and cash equivalents.	80,192	24,666
Restricted cash	11,950	12,466
Mortgage loans held for-for-sale	497,605	782,143
Mortgage servicing rights	24,328	41,470
Derivatives assets, lending, net	863	7,791
Securitized mortgage collateral	2,248,813	2,628,064
Liabilities
Warehouse borrowings	501,086	701,563
MSR financings	15,000
Convertible notes	24,999	24,996
Long-term debt	39,632	45,434
Securitized mortgage borrowings	2,238,208	2,619,210
Derivative liabilities, lending, net		651
Estimated Fair Value \| Level 1
Assets
Cash and cash equivalents.	80,192	24,666
Restricted cash	11,950	12,466
Estimated Fair Value \| Level 2
Assets
Mortgage loans held for-for-sale	497,605	782,143
Liabilities
Warehouse borrowings	501,086	701,563
Derivative liabilities, lending, net		651
Estimated Fair Value \| Level 3
Assets
Mortgage servicing rights	24,328	41,470
Derivatives assets, lending, net	863	7,791
Securitized mortgage collateral	2,248,813	2,628,064
Liabilities
MSR financings	15,000
Convertible notes	24,999	24,996
Long-term debt	39,632	45,434
Securitized mortgage borrowings	$ 2,238,208	$ 2,619,210

Fair value portion of restricted cash.

No definition available.

Name:	imh_RestrictedCashFairValueDisclosure
Namespace Prefix:	imh_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value portion of securitized mortgage borrowings.

+ References

No definition available.

+ Details

Name:	imh_SecuritizedMortgageBorrowingsFairValueDisclosure
Namespace Prefix:	imh_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Fair value portion of securitized mortgage collateral.

+ References

No definition available.

+ Details

Name:	imh_SecuritizedMortgageCollateralFairValueDisclosure
Namespace Prefix:	imh_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value portion of warehouse borrowings.

+ References

No definition available.

+ Details

Name:	imh_WarehouseBorrowingsFairValueDisclosure
Namespace Prefix:	imh_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_AssetsFairValueDisclosureAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Fair value portion of currency on hand as well as demand deposits with banks or financial institutions. Includes other kinds of accounts that have the general characteristics of demand deposits. Also includes short-term, highly liquid investments that are both readily convertible to known amounts of cash and so near their maturity that they present insignificant risk of changes in value because of changes in interest rates.

+ References

No definition available.

+ Details

Name:	us-gaap_CashAndCashEquivalentsFairValueDisclosure
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value portion of borrowing which can be exchanged for a specified number of another security at the option of the issuer or the holder, for example, but not limited to, the entity's common stock.

+ References

No definition available.

+ Details

Name:	us-gaap_ConvertibleDebtFairValueDisclosures
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Fair value, after the effects of master netting arrangements, of a financial asset or other contract with one or more underlyings, notional amount or payment provision or both, and the contract can be net settled by means outside the contract or delivery of an asset. Includes assets not subject to a master netting arrangement and not elected to be offset.

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 55
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=99393222&loc=SL20226008-175313

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 825
-SubTopic 10
-Section 50
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=118260190&loc=d3e13433-108611

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 50
-Paragraph 3
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=51824906&loc=SL20225862-175312

+ Details

Name:	us-gaap_DerivativeAssets
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value, after the effects of master netting arrangements, of a financial liability or contract with one or more underlyings, notional amount or payment provision or both, and the contract can be net settled by means outside the contract or delivery of an asset. Includes liabilities not subject to a master netting arrangement and not elected to be offset.

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 50
-Paragraph 3
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=51824906&loc=SL20225862-175312

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 825
-SubTopic 10
-Section 50
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=118260190&loc=d3e13433-108611

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 55
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=99393222&loc=SL20226008-175313

+ Details

Name:	us-gaap_DerivativeLiabilities
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_LiabilitiesFairValueDisclosureAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Fair value portion of contractual arrangement with a lender under which borrowings can be made up to a specific amount at any point in time.

+ References

No definition available.

+ Details

Name:	us-gaap_LinesOfCreditFairValueDisclosure
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

The fair value amount of long-term debt whether such amount is presented as a separate caption or as a parenthetical disclosure. Additionally, this element may be used in connection with the fair value disclosures required in the footnote disclosures to the financial statements. The element may be used in both the balance sheet and disclosure in the same submission.

+ References

No definition available.

+ Details

Name:	us-gaap_LongTermDebtFairValue
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Fair value portion of mortgage loans held-for-sale.

+ References

No definition available.

+ Details

Name:	us-gaap_MortgagesHeldForSaleFairValueDisclosure
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value of an asset representing net future revenue from contractually specified servicing fees, late charges, and other ancillary revenues, in excess of future costs related to servicing arrangements.

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 860
-SubTopic 50
-Section 45
-Paragraph 2
-URI http://asc.fasb.org/extlink&oid=6469459&loc=d3e122501-111745

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 860
-SubTopic 50
-Section 50
-Paragraph 3
-Subparagraph (a)(1)
-URI http://asc.fasb.org/extlink&oid=116651436&loc=d3e122625-111746

+ Details

Name:	us-gaap_ServicingAssetAtFairValueAmount
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Details

Name:	us-gaap_FairValueByMeasurementBasisAxis=us-gaap_CarryingReportedAmountFairValueDisclosureMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByMeasurementBasisAxis=us-gaap_EstimateOfFairValueFairValueDisclosureMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByFairValueHierarchyLevelAxis=us-gaap_FairValueInputsLevel1Member
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByFairValueHierarchyLevelAxis=us-gaap_FairValueInputsLevel2Member
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByFairValueHierarchyLevelAxis=us-gaap_FairValueInputsLevel3Member
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type: