Fair Value of Financial Instruments - Assets and Liabilities Measured at Fair Value on a Recurring Basis (Details) - USD ($) $ in Thousands	Mar. 31, 2020	Dec. 31, 2019	Dec. 31, 2018
Assets
Mortgage servicing rights	$ 24,328	$ 41,470	$ 64,728
Level 3
Fair Value Measurements
Percentage of level three assets to total assets measured at fair value	82.00%	77.00%
Percentage of level three liabilities to total liabilities measured at fair value	99.00%	99.00%
Recurring basis \| Level 2
Assets
Mortgage loans held for-for-sale	$ 497,605	$ 782,143
Total assets at fair value	497,605	782,143
Liabilities
Derivative liabilities, lending, net		651
Total liabilities at fair value		651
Recurring basis \| Level 3
Assets
Derivatives assets, lending, net	863	7,791
Mortgage servicing rights	24,328	41,470
Securitized mortgage collateral	2,248,813	2,628,064
Total assets at fair value	2,274,004	2,677,325
Liabilities
Securitized mortgage borrowings	2,238,208	2,619,210
Long-term debt	39,632	45,434
Total liabilities at fair value	2,277,840	2,664,644
Recurring basis \| Derivative assets, lending, net \| Interest rate lock commitments. net (IRLCs) \| Level 3
Assets
Total assets at fair value	$ 863	$ 7,800

Represents the percentage of Level 3 assets measured at estimated fair value to total assets measured at fair value.

No definition available.

Name:	imh_FairValueMeasurementWithUnobservableInputsPercentageOfLevelThreeAssetsToTotalAssetsMeasuredAtFairValue
Namespace Prefix:	imh_
Data Type:	num:percentItemType
Balance Type:	na
Period Type:	instant

- Definition

Represents the percentage of Level 3 liabilities measured at estimated fair value to total liabilities measured at fair value.

+ References

No definition available.

+ Details

Name:	imh_FairValueMeasurementWithUnobservableInputsPercentageOfLevelThreeLiabilitiesToTotalLiabilitiesMeasuredAtFairValue
Namespace Prefix:	imh_
Data Type:	num:percentItemType
Balance Type:	na
Period Type:	instant

- Definition

Fair value portion of securitized mortgage borrowings.

+ References

No definition available.

+ Details

Name:	imh_SecuritizedMortgageBorrowingsFairValueDisclosure
Namespace Prefix:	imh_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Fair value portion of securitized mortgage collateral.

+ References

No definition available.

+ Details

Name:	imh_SecuritizedMortgageCollateralFairValueDisclosure
Namespace Prefix:	imh_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value portion of probable future economic benefits obtained or controlled by an entity as a result of past transactions or events.

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 820
-SubTopic 10
-Section 50
-Paragraph 2
-Subparagraph (a)
-URI http://asc.fasb.org/extlink&oid=117815213&loc=d3e19207-110258

+ Details

Name:	us-gaap_AssetsFairValueDisclosure
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_AssetsFairValueDisclosureAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Fair value, after the effects of master netting arrangements, of a financial asset or other contract with one or more underlyings, notional amount or payment provision or both, and the contract can be net settled by means outside the contract or delivery of an asset. Includes assets not subject to a master netting arrangement and not elected to be offset.

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 55
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=99393222&loc=SL20226008-175313

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 825
-SubTopic 10
-Section 50
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=118260190&loc=d3e13433-108611

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 50
-Paragraph 3
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=51824906&loc=SL20225862-175312

+ Details

Name:	us-gaap_DerivativeAssets
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value, after the effects of master netting arrangements, of a financial liability or contract with one or more underlyings, notional amount or payment provision or both, and the contract can be net settled by means outside the contract or delivery of an asset. Includes liabilities not subject to a master netting arrangement and not elected to be offset.

+ References

Reference 1: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 50
-Paragraph 3
-Subparagraph (c)
-URI http://asc.fasb.org/extlink&oid=51824906&loc=SL20225862-175312

Reference 2: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 825
-SubTopic 10
-Section 50
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=118260190&loc=d3e13433-108611

Reference 3: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 210
-SubTopic 20
-Section 55
-Paragraph 10
-URI http://asc.fasb.org/extlink&oid=99393222&loc=SL20226008-175313

+ Details

Name:	us-gaap_DerivativeLiabilities
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Line items represent financial concepts included in a table. These concepts are used to disclose reportable information associated with domain members defined in one or many axes to the table.

+ References

No definition available.

+ Details

Name:	us-gaap_FairValueAssetsAndLiabilitiesMeasuredOnRecurringAndNonrecurringBasisLineItems
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

Fair value of financial and nonfinancial obligations.

+ References

+ Details

Name:	us-gaap_LiabilitiesFairValueDisclosure
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- References

No definition available.

+ Details

Name:	us-gaap_LiabilitiesFairValueDisclosureAbstract
Namespace Prefix:	us-gaap_
Data Type:	xbrli:stringItemType
Balance Type:	na
Period Type:	duration

- Definition

The fair value amount of long-term debt whether such amount is presented as a separate caption or as a parenthetical disclosure. Additionally, this element may be used in connection with the fair value disclosures required in the footnote disclosures to the financial statements. The element may be used in both the balance sheet and disclosure in the same submission.

+ References

No definition available.

+ Details

Name:	us-gaap_LongTermDebtFairValue
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	credit
Period Type:	instant

- Definition

Fair value portion of mortgage loans held-for-sale.

+ References

No definition available.

+ Details

Name:	us-gaap_MortgagesHeldForSaleFairValueDisclosure
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Definition

Fair value of an asset representing net future revenue from contractually specified servicing fees, late charges, and other ancillary revenues, in excess of future costs related to servicing arrangements.

+ References

Reference 1: http://www.xbrl.org/2003/role/disclosureRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 860
-SubTopic 50
-Section 45
-Paragraph 2
-URI http://asc.fasb.org/extlink&oid=6469459&loc=d3e122501-111745

Reference 2: http://fasb.org/us-gaap/role/ref/legacyRef
-Publisher FASB
-Name Accounting Standards Codification
-Topic 860
-SubTopic 50
-Section 50
-Paragraph 3
-Subparagraph (a)(1)
-URI http://asc.fasb.org/extlink&oid=116651436&loc=d3e122625-111746

+ Details

Name:	us-gaap_ServicingAssetAtFairValueAmount
Namespace Prefix:	us-gaap_
Data Type:	xbrli:monetaryItemType
Balance Type:	debit
Period Type:	instant

- Details

Name:	us-gaap_FairValueByFairValueHierarchyLevelAxis=us-gaap_FairValueInputsLevel3Member
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByMeasurementFrequencyAxis=us-gaap_FairValueMeasurementsRecurringMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByFairValueHierarchyLevelAxis=us-gaap_FairValueInputsLevel2Member
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FairValueByAssetClassAxis=imh_DerivativeAssetsLendingMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type:

- Details

Name:	us-gaap_FinancialInstrumentAxis=us-gaap_InterestRateLockCommitmentsMember
Namespace Prefix:
Data Type:	na
Balance Type:
Period Type: