<?xml version="1.0" encoding="us-ascii"?><InstanceReport xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Version>2.4.0.8</Version><ReportLongName>1060 - Disclosure - Impairment of Long-Lived Assets</ReportLongName><DisplayLabelColumn>true</DisplayLabelColumn><ShowElementNames>false</ShowElementNames><RoundingOption /><HasEmbeddedReports>false</HasEmbeddedReports><Columns><Column FlagID="0"><Id>1</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode /><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol /><contextRef><ContextID>D2013Q2YTD</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0001318484</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2013-02-03T00:00:00</PeriodStartDate><PeriodEndDate>2013-08-03T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS /><CurrencyCode /><OriginalCurrencyCode /></MCU><CurrencySymbol /><Labels><Label Key="CalendarSupplement" Id="0" Label="6 Months Ended" /><Label Key="Calendar" Id="1" Label="Aug. 03, 2013" /></Labels></Column></Columns><Rows><Row FlagID="0"><Id>1</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>1</Level><ElementName>ctrn_ImpairmentOfLongLivedAssetsDisclosureAbstract</ElementName><ElementPrefix>ctrn_</ElementPrefix><IsBaseElement>false</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Impairment of Long-Lived Assets</Label></Row><Row FlagID="0"><Id>2</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_AssetImpairmentChargesTextBlock</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terseLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="D2013Q2YTD" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText>&lt;div style="font-size:10.0pt;font-family:Times New Roman;"&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;6. &lt;u&gt;Impairment of Long-Lived Assets&lt;/u&gt;&lt;/font&gt;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;If facts and circumstances indicate that a long-lived asset may be impaired, the carrying value is reviewed. If this review indicates that the carrying value of the asset will not be recovered as determined based on projected undiscounted cash flows related to the asset over its remaining life, the carrying value of the asset is reduced to its estimated fair value.&amp;#160; Non-cash impairment expense related to leasehold improvements and fixtures and equipment at underperforming stores totaled $0.7 million in the twenty-six and thirteen week periods ended August&amp;#160;3, 2013.&amp;#160; There was no impairment expense in the twenty-six weeks ended July&amp;#160;28, 2012.&lt;/font&gt;&lt;/p&gt;
&lt;/div&gt;</NonNumbericText><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>nonnum:textBlockItemType</ElementDataType><SimpleDataType>na</SimpleDataType><ElementDefenition>The entire disclosure for the details of the charge against earnings resulting from the aggregate write down of all assets from their carrying value to their fair value. Disclosure may also include a description of the impaired asset and facts and circumstances leading to the impairment, amount of the impairment loss and where the loss is located in the income statement, method(s) for determining fair value, and the segment in which the impaired asset is reported.</ElementDefenition><ElementReferences>No definition available.</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Impairment of Long-Lived Assets</Label></Row></Rows><Footnotes /><IsEquityReport>false</IsEquityReport><ReportName>Impairment of Long-Lived Assets</ReportName><MonetaryRoundingLevel>UnKnown</MonetaryRoundingLevel><SharesRoundingLevel>UnKnown</SharesRoundingLevel><PerShareRoundingLevel>UnKnown</PerShareRoundingLevel><ExchangeRateRoundingLevel>UnKnown</ExchangeRateRoundingLevel><HasCustomUnits>true</HasCustomUnits><IsEmbedReport>false</IsEmbedReport><IsMultiCurrency>false</IsMultiCurrency><ReportType>Sheet</ReportType><RoleURI>http://www.cititrends.com/role/DisclosureImpairmentOfLongLivedAssets</RoleURI><NumberOfCols>1</NumberOfCols><NumberOfRows>2</NumberOfRows></InstanceReport>
