<?xml version="1.0" encoding="us-ascii"?><InstanceReport xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Version>2.4.0.8</Version><ReportLongName>060600 - Disclosure - Restructuring Charges</ReportLongName><DisplayLabelColumn>true</DisplayLabelColumn><ShowElementNames>false</ShowElementNames><RoundingOption /><HasEmbeddedReports>false</HasEmbeddedReports><Columns><Column FlagID="0"><Id>1</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode /><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol /><contextRef><ContextID>c20130101to20130630</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0000872912</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2013-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2013-06-30T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS /><CurrencyCode /><OriginalCurrencyCode /></MCU><CurrencySymbol /><Labels><Label Key="CalendarSupplement" Id="0" Label="6 Months Ended" /><Label Key="Calendar" Id="1" Label="Jun. 30, 2013" /></Labels></Column></Columns><Rows><Row FlagID="0"><Id>1</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>1</Level><ElementName>us-gaap_RestructuringAndRelatedActivitiesAbstract</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Restructuring Charges [Abstract]</Label></Row><Row FlagID="0"><Id>2</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_RestructuringAndRelatedActivitiesDisclosureTextBlock</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terseLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="c20130101to20130630" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText>&lt;div style="font-family: 'Times New Roman', Times, serif; font-size: 10pt;"&gt;&lt;div style="text-align: justify;"&gt;&lt;table cellpadding="0" cellspacing="0" class="DSPFListTable" style="width: 100%; font-family: 'Times New Roman', Times, serif; font-size: 10pt;"&gt;&lt;tr&gt;&lt;td style="width: 18pt; font-family: ''Times New Roman'', Times, serif; font-size: 10pt; vertical-align: top; font-weight: bold; align: right;"&gt;(6)&lt;/td&gt;&lt;td style="text-align: justify; width: auto; font-family: ''Times New Roman'', Times, serif; font-size: 10pt; vertical-align: top; font-weight: bold;"&gt;Restructuring Charges&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div style="text-align: justify; font-family: ''Times New Roman'', Times, serif; margin-left: 18pt; font-size: 10pt;"&gt;During the six months ended June 30, 2013, the Company implemented restructurings of its workforce to better focus the Company&amp;#8217;s organizational structure, increase efficiency and concentrate financial resources on its clinical development program and European commercialization activity. This resulted in a reduction in the Company&amp;#8217;s workforce by 27 employees. As a result of termination benefits given to the impacted employees, the restructuring activities resulted in a total cost of approximately $1.5 million which is reflected on the Condensed Consolidated Statements of Operations in both &amp;#8220;Selling, general and administrative expenses&amp;#8221; and &amp;#8220;Research and development expenses&amp;#8221;, as appropriate. The $1.1 million in accrued severance expenses at June 30, 2013 is included in &amp;#8220;Accrued Expenses&amp;#8221; on the Condensed Consolidated Balance Sheets.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;table align="center" border="0" cellpadding="0" cellspacing="0" style="width: 80%; font-family: 'Times New Roman', Times, serif; font-size: 10pt;"&gt;&lt;tr&gt;&lt;td valign="bottom" style="padding-bottom: 2px; width: 68%; vertical-align: bottom;"&gt;&lt;div style="text-align: left; font-style: italic; text-indent: -7.2pt; font-family: ''Times New Roman'', Times, serif; margin-left: 7.2pt; font-size: 10pt;"&gt;(in thousands)&lt;/div&gt;&lt;/td&gt;&lt;td valign="bottom" style="padding-bottom: 2px; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;td colspan="2" valign="bottom" style="border-bottom: #000000 2px solid; vertical-align: bottom;"&gt;&lt;div style="text-align: center; font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;June 30, 2013&lt;/div&gt;&lt;/td&gt;&lt;td nowrap="nowrap" valign="bottom" style="text-align: left; padding-bottom: 2px; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td valign="bottom" style="background-color: #cceeff; width: 68%; vertical-align: bottom;"&gt;&lt;div style="text-align: left; text-indent: -7.2pt; font-family: ''Times New Roman'', Times, serif; margin-left: 7.2pt; font-size: 10pt;"&gt;Severance and restructuring expenses&lt;/div&gt;&lt;/td&gt;&lt;td valign="bottom" style="background-color: #cceeff; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;td valign="bottom" style="text-align: left; background-color: #cceeff; width: 1%; vertical-align: bottom;"&gt;&lt;div style="font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;$&lt;/div&gt;&lt;/td&gt;&lt;td valign="bottom" style="text-align: right; background-color: #cceeff; width: 9%; vertical-align: bottom;"&gt;&lt;div style="font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;1,470&lt;/div&gt;&lt;/td&gt;&lt;td nowrap="nowrap" valign="bottom" style="text-align: left; background-color: #cceeff; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td valign="bottom" style="padding-bottom: 2px; background-color: #ffffff; width: 68%; vertical-align: bottom;"&gt;&lt;div style="text-align: left; text-indent: -7.2pt; font-family: ''Times New Roman'', Times, serif; margin-left: 16.2pt; font-size: 10pt;"&gt;Restructuring expenses paid by June 30, 2013&lt;/div&gt;&lt;/td&gt;&lt;td valign="bottom" style="padding-bottom: 2px; background-color: #ffffff; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;td valign="bottom" style="border-bottom: #000000 2px solid; text-align: left; background-color: #ffffff; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;td valign="bottom" style="border-bottom: #000000 2px solid; text-align: right; background-color: #ffffff; width: 9%; vertical-align: bottom;"&gt;&lt;div style="font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;(376&lt;/div&gt;&lt;/td&gt;&lt;td nowrap="nowrap" valign="bottom" style="text-align: left; padding-bottom: 2px; background-color: #ffffff; width: 1%; vertical-align: bottom;"&gt;&lt;div style="font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;)&lt;/div&gt;&lt;/td&gt;&lt;/tr&gt;&lt;tr&gt;&lt;td valign="bottom" style="padding-bottom: 4px; background-color: #cceeff; width: 68%; vertical-align: bottom;"&gt;&lt;div style="text-align: left; text-indent: -7.2pt; font-family: ''Times New Roman'', Times, serif; margin-left: 7.2pt; font-size: 10pt;"&gt;Total restructuring expenses accrued as of June 30, 2013&lt;/div&gt;&lt;/td&gt;&lt;td valign="bottom" style="padding-bottom: 4px; background-color: #cceeff; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;td valign="bottom" style="border-bottom: #000000 4px double; text-align: left; background-color: #cceeff; width: 1%; vertical-align: bottom;"&gt;&lt;div style="font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;$&lt;/div&gt;&lt;/td&gt;&lt;td valign="bottom" style="border-bottom: #000000 4px double; text-align: right; background-color: #cceeff; width: 9%; vertical-align: bottom;"&gt;&lt;div style="font-family: ''Times New Roman'', Times, serif; font-size: 10pt;"&gt;1,094&lt;/div&gt;&lt;/td&gt;&lt;td nowrap="nowrap" valign="bottom" style="text-align: left; padding-bottom: 4px; background-color: #cceeff; width: 1%; vertical-align: bottom;"&gt;&amp;#160;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;div style="text-align: justify; font-family: ''Times New Roman'', Times, serif; margin-left: 18pt; font-size: 10pt;"&gt;Of the $1.1 million of severance payments remaining in accruals, approximately $0.9 million will be paid out through June 2014.&lt;/div&gt;&lt;div&gt;&lt;br /&gt;&lt;/div&gt;&lt;/div&gt;</NonNumbericText><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>nonnum:textBlockItemType</ElementDataType><SimpleDataType>na</SimpleDataType><ElementDefenition>The entire disclosure for restructuring and related activities. Description of restructuring activities such as exit and disposal activities, include facts and circumstances leading to the plan, the expected plan completion date, the major types of costs associated with the plan activities, total expected costs, the accrual balance at the end of the period, and the periods over which the remaining accrual will be settled.</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Statement of Financial Accounting Standard (FAS)

 -Number 146

 -Paragraph 20

 -LegacyDoc This reference is SUPERSEDED by the Accounting Standards Codification effective for interim and annual periods ending after September 15, 2009.  This reference is included to help users transition from the previous accounting hierarchy and will be removed from future versions of this taxonomy.



Reference 2: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 420

 -SubTopic 10

 -Section S99

 -Paragraph 2

 -Subparagraph (SAB TOPIC 5.P.4)

 -URI http://asc.fasb.org/extlink&amp;oid=6394695&amp;loc=d3e140904-122747



Reference 3: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 420

 -SubTopic 10

 -Section S99

 -Paragraph 1

 -Subparagraph (SAB TOPIC 5.P.3)

 -URI http://asc.fasb.org/extlink&amp;oid=6394695&amp;loc=d3e140864-122747



Reference 4: http://www.xbrl.org/2003/role/presentationRef

 -Publisher SEC

 -Name Staff Accounting Bulletin (SAB)

 -Number Topic 5

 -Section P

 -Subsection 3, 4



Reference 5: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 420

 -SubTopic 10

 -Section 50

 -Paragraph 1

 -URI http://asc.fasb.org/extlink&amp;oid=6394359&amp;loc=d3e17939-110869



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Restructuring Activity</Label></Row></Rows><Footnotes /><IsEquityReport>false</IsEquityReport><ReportName>Restructuring Charges</ReportName><MonetaryRoundingLevel>UnKnown</MonetaryRoundingLevel><SharesRoundingLevel>UnKnown</SharesRoundingLevel><PerShareRoundingLevel>UnKnown</PerShareRoundingLevel><ExchangeRateRoundingLevel>UnKnown</ExchangeRateRoundingLevel><HasCustomUnits>true</HasCustomUnits><IsEmbedReport>false</IsEmbedReport><IsMultiCurrency>false</IsMultiCurrency><ReportType>Sheet</ReportType><RoleURI>http://delcath.com/role/RestructuringCharges</RoleURI><NumberOfCols>1</NumberOfCols><NumberOfRows>2</NumberOfRows></InstanceReport>
