<?xml version="1.0" encoding="us-ascii"?><InstanceReport xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Version>2.4.0.8</Version><ReportLongName>1060 - Disclosure - Restricted Stock Units and Long Term Incentive Plan</ReportLongName><DisplayLabelColumn>true</DisplayLabelColumn><ShowElementNames>false</ShowElementNames><RoundingOption /><HasEmbeddedReports>false</HasEmbeddedReports><Columns><Column FlagID="0"><Id>1</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode /><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol /><contextRef><ContextID>D2013Q2YTD</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0001103837</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2013-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2013-06-30T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS /><CurrencyCode /><OriginalCurrencyCode /></MCU><CurrencySymbol /><Labels><Label Key="CalendarSupplement" Id="0" Label="6 Months Ended" /><Label Key="Calendar" Id="1" Label="Jun. 30, 2013" /></Labels></Column></Columns><Rows><Row FlagID="0"><Id>1</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>1</Level><ElementName>us-gaap_CompensationRelatedCostsAbstract</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Restricted Stock Units and Long Term Incentive Plan</Label></Row><Row FlagID="0"><Id>2</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_CompensationRelatedCostsGeneralTextBlock</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terseLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="D2013Q2YTD" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText>&lt;div style="font-size:10.0pt;font-family:Times New Roman;"&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&lt;b&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt; FONT-WEIGHT: bold;" size="2"&gt;6. Restricted Stock Units and Long Term Incentive Plan&lt;/font&gt;&lt;/b&gt;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&lt;b&gt;&lt;i&gt;&lt;font style="FONT-STYLE: italic; FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt; FONT-WEIGHT: bold;" size="2"&gt;Restricted Stock Units&lt;/font&gt;&lt;/i&gt;&lt;/b&gt;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 0.2in; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;The Company recorded $84,000 and $78,000 of compensation expense associated with restricted stock units (&amp;#8220;RSUs&amp;#8221;) during the three months ended June&amp;#160;30, 2012 and 2013, respectively, which have been included in selling, general and administrative expense on the accompanying condensed consolidated statements of operations. The Company recorded $155,000 and $124,000 of compensation expense associated with restricted stock units during the six months ended June&amp;#160;30, 2012 and 2013, respectively, which have been included in selling, general and administrative expense on the accompanying condensed consolidated statements of operations.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 0.2in; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 0.25in; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;The closing price of a share of the Company&amp;#8217;s Common Stock, which is used to calculate the period end RSU liability, was $1.85 and $2.47 on December&amp;#160;31, 2012, and June&amp;#160;30, 2013, respectively. As of December&amp;#160;31, 2012 and June&amp;#160;30, 2013, there was unrecognized compensation cost, related to non-vested RSUs granted to the Company&amp;#8217;s directors, in the amount of $99,000 and $62,000, respectively, using the aforementioned stock prices. Actual compensation costs recognized in future periods may vary based upon fluctuations in stock price and forfeitures.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 0.25in; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 0.2in; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;The Company issued no cash settlements related to the RSUs during the three and six months ended June&amp;#160;30, 2012 and 2013, respectively.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 0.2in; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&lt;b&gt;&lt;i&gt;&lt;font style="FONT-STYLE: italic; FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt; FONT-WEIGHT: bold;" size="2"&gt;Long Term Incentive Plan&lt;/font&gt;&lt;/i&gt;&lt;/b&gt;&lt;/p&gt;
&lt;p style="MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;In the first quarter of 2010, the Company entered into Long Term Incentive Compensation Agreements (&amp;#8220;LTI Agreements&amp;#8221;) granting incentive compensation ranging from $25,000 to $536,000 per employee.&amp;#160; The 50% Employment Awards vested and were settled in cash on August&amp;#160;31, 2012, in the aggregate amount of $1.2 million.&amp;#160; A portion of the Performance Awards vested on December&amp;#160;31, 2012, and were settled during the first quarter of 2013 in the aggregate amount of $640,000; the remainder did not vest.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;In the second quarter of 2011, the Company entered into LTI Agreements granting incentive compensation ranging from $23,000 to $550,000 per employee.&amp;#160; Each of the 50% Employment Awards will vest on August&amp;#160;31, 2013, subject to continued employment, and will be settled in cash by September&amp;#160;30, 2013.&amp;#160; Each of the 50% Performance Awards will vest on December&amp;#160;31, 2013, subject to the Company&amp;#8217;s achievement of financial performance criteria, and will be settled in cash by the later of January&amp;#160;30, 2014, or 15 days after the Company issues its audited financials for 2013, but no later than March&amp;#160;15, 2014. Each award is also subject to earlier pro rata settlement as provided in each LTI Agreement.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;In the first quarter of 2012, the Company entered into LTI Agreements granting incentive compensation ranging from $22,000 to $652,000 per employee.&amp;#160; Each of the 50% Employment Awards will vest on August&amp;#160;31, 2014, subject to continued employment, and will be settled in cash by September&amp;#160;30, 2014.&amp;#160; Each of the 50% Performance Awards will vest on December&amp;#160;31, 2014, subject to the Company&amp;#8217;s achievement of financial performance criteria, and will be settled in cash by the later of January&amp;#160;30, 2015, or 15 days after the Company issues its audited financials for 2014, but no later than March&amp;#160;15, 2015. Each award is also subject to earlier pro rata settlement as provided in each LTI Agreement.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;In the first quarter of 2013, the Company entered into LTI Agreements granting incentive compensation ranging from $20,000 to $680,000 per employee.&amp;#160; Each of the 40% Employment Awards, subject to continued employment, and each of the 60% Performance Awards, subject to the Company&amp;#8217;s achievement of financial performance criteria, will vest on December&amp;#160;31, 2015, and be settled in cash the later of January&amp;#160;30, 2016, or 15 days after the Company issues its audited financials for 2015, but no later than March&amp;#160;15, 2016. Each award is also subject to earlier pro rata settlement as provided in each LTI Agreement.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;During the third quarter of 2012, the Company entered into LTI Agreements granting $20,000 incentive compensation to each independent director, subject to continued membership on the board. The related financial performance criteria for 2012 were achieved. On December&amp;#160;31, 2012, the performance criteria were achieved.&amp;#160; As each independent director must continue to perform service through August&amp;#160;16, 2013, the vesting date, the Company has recognized, and will continue to recognize, expense over the service period. Each award is also subject to earlier pro rata settlement as provided in each LTI Agreement.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 14pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;Additionally, in August&amp;#160;2012, the Company entered into LTI Agreements granting $50,000 incentive compensation to each independent director, subject continued membership on the board through December&amp;#160;31, 2014, the vesting date, and achievement of the financial performance criteria. Each of the awards will be settled in cash by the later of January&amp;#160;30, 2015, or 15 days after the Company issues its audited financials for 2014, but no later than March&amp;#160;15, 2015. Finally, in March&amp;#160;2013, the Company entered into LTI agreements granting $50,000 incentive compensation to each independent director, subject to continued membership on the board through December&amp;#160;31, 2015, the vesting date, and achievement of the financial performance criteria. Each award will be settled in cash by the later of January&amp;#160;30, 2016, or 15 days after the Company issues its audited financials for 2015, but no later than March&amp;#160;15, 2016.&amp;#160; Each award is also subject to earlier pro rata settlement as provided in each LTI Agreement.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-ALIGN: center; MARGIN: 0in 0in 0pt;" align="center"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;Vesting of the 2011, 2012 and 2013 LTI Performance Awards will be determined in accordance with the Company performance criteria concerning adjusted EBITDA and cash flow. Each award is subject to earlier pro rata settlement as provided in the related LTI Agreement.&lt;/font&gt;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&amp;#160;&lt;/p&gt;
&lt;p style="TEXT-INDENT: 13.5pt; MARGIN: 0in 0in 0pt;"&gt;&lt;font style="FONT-FAMILY: Times New Roman; FONT-SIZE: 10pt;" size="2"&gt;In recognition of these LTI Agreements, the accompanying condensed consolidated statements of operations include $739,000 and $966,000 among selling, general and administrative expense for the three months ended June&amp;#160;30, 2012 and 2013, respectively. Likewise, the accompanying condensed consolidated statements of operations include $1.3 million and $1.7 million among selling, general and administrative expense for the six months ended June&amp;#160;30, 2012 and 2013, respectively. Related liabilities of $3.6 million and $4.7 million at December&amp;#160;31, 2012, and June&amp;#160;30, 2013, respectively, are included in accounts payable and accrued liabilities in the accompanying condensed consolidated balance sheets.&lt;/font&gt;&lt;/p&gt;
&lt;/div&gt;</NonNumbericText><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>nonnum:textBlockItemType</ElementDataType><SimpleDataType>na</SimpleDataType><ElementDefenition>The entire disclosure for compensation costs, including compensated absences accruals, compensated absences liability, deferred compensation arrangements and income statement compensation items.  Deferred compensation arrangements may include a description of an arrangement with an individual employee, which is generally an employment contract between the entity and a selected officer or key employee containing a promise by the employer to pay certain amounts at designated future dates, usually including a period after retirement, upon compliance with stipulated requirements. This type of arrangement is distinguished from broader based employee benefit plans as it is usually tailored to the employee. Disclosure also typically includes the amount of related compensation expense recognized during the reporting period, the number of shares (units) issued during the period under such arrangements, and the carrying amount as of the balance sheet date of the related liability.</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 718

 -SubTopic 10

 -Section 50

 -Paragraph 1

 -URI http://asc.fasb.org/extlink&amp;oid=6415400&amp;loc=d3e5047-113901



Reference 2: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 718

 -SubTopic 10

 -Section 50

 -Paragraph 2

 -URI http://asc.fasb.org/extlink&amp;oid=6415400&amp;loc=d3e5070-113901



Reference 3: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 718

 -SubTopic 10

 -Section S99

 -Paragraph 1

 -Subparagraph (SAB TOPIC 14.F)

 -URI http://asc.fasb.org/extlink&amp;oid=6793087&amp;loc=d3e301413-122809



Reference 4: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 715

 -SubTopic 70

 -Section 50

 -Paragraph 1

 -URI http://asc.fasb.org/extlink&amp;oid=6414718&amp;loc=d3e28014-114942



Reference 5: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 715

 -SubTopic 20

 -Section 50

 -Paragraph 1

 -URI http://asc.fasb.org/extlink&amp;oid=21915506&amp;loc=d3e1928-114920



Reference 6: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Statement of Financial Accounting Standard (FAS)

 -Number 123R

 -Paragraph 64, 65

 -LegacyDoc This reference is SUPERSEDED by the Accounting Standards Codification effective for interim and annual periods ending after September 15, 2009.  This reference is included to help users transition from the previous accounting hierarchy and will be removed from future versions of this taxonomy.



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Restricted Stock Units and Long Term Incentive Plan</Label></Row></Rows><Footnotes /><IsEquityReport>false</IsEquityReport><ReportName>Restricted Stock Units and Long Term Incentive Plan</ReportName><MonetaryRoundingLevel>UnKnown</MonetaryRoundingLevel><SharesRoundingLevel>UnKnown</SharesRoundingLevel><PerShareRoundingLevel>UnKnown</PerShareRoundingLevel><ExchangeRateRoundingLevel>UnKnown</ExchangeRateRoundingLevel><HasCustomUnits>true</HasCustomUnits><IsEmbedReport>false</IsEmbedReport><IsMultiCurrency>false</IsMultiCurrency><ReportType>Sheet</ReportType><RoleURI>http://www.hallmarkchannel.com/role/DisclosureRestrictedStockUnitsAndLongTermIncentivePlan</RoleURI><NumberOfCols>1</NumberOfCols><NumberOfRows>2</NumberOfRows></InstanceReport>
