<?xml version="1.0" encoding="us-ascii"?><InstanceReport xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"><Version>2.4.0.8</Version><ReportLongName>027 - Disclosure - Note 4 - Stockholders' Equity (Details) - Changes in the Carrying Amount of the Non-Controlling Interest</ReportLongName><DisplayLabelColumn>true</DisplayLabelColumn><ShowElementNames>false</ShowElementNames><RoundingOption>In Thousands, unless otherwise specified</RoundingOption><HasEmbeddedReports>false</HasEmbeddedReports><Columns><Column FlagID="0"><Id>1</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><CurrencyCode>USD</CurrencyCode><FootnoteIndexer /><hasSegments>false</hasSegments><hasScenarios>false</hasScenarios><MCU><KeyName /><CurrencySymbol>$</CurrencySymbol><contextRef><ContextID>c3_From1Jan2013To30Jun2013</ContextID><EntitySchema>http://www.sec.gov/CIK</EntitySchema><EntityValue>0001089907</EntityValue><PeriodDisplayName /><PeriodType>duration</PeriodType><PeriodStartDate>2013-01-01T00:00:00</PeriodStartDate><PeriodEndDate>2013-06-30T00:00:00</PeriodEndDate><Segments /><Scenarios /></contextRef><UPS><UnitProperty><UnitID>usd</UnitID><UnitType>Standard</UnitType><StandardMeasure><MeasureSchema>http://www.xbrl.org/2003/iso4217</MeasureSchema><MeasureValue>USD</MeasureValue><MeasureNamespace>iso4217</MeasureNamespace><MeasureDescription /><MeasureType /></StandardMeasure><Scale>0</Scale></UnitProperty></UPS><CurrencyCode>USD</CurrencyCode><OriginalCurrencyCode>USD</OriginalCurrencyCode></MCU><CurrencySymbol>$</CurrencySymbol><Labels><Label Key="CalendarSupplement" Id="0" Label="6 Months Ended" /><Label Key="Calendar" Id="1" Label="Jun. 30, 2013" /></Labels></Column></Columns><Rows><Row FlagID="0"><Id>1</Id><IsAbstractGroupTitle>true</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>1</Level><ElementName>swkhob_ChangesInTheCarryingAmountOfTheNonControllingInterestAbstract</ElementName><ElementPrefix>swkhob_</ElementPrefix><IsBaseElement>false</IsBaseElement><BalanceType>na</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="" UnitID=""><Id>1</Id><IsNumeric>false</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>0</NumericAmount><RoundedNumericAmount>0</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:stringItemType</ElementDataType><SimpleDataType>string</SimpleDataType><IsTotalLabel>false</IsTotalLabel><UnitID>0</UnitID><Label>Changes in the Carrying Amount of the Non-Controlling Interest [Abstract]</Label></Row><Row FlagID="0"><Id>2</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_MinorityInterest</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>credit</BalanceType><PeriodType>instant</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>true</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>periodStartLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="c1_AsOf31Dec2012" UnitID="usd"><Id>1</Id><IsNumeric>true</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>7000000</NumericAmount><RoundedNumericAmount>7000</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode>USD</CurrencyCode><CurrencySymbol>$</CurrencySymbol><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>true</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:monetaryItemType</ElementDataType><SimpleDataType>monetary</SimpleDataType><ElementDefenition>Total of all stockholders' equity (deficit) items, net of receivables from officers, directors, owners, and affiliates of the entity which is directly or indirectly attributable to that ownership interest in subsidiary equity which is not attributable to the parent (that is, noncontrolling interest, previously referred to as minority interest).</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 210

 -SubTopic 10

 -Section S99

 -Paragraph 1

 -Subparagraph (SX 210.5-02.31)

 -URI http://asc.fasb.org/extlink&amp;oid=6877327&amp;loc=d3e13212-122682



Reference 2: http://www.xbrl.org/2003/role/presentationRef

 -Publisher SEC

 -Name Regulation S-X (SX)

 -Number 210

 -Section 02

 -Paragraph 27

 -Article 5



Reference 3: http://www.xbrl.org/2003/role/presentationRef

 -Publisher SEC

 -Name Regulation S-X (SX)

 -Number 210

 -Section 03

 -Paragraph 20

 -Article 7



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>2</UnitID><Label>Balance at</Label></Row><Row FlagID="0"><Id>3</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_NetIncomeLossAttributableToNoncontrollingInterest</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>debit</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>terseLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="c3_From1Jan2013To30Jun2013" UnitID="usd"><Id>1</Id><IsNumeric>true</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>214000</NumericAmount><RoundedNumericAmount>214</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:monetaryItemType</ElementDataType><SimpleDataType>monetary</SimpleDataType><ElementDefenition>Amount of Net Income (Loss) attributable to noncontrolling interest.</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 810

 -SubTopic 10

 -Section 50

 -Paragraph 1A

 -URI http://asc.fasb.org/extlink&amp;oid=18733093&amp;loc=SL4573702-111684



Reference 2: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 810

 -SubTopic 10

 -Section 55

 -Paragraph 4J

 -URI http://asc.fasb.org/extlink&amp;oid=31814832&amp;loc=SL4591551-111686



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>2</UnitID><Label>Add: Income attributable to non-controlling interests</Label></Row><Row FlagID="0"><Id>4</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_MinorityInterestDecreaseFromDistributionsToNoncontrollingInterestHolders</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>debit</BalanceType><PeriodType>duration</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>false</IsEndingBalance><IsReverseSign>true</IsReverseSign><PreferredLabelRole>negatedLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="c3_From1Jan2013To30Jun2013" UnitID="usd"><Id>1</Id><IsNumeric>true</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>-866000</NumericAmount><RoundedNumericAmount>-866</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode /><CurrencySymbol /><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>false</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:monetaryItemType</ElementDataType><SimpleDataType>monetary</SimpleDataType><ElementDefenition>Decrease in noncontrolling interest balance from payment of dividends or other distributions by the non-wholly owned subsidiary or partially owned entity, included in the consolidation of the parent entity, to the noncontrolling interest holders.</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 505

 -SubTopic 10

 -Section S99

 -Paragraph 1

 -Subparagraph (SX 210.3-04)

 -URI http://asc.fasb.org/extlink&amp;oid=27012166&amp;loc=d3e187085-122770



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>2</UnitID><Label>Less: Cash distribution to non-controlling interest</Label></Row><Row FlagID="0"><Id>5</Id><IsAbstractGroupTitle>false</IsAbstractGroupTitle><LabelSeparator>

</LabelSeparator><Level>2</Level><ElementName>us-gaap_MinorityInterest</ElementName><ElementPrefix>us-gaap_</ElementPrefix><IsBaseElement>true</IsBaseElement><BalanceType>credit</BalanceType><PeriodType>instant</PeriodType><IsReportTitle>false</IsReportTitle><IsSegmentTitle>false</IsSegmentTitle><IsCalendarTitle>false</IsCalendarTitle><IsEquityPrevioslyReportedAsRow>false</IsEquityPrevioslyReportedAsRow><IsEquityAdjustmentRow>false</IsEquityAdjustmentRow><IsBeginningBalance>false</IsBeginningBalance><IsEndingBalance>true</IsEndingBalance><IsReverseSign>false</IsReverseSign><PreferredLabelRole>periodEndLabel</PreferredLabelRole><FootnoteIndexer /><Cells><Cell FlagID="0" ContextID="c0_AsOf30Jun2013" UnitID="usd"><Id>1</Id><IsNumeric>true</IsNumeric><IsRatio>false</IsRatio><DisplayZeroAsNone>false</DisplayZeroAsNone><NumericAmount>6348000</NumericAmount><RoundedNumericAmount>6348</RoundedNumericAmount><NonNumbericText /><FootnoteIndexer /><CurrencyCode>USD</CurrencyCode><CurrencySymbol>$</CurrencySymbol><IsIndependantCurrency>false</IsIndependantCurrency><ShowCurrencySymbol>true</ShowCurrencySymbol><DisplayDateInUSFormat>false</DisplayDateInUSFormat></Cell></Cells><ElementDataType>xbrli:monetaryItemType</ElementDataType><SimpleDataType>monetary</SimpleDataType><ElementDefenition>Total of all stockholders' equity (deficit) items, net of receivables from officers, directors, owners, and affiliates of the entity which is directly or indirectly attributable to that ownership interest in subsidiary equity which is not attributable to the parent (that is, noncontrolling interest, previously referred to as minority interest).</ElementDefenition><ElementReferences>Reference 1: http://www.xbrl.org/2003/role/presentationRef

 -Publisher FASB

 -Name Accounting Standards Codification

 -Topic 210

 -SubTopic 10

 -Section S99

 -Paragraph 1

 -Subparagraph (SX 210.5-02.31)

 -URI http://asc.fasb.org/extlink&amp;oid=6877327&amp;loc=d3e13212-122682



Reference 2: http://www.xbrl.org/2003/role/presentationRef

 -Publisher SEC

 -Name Regulation S-X (SX)

 -Number 210

 -Section 02

 -Paragraph 27

 -Article 5



Reference 3: http://www.xbrl.org/2003/role/presentationRef

 -Publisher SEC

 -Name Regulation S-X (SX)

 -Number 210

 -Section 03

 -Paragraph 20

 -Article 7



</ElementReferences><IsTotalLabel>false</IsTotalLabel><UnitID>2</UnitID><Label>Balance at</Label></Row></Rows><Footnotes /><IsEquityReport>false</IsEquityReport><ReportName>Note 4 - Stockholders' Equity (Details) - Changes in the Carrying Amount of the Non-Controlling Interest (USD $)</ReportName><MonetaryRoundingLevel>Thousands</MonetaryRoundingLevel><SharesRoundingLevel>UnKnown</SharesRoundingLevel><PerShareRoundingLevel>UnKnown</PerShareRoundingLevel><ExchangeRateRoundingLevel>UnKnown</ExchangeRateRoundingLevel><HasCustomUnits>true</HasCustomUnits><IsEmbedReport>false</IsEmbedReport><IsMultiCurrency>false</IsMultiCurrency><ReportType>Sheet</ReportType><RoleURI>http://www.swkhold.com/role/ChangesintheCarryingAmountoftheNonControllingInterestTable</RoleURI><NumberOfCols>1</NumberOfCols><NumberOfRows>5</NumberOfRows></InstanceReport>
