Comments re: SR-NASD-2002-21 and SR-NYSE-2002-09


Joe Gatto, Founder and CEO
StarMine Corporation
715 Bryant Street, Suite 100
San Francisco, CA 94107


I am the founder and CEO of StarMine Corporation whose clients include leading professional money management firms and sell-side research firms. We also provide limited analyst performance information for free to the general public on our web site:

StarMine's two-fold mission remains the same since inception:

1. To objectively measure the performance of equity analysts (the accuracy of their earnings estimates and the profitability of their recommendations), and

2. To provide professional investors with tools and analyses to profit from variance among analyst track records and current opinions.

More information is available at


As a vendor of analyst performance information StarMine may profit from the suggestions I outline; however, we may profit an equal amount from the proposals as currently drafted. For example, StarMine maintains a database of the historical analyst recommendations (through arrangement with Thomson Financial/First Call), the historical price data (from Reuters), and already includes similar price/recommendation charts in its products. Moreover we provide a hosted solution where clients could generate a chart on the fly from Word or other research-authoring environments to allow relatively easy integration into a research report.

Scope of Comments

We agree with many of the comments expressed by the SIA, and those made public on the SEC website, but will not repeat those here. While the problems addressed are real, we echo concerns about the current proposals: of burdensome cost and questionable benefit. That said, I will focus on just the set of issues we feel most competent to comment on, namely, those that relate to analyst past performance-disclosure of and tying compensation to.

Compensation Conflicts

The problems the proposals attempt to address are: that there is growing concern that analysts are sometimes dishonest (usually phrased more euphemistically as "unduly optimistically biased"). That analysts sometimes publish a positive research opinion (such as a Strong Buy or a Buy) when they hold a contrary opinion about the company's prospects. That this dishonest gap between what they believe and what they publish stems from overwhelming economics due to their firm's current or prospective investment banking relationships with the company.

The proposal fails to neutralize the economic incentives of dishonesty

The proposed regulation would prohibit tying analyst compensation to specific investment banking activity. Yet, by allowing an analyst to participate in bonuses related generally to investment banking activity, the analyst would still be incentivized to favorably promote investment-banking clients. If the analyst endeavors to win the banking business, the firm profits and the analyst profits. With continued dominance of compensation, specific or general, from investment banking, the potent economic incentives for dishonesty may persist. The detailed pages of proposed disclosure don't eliminate this very real and very strong economic reality.

An alternative that would neutralize the incentive for dishonesty

Suppose that a member firm would represent (and subject to audit) that it maintained an analyst monitoring and compensation procedure that assured that an analyst's potential and realized compensation from investment banking activity must be less than the analyst's potential and actual compensation from Research Quality Factors. Those Research Quality Factors could include buy-side "votes", analyst's performance in surveys such as Institutional Investor or Greenwich Associates, and perhaps objective measurements of performance including the accuracy of their earnings estimates and/or profitability of their buy/sell/hold opinions, provided, for example, by StarMine.

To be sure, details would have to be worked out (including gaming of future "base" salaries), but is it not necessary (and perhaps sufficient) to attack the problem at the root: economic incentives for an analyst to say one thing while believing something else? And would not assurance of a relative cap on all investment-banking related compensation fix that very root problem? If an analyst had more to gain from performance of her opinions along with perceived customer service (that the qualitative polls like II and Greenwich measure), than to gain from investment banking activity, wouldn't the incentive for dishonesty disappear?

Our proposal might be controversial; however, suppose that the NASD/NYSE proposals were amended to allow an optional "out" from some of the more expensive and detailed disclosures for a firm who agreed to our proposal. Then the choice would be up to the firm. Either subject itself to the expensive tracking and "affiliation" and "ownership" disclosure requirements proposed, or take an alternative path. If our proposed "out" were to be adopted by firms, then the desired result (removal of conflict) would have been achieved with greater effectiveness (our opinion) at less cost (in the broker's rational opinion, not ours) than the current proposal. And if no firm chose our proposed "out", then no harm has been done by including it as an option.

But our proposal should not be seen as controversial. The SIA itself in its Best Practices for Research, published June 2001, stated that analyst compensation should be tied at least in part to the performance of analyst's recommendations. While this is indeed "best" practice, our experience with sell-side research departments shows that it is not common practice. A notable exception is Merrill Lynch who has recent become the first firm to announce its intention to do so.

Disclosure of Past Performance

Again, StarMine already produces historical price/recommendation charts and makes them available to our clients. As such, we would likely profit from this proposal since many firms have indicated interest in contracting us to provide these charts on a service basis. Nonetheless, we question the appropriateness of the chart.

Price chart conceptual flaw: focus on single stock

The price chart requirement is flawed in that it only requires information about a single stock.

Several norms including industry practice, portfolio theory, "no touting" guidelines, and AIMR performance presentation guidelines all emphasize that one may not highlight a subset of stock purchases or recommendations for purposes of self-promotion. One must show the "whole picture". If this rule is so important for promotion, why does it not hold for disclosure contemplated by this proposal?

The difficulty and variability of predicting stock price movements dictate that it is meaningless to measure a professional's performance on a single stock, whether that professional is a portfolio manager or an analyst.

It would be more valuable, more meaningful, more statistically significant, and consistent with other industry practices to report the analyst's full and overall track record instead of performance of recommendation(s) on a single stock. Two alternatives are proposed in the next section.

Modern portfolio theory shows that most of the variance of a stock price is explained by its correlation to market and industry moves (systematic risk). Stock-specific price movements have been described with statistical appropriateness as ripples riding on a wave on top of a tide level. Should an analyst's stock opinion (or worse yet, price target) be held to account for market and industry fluctuations, or merely to the relative price movements within an industry?

Two Alternatives to the single-stock price chart

1. A single metric that captures the past performance of all an analyst's recommendations

2. A table displaying historical distribution of and performance of analyst's recommendations

A problem has been raised by some research firms that in multi-stock or industry-roundup reports by analysts, the number of charts required would add tremendously to report length. Requiring, instead, a single metric, or one table with a relevant, more useful, summary of all of an analyst's ratings would require the death of fewer trees, and more importantly, yield more relevant insight as to the distribution of an analysts opinions and the performance thereof.

1. A single overall performance measure-Value Add

The most important and most read page in StarMine Monitor (sell-side product for measuring the performance of a firm's analysts) is the one that summarizes an analyst's performance stock-by-stock and overall. Here is an example for Bryan Maher, one of the best performing stock pickers in 2001:

We present this as information to readers so they know what is possible (or in this case, already built). We are not proposing this level of detail in disclosure, but present it as a real world example to illustrate the two proposed approaches.

This table above displays one row for each stock. In the first row in the Summary by Stock section, we see that Maher followed Four Seasons Hotel (FS) for the entire year (i.e., Coverage = 100%). We further see that when Maher had a Buy on FS, it rose 47%, and when he had a Hold on FS, it was down 58%. At the bottom of the Summary by Stock table, we tally the performance of stocks by recommendation level. His Strong Buys were up 12%, his Buys were down 15%, and his Holds were down 58%.

Maher rates as an outstanding stock picker. Why? The stocks to which he was assigned were down an average of 15% during 2001, yet, a simulated portfolio of his picks (owning his "buys", owning twice as much of each "strong buy", owning cash instead of each "hold" and shorting any sell or strong sell (he had none), would have provided an "Active Return" of 20%. We call the difference between 20% gain (his active return or simulated portfolio return) and -15% (his benchmark or "coverage return") his Value Add of 35%. In this example the return corresponds to the calendar year 2001, but the same approach can be applied to a longer or shorter time horizon.

Similar but alternative approaches could be used and could reflect market cap weighting, adjustment for risk or other variations, but the essence of the Value Add approach is to provide a single number that compares a simulated portfolio based on an analyst's recommendations to the average performance of the stocks the analyst covered. The result is a fair comparison to the most appropriate benchmark (the stocks he covers), and a single number that captures his overall performance.

The benefits of Value Add are that it represents a single number representative of the performance of the analyst's entire coverage list, not just performance on a single stock, and that the number is instantly understandable and need not be deciphered like a price chart.

2. Alternative to single-stock chart: Summary by Rating Level

Value Add is the single best number to represent an analyst's performance over all her picks. However, next best is to provide a summary of performance of stocks while at each recommendation level. In the example above, Maher's Strong Buys were up 12%, his Buys down 15% and holds down 58%. These numbers give the reader good insight into the meaning and effectiveness of his ratings. (Not so good if you take a "hold" at face value, but excellent considering that it is the most negative rating that he employed.)

The page above would be too much detail for a research report. Below, however, is a suggestion that brings together the idea of (h) (5) (b) (disclosure of percentage of companies at various ratings levels, a good idea) with a concise summary of analyst's distribution of ratings levels (both overall and for the stock in question)

    Stock Stock Analyst's 21 stocks Firm
    Proportion Return Proportion Return Proportion
1 Strong Buy     8% 12% 25%
2 Buy 30% 47% 40% -15% 40%
3 Hold 70% -58% 52% -58% 30%
4 Sell         5%
5 Strong Sell          
  No Rating          
  Simple return of stock(s) -37%   -15%  

From the chart, the reader could easily answer the most relevant questions:

1. How often does the analyst assign this rating? We see from the third column that Maher had 8% Strong Buys, 40% Buys, and 52% Holds, on a time and stock weighted basis. For this particular stock, we see from column 1 that he rated it a "buy" for 30% of the past 12 months, and a "hold" the other 70% of the year.

2. How often does this firm assign this rating? In this example, the numbers are illustrative only and shown in the far right column.

3. How well have stocks to which he assigned this rating performed? We see from the fourth column that his Strong Buys rose 12%, his Buys were down 15%, and his Holds were down 58%.

4. How have the stocks he covered performed on average? The stocks that he was assigned to (most analysts at major sell side firms are assigned most of their coverage stocks, they don't get to choose them) were down 15%. This information puts the previous information in context. His Strong Buys (up 12%) significantly outperformed and his holds (down 58%) significantly under performed the average return of the stocks he followed (which were down 15% on average).

About Rating Schemes: prevailing approaches

The point of this section is to identify and categorize three very different approaches that are currently being used on the Street. Any proposal should be mindful of the implications of such policy in the context of these different approaches.

Three different approaches

1. Buy/Sell/Hold (zero relative)

Used by firms like Prudential, this approach rates a stock relative to zero return. Buys are expected to return a certain amount and Sells are expected to decline a certain amount.

2. Industry or Sector Relative

Some firms use a system with levels like "sector outperform", "sector perform", "sector under perform". A stock that was down 10% (in an industry that was down 30%) would have been a great "sector outperform" call, but a poor "buy" call. The interpretation is clear, yet different from the other two approaches in this section. Morgan Stanley's new "Overweight", "Equal Weight", and "Underweight" approach falls in this category. From a brief by MS on March 18, 2002 explaining their new system, "Equal weight is very different [from the "hold" of our old system]: it means that the analyst expects the total return for this stock to approximate the average (equally weighted) total return for his or her coverage universe on a risk-adjusted basis over the next 12-18 months."

3. Market Relative

Some firms use a system with terms like "Market Outperform", Market Perform, and Market Under Perform. The meanings here are clear, yet clearly different from the other two approaches outlined immediately above.

Should not force firm to map ratings to "buy" or "sell"

After allowing firms to use whatever system they choose, and requiring such system to be consistent with plain English meanings of the definitions the proposal confuses matters by proposing that those terms are mapped to buy, hold/neutral, and sell.

The above example table could just as well as used the designations of A, B, C, D, and E:

Why not display the proportions of whatever ratings levels are used. For example, when Morgan Stanley published their new system, they announced that their "overweight", "equal weight", and "underweight" ratings were distributed 33, 45, and 22% respectively at that time.

However, the same report (and associated press releases) vehemently express that an "underweight" rating does not necessarily mean sell. "There is no one to one correspondence." An "underweight" could mean the stock may appreciate especially if the industry call is positive. Conversely, an "overweight" position may be consistent with a falling stock price if the industry call is negative.

Such accounting might make sense if the only approach were zero-relative. Given the increasing popularity of market and industry-relative rating schemes, this particular proposal (forcing a mapping to buy, sell, and hold) makes no sense.

Yes, show the distribution of returns (as currently proposed, or better yet in the table we propose above), but allow the broker to do so in their native language. Also require that they rank the ratings (i.e., overweight is more positive than equal weight which is more positive than underweight; or A is more positive than B, etc.). Those two provisions are powerful and sufficient with out requiring a clumsy, arbitrary, and inappropriate mapping to buy, hold, and sell.

If the Price Chart is required

While we believe that the price chart is not as useful as the proposed table at indicating analyst past performance, we offer some comments on its details in case that provision is approved.

Plot consensus recommendation for relative comparison.

By plotting the consensus (average) recommendation of all brokers following the stock, the reader would be able to put the analyst's opinion in context. A strong buy rating is quite optimistic if all other analysts are at a "hold"; conversely, a strong buy rating is not so extraordinary when most other analysts also have buys and strong buys.

Chart interpretation is difficult. Allow companion table?

The charts can be hard to read. Viewing 3 years of stock prices on a single chart leads to information compression.

To facilitate interpretation, firms should be allowed to call out performance by using in addition, or instead, a table of recommendation changes for the stock which included the date of an initiation/change, the new recommendation, the closing price on that day, the length of time the recommendation was maintained, and the Total Return over the time period held expressed as a percentage. This is a standard feature of our product, but one firm was worried that this might run afoul of "no touting" guideline. (Guidance relative to that guideline would help firms.)

The graphic below summarizes recommendations for Four Seasons for the same analyst highlighted above. In this plot, extracted from StarMine Monitor, a web-based application sold to research firms to measure the performance of their analysts, the stock price is shown in gray, in the background, and plotted on the right axis; Maher's recommendations are plotted in the blue line against the left axis. The consensus recommendation is shown in bronze. For many readers, the table is easier to interpret than the chart, since the performance calculations are already tallied.

"Price" plotted should be "Total Return" including dividends and distributions.

We recommend that the "price" that is plotted be a Total Return plot including dividends and distributions. E.g., if the stock traded at $100 per share on Jan 14, paid a dividend of $10/share on Jan 15, and closed Jan 30 at $100, the price should be plotted as $100 on Jan 30, but reduced to approximately $91 on Jan 14. The plot would thereby show the shareholder value increasing by 10% to from Jan 14 to Jan 30 to reflect the dividend paid.

Should/may price chart include a benchmark price?

For firms who employ an industry-relative or market-relative scheme, plotting the price relative to the appropriate index (industry or market or analyst's coverage list) might align the interpretation closer to intention. The proposal should spell out the appropriateness of this enhancement.

Comments on Data Sources

Two sources of data for ratings

Internal databases

Many firms do not have well-maintained historical databases of analyst ratings.

Databases available from Research Aggregators

Each broker distributes research to aggregators such as Thomson First Call and Multex. The aggregators, at the direction of brokers, standardize ratings on a 1-5 scale, so as to facilitate comparison of ratings on a stock among different brokers.

StarMine, and other potential third party providers of tools to support compliance with the current proposal, would naturally make use of data from aggregators when appropriate. Since the aggregators standardize levels and data delivery means, using such data would allow a solution built, by a third party like StarMine, for one broker to be standardized and provided to another broker, thus reducing industry-wide cost of implementation of systems required for compliance.

Treatment of non-unique mappings

While firms vary in the language used to describe their levels of recommendations, a standard 1-5 mapping currently does exist. The broker dictates to the aggregators how verbal ratings map to numerical levels.

Not all brokers use all 5 levels. For example, a firm with a simple rating system which allows only a "buy", "hold", or "sell" may map these ratings to 2, 3, and 4, respectively, not utilizing 1s or 5s.

More importantly, in some cases, a broker will have two ratings that map to the same numerical standardized rating. E.g., a single firm may map "long term attractive" and "accumulate" both to a "2"

First Call does not preserve and therefore does not redistribute degrees of distinction within a numerical level. That is, the distinction between "long term attractive" and "accumulate" could not be made by StarMine or by other third parties that relied on such aggregated data.

Requiring a firm to distinguish between these two levels of "2" in a chart would mean that they would be unable to turn to third party standardized, lower cost, solutions as will be provided by StarMine, if the price chart proposal is finally adopted.

We suggest that firms who have such non-unique mappings of ratings to numerical scales be allowed to combine the ratings for purposes of the Price Chart disclosure proposed in (h) (5) (b) as long as the names of the two ratings that share the number are called out near the chart.

Alternative Disclosure Locations

Again, a concern of many firms is the impact on the length of research reports (and/or monthly research summaries) that all the disclosures, including price charts, would cause.

Above, I proposed a compensation-based approach that would remove the need for the disclosures.

As an alternative (which does not save the cost of monitoring, but does remove the clumsiness factor from a report), a simple idea here is to allow the firm, at its option, to publish on the research report the web address of a web site accessible by the reader of the report which contains all the appropriate disclosure material.

This approach might also get around the TV appearance problem. A broadcast featuring an analyst would not likely itemize detailed disclosures on air, but it is reasonable to think that they would show the web address on screen while the analyst is being interviewed.

This short hand pointer to disclosure statements could also be used in news releases about a recommendation or in morning notes or in communications with clients via email.

Thank you for consideration of these comments.


Joe Gatto