Risk Modeling at the SEC:  The Accounting Quality Model

Speech

Risk Modeling at the SEC:  The Accounting Quality Model

Craig M. Lewis

Chief Economist and Director, Division of Risk, Strategy, and Financial Innovation
U.S. Securities & Exchange Commission

Financial Executives International Committee on Finance and Information Technology

Dec. 13, 2012

Introduction

Thank you so much for inviting me here to speak with you. Before I begin my remarks, I must make clear that the views I express today are mine alone and do not necessarily reflect the views of the Commission or of my colleagues on the Commission Staff.*

The Division of Risk, Strategy and Financial Innovation, or “RSFI”, was formed, in part, to integrate rigorous data analytics into the core mission of the SEC. Often referred to as the SEC’s “think tank,” RSFI consists of highly trained staff from a variety of backgrounds with a deep knowledge of the financial industry and markets.  We are involved in a wide variety of projects across all Divisions and Offices within the SEC and I believe we approach regulatory issues with a uniquely broad perspective. 

Because my Division has a slightly cumbersome name – which is why you might hear us colloquially called “RiskFin” (though I prefer the more inclusive and accurate “RSFI,” as you can see) – today in my remarks I thought I’d focus on one word in our magisterial title:  “Risk.”  Risk, particularly as relates to the financial markets, can be a capacious term, and my Division certainly touches on many of those various meanings.  But we are particularly focused on developing cutting-edge ways to integrate data analysis into risk monitoring. 

To that end, I created RSFI’s Office of Quantitative Research (OQR), which develops custom analytics intended to inform monitoring programs across the SEC. The best way to illustrate OQR’s role at the Commission is by a concrete example. Recently, OQR staff developed a model used by the Division of Enforcement’s Asset Management Unit. For that project, OQR, together with the Office of Compliance, Inspections, and Examinations (OCIE), developed an analytical model that uses performance data to identify hedge fund advisers worthy of further review by either OCIE or the Asset Management Unit. In addition, OQR staff provides ongoing analytical support to and performs ad hoc research at the request of OCIE and Enforcement. A number of successful cases have been brought based on our work, and I believe that this project successfully demonstrates the value of the coordinated application of analytics across Divisions and Offices.

This success has only fed our ambition for what we can do with sophisticated data-driven monitoring programs. 

As you know, the SEC has a veritable treasure trove of information from various registrant filings.  We are mining that rich vein of information and are applying the same quantitative approach to develop various ways to evaluate registrant filings and search for potential areas of risk.  While we have several projects in development, we are particularly excited about what we call an “Accounting Quality Model” (AQM).  This model is being designed to provide a set of quantitative analytics that could be used across the SEC to assess the degree to which registrants’ financial statements appear anomalous. 

While I’ll go into more detail later about what, exactly, the AQM is, at the highest level of generality it is a model that allows us to discern whether a registrant’s financial statements stick out from the pack, while taking into account the contemporaneous attributes of that pack.  The goal is to facilitate comparison across firms within their industry while accounting for and illustrating industry differences as well. 

Such assessments will be incredibly useful in many areas of the Commission.  For example, the Division of Corporation Finance (CorpFin) could use these analytics to inform their filings review process.  OCIE could use these analytics during registrant examinations as supplemental information to inform the pre-exam process.  Enforcement could use the analytics to focus their investigative process.  Finally, staff across the SEC could use these metrics to evaluate claims made by tipsters as part of our Tips, Complaints, and Referrals process. 

The Accounting Quality Model project aligns closely with our underlying approach to data-driven analytics:  Given a broad source of data, we use quantitative models to evaluate individual registrants and consistently estimate peer-level risk metrics.  Model results can be used to generate risk-based rankings that can then be used, in turn, to conduct outlier analysis.  In addition, staff can develop a better empirically based understanding of an individual registrant business and how it relates to peers.  For example, firms within a particular industry tend to share accounting characteristics and approaches.  These shared characteristics might reflect features specific to the industry or they might indicate similar industry level accounting choices.  We take a modeling approach that quantitatively accommodates these types of industry-level effects. 

Why Accounting Quality?

I don’t mean to keep deferring the main course here, but before I actually describe the Accounting Quality Model and what it can do for the SEC, it’s probably worthwhile to consider why we’ve chosen to focus on accounting quality and to honestly confront some of the challenges in doing so. 

Academics in finance and accounting have long-studied the information contained in financial statements to better understand the discretionary accounting choices that are made when presenting financial information to shareholders.  You may even have heard the euphemistic phrase for the aggressive use of that discretion in manipulative or even potentially fraudulent accounting practices: “earnings management.” Importantly, the phrase “earnings management” is broad enough to capture both aggressive accounting practices that fall within GAAP and fraudulent accounting practices that violate GAAP.  As I mentioned above, the metrics we are developing can help identify firms that stick out from the pack.  In the context of earnings management measures, sticking out from the pack could indicate the need for further review.  Unfortunately, there is a perception in the academic literature that standard models often fail to reliably identify earnings management.  Moreover, as I just noted, it is not clear that once earnings management is identified, it is necessarily indicative of fraudulent accounting practices. It may simply reflect a full and creative – but ultimately permissible – use of the professional judgment that is permitted under GAAP.  Both issues are important and the concerns described real. Regardless of the intended purpose for the Accounting Quality Model – whether to root out potential fraud or simply identify aggressive uses of the discretion allowed under GAAP – a poorly conceived model might produce a significant number of false positives. A false positive occurs when the model incorrectly identifies a filer as an outlier.  The identification of false positives can be costly, not only for the registrant erroneously tagged as engaging in earnings management, but for staff who has expended resources to investigate further. 

However, if the number of false positives can be kept to a manageable level, the use of quantitative models regarding discretionary accounting choices could be a powerful tool for staff who may be interested in the full range of behaviors associated with earnings management, and not merely as a way to potentially identify fraud. Indeed, for example, identifying reporting anomalies could be useful to SEC review teams when completing reviews of corporate filers and allow staff to assist them in providing disclosures that comply with GAAP.  But, of course, it also has the potential to be used by the Enforcement or OCIE to identify firms with intentional potentially fraudulent reporting practices. 

The Accounting Quality Model

We believe we are designing an Accounting Quality Model that has the potential to limit these costly false positives.  One of the unique aspects of this project is the organic nature of the Model’s development, as we have been incredibly fortunate to work with staff across the SEC to integrate feedback based on individual experiences with financial reporting.  For example, a common test of the efficacy of accounting quality-type models is to estimate their power to predict accounting fraud by evaluating how well they predict SEC Accounting and Auditing Enforcement Releases (AAERs). AAERs are notices that financial reporting related enforcement actions concerning civil lawsuits have been brought by the Commission in federal court and notices and orders concerning the institution and/or settlement of administrative proceedings. In developing our model, we not only replicated those results but also actually integrated SEC staff experience into the design of the model.  By incorporating feedback from the very staff that the model is intended to serve, it allows us to adjust the model to accommodate evolving staff experiences and priorities.

And this is where I will start to get a little more technical.  While I obviously can’t give away the store, I would like to give you some sense of how our Model actually works.  Bear with me as I set up the precursors of our Model.

First, the premise of all models that seek to identify earnings management is that firms have strong incentives to manage earnings.  The evidence broadly follows two strains:  One, investors respond to earnings announcements and, two, earnings management by the firm influences market information about the firm’s future performance and investment prospects. 

Second, we need to understand generally where it is possible to discern the effects of earnings management.  Typically, they manifest in the discretionary choices that management can make under GAAP when reporting its financials.  In accounting jargon, total accruals are the difference between free cash flows and income before extraordinary items.  It is the difference between what accountants recognize as revenue and expenses and the actual cash flows available to shareholders.  We can decompose total accruals into two broad categories: discretionary accruals and non-discretionary accruals.  Non-discretionary accruals are accounting adjustments made in strict adherence to GAAP and are relatively objective.  Discretionary accruals however, may be subjective and require the preparer to exercise considerable accounting judgment.  As is generally recognized, this influence over the potential accrual values can allow for opportunities to, for example, smooth income and therefore, manage earnings

Thus, bringing it back to accounting quality models, in a nutshell, outlier discretionary accruals can be a powerful indicator of attempts to manage earnings.  The trick is to identify those outliers. 

For those of you not conversant with the vast existing work on earnings management detection, traditional models – often based on the popular “Jones” model or the “modified Jones” model – generally attempt to isolate the effect of discretionary accruals by regressing total accruals (a reminder: the difference between net income and free cash flows) on factors that proxy for non-discretionary accruals and treating the unexplained portion (the residual) as an estimate of discretionary accruals.

Our Accounting Quality Model extends the traditional approach by allowing discretionary accrual factors to be a part of the estimation.  Specifically, we take filings information across all registrants and estimate total accruals as a function of a large set of factors that are proxies for discretionary and non-discretionary components.  Further, we decompose the discretionary component into factors that fall into one of two groups:  factors that indicate earnings management or factors that induce earnings management.  Discretionary accruals are calculated from the model estimates and then used to screen firms that appear to be managing earnings most aggressively. 

If you’ve followed me so far, you can see that our approach necessitates the classification of factors into those that explain either discretionary accruals or non-discretionary accruals.  The classification process should be informed by staff experience, intellectual capital, and the substantial accounting literature related to earnings quality and discretionary accruals.  As I described above, by integrating actual staff experiences and knowledge into the Model, we have a powerful method for identifying those factors that can indicate outliers.

So, the obvious question is, then, what are some of the factors that we take into account when trying to identify outlier discretionary accruals?  We can characterize discretionary accruals as different types of risk indicators and risk inducers.  Risk indicators are factors that are directly associated with earnings management while risk inducers are factors that are associated with strong firm incentives to manage earnings. 

In our model, for example, the choice of accounting policy and firm interactions with independent auditors may be indicative of specific types of earnings management.  An accounting policy that could be considered a risk indicator (and consistently measured) would be an accounting policy that results in relatively high reported book earnings, even though firms simultaneously select alternative tax treatments that minimize taxable income.  Another accounting policy risk indicator might be a high proportion of transactions structured as “off-balance sheet.”  Although the vast majority of firms use off-balance sheet financing for legitimate business purposes, many of the largest accounting scandals used off-balance sheet activities to hide poor financial performance. In both instances, the metrics associated with accounting policies can be consistently estimated from filings data and compared to peers.  Another risk indicator could be the frequency and types of conflicts with independent auditors, as measured by changes in auditors or delays in the release of financial statements or earnings.  Again, these risk indicators could be consistently estimated from filings data and compared to peers.

On the other hand, risk inducers are designed to capture managerial incentives to mask poor absolute or relative performance. For example, a firm may be losing market share or it may be less profitable than its competitors. A firm experiencing performance problems, particularly those it considers transient, may induce a response that inflates current earnings numbers in exchange for lower future earnings. 

The factor-based approach is a flexible modeling framework that easily accommodates new modeling factors as we add and delete proxies for potential earnings management.  The additional flexibility lets us efficiently respond to model feedback and customize the model to suit different missions within the Commission while allowing for sensitivity to the nuances of those differing goals. 

Integration of the Accounting Quality Model into the SEC’s Mission

I believe that our work here is vital to the core mission of the SEC.  The SEC has responsibility for facilitating the provision of accurate and fairly stated financial statements to the public.  As a result, there is a cross-agency need to identify and then investigate potentially misstated or even fraudulent financial statements.  SEC Staff successfully engages in these efforts in many ways, including – as I described earlier – filer reviews, examinations, and enforcement actions. 

The Accounting Quality Model I’ve been talking about today represents an additional approach to estimating discretionary accruals and identifying outliers, which can, in turn, assist in the effort to identify earnings management.  This is not to say that we’ve built a model that can “detect fraud.”  Far from it.  Rather, we hope to provide one more tool that the already sophisticated staff of the SEC can use in its efforts to ensure high quality financial statements.  As staff become more comfortable using models we will apply our analytical expertise to other registrant practices that might be of interest to staff across the SEC. 

Thank you again for having me here to speak with you today and I am happy to answer any questions you may have.

* The Securities and Exchange Commission, as a matter of policy, disclaims responsibility for any private statement by any of its employees. The views expressed herein are those of the speaker and do not necessarily reflect the views of the Commission or of the speaker’s colleagues upon the staff of the Commission.


Last modified: Dec. 17, 2012