Insights into the SEC’s Risk Assessment Programs
Mark J. Flannery, Chief Economist and Director, Division of Economic and Risk Analysis
Global Association of Risk Professionals 16th Annual Risk Assessment Convention<br/>New York, NY
Feb. 25, 2015
Thank you so much for inviting me here today. I will be speaking on an important topic that has received considerable attention since the financial crisis: market risk assessment. Before I begin my remarks, I must make clear that the views I express today are mine alone and do not necessarily reflect the views of the Commission or of my colleagues on the Commission Staff.
I would like to share with you some thoughts about two distinct, but related, types of risk assessment that we in the Division of Economic and Risk Analysis (DERA) spend a lot of time thinking about. The first type of risk involves various forms of market misconduct. The cost of these risks falls on investors and on the integrity of financial markets, and the Commission spends much time and effort trying to detect market misconduct. By this I mean conduct — often impermissible — that generates investment risk unrelated to the merit of projects underlying the investment. These situations are frequently motivated by conflicts of interest, and they are often undertaken in the presence of asymmetric information between various market participants. For example, when hedge fund managers misvalue assets to inflate or smooth their returns. Or corporate issuers misstate earnings. Or brokers privilege some investors over others when allocating trades — a practice called cherry picking.
While these provide just a flavor of the types of misconduct that SEC staff seek to identify and remedy, they represent the SEC’s core mission to protect investors and ensure fair and orderly U.S. financial markets. The SEC’s success in these areas makes it easier for businesses, entrepreneurs, funds and other issuers of securities to obtain finance for legitimate investment projects. Curtailing instances of market misconduct reduces the cost of capital because potential investors need only worry about the merit of legitimate investment opportunities.
The second kind of risk assessment deals with understanding and identifying market-wide, or “systemic” risks. These risks may flow from the correlated activities of many market participants. These risks can propagate through the entire market or a segment of it, adversely affecting many entities, including those that did not contribute to the activity causing the market-wide risk.
An example of this type of risk would be the potential for knock-on effects if one firm’s failure weakened another. This might occur in a number of ways. One avenue would be the use of derivative securities such as over-the-counter (OTC) swap agreements, for which inadequate margining might leave customers exposed to counterparty failure. As we all know, leverage in the affected parties tends to exacerbate the potential for spillover risks. By disrupting market activities that investors rely on, such risks threaten access to, and the cost of, capital required to finance worthwhile investment opportunities in the economy.
Staff at the Commission think carefully about these risks, and incorporate that thinking into our risk assessment programs, as well as our rulemaking recommendations. Recent manifestations of this concern can be found in SEC rules covering OTC derivatives, the use of asset-backed securities, and appropriate trading activities.
These are risks that I’ve given significant thought to. Indeed, even before I came to the SEC, I’d been conducting research on market risk related to financial intermediation for more than 30 years, and have spent a fair bit of my career working with regulators on this issue, including the Fed, FDIC, and OFR, before joining at the SEC. So, although it is only recently that I have had to think about applying this work to financial market regulation, I came ready to think creatively and rigorously about how to identify and assess both types of relevant market risks.
Detecting market misconduct
First I will discuss the detection of market misconduct. This type of risk assessment is widespread at the Commission, in the form of supervisory and investigative functions of several offices and divisions. Many of the tools used to facilitate this type of risk assessment combine the analytic and financial modeling capabilities of DERA’s quantitative staff with the institutional market expertise of other Commission staff. The (recently created) Office of Risk Assessment within DERA is home to this development activity, and this office currently administers two recently implemented programs.
The first is the Corporate Issuer Risk Assessment (CIRA) program, which expands upon an “accounting quality model” that had been developed to detect anomalous patterns in financial reporting. Tools developed under this program seek to identify situations or activities at corporate filers that warrant further inquiry. The original (AQM) effort focused on estimates of earnings quality and indications of inappropriate managerial discretion in the use of accruals. This is now only one of more than over one hundred custom metrics provided to SEC staff through an intuitive dashboard. For example, we can look at how inventory at a manufacturing company is moving relative to sales, because an unusual inventory buildup might lead managers to be aggressive in their accounting. When combined with other risk indicators, SEC staff may decide to focus more attention on the reporting firm. Members of the SEC’s cross-agency Fraud Task Force are frequent users of the CIRA system.
The SEC regularly receives massive amounts of data reported by corporate registrants — far too much data to be examined efficiently by individual reviewers. The CIRA dashboard is intended to act as a flexible and user-friendly interface between the quantitative modeling underlying CIRA and the expert staff who want to use it to facilitate their review of issuer filings.
Similar modeling techniques underpin our Broker Dealer Risk Assessment program. This analytical tool was developed in conjunction with the Office of Compliance, Inspections, and Examinations to help prioritize inspections. By focusing staff attention on the riskiest areas of the market, this tool can assist the SEC to conserve limited resources, and to help focus the attention of examiners where they are most likely to detect inappropriate risk or fraudulent activity.
The process works as follows. BDs are first classified by their type of dealing activity — for instance whether or not they carry customer securities on their books (i.e., a “carrying broker”). This allows staff to analyze how a firm’s behavior compares to its peers. We then look for predictors of potentially anomalous or concerning behavior, which include potential risks related to, for example, its operations, financing, workforce, or firm structure. Risk factors in each of these areas are created from information collected in mandatory disclosure by BDs, such as from the FOCUS, BD, BR, and U4-5 filings. A score card rates how each firm’s activity in each of these areas compares to its peer firms, and results from these score cards are used to help prioritize the sequence of BD inspections as well as areas for examiners to focus on.
Another ongoing initiative follows up on some of the early successes of the Aberrational Performance Inquiry model, which has led to a number of actions taken against hedge funds for misconduct. We are currently extending risk models to identify a variety of risks at registered investment advisors, using information they provide on their annual Form ADV. In these and in other areas, our model development activity is ongoing. The next generation tools are informed by the users of current tools. To that end, much of what DERA currently does entails listening to what review, investigative, and inspection staff need from these models so that we can better leverage their expert knowledge. Our goal is to help make SEC staff more nimble in their analyses by making relevant market participant information more accessible to interpretation of potential risks and misconduct. I believe that this kind of data-driven risk assessment is necessary to operate effectively in increasingly sophisticated financial markets.
As an aside, I’d like to take this opportunity to correct the record on something. Somehow the press has coined the term “Robocop” to describe these types of modeling tools — as if a machine makes the important decisions in identifying important risks. This implied perspective is at best inaccurate and at worst misleading. While these activities use quantitative analytics designed to help prioritize limited agency resources, the tools we in DERA are developing do not — indeed cannot — work are on their own. They require expert “human” oversight, which is abetted by data-driven models.
Assessing market-wide risks
Now let me focus on a second type of risk: those that are market-wide and not necessarily related to misconduct or fraud by market participants. These types of risk have garnered increased attention and scrutiny since the financial crisis, particularly to the extent that they are perceived to threaten financial stability. Unlike the risks associated with market misconduct, market-wide risks could pose a threat to financial stability by complex interactions among market participants. Such risks are challenging to measure and evaluate.
At one time, this sort of risk was stylized as the impact of a firm’s failure on other firms and investors doing business with the failed firm: Could the losses associated with one failure cause problems further down the line, in a domino-like fashion? Since the financial crisis we hear more about the fire sale of assets, whose reduced value affects the solvency of other firms holding the same or similar assets. Their solvency issues might then prevent them providing loans or funding good investment ideas through the capital markets. Such real effects constitute the ultimate cost of financial instability.
The Commission, along with other financial regulators, takes steps to both monitor and address these kinds of broader risks. In addition to its ongoing market monitoring activities, several of the rules issued by the Commission directly address such risks. The most recent example is, of course, the Commission’s ongoing work regulating the previously opaque security-based swaps market. Only last month, the Commission adopted rules that will bring much-needed visibility into these markets by requiring both regulatory reporting and public dissemination of certain trade information. This will enable the Commission and other market observers to see transaction flows and potential risk buildups with a clarity that was heretofore simply impossible.
And the Commission is continuing to consider these types of broader risk questions. For example, some observers suggest that one fund’s inability to meet investor redemptions promptly, particularly during times of market stress, might generate concern among investors at unaffiliated funds. If those investors rush to liquidate their holdings, even healthy funds could be forced to sell assets, perhaps at prices below their intrinsic value. We sometimes refer to this as a fire sale, but not the good kind that are arranged by retailers after the holidays. Understanding these types of market movements, the SEC is considering whether to revise its liquidity rules for mutual funds. As part of this process, one of the important questions for the Commission to consider is whether widespread investor flight is likely enough that regulators should take preventative steps now.
The centrality of high-quality and useable data to risk assessment
The quantitative risk assessment programs currently underway at the Commission rely on high-quality, structured financial information. Modelling potential misconduct or market-wide risks requires quantifiable measures of market behavior and market participant characteristics. That is, we must be able to generate quantifiable measures of what the market is currently doing and who is doing it. Collecting this information and putting it into a usable format is often the most time consuming and labor intensive part of the model-building process. Accordingly, DERA oversees several projects that will improve the quantity and quality of financial information available for risk assessment and other SEC activities.
Some of the most important financial information about firms comes from mandated disclosures by market participants. Filings made with the Commission contain valuable information, not least because the disclosures were designed with the SEC’s mission in mind. That is, mandatory disclosures required through Commission rulemaking are premised on their relevance to the maintenance of investor protection, the operation of fair, orderly, and efficient markets, and the facilitation of capital formation.
However, requiring disclosure of the information is only the first step; to be used for risk analytics, the data also must be presented in a format that is both accessible and usable by analysts. Currently, only some disclosures are available in a machine-readable format, and even they may require significant processing before the information can be used for analytical purposes. Moreover, risk analytics often require that disclosed financial information be combined (or merged) with other financial information before an interpretation can be made. For instance, understanding a corporate issuer’s growth opportunities may require a comparison of the book value of its assets (from the balance sheet) to the market value of its equity (obtained from trading activity). The Commission also receives valuable market information from other sources, including SROs and data aggregators that package and sell financial market data.
DERA’s Office of Structured Disclosure works to make disclosed information more widely available, and more easily combined with other data sources. Structuring the available information provides data for our economists; it also aids other divisions and offices by enabling them to expand their insights into SEC registrants.
Presenting the contents of large and sophisticated datasets in a user-friendly environment often requires specialized expertise and intensive computing resources. To leverage the Division’s work with outside (contracting) service providers, DERA launched a groundbreaking Quantitative Research Analytical Data Support (“QRADS”) Program. This program supports a variety of complicated data-driven initiatives, including the generation of standardized quantitative reports about financial markets and registrant activity. Currently, QRADS is processing and analyzing FINRA-produced financial market data, OTC security-based swap data, and mutual fund flow data. But these are only a few of our ongoing, large scale data projects. The QRADS program is flexible and is designed to meet the Commission’s growing and changing data needs.
In conclusion, I hope I have made clear that DERA embraces a data-driven approach to monitoring all kinds of risks in the market. In my view, the SEC cannot be an effective supervisor of financial markets unless it understands both the risks facing small retail investors and those potentially building across an entire industry. I look forward to being part of this ongoing creative and effective work.
 The Securities and Exchange Commission, as a matter of policy, disclaims responsibility for any private publication or statement by any of its employees. The views expressed herein are those of the author and do not necessarily reflect the views of the Commission or of the author’s colleagues upon the staff of the Commission.