This document is an HTML formatted version of a printed document. The printed document may contain agency comments, charts, photographs, appendices, footnotes and page numbers which may not be reproduced in this electronic version. If you require a printed version of this document contact the United States Securities and Exchange Commission, Office of Inspector General, Mail Stop 11-7, 450 Fifth Street N.W., Washington, D.C. 20549 or call (202) 942-4460.

BROKER-DEALER RISK ASSESSMENT PROGRAMS

Audit No. 354
August 13, 2002

EXECUTIVE SUMMARY

We found that the effectiveness of the broker-dealer Risk Assessment Program has been compromised by the lack of a supervisor (since October 2000). Our recommendations include appointing a supervisor for the program, reviewing its workload and priorities, using support staff better, updating the program's rules, improving documentation and data back-up, and ensuring that all required filings are made.

During the audit, Commission officials took steps to address our recommendations. We commend them for their attentiveness and timely responses.

SCOPE AND OBJECTIVES

Our audit objectives were to evaluate the overall effectiveness of the Commission's Risk Assessment Program and identify potential improvements. We interviewed officials in the program and reviewed related documentation.

The audit was performed from March 2002 to May 2002 in accordance with generally accepted government auditing standards.

BACKGROUND

In 1990, unregulated affiliates of Drexel Burnham Lambert (a then prominent broker-dealer) experienced financial difficulties. Drexel's holding company transferred funds to the affiliates from Drexel. Eventually, the holding company collapsed, which necessitated the liquidation of Drexel.

As a result, the Market Reform Act of 1990 added section 17(h) to the Securities Exchange Act of 1934. This section provided the Commission with authority to obtain information regarding the financial activities of broker-dealer affiliates.

In July 1992 the Commission adopted temporary rules 17h-1T and 17h-2T. Rule 17h-1T requires certain broker-dealers to maintain and preserve records and other information concerning certain associated persons of the broker-dealers. Rule 17h-2T requires certain broker-dealers to file with the Commission quarterly reports concerning the information required to be maintained and preserved under Rule 17h-1T.

These broker-dealers are required to file quarterly risk assessment reports for broker-dealers (Form 17-H), consolidated and consolidating financial statements for the broker-dealer and holding company, disclosures about material associated persons, organization charts, policies and procedures, and legal representations, as necessary. As of March 31, 2002, approximately 170 broker-dealers were filing reports ¹.

The Office of Filings and Information Services (OFIS) logs limited information about the 17-h filings into EDGAR, the Commission's automated database of filings.

The Risk Assessment Program in MR is implemented by a group of five accountants, two economists and two support staff (which includes a security regulation specialist). Its purpose is to monitor the activities of affiliates to identify potential adverse effects on broker-dealers.

The program's accountants review quarterly 17-h risk assessment filings and keep apprised of significant events that could adversely affect the broker-dealers. The economists provide technical assistance in understanding complex risk assessment tools used by the larger, more prominent broker-dealers, and in particular, review and analyze the models used to manage market risk.

The security regulation specialist developed and maintains a Microsoft Access database containing filing information and historical data about reporting broker-dealers. Both support staff assist with day-to-day operations.

The program's accountants and economists participate in monthly risk assessment meetings with some of the largest broker-dealers to review their market and credit risk exposures, examine broker-dealers applying for broker-dealer "lite" ² status, provide rulemaking assistance and participate in several ad hoc projects. These projects include:

• Monitoring current events and identifying trends and developments in the securities industry to assess their impact on broker-dealers and the industry as a whole;

• On-site comprehensive reviews and continuous monitoring of over-the-counter (OTC) derivatives dealers;

• Involvement in international groups to ensure Commission input into international supervisory papers and pronouncements; these initiatives sometimes involve overseas travel;

• Research and exploration of international "best practice" guidelines and standards.

• Contact with broker-dealers engaged in high-volume, day-trading activities and on-line broker-dealers for the purpose of identifying and mitigating risks;

• Identification of non-reporting broker-dealers;

• Monitoring and periodic audits of the Securities Investor Protection Program (SIPC); and

• Post September 11^th follow-up with broker-dealers to assess the impact of the terrorist attacks on these entities.

Additionally, Commission staff are planning a rule that would permit certain broker-dealers to compute their capital requirements using statistical models in exchange for increased Commission scrutiny. This would likely result in lower capital requirements. If adopted, this rule is expected to significantly increase the workload of staff in the Risk Assessment Program.

The program has been without a direct supervisor since October 2000.

AUDIT RESULTS

The Risk Assessment Program is staffed with a collegial group of highly motivated individuals. Despite increased workloads and in the absence of an immediate supervisor, the group has taken proactive steps to identify and monitor risks faced by broker-dealers as well as target and review firms with the greatest perceived risks. The group also participates in several ad hoc projects to better understand and monitor risks faced by broker-dealers. We commend the initiative displayed by the staff.

However, since the program's direct supervisor left the Commission in October 2000, the ability of the program to achieve its objectives has deteriorated because of the absence of a supervisor. In addition, program and support staff have not been used as effectively as possible.

Program staff and other officials stated that a lack of staff resources has hindered their ability to better structure the program. The staff do not have time to review in detail all 17-h filings and also participate extensively in important ad hoc projects. Restructuring the program would allow better use of the limited staff available. We recommend that MR review the program's mission, structure, and workload, improve program management, and prioritize tasks.

Other recommendations include updating the Commission's rules governing the program, establishing a formal system to document staff reviews of 17-h filings, and better ensuring that firms subject to the rules file required forms with the Commission.

SUPERVISION

The program's direct supervisor left the Commission in October 2000. Since that time, MR twice posted the position, but has not appointed an acting or permanent supervisor.

There has been insufficient program focus and direction, especially for the accountants and support staff. Certain program functions have deteriorated and day-to-day decisions regarding program operations have been put on hold. The program staff are on a different floor in headquarters than most MR staff, increasing their isolation.

Although there is no day-to-day supervisor, an Associate and Assistant Director in MR understand and oversee the program and participate with program staff in certain risk assessment initiatives. The accountants and economists are comfortable approaching these officials with major issues such as risks or operational emergencies. However, the staff generally do not discuss day-to-day issues with supervisory personnel.

Recommendation A

MR should appoint an acting or permanent supervisor for the Risk Assessment Program as soon as possible.

SUPPORT STAFF

The two support staff have been especially affected by the absence of a supervisor and need more guidance in performing day-to-day operations. Unlike the group's accountants and economists, the nature of their duties provides them limited opportunities to discuss most issues with upper management.

These officials would like added duties and responsibilities, including the opportunity to provide additional assistance to the program's professional staff. They would like to identify and contact subject broker-dealers that fail to file 17-h forms and one would like training as a back-up for maintaining the Access database.

Recommendation B

MR should provide additional supervision and consider delegating additional responsibilities to the program's support staff, as discussed above.

17-H REVIEWS

Since the Risk Assessment Program's inception, the accountants in the program have reviewed only a limited number of 17-h filings. They concentrate on filings from firms with the perceived highest risk, either because of their size or because they engage in activities such as derivatives transactions.

Firms subject to filing were last allocated among the accountants approximately two years ago. One of the five accountants is not currently responsible for any firms. In addition, the number of reviews and review goals differs among accountants (e.g., some try to review all assigned firms at least once a year; others do not).

Participation in special projects takes precedence over routine reviews of 17-h filings. This often precludes adequate or timely review of 17-h filings.

We believe the review process would be more effective if it was more consistently implemented with enhanced guidance to staff. For example, the following considerations are important:

• • Should there be a reallocation of firms among the five accountants?

  • Approximately how many or what types of firms should be reviewed?

  • Should the accountants review all subject firms at least once a year?

  • What amount of resources should be devoted to reviewing the smallest firms? (Less frequent reviews may be appropriate.)

  • How should the accountants divide their time between routine 17-h reviews and special projects?

Recommendation C

MR should implement changes to enhance the effectiveness of the 17-h review process. At a minimum, the considerations listed above should be resolved. Any changes should be in line with current or anticipated staff resources.

PROGRAM RULES

Temporary rules 17h-1T and 17h-2T have not been updated or made permanent since their issuance in 1992, despite significant changes in broker-dealer operations and resulting risks. MR agrees that the rules need revision, but limited staff resources (in part) have prevented issuance of revised, permanent rules.

According to Program staff, certain rule provisions may be outmoded and could be more effective. Consideration should be given to the following:

• The rules could be updated to better allow for the collection of information about risks posed by newly developed derivatives products and transactions.

• Form 17-H, part II could be revised so that it captures relevant information in accordance with current risk management and measurement criteria.

• The time allotted to firms to send quarterly and annual filings could be shortened to accommodate more timely receipt of information.³

Additionally, the rules required MR to issue a study evaluating their effectiveness within 90 days after the rules were fully operative for two years. The purpose of this study was to provide the industry with an opportunity to comment on the rules so the Commission could better evaluate and modify the rules. This study was drafted in 1996 but was never finalized or issued.

Recommendation D

MR should consider the points above, evaluate the effectiveness of the current rules, and update and/or finalize temporary rules 17h-1T and 17h-2T.

DOCUMENTATION

No formal procedure to document the accountants' review of 17-h filings is in place, although some staff make informal notes. Documentation would help show what the program has accomplished, ensure uniform review of filings, identify trends and at-risk broker-dealers, and provide information for staff performance ratings and reporting under the Government Performance and Results Act.

The accountants agreed reviews should be documented, but in a cost effective manner. A supervisor for the program would be in the best position to coordinate the development of useful documentation procedures.

Recommendation E

MR should establish a standard method for documenting staff 17-h reviews.

TREND ANALYSIS

The program's economists recently began comparing risk exposures and other financial trends among four prominent broker-dealers. The process is still evolving and the economists plan to expand their analyses.

One option is to create a database containing selected financial information from several broker-dealers (including the four prominent broker-dealers). Using this information, program staff could then identify financial trends among the broker-dealers, rank them by financial condition, and identify broker-dealers falling outside established normal ranges. The staff could also develop a watch list of broker-dealers warranting immediate review.

Program officials indicated it might be efficient to limit the analyses to larger and medium-sized firms.

Recommendation F

MR should consider expanding the current trend analyses, including adding additional broker-dealers to the trend analyses.

NON-FILING BROKER-DEALERS

Several of the 170 broker-dealers known to be subject to reporting have not filed consistently. Program staff indicated they have not contacted these delinquent firms because of limited resources.

Recommendation G

MR should timely contact all known broker-dealers subject to the 17-h filing requirements that have not filed.

In addition to the above-mentioned broker-dealers, there are other (unknown) broker-dealers subject to the Commission's 17-h filing requirements that do not file. The accountants recently started a project to identify these firms (after at least a three year hiatus). The accountants believe this should be done annually to be effective.

Recommendation H

On a routine basis (e.g., annually), program officials should identify and contact (unknown) firms that are subject to the Commission's 17-h filing rules and are not consistently filing.

ELECTRONIC FILING

Program staff said it would be useful if broker-dealers had the option of submitting their 17-h filings electronically (on CD-Rom, diskette or via e-mail). Additionally, MR has discussed with the Office of Information and Technology (OIT) the feasibility of setting up a risk assessment filing system capable of supporting the electronic filing of 17-h materials.⁴

Recommendation I

MR, in conjunction with OIT and OFIS, should continue to explore the feasibility of electronic filing of 17-h materials. 

DATA RETENTION

A Commission contractor copies the paper 17-h filings onto CD-Rom, and then shreds the hard copy. OFIS (in the Operations Center) maintains the CD-Roms. MR program staff normally destroy their hard copy of the filing in one or two years.

MR (in headquarters) should also retain a copy of the 17-h filings on CD-Rom because this information could be necessary for research purposes and may be useful for contingency planning purposes.

Recommendation J

OFIS should send MR a copy of the 17(h) filings on CD-Rom.

BROKER-DEALER REPORTING PROCESS

Many broker-dealers are apparently unaware of the filing requirements for Form 17-h. Instead of sending these confidential filings to OFIS, many broker-dealers address filings to the Office of the Secretary, MR and to officials that no longer work in the program. Some only send one copy, instead of three, as the Commission has requested. As a result, some filings move around the Commission unnecessarily⁵ or do not get logged into EDGAR or MR's database. Some subject broker-dealers do not file at all.

We believe it would be useful if MR reminded all non-compliant reporting broker-dealers of the following general 17-h filing requirements:

• the reason for the filing;

• the address to which the filings should be sent;

• the name and phone number of a designated contact official;

• the Commission's request that the broker-dealers send more than one filing; and

• any other relevant information that program officials believe the broker-dealers should be aware of.

One way to inform broker-dealers would be to send all non-compliant reporting broker-dealers a letter describing the 17-h reporting requirements above. The letter would help ensure that broker-dealers continue to file, despite staff turnover. ⁶

Recommendation K

MR should remind all non-compliant broker-dealers subject to the 17-h reporting requirements of the above information.

DATA DISCREPANCIES

We compared 17-h filing information in EDGAR and MR's Access database, and identified the following discrepancies:

• some filings were listed in only one system; and

• the two systems did not always agree as to whether a filing was a quarterly or annual filing.

Recommendation L

OFIS and MR staff should correct the data discrepancies we identified.

During the audit, OFIS and MR staff began researching these discrepancies and implemented procedures to prevent further discrepancies.

GOVERNMENT PERFORMANCE AND RESULTS ACT

One of the Commission's Government Performance and Results Act (GPRA) measures relates to the Risk Assessment Program. The measure tracks the percentage of broker-dealers for which staff surveillance procedures resulted in follow-up action. The purpose of the follow-up is to determine whether the broker-dealer and its customers were exposed to significant risks.

A program official pointed out that fewer follow-ups would actually indicate that the program is better accomplishing its objective, since program staff typically follow-up with broker-dealers only when a problem develops. Thus, the GPRA measure could be misleading. Furthermore, the measure is limited to staff review of quarterly filings, which is only one component of the program.

Recommendation M

MR should consider modifying the current GPRA measure for the Risk Assessment Program, and adding additional measures to reflect program performance and results.

POLICIES AND PROCEDURES

The Risk Assessment Program does not currently have adequate written policies and procedures. Documented policies and procedures would provide clearer direction to staff (especially new and support staff).

The policies and procedures should include issues discussed in this report and requirements in Commission rules. At a minimum, they should contain guidance on reviewing 17-h filings, documenting 17-h reviews, updating and maintaining the Access database used by the Risk Assessment program, what reports to produce and how often to produce them, follow-up procedures for contacting broker-dealers that fail to file, and a description of the program's involvement with other divisions/offices, including OFIS.

Recommendation N

MR should develop written policies and procedures for the Risk Assessment Program, as discussed above.

_____________________________________________

¹ At the time of our review, program staff were responsible for monitoring approximately 170 firms. This number is expected to increase as a result of the identification (by program officials) of several other firms that should be, but are not currently, filing 17-h documents with the Commission.

² This option is available to certain broker-dealers engaged in over-the-counter derivatives trading and applying for permission to compute their capital requirements using a statistical model (as opposed to abiding by the Commission's standard capital requirements).

³ Currently, firms have 60 and 105 days, respectively, to send quarterly and annual filings.

⁴ By the end of fiscal year 2003, the Government Paperwork and Elimination Act requires Federal agencies to allow entities to transact business with the Federal government electronically and to maintain electronic records.

⁵ Nothing came to our attention indicating that the confidentiality of the filings was compromised.

⁶ Program staff said that broker-dealers often stop filing 17-h forms when the person in charge of sending the filing leaves the firm. The new person is often unaware of the requirement.