This document is an HTML formatted version of a printed document. The printed document may contain agency comments, charts, photographs, appendices, footnotes and page numbers which may not be reproduced in this electronic version. If you require a printed version of this document contact the United States Securities and Exchange Commission, Office of Inspector General, Mail Stop 11-7, 450 Fifth Street N.W., Washington, D.C. 20549 or call (202) 942-4460.
IRM Planning and Execution
Audit Report No. 220
March 26, 1996
We found that until recently, Commission Information Resources Management (IRM) planning and execution were not adequate, due to a number of factors common throughout the government. Consequently, significant IRM projects were frequently over budget or delayed, or did not fully meet Commission needs.
New management in the Office of Information Technology (OIT) is making significant improvements, however. According to the Office of the Executive Director, these improvements were requested by the Executive Director. The Commission staff we interviewed feel that OIT is now heading in the right direction.
We are making several recommendations to further that progress. Generally, OIT concurs with these recommendations and has already implemented or is actively working on many of them.
Our recommendations include upgrading the skills and defining the accountability of staff within the Office of Information Technology; assigning qualified people to major projects; increasing the role of senior management; improving management controls (including the setting of priorities, project management and reporting, contracting, and written guidance); and improving communication and working relationships within OIT and between OIT and other offices (e.g., by conducting customer surveys).
OIT and the Office of the Executive Director (OED) also note (in their response to an earlier draft report) that they have, among other steps, hired several new skilled managers, established a clear strategic direction, provided written guidance, focused upon customer service, and established strategic technical infrastructure standards.
The Audit Results section contains the detailed responses of OIT and OED to our recommendations. The Appendix contains OED's final written response, as well as that of the Office of Administrative and Personnel Management. The Office of the Chairman provided verbal comments on our draft report. We have modified the report to reflect the comments.
OBJECTIVES AND SCOPE
Our objectives were to determine if the Commission's information resources management planning for automated systems was adequate, and if controls over the execution of IRM plans were effective. Our approach focused on promoting best practices identified at other organizations.
During the audit, we interviewed numerous Commission staff in the Office of Information Technology and other offices, and examined available IRM documentation. The audit also relied on earlier and concurrent OIG audit and investigative work.
The audit was conducted in accordance with generally accepted government auditing standards from November 1994 through December 1995.
Over the last decade, the Commission planned and executed changes in information resources management which dramatically affected its work methods. The basic automation environment changed from one composed primarily of mainframe computers, stand-alone personal computers, and applications, to an integrated environment of mainframes and networked personal computers.
Many Commission employees contributed to this change, particularly those in the Office of Information Technology. / They acquired and installed the necessary hardware and software; developed new applications, either in-house or through contractors (e.g., EDGAR); and set up an expanded Operations Center in Alexandria, Virginia, all the while maintaining routine operations on existing ("legacy") systems. OIT also undertook a Strategic Automation Modernization (SAM) initiative to modernize the legacy systems (many of which had become obsolete and difficult to maintain) and to develop new applications.
During this time, the Commission developed an organizational structure for IRM planning and execution. Within OIT, an IRM Office has recently been established reporting to the Chief Information Officer (CIO). / An End-User Advisory Committee, composed of selected representatives from Commission offices and divisions, was also established.
In the past, a number of factors have hindered Commission IRM planning and execution. These include organizational issues and weaknesses in leadership, skills, and management controls.
The General Accounting Office (GAO) has identified similar difficulties at many other government agencies. Its recommended solutions ("best practices") are included in the Appendix, and should assist the Commission in developing corrective actions.
GAO identified these solutions by examining practices that worked for many different organizations, including state and federal agencies, and private firms. Many of our recommendations relate to GAO's best practices.
The difficulties in IRM planning and execution have led in some cases to excess costs, delays, and inadequate systems. A recent example follows.
Strategic Automation Modernization (SAM)
This initiative / was concurrently reviewed by an OIG audit/investigation. The results of that work indicate that both the planning and execution of this significant and expensive initiative (several million dollars in contractor costs, besides the cost of Commission staff time) were materially deficient.
The deficiencies noted in contractor development work, which was a major portion of the SAM initiative, included:
- Inadequate definition of contract expectations;
- Inadequate monitoring of the contractor's performance, and of contract deliverables and costs;
- Inefficient and wasteful methods for achieving the project's objectives;
- Poor planning and an extreme increase in the contract's size / (from $2.5 to $12.5 million) without a corresponding increase in controls over project development;
- Inadequate reporting of progress to senior management; and
- Ineffective project management and a lack of accountability for its success.
According to the Office of the Executive Director, the SAM initiative continues with renewed direction and improved oversight, and is showing early success. OED cites the installation of modern personal computers throughout the Commission; deployment of several new client server applications for the Offices of the Secretary and Investor Education and Assistance; recent infrastructure improvements improving network reliability and management; and the establishment of strategic technical standards for applications development, communications, and office automation.
The Executive Director recently recruited a new Chief Information Officer. According to the OED, the Executive Director asked the CIO to address numerous areas needing improvement, including IRM planning and execution.
The new OIT management has made numerous changes to improve IRM planning and execution, and further improvements are planned. OIT has been reorganized to better use its staff's abilities and to enhance internal communication and effectiveness. For example, data communications and networks are now in the same office within OIT, and a separate IRM Office reporting to the CIO has been created.
Other changes include development of a new strategic plan (currently in draft); improvements in contracting controls; staff hires and reassignments; improved internal status reporting; more frequent staff meetings; and designation of liaisons to the divisions. OIT's recent accomplishments include putting EDGAR filings on the Internet; acquiring Pentium personal computers; and enhancing network response time and capability.
OIT and user staff we interviewed were generally pleased with the direction OIT is now going, although some users expressed a "wait and see" attitude about OIT accomplishments. The open management style and concern for customer service of current OIT management were also praised.
Our recommendations are listed below, followed by a discussion and the responses. OIT, in cooperation with the Offices of the Executive Director and Administrative and Personnel Management, has begun improving IRM planning and execution, as summarized above. These offices should continue this cooperation as they implement the recommendations. OIT should establish target dates for corrective actions.
The Office of Information Technology should upgrade the skills and define the responsibilities and accountability of its staff.
OIT needs to consider a variety of steps to improve the skills of its staff and increase the staff's accountability. It could hire new staff with the required skills (i.e., IRM planning and execution, project management, staff management, contracting, and communications); provide training to its existing staff; take personnel actions (e.g., reassigning, promoting, demoting, or terminating staff, revising position descriptions and performance plans); or privatize certain functions (i.e., contract them out on a competitive basis).
OED and OIT Response: It should be noted that OIT has already begun to address this. The new CIO has solicited planning involvement throughout the organization in several ways. In addition, new training has been provided to most of the applications development staff and many others as funding has allowed. Several new staff members have been hired bringing new skills to the organization. New mission statements have been defined. A reorganization plan was approved to better position staff to complete strategic initiatives and match skills to project requirements. Position descriptions and performance standards are being revised to reflect actual duties and desk audits have been underway for some time to better relate position ranges with position duties. OIT has also begun investigating the use of facilities contracts for some functions.
The Offices of Information Technology, Administrative and Personnel Management, and the Executive Director should improve management controls over significant IRM projects.
OIT needs to define a threshold (e.g., dollars or some other criterion; see OMB Circulars A-109 and A-130) for significant projects, so it can establish priorities and handle significant projects differently than more routine initiatives. Management controls over IRM projects should be more elaborate when the risks are greater.
OIT should develop written procedures to ensure that significant projects are adequately planned and monitored; identify and meet user needs; and are developed efficiently and on time. The procedures should define under what circumstances a project is significant; describe required documentation, both for the planning (e.g., user requirements, design specifications) and execution of the project (e.g., justifications, milestones, budgets, reports); and specify staff qualifications, reporting relationships, and accountability.
The roles of other offices involved (e.g., the Chairman's Office, OAPM, OED, and user offices) in significant projects also need to be defined. The OED and the Chairman's Office should have primary responsibility for this role definition.
OED and OIT Response: Currently, major program status is reported to the OED on a regular basis as well as being reported in the bi-weekly status report that is shared among all OIT staff. In addition, major program status is discussed on a regular basis with OIT senior management and the OED to ensure projects remain on track and within scope. Use of programmatic budgeting to improve project management accountability is being developed. In addition, all major OIT projects are being tracked in a project management tool. All new projects require that a Project Initiation Form (PIF) be submitted to senior management so that objectives, impacts, costs, resources, and relationship to overall organizational strategies and plans can be discussed in advance of any work commencing. PIFs are required if activities will require a specified amount of budget or personnel resources or will extend through a certain duration.
The Offices of Information Technology and Administrative and Personnel Management should improve management controls over contracting and contractors.
OIT and OAPM have already taken several steps to improve controls over contracting, partly in response to a recent Office of Inspector General audit (No. 235), and further actions are planned or under consideration. Many of these steps were implemented or begun at the direction of OED and OIT management before audit No. 235 was completed. For example, the project manager, Contracting Officer's Technical Representative (COTR), and the Contract Specialist were changed for the SAM initiative, and the contractor involved was monitored more closely.
OAPM provided training for OIT COTRs in October 1995, and has set up a task force to improve controls over contractor staff. OIT is also providing General Services Administration sponsored Trail Boss training to its IRM staff.
OIT is considering methods to further improve its contracting methods. These include enhancing competition in the award of task orders, issuing task orders on a fixed price rather than level of effort basis (where feasible), and avoiding the appearance of favoritism towards contractors or treating their employees like government employees (i.e., personal service contracts).
In the past, OAPM did not take an active role in contract administration unless problems developed, delegating this responsibility to the COTR and project manager. However, OAPM as the contracting office has an overall responsibility to ensure that contracts are meeting their objectives efficiently and in accordance with regulations.
OAPM needs to develop procedures to enhance its oversight of major IRM contracts (e.g., through monitoring reports, interviews with COTRs, and review of COTR files). It should train its staff on these procedures, and hold them accountable for performing this function. Also, OAPM should evaluate whether its current staff lack certain necessary skills (e.g., price analysis, contract negotiations, communication skills), and if so, should consider hiring new staff or training existing staff.
In its written response to our draft report, OAPM indicated that it has issued formal procedures for monitoring contractors' performance and costs. It is also assessing the skills of its contracting staff, and will provide any needed training.
The Offices of the Chairman and the Executive Director should increase the role of senior management in IRM planning and execution.
Formal mechanisms for involving senior management (i.e., the Offices of the Chairman and the Executive Director) in IRM projects are not sufficient. Currently, senior management is most involved in IRM projects only when problems arise.
Senior management should define its role in writing, as well as the mechanisms by which it will stay involved and informed. For example, a senior IRM Committee could be established, consisting of the Chairman's Chief of Staff, the Executive Director, and the CIO. This committee could meet periodically to discuss the status of major IRM projects, based on written reports from OIT.
Based on our interviews, the members of the End-Users Advisory Committee generally represent their own office's interest, not the Commission's overall interest. Consequently, this committee has not been able to provide sufficient oversight over major IRM projects. The Chairman's Office and the Office of the Executive Director need to clearly define the role of this committee.
OED and OIT Response: The Executive Director and the CIO could jointly prioritize projects and contracts -- with input from programs and the End Users Advisory Committee. The initiation of new projects and the status of major IRM projects could be communicated to the Chairman's Office (i.e., Chief of Staff and the Chairman), when appropriate, in (1) written monthly reports, (2) briefing sessions held with the ED in the normal course of business, (3) special briefing sessions and written proposals on new, major projects and (4) the operating budget process.
The Office of Information Technology should take steps to improve communications and service to users.
OIT should develop a mission statement emphasizing customer service, and expand its surveys of users and its own staff. It should develop a procedure for establishing priorities for its work (with advice from senior management and users), and explain these priorities to other offices. It should hire or train staff with adequate communications skills.
The CIO should hold the heads of OIT's offices responsible for communicating effectively. OIT should consider whether any further revisions to its organizational structure are needed to improve communications and efficiency. For example, in accordance with industry practice, the Information Technology Security Officer should report directly to the CIO.
OED and OIT Response: While always striving for improved customer support, OIT recently reorganized and established an Office of Communications and Systems Support. A new Branch was established within this office called Customer Support which is charged with the responsibility of improving OIT customer support throughout the agency. One of the newly established functions is Consulting Services in which an OIT support representative is assigned to each Commission office to serve as a facilitator, to oversee problem resolution, and to ease new service deployment. OIT is hopeful that this new customer attention will result in greater user satisfaction and expanded use of modern information technology throughout the Commission. In addition, OIT is using 'details' from other SEC organizations to represent user needs in project direction and application design. OIT plans to work with users to explore the use of service level agreements for product and service delivery. All new system development efforts will have a designated OIT project lead and a program area owner to ensure requirements are accurately defined and met.
The Office of Information Technology should take actions to significantly enhance communication and teamwork within the Office.
To enhance communication and teamwork, OIT could conduct team building exercises. In addition, changes in performance appraisals of managers could be developed to enhance managers' focus on office goals. For example, half of the rating could be based on the Executive Director's or users' assessment of OIT's success as a whole, and the other half based on the manager's personal efforts and accomplishments.
Other Federal agencies have experience rating staff members on the performance of their team. OIT might need outside assistance (e.g., from OAPM) to implement this initiative.
OED and OIT Response: OIT has already taken several measures to improve communications, involve staff in more activities, develop leadership skills among its staff, and strive for delivering higher quality service. There has been a greater use of teams for development of new systems. Several all-hands meetings have been held to stimulate teambuilding and focus organizational efforts toward a common set of goals. Staff have participated in providing input into the OIT strategic document and long range plan. Meetings are held between staff (several staff from each OIT branch on a rotating basis) and the CIO to share ideas and solicit new ways of improving OIT services.
The Office of Information Technology should issue written guidance on IRM planning and execution.
OIT has issued little written guidance for its staff and users on IRM issues. By a deadline approved and monitored by senior management, OIT should issue a Commission regulation covering its major functions: system development, security, user service, contractors, organizational structure and accountability, reports, and planning. Several Office of Internal Audit and Office of Inspector General recommendations on written guidance are pending, some since 1988.
The guidance should define the threshold for major IRM projects, and describe the additional controls required for them.
The Office of the Executive Director should develop procedures to ensure leadership continuity when the CIO's position is vacant.
With advice from the CIO and senior management, the Executive Director should consider establishing a Deputy Director position within OIT. At a minimum, a single individual should be designated to serve as Acting CIO when the CIO is unavailable, or the position is vacant.
OED and OIT Response: The CIO will appoint an acting Director during brief absences due to leave or travel. In the event the position is vacant, the OED will appoint an acting CIO until a replacement can be recruited. There is no plan to establish a Deputy OIT Director at this time due to budget and programmatic reasons.
The Office of the Executive Director should strengthen its ADP expertise.
The OED should have at least one employee with ADP expertise on staff. It should hire, transfer, or train an appropriate staff member. The employee should be assigned responsibility for monitoring IRM planning efforts and projects. If resources permit, the employee should work solely or mostly on IRM issues.
OED and OIT Response: In September 1995, the OED hired a management analyst with significant ADP expertise. Prior to her being hired, she was employed by the U.S. Senate Computer Center. To date, her assignments have been almost exclusively devoted to IRM project coordination and IRM budgeting.
In consultation with other Commission offices, the Office of Information Technology should update the Commission's five year IRM strategic plan.
The previous strategic plan is being revised to reflect OIT's change in direction. The revision should comply with all OMB guidance.
OED and OIT Response: The SEC Five-Year Automation Plan is currently in draft and a copy has been provided to the OIG for review. This is an annual process which OIT intends to continue.