Executive Summary

Objective and Scope

Background

Results

Conclusion

Appendix I - Detailed Objective, Scope, and Methodology

Appendix II - Major Contributors to This Report

Appendix III - Report Distribution List

Appendix IV - Management’s Response to the Draft Report

Executive Summary

The Year 2000 (Y2K) presents a significant challenge to Internal Revenue Service (IRS) operations. The IRS Chief Information Officer created the Century Date Change (CDC) Project Office to ensure that all current and future IRS information systems will be operational and functioning as intended on January 1, 2000. The overall objective of this review was to assess the CDC Project Office’s effectiveness in providing management oversight to ensure that the goals and objectives of the Y2K conversion effort will be met.

Management controls established by the CDC Project Office provide an appropriate control framework to ensure that the objectives and goals of the IRS’ Y2K conversion effort will be met. The CDC Project Office provides guidance and direction and monitors the status of Y2K conversion efforts assigned to various project owners and responsible executives. However, the effectiveness of management oversight by the CDC Project Office can be strengthened through its independent validation and verification of information reported by the working groups; its establishment of a control technique to identify schedule slippage and eliminate what could become crisis management; and its timely identification of risk implications associated with requests to delay conversion decisions.

The CDC Project Office does not independently validate or verify inventory information reported on the Integrated Network and Operations Management System (INOMS). This inventory information is used by the CDC Project Office to monitor the conversion effort. The Treasury Inspector General for Tax Administration (TIGTA) has made reference in a number of reports that the INOMS inventory is incomplete or inaccurate, and reliance on this information could cause risk to the conversion effort. However, IRS management does not believe that the independent validation and verification of conversion status information is part of the CDC Project Office’s responsibilities, thereby solely relying on the INOMS information received from the working groups. Further, IRS management relies too heavily on external auditors, such as the General Accounting Office and TIGTA to report issues and concerns with the conversion effort. Independent validation and verification of reported information strengthens management oversight and helps ensure that inaccurate information will be prevented or detected and corrected.

The CDC Project Office does not have a technique to identify critical schedule slippage and the possible impact on the overall conversion. Between now and December 31, 1999, the CDC Project Office must oversee a number of significant conversion events and activities, many of which have interdependencies among them. Although the CDC Project Office schedules provide specific conversion information, they do not provide how milestone delays may affect the overall project. The lack of an integrated master schedule of all critical Y2K activities does not afford the CDC Project Office the IRS-wide perspective needed to steer the project partners, anticipate changes in Information Systems resources, develop alternative conversion and testing schedules, and help eliminate what may become crisis management.

The CDC Project Office is unable to timely evaluate requests for waivers and identify the potential impact on end-to-end testing and/or contingency planning. The CDC Project Office initially established the waiver process to evaluate any system or component requiring an exemption from conversion and which would remain in-service after December 31, 1999. In August 1998, the CDC Project Office expanded the waiver process to include the movement to Phases 6 and 7 of components to be retired after January 31, 1999; components to be converted but not completed by January 31, 1999; or new developments. The result was a significant increase in the number of waiver requests received near the end of Phase 5. As of March 15, 1999, the CDC Project Office had 135 open waivers that had not been reviewed or approved. This number equates to a workload of approximately 19 weeks to complete the review for potential risks and impact. In February 1999, Booz-Allen Hamilton was contracted by the CDC Project Office to assist in the waiver review process.

To strengthen management oversight of the Y2K conversion effort, the CDC Project Office should: ensure the accuracy and reliability of information used to manage the conversion effort; identify interdependencies among events, activities and schedule slippage to better manage delays within the overall Y2K project; and continue to use contractor support to evaluate waiver requests. In addition, the CDC Project Office should prioritize non-compliant systems and focus resources on the highest priority projects requiring contingency planning.

Management’s Response: IRS management generally agreed with the facts cited in the report and is taking appropriate corrective action. Management’s comments are included in the body of the report, where appropriate, and a complete text appears as Appendix IV.

Objective and Scope

The overall objective of this review was to assess the Internal Revenue Service’s (IRS) Century Date Change (CDC) Project Office’s effectiveness in providing management oversight to ensure that the goals and objectives of the Year 2000 (Y2K) conversion efforts will be met. The review focused on seven key control areas within the CDC Project Office, which consist of conversion tracking, the master schedule, independent validation and verification, budget, contingency planning, risk management, and waivers.

We conducted audit work from November 1998 to July 1999. Audit work was conducted in accordance with Government Auditing Standards. Appendix I contains the detailed objective, scope, and methodology for our review. A listing of major contributors to this report is shown in Appendix II.

Background

The IRS established the CDC Project Office in 1996 to provide oversight and to direct the conversion of all IRS Information Technology (IT) systems and Non-IT elements that would be affected by the Y2K date change. The primary goal of the CDC Project Office is to ensure that all current and future IRS systems are compliant prior to January 1, 2000.

In helping to achieve this goal, the CDC Project Office is assisted in the conversion of IRS components by support organizations, work groups, teams, and steering committees.

In carrying out its oversight responsibilities, the CDC Project Office: established overall project policies and procedures; issued conversion standards; developed methodologies used for measuring conversion progress; and created conversion schedules and milestones. The Integrated Network and Operations Management System (INOMS), an inventory which contains information for all IRS applications and Commercial Off-the-Shelf (COTS) products, was modified to meet the CDC Project Office’s need for an automated tracking mechanism and serves as the primary control for tracking the overall conversion effort.

Currently, the IRS uses a 2-digit year date format representation in its computer systems, which may affect mathematical calculations and other critical computer operations after December 31, 1999. Consequently, the CDC Project Office adopted the National Institute of Standards and Technology standard of expanding all 2-digit year fields to 4-digit year fields. To ensure all existing and developmental IRS applications are operational and functioning as intended to this standard, the CDC Project Office established a 14-step process to be followed for components listed in INOMS which require conversion. The primary activities within the 14 steps involve identifying the components for conversion, performing impact analysis to assess the scope of conversion requirements, determining source code compliance, performing the conversion, unit and system testing, and certifying the converted components. As each component completes a given milestone, field site personnel update INOMS to reflect the completion date. To account for those instances when a component is not taken through the 14-step process and will remain in service after December 31, 1999, an approved waiver must be obtained.

Each week the CDC Project Office assigns a status indicator based on the variance between the planned and actual percentage of conversion activities. The CDC Project Office uses status information reported on INOMS to create the Dashboard. The Y2K Project Dashboard is a one-page report that is color-coded red (more than 15 percent late), yellow (greater than 5 and less than 15 percent late), and green (less than 5 percent late) to indicate the progress status for the 13 major CDC project areas. During a phase, progress can be assessed for the conversion activities by dividing actual number of days that have lapsed by the number of days planned to complete a phase, to reflect the percentage completed for each phase. For those components in yellow or red status, the CDC Project Office requests confirmation from the owners as to the current status of the component and to also determine how the project intends to get back on schedule.

Results

The CDC Project Office established a number of management controls to provide an appropriate control framework to ensure that the goals and objectives of the conversion effort will be met. The CDC Project Office also established an intranet website used to disseminate policies and procedures and provide overall project information. Further, the CDC Project Office holds numerous weekly meetings with responsible executives from Information Systems (IS) Division and field and customer organizations to discuss conversion progress for software applications, COTS products, inventory issues, External Trading Partners, and century date issues and risks. Specifically in these meetings, the CDC Project Office provides guidance and direction to each of the supporting organizations and project offices.

Although the management controls established by the CDC Project Office provide an appropriate control framework to ensure that the organization’s goals and objectives will be met, the effectiveness of management oversight can be strengthened.

The effectiveness of management oversight can be strengthened by: the independent validation and verification of information reported to the CDC Project Office; the establishment of a control technique to identify overall schedule slippage and to eliminate what could become crisis management; and a more timely identification of the impact in delaying Y2K conversion decisions for certain components.

Controls Are Needed to Ensure the Reliability of Information Used to Manage the Conversion

The CDC Project Office does not independently validate or verify INOMS information that it uses to monitor the conversion effort. The information reported on INOMS reflects the current status of the conversion and certification of all IRS IT components. This information is also used to periodically report on the overall progress of the IRS conversion internally, and to the Department of the Treasury, the Office of Management and Budget, the General Accounting Office (GAO) and the Congress.

Since 1998, the former IRS Office of Chief Inspector and the Treasury Inspector General for Tax Administration (TIGTA) have reported several instances where information entered into INOMS and reported to the CDC Project Office was incorrect. In many instances, the information reported indicated a favorable conversion status; however, this information was later determined to be inaccurate. For example, we identified:

• The Tier II Program Office monitors 66 minicomputer systems for Y2K compliance. These systems were scheduled to be converted by January 31, 1999. TIGTA identified at least 22 Tier II systems which did not meet the January 31, 1999, due date for conversion, 12 of which are considered mission critical. Rather than classifying these systems as red, which indicated the milestones as past due and would require enhanced oversight by the CDC Project Office, many of the milestones were extended by the various systems Project Offices, resulting in systems being classified as either green or yellow. Five of the systems’ conversion dates are now scheduled for September and October 1999.
• The CDC Project Office requires that three milestone dates be entered into INOMS to assist in tracking component progress. We identified inaccuracies in some milestone dates on INOMS related to the Y2K conversion efforts being coordinated with banks and other private organizations and government agencies (i.e., "external trading partners"). In 125 data files reviewed, we found that 28 (22 percent) had at least 1 milestone date inaccurately recorded as completed.
• TIGTA has made reference in a number of reports that significant risk exists that the INOMS inventory may be incomplete or inaccurate, and reliance on the information is insufficient when monitoring the conversion. The Tier II Site Coordination Office is currently using a phased-milestone approach to complete its Y2K activities. The final phase is the process of transmitting the application to each site where it will be installed. In February 1999, the Tier II Site Coordination Office indicated the inventory was 99 percent accurate. However, no reconciliation had been performed between the INOMS inventory and the application and operational owners.
• The Telecommunications Project Office has also relied on INOMS to identify its equipment items needing conversion. However, the Telecommunications Project Office did not reconcile its inventory listing with the physical inventory located in the service centers. Consequently, a significant amount of telecommunication equipment could not be found on the inventory maintained by the IRS. TIGTA identified that the Telecommunication’s inventory was 43 percent inaccurate. IRS’ Inventory Steering Committee confirmed this error rate. The omission of items from the inventory will hamper the IRS’ efforts to furnish a telecommunications environment that will operate without problems in the year 2000.

Each week the CDC Project Office assigns a status indicator based on the variance between the planned and actual percentage of conversion activities. Upon completion of its review of the status information, the CDC Project Office contacts the owners to request a narrative explanation as to why component status is yellow or red. In many instances, responses from project partners are either not received or do not contain the information regarding why they are behind schedule, how they plan to bring the project back on schedule, and the revised completion dates.

The Deputy Chief Information Officer, Systems, and CDC Project Office management stated that independent validation and verification are not considered a part of their responsibilities to be performed. The CDC Project Office relies upon the accuracy of information provided by the project owners and responsible executives and does not perform independent validation and verification of the information reported. Further, the IRS has relied on GAO and TIGTA to provide both independent validation and verification of information and to point out issues and concerns with the conversion effort. GAO’s report, Year 2000 Computing Crisis, indicated that agencies should consider subjecting their Y2K program to independent validation and verification efforts.

By placing reliance on information which is not independently validated or verified when received from the project partners, the CDC Project Office may not be ensuring the effectiveness of established controls, such as a Risk Management and Contingency Management Plan. Incorrect information could lessen the perceived severity level of a potentially high-risk area. Further, incorrect information could delay the development of a Contingency Management Plan. For example:

• The IRS implemented a structured process to manage risks associated with the CDC conversion effort. A risk is the probability of an event occurring that adversely impacts the CDC project mission. The Director, CDC Project Office, subjectively sets risk severity levels and makes the final decision whether to remove a risk from the risk management database or lower the severity rating. These decisions are based on information that is received from the working groups, and inaccurate information may adversely influence those decisions.
• The CDC Contingency Management Plan Procedures require that when a conversion effort is in red status and more than 40 percent of the phase is complete, the CDC Project Office should send out an Early Warning Memorandum (EWM). The EWM directs the technical owners to take immediate action to get the conversion back on schedule. When the phase reaches 60 percent complete, the CDC Project Office issues a Contingency Plan Request Memorandum (CPRM) for conversion efforts in red status. When this occurs, technical owners are required to provide a response to the CDC Project Office within 60 days. The response should contain a contingency plan alternative, resources required for implementation, impact on ongoing conversion efforts, and contingency plan milestones.

Between June 30, 1998, and February 16, 1999, 133 CPRMs were generated, of which 128 were either cancelled or suspended as the projects moved from red to green status within 30 days of issuance. If projects reflect an optimistic view by changing the status, the CDC Project Office may fail to detect the need for a Contingency Management Plan.

1. The CDC Project office should establish an effective control which ensures the accuracy and reliability of information used to manage the conversion effort.

Management’s Response: The CDC Project Office implemented several initiatives to validate information provided to manage the conversion effort. These initiatives include independent verification and validation, independent audit and readiness verification, a wall-to-wall inventory, audits of system owner sponsored end-to-end tests, follow-up on TIGTA/GAO audits, and various oversight meetings.

The Century Date Change Project Office Needs to Develop a Control Technique to Identify Interdependencies Among Events, Activities, and Schedule Slippage

The CDC Project Office does not have a technique to identify critical schedule slippage which would identify the interdependencies among events, activities, and schedule slippage, as well as, the possible impact on the overall Y2K conversion. As January 1, 2000, approaches, increased demands will be placed upon resources required to complete the conversion of IRS systems. In addition, between now and December 31, 1999, the CDC Project Office must oversee a number of significant conversion events and activities, many of which will have interdependencies among them. For example, testing that is normally performed prior to the filing season and the certification that the systems are working properly will have to be completed much earlier because of demands for end-to-end testing and issues surrounding the Y2K conversion. In addition, delays in reviewing programming changes as a result of delays in completing the IRS inventory cannot immediately be assessed to determine the impact on the overall project. Although the CDC Project Office schedules provide specific conversion information, they do not provide how milestone delays may affect the overall project. In addition, the CDC Project Office does not have a technique to identify critical schedule slippage in time for corrective action.

Typically, a large project encompasses a long series of milestones, which can be cumbersome. However, crucial milestones, referred to as a critical path, are those that, if not completed on schedule, can cause slippage in subsequent events and milestones. A milestone schedule provides the basis for scheduling, establishing risk, and coordination of objectives and control among projects. The level of detail should be sufficient to reflect the integration of efforts. However, the milestone schedule need not be at a level which ends up having a cost much higher than the benefit to be derived from its use. The milestone schedule should show the interdependencies between events and activities to provide an up-to-date picture of operations at all times. Finally, the milestone schedule can greatly help to reduce or eliminate crisis management.

Any delays experienced by the CDC Project Office could affect significant activities or events, such as the end-to-end testing, and put the entire Y2K conversion at risk. Establishment of a milestone schedule could show the CDC Project Office the impact of any delays. Our assessment of the Y2K Conversion Process identified instances where a milestone schedule would have indicated a delay in the overall project conversion:

• The IS Division, Systems Software Branch, was not aware of what progress the field sites were making in implementing upgrades and fixes relating to whether products were Y2K compliant, and whether critical or priority levels of implementation and installation had been performed for mainframe computer and COTS products. Further, the CDC Project Office was not ensuring that the operational owners were establishing timelines and deadlines for their implementation schedules. Delays in the implementation and installation of fixes and upgrades could directly impact end-to-end testing initiatives.
• Within Tier II, the Site Coordination Office identified 49 mission critical and 53 non-mission critical computers which required replacement or where hardware upgrades were required to support the Y2K environment. The Site Coordination Office is currently using a 5-milestone approach. Milestone 4 is the process of upgrading the production platforms at each site and making them Y2K compliant. The Milestone 4 target date of February 13, 1999, was not met and the impact to the Y2K initiative has not been established.

GAO and Booz-Allen Hamilton, a contractor hired by the CDC Project Office, have recommended developing an integrated master schedule of all critical Y2K activities. Specifically, Booz-Allen Hamilton indicated that without a high-level chart of the dependencies, the CDC Project Office lacks the IRS-wide perspective needed to steer the project partners. In addition, the CDC project partners cannot anticipate changes in IS’ staff resources or develop alternative conversion and testing schedules.

The CDC Project Office does maintain a master schedule that rolls together the work breakdown structure for all of the supporting organizations. Additionally, the CDC Project Office has developed a schedule to show the subset of program activities that will be performed at or directly impact a specific location. Both of these schedules can be used to identify missed milestones, but neither schedule is able to indicate to the CDC Project Office when missed milestones will affect the overall project.

1. The CDC Project Office should develop a technique to identify interdependencies among events, activities, and schedule slippage to allow it to better manage delays within the overall Y2K project.

Management’s Response: The CDC Project Office will monitor dependencies at weekly progress meetings, bi-weekly oversight and risk meetings, and review implementation dates identified in the Location Specific Deployment Schedule and the Infrastructure Matrix.

The Impact of Delaying Year 2000 Conversion Decisions for Certain Components Should be More Timely Assessed

The CDC Project Office established the waiver process to evaluate any system or component not taken through the 14-step conversion which would remain in-service after December 31, 1999. In August 1998, the CDC Project Office expanded the waiver process to include the movement of components into Phases 6 and 7 for components to be retired after January 31, 1999; components to be converted but not completed by January 31, 1999; or new development. Components may not be moved to a later phase without an approved waiver. A risk analysis to determine the impact on end-to-end testing and an evaluation to determine the need for a contingency plan must first be completed before a waiver is granted.

Between the end of Phase 5 and the beginning of Phase 6, the time period between January through March 1999, waiver requests received by the CDC Project Office increased 373 percent (26 to 123). The increase in the number of requests for waivers placed a significant burden on the CDC Project Office’s ability to timely review and approve these requests.

Between October 1, 1998, and January 25, 1999, there were an average of 26 open waivers which were in the process of being reviewed and awaiting approval. However, during the 6 weeks after the end of Phase 5, there were an average of 123 open waivers. Our analysis of weekly status reports indicated that the CDC Project Office processed an average of seven waivers each week. At this rate, it will take the CDC Project Office approximately 19 weeks to process the remaining waivers. This workload does not take into account any additional waivers still to be received by the CDC Project Office.

Unable to timely process these waivers, the Director, CDC Project Office, instructed organizations to move components into the later phases pending approval of their waivers. As a result, the movement of a component or system to Phases 6 or 7 would take them from being behind schedule to being on schedule for the purpose of conversion tracking. The established waiver process allowed movement of a component, which adjusted the conversion dates and gave the component a new deadline.

The CDC Project Office’s conversion tracking system measures weekly progress as a percentage of activities completed by phase. As a result, revising the waiver process and the resulting movement of components to later phases was not necessary. Any delays of component or system completion could be identified with a red status through conversion tracking and the risk evaluated at that time. However, the CDC Project Office cited that an evaluation of the components placed into Phases 6 and 7 was needed in order to determine impact on end-to-end testing or the need for contingency planning.

In February 1999, the CDC Project Office tasked Booz-Allen Hamilton and SRA to review waivers for impact on end-to-end testing and contingency planning. Booz-Allen Hamilton is reviewing 177 previously approved waivers, which includes those waivers retiring or converting components after January 31, 1999. SRA will review waivers still awaiting approval. On March 15, 1999, during our review, the total number of open waivers was 135.

With the year 2000 quickly approaching, the CDC Project Office must be realistic when determining the amount of work that can be completed in the time remaining. The IRS could be at risk of major computer problems if the systems are not converted or contingency plans established. Therefore, emphasis on those components needing contingency plans should be prioritized.

Recommendations

2. The CDC Project Office should continue to use the contractor to evaluate the waivers for impact on end-to-end testing and contingency planning.

Management’s Response: The CDC Project Office will continue to use the contractor to evaluate the waivers for impact on end-to-end testing and contingency planning. In addition, the CDC Project Office is considering merging the two SRA teams in order to consolidate the process. All information necessary to determine approval or denial of the waiver, as well as to analyze the waiver request for potential risk requiring a contingency plan request memo to be issued, would be gathered at one time.

3. The CDC Project Office should prioritize non-compliant systems and concentrate resources on the highest priority projects requiring contingency planning.

Management’s Response: The CDC Project Office will use fact sheets to gather information in order to prioritize requests for waivers and to determine the need to revise or add a contingency plan or whether no additional action is warranted. This process assures an efficient use of the CDC Project Office’s resources in tracking the progress of CPRM actions.

Conclusion

The Y2K conversion effort continues to present a significant challenge to the IRS’ operations. The CDC Project Office’s management oversight is necessary to ensure that the goals and objectives of the conversion effort will be met. The strengthening of IRS management oversight will help ensure that the Y2K conversion is effectively monitored and accurately reported.

Detailed Objective, Scope, and Methodology

The overall objective of our review was to assess the Internal Revenue Service’s (IRS) Century Date Change (CDC) Project Office’s effectiveness in providing management oversight to ensure that the goals and objectives of the IRS’ Year 2000 (Y2K) efforts will be met. Specifically, we:

1. Determined whether the CDC Project Office had identified and implemented adequate program management controls to manage the IRS’ Y2K conversion efforts. Major emphasis was focused in the areas of: conversion tracking, the master schedule, independent validation and verification, budget, contingency planning, risk management, and waivers. These key controls were initially identified by the Treasury Inspector General for Tax Administration (TIGTA) and later confirmed with the CDC Project Office.

1. Reviewed governmental and departmental documents (i.e., General Accounting Office (GAO) and Treasury guidelines) to identify recommended management practices that agencies could implement to aid in administering Y2K programs.
2. Determined the types of management controls established within the IRS’ Y2K program.

1. Determined whether the desired objectives of program management controls are preventive, detective, limiting, or corrective in nature.
2. Determined the type of risks each control was designed to alleviate.
3. Performed walk-throughs on selected controls to determine whether they have been established and implemented, and documented how they were intended to work within a given process.

1. Obtained internally developed documents on key program management controls to aid in the audit testing and evaluation process.

1. Reviewed GAO and TIGTA audit reports issued from prior Y2K audits to identify issues and corrective actions relating to the improvement of selected program management controls.

1. Determined whether the respective corrective actions have been implemented.
2. Determined whether any additional program management controls were, or should have been, implemented as a result of management’s corrective actions.

1. Interviewed IRS senior management affiliated with the CDC effort and personnel within selected CDC working groups to validate the Project Office’s current program level roles and responsibilities, and determined whether clear lines of responsibility have been established.

1. Identified the role and responsibilities of the CDC Project Office. Also, confirmed the duties of this office with assigned CDC Project Office management.
2. Determined whether select CDC Working Groups, which support major Y2K functions, understand the responsibilities of the Project Office, and whether the groups have fully accepted their assigned conversion requirements.
3. Determined whether and how often the CDC Project Office coordinates with responsible working groups/support organizations to clarify duties and conversion requirements.

1. Determined whether key program management controls were effective and provided the CDC Project Office with accurate information upon which to make decisions.

1. Analyzed the program management controls and identified those that pose the highest/greatest potential risk to the Y2K program, if not working as intended.
2. Assessed certain key program management controls, based on the associated risk factor(s), to determine whether the controls allow management to meet program objectives.

1. Determined whether the system of program level controls was documented.
2. Determined whether controls have been established to validate or verify reported information.
3. Determined whether information reported to the CDC Project Office, was accurate, reliable, timely, complete, and useful.

1. Determined whether management had established controls to monitor the implementation of policies and procedures.
2. Determined whether CDC Project Office management performed periodic assessments of the controls to ensure they were working as intended.

Major Contributors to This Report

Scott Wilson, Associate Inspector General for Audit (Information Systems Programs)

Michael Phillips, Director

Edward Coleman, Audit Manager

Vincent J. Dell’Orto, Audit Manager

S. Kent Johnson, Senior Auditor

Barbara Sailhamer, Acting Senior Auditor

Ruth Ware, Senior Auditor

Bobbie Draudt, Auditor

Art Granger, Auditor

Melvin Lindsey, Auditor

Report Distribution List

Deputy Commissioner Modernization C:DM

Office of the Chief Counsel CC

Chief Information Officer IS

Deputy Chief Information Officer, Systems IS

Assistant Commissioner (Program Evaluation and Risk Analysis) M:OP

Director, Century Date Change Program Office IS:CD

National Director for Legislative Affairs CL:LA

Office of Management Controls M:CFO:A:M

Audit Liaison (Attn: Frank Guarino) IS:I:IS:O:A

Audit Liaison, Office of Program Oversight IS:PM:O

Management’s Response to the Draft Report