Management Advisory Report: Improved Project Management is Needed in Modernizing the Internal Revenue Service’s Walk-In Program Management Information System
May 2000
Reference Number: 2000-40-067
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
May 11, 2000
MEMORANDUM FOR COMMISSIONER ROSSOTTI
FROM: Pamela J. Gardiner /s/ Pamela J. Gardiner
Deputy Inspector General for Audit
SUBJECT: Final Management Advisory Report - Improved Project Management is Needed in Modernizing the Internal Revenue Service’s Walk-In Program Management Information System
This report presents the results of our review of the Internal Revenue Service’s (IRS) Walk-In Program Management Information System. Our review focused on Customer Service’s acquisition of
the Q-Matic system to improve Walk-In Program management information. We reviewed the investment management process used to implement Q-Matic.In summary, we found that Customer Service has not met all requirements of a disciplined investment justification process in acquiring the Q-Matic system, including the development of a complete and accurate business case. Specifically, implementation costs have not been fully identified and documented, and funding for the project has not been secured. Plans to address problems identified during a test period have not been formally documented to ensure the problems are monitored and resolved. Our recommendations will assist the IRS in ensuring that the Q-Matic system delivers expected outcomes within expected implementation costs.
IRS management agreed to the findings and recommendations presented in this report. Management’s comments have been incorporated into the report where appropriate, and the full text of their comments is included as an appendix.
Copies of this report are also being sent to the IRS managers who are affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions, or your staff may call Walter E. Arrison, Associate Inspector General for Audit (Wage and Investment Income Programs), at (770) 455-2475.
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Management’s Response to the Draft Report
This management advisory report provides information about and recommendations for improving the project management of the Internal Revenue Service’s (IRS) Walk-In Program Management Information System (MIS).
The IRS Restructuring and Reform Act of 1998 resulted in a new IRS mission statement which reflects a shift in tax administration that emphasizes taxpayer service. The emphasis on service to taxpayers to accomplish the agency’s mission has elevated the significance of the IRS’ Customer Service programs including the Walk-In Program. Each year, approximately 10 million taxpayers seek tax assistance at more than 400 walk-in sites.
The General Accounting Office (GAO), IRS Inspection Service (now the Treasury Inspector General for Tax Administration), and Customer Service have reported that the Walk-In Program does not have accurate or sufficient management information to make informed program decisions, and that the Program does not have a system to determine the quality of walk-in service. We initiated this audit to determine the effectiveness of the Walk-In Program MIS in capturing information needed to produce an effective customer-driven program. Accurate data regarding the reasons taxpayers seek face-to-face assistance and the quality of that assistance are critical if the Walk-In Program is to support the IRS’ mission of providing top quality service to taxpayers.
Results
Customer Service management implemented a quality review process that should provide management with information needed to measure the quality of service provided to taxpayers at walk-in locations. Customer Service has also begun implementing the Q-Matic, a technologically advanced MIS to replace the Walk-In Program’s current MIS, the Resource Management Information System (RMIS). However, Customer Service needs to implement the Q-Matic in accordance with Office of Management and Budget requirements to ensure that it delivers all expected outcomes within prescribed costs and time frames.
Customer Service Can Better Manage Efforts to Improve the Walk-In Program Management Information System
Customer Service is implementing the Q-Matic at an estimated cost of approximately $5.5 million. When fully implemented, all individual Q-Matic systems installed at walk-in sites will be electronically linked to form the MIS.
However, this acquisition is not being managed according to all requirements of an investment management process, including the development of a complete and accurate business case. Implementation costs have not been fully identified and documented, and funding for the Project has not been secured. Plans to address problems identified during a test period have not been formally documented to ensure the problems are monitored and resolved. As a result, the $5.5 million cost estimate may be even higher, continued funding for the Project is uncertain, and project management controls are not in place to ensure that the system will deliver the expected outcome of an improved MIS that can replace the RMIS.
Summary of Recommendations
To ensure that the Q-Matic will be an effective Walk-In Program MIS, Customer Service should update the Q-Matic business case or project plan to reflect complete and accurate implementation costs and to secure full funding for the Project. Management needs to determine and document how the individual Q-Matic systems will be networked, whether crucial consolidated reports can be delivered, and who will be responsible for these tasks, the IRS or the vendor. Additionally, management needs to identify the type of system to be used for small walk-in offices and ensure that accurate data on all taxpayer assistance are entered into the Q-Matic.
Management’s Response: Customer Service management agreed with the findings and recommendations in this report and has implemented or planned corrective actions. The Q-Matic project plan will be updated with the necessary revisions. Customer Service is working with Information Systems and has prepared a Request for Information Services to develop a network solution. Once the network has been established, Q-Matic will be able to provide consolidated reports. Customer Service will identify and test an appropriate Q-Matic system for the small walk-in offices.
To improve the accuracy of taxpayer assistance data, Customer Service will continue to explore ways to use Q-Matic to capture information for taxpayers served at workstations located away from the area where the Q-Matic equipment is located. They have created a training class to instruct all Customer Service Representatives (CSR) on the proper use of Q-Matic closing codes. Customer Service will also establish a process of examining monthly closing code reports and contacting CSR managers to determine reasons for using the "Other Issues" code and if additional closing code(s) should be established.
Our overall objective was to evaluate the effectiveness of the Internal Revenue Service’s (IRS) Walk-In Program Management Information System (MIS) in capturing information needed to produce an effective customer-driven program. Accurate data regarding the reasons taxpayers seek face-to-face assistance and the quality of that assistance are critical if the Walk-In Program is to support the IRS’ mission of providing top quality service to taxpayers.
To accomplish this objective, we evaluated the effectiveness of management actions to:
We conducted the audit at the Customer Service Office of the National Director, Education, Walk-In and Correspondence Improvement Division, and the Upstate New York District Office. The audit was performed from December 1998 to October 1999. We interviewed Walk-In Program analysts and reviewed documentation available at the National Office related to the quality review process and the acquisition and testing of the Q-Matic system. This included key documents titled, "Q-Matic Business Case," "FY 98 Q-Matic Test Report," and "Walk-In Q-Matic Project Scope."
Details of our audit objective, scope, and methodology are presented in Appendix I. Major contributors to this report are listed in Appendix II.
Each year approximately 10 million taxpayers seek tax assistance at more than 400 IRS walk-in sites. General Accounting Office (GAO) and IRS Inspection Service (now the Treasury Inspector General for Tax Administration) reports as well as Customer Service’s own initiatives have shown that the IRS cannot deliver the most effective customer service at its walk-in sites unless it can measure the quality of service and improve the accuracy and completeness of its management information regarding customer needs. In response to this assessment, Customer Service management (hereafter referred to as Customer Service), is developing a Walk-In Quality Review Program. Customer Service is also implementing the Q-Matic, a technologically advanced MIS to replace the RMIS.
The Q-Matic system is designed to provide more accurate and detailed data than the RMIS regarding taxpayers served and the type of service provided. This is critical for identifying customer needs and the ability to tailor a customer-driven program. The Q-Matic also has other features beneficial to improving customer service that are not provided by the RMIS. These include real-time management reports of walk-in site activity, remote monitoring of several walk-in sites simultaneously, informing customers of their expected wait time, and calling them at the appropriate time. As of the end of Fiscal Year (FY) 1999, the Q-Matic system had been placed in 33 walk-in sites.
This management advisory report provides information about and recommendations for improving the project management of the IRS’ Walk-In Program MIS.
Customer Service has taken significant actions to begin measuring the quality of service provided to taxpayers at walk-in sites. Customer Service has developed a quality review process consisting of filing season Walk-In Readiness Visits and Customer Simulation Visits. Continued efforts to formulate and refine the quality review process should enable the IRS to fill the gap that has existed in evaluating the quality of its walk-in service.
A properly implemented Q-Matic system should result in improved management information. However, Customer Service’s management of the implementation of this system needs improvement. Specifically, an adequate business case was not prepared to accurately identify the Q-Matic implementation costs and to identify and correct problems encountered during the implementation. Additionally, funding for the complete implementation of the Q-Matic has not been secured.
Customer Service Can Better Manage Efforts to Improve the Walk-In Program Management Information System
Customer Service has not prepared an adequate business case for the Q-Matic acquisition. Specifically, the Customer Service document entitled, "Q-Matic Business Case" does not meet the Office of Management and Budget’s (OMB) business case requirements as stated in the IRS Investment Handbook. It does not provide adequate data needed by Customer Service to make informed business decisions. The "Walk-In Q-Matic Project Scope" document also lacks specific costs and details concerning resolutions to identified problems.
The IRS Investment Handbook states that the OMB requires sufficient justification be provided for major acquisitions. A business case is the vehicle for documenting the justification. It is a structured proposal for business improvement and forms the basis for understanding all potential costs, benefits, alternatives, and risks. A properly developed business case should:
The lack of an adequate business case for the Q-Matic acquisition occurred because Customer Service did not follow guidelines contained in the IRS Investment Handbook, which is the IRS’ guide for initiating, analyzing, and managing all major investments.
As stated below, insufficient documented data pose specific risks to the successful implementation of the Q-Matic system.
Costs identified in the business case were not complete
The business case states that the estimated cost to equip walk-in offices with the Q-Matic system is approximately $5.5 million. This figure included only the hardware and software costs of the Q-Matic systems. The estimate does not include telecommunications costs, furniture and office redesign costs, the cost of interconnecting the systems, operations and maintenance costs, and labor costs. Labor costs include Information Systems (IS) field function costs, contractor costs, and training and travel.
Implementation costs through the end of FY 2000 will exceed $4.7 million for 148 sites, or approximately one third of the walk-in sites that will require a Q-Matic system. Estimated costs have not been identified for 46 sites that management has determined will require major office redesign to install the Q-Matic, and for more than 200 small walk-in sites with fewer than three assistors that will also require Q-Matic systems. The particular system to be acquired for the small walk-in sites, those that do not require full-scale Q-Matic systems, has not yet been identified. Therefore, the costs are unknown as well as the manner in which they will be networked with the other Q-Matic systems.
For proper monitoring, the total cost of implementing the system through the expected project completion date of December 31, 2002, should be identified and updated as necessary.
A formal documented plan for resolving problems identified during the implementation of the Q-Matic system does not exist
The Q-Matic system is expected to help Customer Service deliver its commitment to provide an accurate accounting of customers’ issues. However, during the 1998 filing season, at five walk-in offices, Customer Service conducted a test of the Q-Matic that identified problems that resulted in incomplete and inaccurate Q-Matic data. There is no formal, documented plan for addressing these still unresolved problems. Three of these problems are discussed below.
The system cannot produce necessary management information reports. During the test, management discovered that the Q-Matic could not produce critical management information reports. At present, the Q-Matic can only provide daily closing code reports of walk-in activity for individual walk-in sites, but cannot consolidate the closing code report into weekly, monthly, or yearly reports.
The vendor informed Customer Service that it can develop the reports for an additional fee. Information Systems management informed Customer Service that National Office programmers can provide the assistance to produce the necessary reports, but they do not have the resources at this time due to Year 2000 computer programming priorities. Customer Service has no documented plan in place for resolving this issue, is not yet certain whether the vendor or IRS personnel will assist with the programming tasks, and at what cost to Customer Service and the time frame.
In addition, until the individual Q-Matic systems are installed in each walk-in site with assistors, including part-time offices and those offices with only one or two assistors, comparative reports produced by the Q-Matic will be incomplete for use in evaluating the Walk-In Program on a district, regional, or national level.
Space limitations at walk-in sites cause incomplete Q-Matic data. Space limitations at one of the five test sites resulted in assistance being given to taxpayers in another area of the building not connected to the Q-Matic system hardware. As a result, data for approximately 25 to 30 percent of the walk-in customers were not entered into the Q-Matic system. Subsequent to the test, additional Q-Matic equipment was purchased and the problem at this site was resolved. However, during a concurrent audit, we observed the same condition at another high volume walk-in site that was not part of the Q-Matic test. The condition at this site remains, and is not easily corrected due to the number of floors that separate the walk-in office from the location where additional assistance is provided during the filing season. The "Walk-In Q-Matic Project Scope" document does not address this problem. Therefore, the extent to which it may exist is not known, nor are any possible solutions and associated costs.
Excessive use of the "Other Issues" closing code diminishes the accuracy of Q-Matic data. After assisting a taxpayer, the Walk-in employee will enter one or more closing codes into the Q-Matic, to reflect the type of assistance provided. One test site reported that the "Other Issues" closing code was used for a large number of taxpayers assisted. This code should be used only when none of the descriptive closing codes match the assistance given to the taxpayer. The more these codes are used, the less accurate Q-Matic data are in reflecting the assistance actually provided to taxpayers. Frequent use of the "Other Issues" code could indicate a need for additional closing codes or that assistors are choosing it for convenience rather than selecting the proper code.
Roll-out of additional Q-Matic systems to other walk-in sites continues even though no formal plan was developed to address how, or if, the limitations identified during the test report will be resolved, by whom, and at what cost.
The Q-Matic implementation has not been fully funded
Funding for the complete implementation of the Q-Matic has not been secured. The "Walk-In Q-Matic Project Scope" document lists funding concerns as a project constraint.
We were advised by National Office Walk-In Program management that the FY 1999 budget of $5 million for the Q-Matic was reduced to $1.2 million in order to fund other IRS budget shortfalls. As a result, Customer Service will use these funds to purchase software upgrades and networking software for the 33 existing systems, and to purchase the Q-Matic systems for 43 additional sites. Beyond this requisition, funding for the more than 300 remaining walk-in sites, including office redesign costs, is uncertain.
Several of the 33 systems already in place were funded not by the National Office as is assumed in the business case estimate, but by regional and district offices that did not want to rely on the uncertainty of National Office funding. The National Office did not have data on the costs to the offices that purchased their own systems. Such a tentative method of funding places the IRS at risk of time and cost overruns in implementing the Q-Matic system.
Until all of the issues raised in this report are resolved, the Q-Matic cannot replace the RMIS. Customer Service will continue expending resources to manually input inaccurate data into the RMIS, while National Office management still will not have the detailed, accurate information necessary to analyze customer needs and improve customer service at the walk-in sites. A properly prepared business case or project plan would document all of these issues as well as the planned solutions. Without such documentation, Customer Service cannot ensure that the cost estimate is accurate or that the Q-Matic will deliver the expected improvements to the current Walk-In Program MIS.
Recommendations
To better manage the implementation of the Q-Matic and to ensure that the Q-Matic will be an effective Walk-In Program MIS, Customer Service should:
Management’s Response: Customer Service agreed to update the project plan making the necessary revisions.
Management’s Response: Customer Service prepared a Request for Information Services (RIS) to develop a network solution. Once the network has been established, the software will be able to provide consolidated national reports.
Management’s Response: Customer Service will meet with members of a modernization redesign team to determine the number of small walk-in offices and the number of employees staffing them. Once they know the smallest offices and where they will be located, Customer Service will test a Q-Matic system that meets their needs.
Management’s Response: As part of the modernization reorganization, Customer Service will be assessing and redesigning space and office locations. They will consider the need to provide for Q-Matic as part of that process. In the interim, in buildings where the Q-Matic system is located in another area, Walk-In personnel will use a manual system to count the number of taxpayers served.
Management’s Response: Customer Service created a training class for Customer Service Representatives (CSR) which includes the proper use of Q-Matic closing codes. Customer Service will establish a process of examining monthly closing code reports and contacting CSR managers to determine reasons for using the "Other Issues" code and if additional closing code(s) should be established.
The IRS has made commitments to provide top quality customer service to taxpayers and to better manage information technology investments. Customer Service can show its support of these commitments by following an investment management process, including preparing an adequate business case document, in implementing the Q-Matic system.
An adequate business case will address the cost identification issues, plans to resolve identified problems, and funding constraints.
Appendix I
Detailed Objective, Scope, and MethodologyOur overall objective was to evaluate the effectiveness of the Internal Revenue Service’s (IRS) Walk-In Program Management Information System (MIS) in capturing information needed to produce an effective customer-driven program. To accomplish this objective, we performed the following audit tests:
Appendix II
Major Contributors to This ReportWalter E. Arrison, Associate Inspector General for Audit (Wage and Investment Income Programs)
Kerry R. Kilpatrick, Director
Richard Hayes, Audit Manager
Mark Nathan, Audit Manager
John O’Rourke, Senior Auditor
Joan Raniolo, Senior Auditor
James McCormick, Auditor
William Simmons, Auditor
Appendix III
Report Distribution ListDeputy Commissioner Operations C:DO
Chief Operations Officer OP
Assistant Commissioner (Customer Service) OP:C
National Director, Education, Walk-In, and Correspondence Improvement OP:C:E
National Director for Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis M:O
Office of the Chief Counsel CC
Office of Management Controls M:CFO:A:M
Audit Liaison (Customer Service) OP:C:E:L
Appendix IV
Management’s Response to the Draft ReportThe response has been removed due to its size. To see the complete response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.