Summary
Millions of taxpayers ask IRS questions about tax law each year. While the accuracy of IRS's answers has improved in some cases, it is still not always what taxpayers or Congress expect. Concerns about accuracy have raised questions about the adequacy of the training IRS provides to its taxpayer assistance staff. Because of these questions, GAO was asked to assess the extent to which IRS's planning and evaluation of its taxpayer assistor training conformed to guidance published by GAO and others. Planning and evaluation are part of a feedback loop whereby lessons from one year can be applied to making improvements in future years.
IRS devotes significant resources to training its tax law assistors to answer questions, by telephone and at walk-in sites, and prepare tax returns. Although IRS cannot separate the costs of training tax law assistors from other assistance staff, the thousands of staff devoted to providing tax law assistance receive training each year. The training includes classroom and computer-based training on such subjects as tax law and communication. While IRS has some data on travel and course development costs associated with training, it does not have data on what is likely the largest cost component, the value of staff time devoted to tax law training. Responsibility for training IRS taxpayer assistance staff is decentralized. IRS's Human Capital Office provides guidance and sets policy. The two divisions responsible for tax law assistance each have a human resources office with technical staff that are assigned to the various taxpayer assistance programs. Generally speaking, the taxpayer assistance programs share responsibility for planning training with human resource staff. The human resource staff are responsible for training evaluations. IRS's planning for taxpayer assistor training could be enhanced by a more strategic approach. To their credit, some taxpayer assistance programs clearly communicated the importance of training to staff and had knowledge and skills inventories. All the units had analyses of some of the individual factors that affect accuracy, such as the quality and use of their taxpayer assistance guidance. However, none of the programs had long-term goals for either accuracy or training or had benchmarked their training efforts against those of other organizations. Nor had they done a combined analysis of the major factors that affect accuracy in order to determine their relative importance. Setting long-term goals and analyzing training needs and relative impacts are important steps in strategic planning. Goals can provide a yardstick for measuring progress and benchmarking can help identify best practices. Analyses of relative impacts can help IRS make informed decisions about a strategy for improving accuracy, including the importance of training compared to other factors in that strategy. The taxpayer assistance programs routinely conducted evaluations of their training efforts. However, with one exception, the evaluations did not include analyses of the impact of training on the accuracy of assistance. Instead, the units conducted less sophisticated analyses of more immediate impacts, such as trainees' satisfaction. Given the importance of accurate answers to taxpayers' questions and the resources spent on training, the four assistance programs would benefit from more sophisticated evaluations of the effectiveness of training. One program had recognized the potential value of a more sophisticated evaluation of training and pilot tested an analysis in 2004. The value of evaluation is that it provides feedback about the effectiveness of one year's training that can be used to plan improvements to future training.
Recommendations
Our recommendations from this work are listed below with a Contact for more information. Status will change
from "In process" to "Implemented" or "Not implemented" based on our follow up work.
Director:
Team:
Phone:
James R. White
Government Accountability Office: Strategic Issues
(202) 512-5594
Recommendations for Executive Action
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should establish a long-term goal for the accuracy of taxpayer assistance.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: Implemented
Comments: Based on our recommendation, IRS established a long-term goal for the accuracy of taxpayer assistance as of November 15, 2006. Based on information from IRS, the goal will be tracked through IRS's Wage and Investment Division's Accounts Management and Assistance program and includes fiscal year targets for customer accounts resolved, toll-free tax law, and toll-free accounts.
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should establish goals and measures for training and development logically linked to accuracy.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: Implemented
Comments: Based on our recommendation, IRS established a long-term goal for the accuracy of taxpayer assistance, including fiscal year targets and measures for customer accounts resolved, toll-free tax law, and toll-free accounts. The goal and related targets and measures were completed as of November 15, 2006.
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should determine and track the relative importance of the various factors, including training, that affect accuracy.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: In process
Comments: IRS has delayed the implementation of this recommendation until October 15, 2008. IRS plans to expand Embedded Quality Reports System to include links to key course content objectives for a more direct assessment of training effectiveness as it relates to performance of specific applications and case types. Modification of EQRS will be included in the fiscal year 2009 planning process.
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should benchmark training and development programs against high-performing organizations.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: In process
Comments: IRS still plans to benchmark its taxpayer assistance training programs and use the results from the benchmarking to assess the state of taxpayer assistance training. Due to organizational changes, IRS does not have a completion date for implementing this recommendation.
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should conduct skills and knowledge gap analyses for all taxpayer assistance programs, to include identifying and comparing current skills to long-term skill needs.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: Implemented
Comments: IRS implemented the field assistance technical assistance battery (FA TAB) in June 2006; all employees are expected to complete the assessment. The FA TAB helps to identify gaps in training and/or reference materials based on compiled FA TAB results so that gaps may be filled to meet the needs of employees, thus improving quality. According to IRS, the implementation of the FA TAB indicates that the values of such assessments are inculcated into its near-term goals and long-term strategies.
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. Specifically, in the area of planning, IRS should consider costs, benefits, ways to mitigate risks, and the appropriate level of investment for training and development efforts.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: Implemented
Comments: According to IRS, its annual training plan formulation process allows business divisions to identify their training priorities based on strategic plans and the level of funding appropriated for that year. When the Service-wide long-term goals are finalized and approved, the training plan will be developed considering costs, benefits, risk mitigation, and appropriate levels of investment for training and development efforts.
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include an analysis of the feasibility and cost effectiveness of alternative level 4 methodologies and a data collection and analysis plan.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: In process
Comments: IRS has delayed the implementation of this recommendation until 4/15/2009. Prior attempts to conduct Level 4 evaluation have not been successful for a variety of reasons, including lack of appropriate data (Level 1-3 information), lack of Level 4 design knowledge, and resource issues (staff, dollars, participants, loss of opportunity, etc.)
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include a comparison of the accuracy benefits to the costs of the training.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: In process
Comments: IRS has delayed the implementation of this recommendation until 4/15/2009. Prior attempts to conduct Level 4 evaluation have not been successful for a variety of reasons, including lack of appropriate data (Level 1-3 information), lack of Level 4 design knowledge, and resource issues (staff, dollars, participants, loss of opportunity, etc.)
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include benchmarking of the analytical methods and the results of the data analysis against high-performing organizations.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: In process
Comments: IRS has delayed the implementation of this recommendation until 4/15/2009. Prior attempts to conduct Level 4 evaluation have not been successful for a variety of reasons, including lack of appropriate data (Level 1-3 information), lack of Level 4 design knowledge, and resource issues (staff, dollars, participants, loss of opportunity, etc.)
Recommendation: The Commissioner of Internal Revenue should take appropriate action to ensure that IRS's planning and evaluations of its taxpayer assistance training and development efforts better conform to guidance for strategically planning and evaluating the training efforts. In the area of evaluation, IRS should continue to pursue the level 4 pilot in TEC and, if that analysis is shown to be feasible, conduct level 4 evaluations for its other taxpayer assistance training and development programs. The evaluations should include an analysis of stakeholder assessments of the impact of training on accuracy.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: In process
Comments: IRS has delayed the implementation of this recommendation until 4/15/2009. Prior attempts to conduct Level 4 evaluation have not been successful for a variety of reasons, including lack of appropriate data (Level 1-3 information), lack of Level 4 design knowledge, and resource issues (staff, dollars, participants, loss of opportunity, etc.)
Recommendation: IRS should replace the defunct Administrative Corporate Education System database, which had been used to store level 2 data, with another database for this purpose.
Agency Affected: Department of the Treasury: Internal Revenue Service
Status: Implemented
Comments: In October 2007, IRS implemented a process to collect level 2 data and store it in the Enterprise Learning Management System (ELMS), which replaces ACES. In addition, IRS's Human Capital's Office of Learning Technology asked IRS training directors to review and update items in the ELMS course catalog to ensure they accurately reflect content which requires a level 2 evaluation.
