Detailed Information on the
Internal Revenue Service Taxpayer Advocate Service Assessment

Improve program performance by developing a unit cost measure for its casework by 2006 (delayed to 2008). Fall 2008 Update: TAS identified data limitations in implementing Phase I of a new time reporting system that automatically captures the amount of time spent on cases when a case is accessed through TAMIS. These limitations will be addressed when Phase II is implemented, which has been delayed to June, 2009 due to conflicting priorities in the MITS organization. When Phase II is completed, TAMIS will provide a graphical front-end screen that will allow users to capture time spent on cases, even when TAMIS is not used (e.g., research, telephone calls, etc.). TAS must have time data, along with fiscal data and case complexity results, before it can develop a costing system. In the interim, two case advocacy cost indicators have been developed: Closed Cases per Case Advocacy Full Time Equivalent (FTE) and Closed Cases per Direct FTE. In FY 2009, TAS will track reliability and utility of Phase II time data results and results from case advocacy cost indicators, and explore opportunities for integrating the data with performance metrics.

Improve program performance by exploring other means to measure effectiveness in solving systemic problems leading to taxpayer hardship. IRS will report its findings in 2006 for possible inclusion in its FY 2008 Budget. (findings delayed until 2009). Fall 2008 Update: During FY 2007, TAS implemented a formal quality review process (QRP) and established baseline measures for quality and timeliness of closed Advocacy Projects and Immediate Interventions. Throughout FY 2008, TAS management used results obtained from the QRP to track program performance and identify opportunities for product improvement. TAS formally reports these measures on a quarterly basis in its Business Performance Review. At the conclusion of the fiscal year, TAS developed performance goals and associated improvement initiatives targeted at specific quality attributes. Systemic Advocacy will administer its second internal customer satisfaction survey by the end of the 1st quarter FY 2009 and anticipates delivering a full analysis of the survey results by the beginning of the third quarter FY 2009. Recommendations resulting from the analysis of the first survey led to major revisions of the 2008 survey instrument. The revisions will yield data that more readily provides information in relation to required process and/or systems improvements, which should lead to improved customer satisfaction. TAS is currently rolling out its Systemic Advocacy Management System (SAMS) Version 2 , which includes program, procedural, and interface improvements. TAS designed the enhancements with the expectation of improving the efficiency of operations and the satisfaction rates of internal (TAS and IRS) submitters. The FY 2009 survey will include questions geared at determining the impact of system enhancements on submitter satisfaction rates. TAS continues to gather data for the purpose of setting baselines for effectiveness measures. In addition, TAS has computed targets for its efficiency measures for FY 2009 through FY 2011 for Advocacy Projects and Immediate Interventions. TAS made progress in its efforts to identify and rectify sources of rework originating from the operating divisions. The two primary areas of focus are Amended Returns and Combined Annual Wage Reporting. TAS continues to work jointly with the operating divisions to improve the systems associated with these issues. The recommendations center on reducing delays from the Examination Operating Division's "Category A" process, reducing general processing timeframes, improving customer service, and modernizing systems. The National Taxpayer Advocate is in consultation with Wage & Investment leadership regarding implementation of the recommendations. TAS's FY 2008 receipts of Combined Annual Wage Reporting (CAWR) and Federal Unemployment Tax Act (FUTA) cases increased by 25% over case receipts received in FY 2007. These cases involve wage and tax reconciliation, and reporting compliance issues. TAS and SB/SE have identified the effect of the CAWR and FUTA Certification Program on TAS receipts as the targets of a second study. The study includes analyzing data to determine the root causes of the increase in TAS referrals, identifying systemic problems and causes, recommending and testing potential solutions, and monitoring implementation. The study team reviewed the CAWR and FUTA processes, discussed systemic problems and the IRS efforts to address them, and developed a data collection instrument (DCI) for analyzing a sample of TAS CAWR and FUTA cases. Initial results suggest a need to modify TAMIS coding to refine the case selection criteria for proper issue focus. The study team is identifying cases for review, perfecting the DCI, and building a database.

Improve program performance by improving case quality to 91.5 percent by 2006, 93 percent by 2009, and 95 percent by 2014. Fall 2008 Update: TAS almost attained the quality goal in FY 2008 with a cumulative quality rate of 89.9 percent. TAS's overall quality rate has remained constant, near 90 percent, for the last five years. Consistently maintaining this level of quality is a remarkable accomplishment considering the significant increase in case receipts, the growing complexity of case issues, and the staffing challenges TAS has experienced. Case receipts increased 47 percent from 2004 to 2007, while the number of advocates available to work cases decreased by 13 percent. In FY 2008, as a result of additional funding, TAS hired 212 Case Advocates and 43 Intake Advocates. Hiring exceeded attrition for the first time in many years. Budget permitting, TAS plans to complete at least attrition hiring in FY 2009. As a result of the challenging operating environment, TAS has revised its long-term case advocacy quality goals. TAS has completed efforts to identify revised quality attributes to enhance TAS's case quality measurement standards by addressing changes in casework and processing that have occurred since TAS began operating in 2000. TAS solicited recommendations from all employees regarding revising quality standards and conducted final focus groups sessions in September 2008. TAS plans to test the revised standards, develop a database, create an application guide, and brief employees on the revised standards in FY 2009. The new quality standards should be implemented in FY 2010. New goals will be developed after a baseline is established. Beginning in June 2009, TAS will include its customer satisfaction measure in its PART assessment; this measure and associated long-term target goals will be reported in the future.

Measure: Case Closure to Receipt Ratio (IRS chose not to use this performance measure.)

Explanation:Shows the Advocate's ability to keep up with its taxpayer problem case workload. Greater than 100 percent indicates case inventory is dropping.

Measure: Casework Quality

Explanation:Quality based on independent evaluation of eight quality standards for a random sample of Advocate taxpayer problem cases. Actuals are based on statistical samples and are accurate within about +/- 1%.

Measure: Customer Satisfaction with Advocate Assistance (IRS chose not to use this performance measure.)

Explanation:Satisfaction of taxpayers receiving assistance from the Advocate based on random surveys (5-point scale with 1 = very dissatisfied and 5 = very satisfied).

Measure: Advocate Systemic Hardship Case Receipts (IRS chose not to use this performance measure.)

Explanation:The Advocate identifies and helps IRS resolve systemic problems that lead to taxpayer hardship. This measure captures the total cases received as a result of IRS systemic problems. This total should drop as a result of Advocate activity.

Is the program purpose clear?

Explanation: The Office of the Taxpayer Advocate (Advocate) was established in legislation to make certain that taxpayers had an independent advocate within the IRS to resolve individual problems and to propose solutions to systemic problems. Its mission is to: 1. Assist taxpayers in resolving problems with the IRS; 2. Identify areas in which taxpayers have problems in dealing with the IRS; 3. Propose changes in the administrative practices of the IRS to mitigate those problems where possible; and 4. Identify potential legislative changes which may be appropriate to mitigate such problems.

Evidence: The functions of the Advocate are prescribed by Internal Revenue Code Section 7803(c)(2). The statute specifies that the Advocate identify and propose both administrative and legislative recommendations that will mitigate taxpayer problems. Cases are accepted into the Advocate program based on criteria established by Internal Revenue Code Section 7811. The specific criteria for Advocate's acceptance of a taxpayer case fall into one of two major categories: financial/economic hardship and systemic hardship. See the National Taxpayer Advocate 2003 Annual Report to Congress.

Does the program address a specific and existing problem, interest or need?

Explanation: As a result of the Senate Finance Committee hearings of 1997, Congress concluded that some taxpayers were not being treated fairly or equitably by the IRS. The IRS Restructuring and Reform Act of 1998 established the Advocate to function as an independent entity within the IRS to represent taxpayers and to protect their rights in their dealings with IRS. The Advocate is responsible for resolving taxpayer problems on both a case-by-case and a systemic basis.

Evidence: Despite continuing improvements in IRS' taxpayer service and strengthened statutory taxpayer rights, the Advocate is still needed in some cases to ensure fair treatment. During FY 2003, the Advocate provided assistance to 205,053 taxpayers who met criteria as defined in Internal Revenue Code Section 7811.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Advocate plays a unique role in IRS. He/she is charged to act on the taxpayer's behalf both in specific cases and system-wide. He/she is the only employee of the IRS who is allowed and required by statute to publicly take positions on taxpayer issues regardless of whether or not they differ from official IRS positions.

Evidence: Other IRS offices provide service and assistance to taxpayers. IRS' Appeals function even operates independently from other IRS enforcement functions to act as an unbiased arbitrator. However, none have the independence and broad authority of the Advocate.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that an alternative structure would be more effective or efficient in delivering advocate services.

Evidence: Improved service by IRS front-line divisions would reduce the need for the Advocate. If IRS handled all taxpayer cases accurately and fairly in the first place, there would be no need for the Advocate's support. However, given the size and complexity of the tax system, this is unlikely.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The Advocate's position was created by statute to make certain that taxpayers with serious problems with IRS had recourse if normal IRS systems failed. IRS uses a variety of means to inform taxpayers of their option of contacting the Advocate, including publishing its toll free number in many publications and notices.

Evidence: The Advocate accepts taxpayer cases based on 7 criteria, five of which are statutory (26 USC 7811(a)). There is some risk that taxpayers will use the advocate to solve normal problems rather than regular IRS systems. This would raise overall costs. In future program evaluations, the Advocate should seek information on this question.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Advocate uses balanced measures to track its success in solving taxpayer problems. This includes case quality, customer satisfaction and closure to receipt ratio. Together these indicate the degree of Advocate success in achieving its goal of helping taxpayers resolve problems with IRS.

Evidence: See IRS's annual performance plans and the Advocate's Strategy and Program Plans and National Taxpayer's Advocate Report to Congress Fiscal Year 2005 Objectives. Both case quality and customer satisfaction (surveys) are based on a random sample of actual cases.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Advocate set long term goals for closure to receipts, case quality and customer satisfaction in its National Taxpayer's Advocate Report to Congress Fiscal Year 2005 Objectives.

Evidence: Goals include 100% closure to receipts for now through 2010, 95% case quality by 2009, and 4.53 (out of 5) customer satisfaction by 2009.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Advocate uses the same balanced measures for annual progress as for long term progress (i.e., quality, customer satisfaction and closure to receipt ratio). It is adding "number of systemic hardship cases" as an efficiency measure starting in 2005. It needs to add a unit cost measure for its casework and explore other possible measures of its systemic advocacy function.

Evidence: See IRS's annual performance plans and the Advocate's Strategy and Program Plans and National Taxpayer's Advocate Report to Congress Fiscal Year 2005 Objectives.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Advocate has ambitious targets for its closure to receipts ratio and quality annual measures. Starting in 2005, it will begin setting ambitious goals for customer satisfaction and introduce a new efficiency measure.

Evidence: See IRS's annual performance plans and the Advocate's Strategy and Program Plans.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The Advocate provides support directly to taxpayers rather than working through partner agencies.

Evidence: The Advocate does cooperate closely with other IRS units in ensuring fair treatment of taxpayers and promoting systemic improvements. IRS also runs several grant programs to assist taxpayers (e.g., Low Income Taxpayer Clinics). However, these are not covered in this evaluation.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Advocate is subject to regular IG audits looking at various aspects of its performance. They often include auditors examining a sample of cases to independently determine whether Advocate case workers reacted correctly. Collectively, these audits provide an independent, quality assessment of the Advocate's success in helping taxpayers resolve IRS problems.

Evidence: Examples of recent IG reports include: The Taxpayer Advocate Service Effectively Responded to Taxpayers Requesting Relief from a Significant Hardship 2001-10-073; The National Taxpayer Advocate Has Improved the Quality of Casework, but Continued Vigilance is Needed to Increase Compliance with Quality Standards 2003-10-074; and The NTA Could Enhance the Management of Systemic Advocacy Resources 2003-10-187.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: IRS has a rigorous planning, budgeting and performance management process that integrates performance planning and budgeting.

Evidence: See Treasury's integrated budget and performance plan. While IRS presents integrated performance plans and budgets, it needs to continue to refine its systems for quantifying the specific performance impacts of specific resource changes. However, IRS has committed to improving this process for the 2006 budget cycle. IRS is also proposing a new budget structure in 2006 which shows the full costs of programs such as the Advocate.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: IRS and the Advocate have processes to focus on and improve management and performance. It has set long term goals and is introducing a new efficiency measure (number of systemic hardship cases). In addition to these efforts, the Advocate should introduce a unit cost measure for its casework and work to find additional independent sources of program evaluation information.

Evidence:  

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Advocate regularly collects and analyzes performance and workload data. It also receives input from several Taxpayer Advocate Panels (citizen advisory committees). This data is used both to improve the Advocate's casework and to guide recommendations for systemic improvements.

Evidence: IRS managers receive monthly reports on the Advocate's and other IRS programs' performance. Every quarter the Advocate personally presents a full performance report to senior IRS leaders. One example of using performance data to improve performance is efforts by the Advocate to improve case quality by tailoring employee training to emphasize areas where quality standards have not been met at acceptable levels. This has contributed to the steady case quality improvements since 2001.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The performance appraisals of Advocate managers are tied to organizational goals by setting "commitments" based on organizational goals for each manager.

Evidence: IRS uses a four step Performance Management System (PMS) process to ensure management accountability for achieving strategic goals. The key features of the PMS process are expectations planning, progress monitoring, performance evaluation and performance recognition. Performance commitments and expectations are developed, during the planning stage, that tie strategic business goals with demonstrable actions. Progress toward business goals are monitored throughout the performance cycle. Managers are evaluated annually and recognized accordingly.????

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The Advocate's budget is primarily salaries and direct expenses. It is annually assigned a financial plan by IRS's CFO and manages against that plan.

Evidence: Per IRS's financial reports, the appropriation which funds the Advocate (Tax Law Enforcement) has ended each year with a responsible level of unobligated balances at year end.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Advocate is adding "number of systemic hardship cases" as an efficiency measure starting in 2005. It has presented data on this measure back to 2001. It should also move to measuring unit cost for its casework.

Evidence: This new efficiency measure will be presented in IRS's 2006 Budget.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Advocate collaborates closely with other IRS divisions in day to day case work and in systemic improvements. It also keeps in close contact with taxpayer representative groups outside of government.

Evidence: Recent examples of the Advocate's coordination include: ensuring taxpayer rights in the design of the private collection agent and earned income tax credit error reduction initiatives and redesign of procedures in the offers in compromise program.

Does the program use strong financial management practices?

Explanation: Weaknesses in IRS' financial systems result in a lack of reliable day-to-day management data. IRS is in the process of implementing a modernized financial management system for its administrative programs.

Evidence: Per GAO 2003 report on the IRS financial audit (GAO-04-126): IRS lacks "a financial management system that can produce timely, accurate, and useful information needed for day to day decisions."

Has the program taken meaningful steps to address its management deficiencies?

Explanation: IRS is in the process of implementing a new administrative financial accounting system (October 2004) to address the problems outlined in question 3.6. It has committed to introduce unit cost measures once this new system is functional.

Evidence: See IRS' Business Systems Modernization spending plans.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: As shown by its performance measures, the Advocate has made progress in improving case quality and in working through its case inventory. It has seen a steady decline in systemic hardship cases which it believes results from its efforts to guide improvements in IRS processes. However, it has not yet shown improvements in customer satisfaction scores (already above a 4 on a 5 point scale).

Evidence: See IRS's annual performance reports and internal Budget and Performance Review System reports. Case quality has improved from 71 percent in 2001 to 90 percent in 2004. Closures have exceeded receipts for three years in a row, reflecting the Advocate's success in reducing its backlog of cases. Customer satisfaction has remained above 4 on a 5 point scale.

Does the program (including program partners) achieve its annual performance goals?

Explanation: As noted in question 4.1 above, the Advocate has made progress in improving case quality and in working through its case inventory. Case quality has improved from 71 percent in 2001 to 90 percent in 2004.

Evidence: See IRS's annual performance reports and internal Budget and Performance Review System reports.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Advocate is introducing a measure of the number of systemic hardship cases referred to the Advocate as a proxy measure for its success in helping IRS to fix problems with its systems that unnecessarily cause taxpayer problems. This measure has shown a steady decline (improvement) since 2001.

Evidence: Systemic hardship cases have declined from 217,000 in 2001 to an 2004 projected level of 131,000. It is possible that factors other than the Advocate's efforts impact this measure (e.g., changes in the quantity or program mix of IRS enforcement efforts). This may reduce the value of this metric for measuring the Advocate's performance.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are a number of other ombudsmen offices for state and federal programs. However, none are close enough in structure or mission to the advocate for valid comparisons.

Evidence: See the Advocate's 2003 study of external ombudsmen within the federal government: Independent Advocacy Agencies Within Agencies: A Survey of Federal Agency External Ombudsmen (Publication 4213, IRS Catalog Number 36988J, available at www.irs.gov).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A number of evaluations by the Tax IG note the Advocate's effectiveness in casework. The Advocate should seek other sources of independent evaluation to supplement the IG's work. It should also seek evaluation efforts on its systemic advocacy efforts.

Evidence: An example of the IG's evaluation efforts includes a 2003 IG report that the Advocate had "improved the quality of taxpayer service and case resolutions," but noted continuing needs to further improve compliance with case quality standards (200310074).