Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to take steps to ensure that agencywide goals clearly align with operating division goals and performance measures for each of the four areas reviewed. Specifically, (1) clearly document the relationship among agencywide goals, operating division goals, and performance measures (the other three program areas may want to consider developing a template similar to the one field assistance developed, shown in figure 4) and (2) ensure that the relationship among goals and measures is communicated to staff at all levels of the organization.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: During their fiscal year 2003 Leadership Conference, IRS managers reviewed the agencywide and operating division goals and measures, ensured there was alignment among them, and documented their relationship--beginning in the fiscal year 2005 Strategy and Program Plan. According to the IRS, the Strategy and Program Plan was shared throughout the organization to ensure communication at all levels. By implementing this recommendation, the IRS has created a bridge between long-term strategic goals and short-term daily operational goals, which can, among other things, be used for holding the IRS and the key filing season program areas accountable for achieving its goals. Further, the link to specific measures provides additional information needed to clearly communicate the alignment of goals and measures throughout the agency, as well as the level of performance the operating division hopes to achieve.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to make the name and definition of several field assistance measures (i.e., "geographic coverage," "return preparation contacts," "return preparation units," "TACs total contacts," "forms contacts," "tax law contacts," "account contacts," "other contacts," "tax law accuracy," "accounts/notices accuracy," and "return preparation accuracy") more clear to indicate what is and is not included in the formula.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: In its 2003 report, GAO recommended that the IRS make the name and definition of several field assistance measures (i.e., "return preparation contacts," "return preparation units," "TACS total contact," "form contacts," "tax law contacts," "other contacts," "tax law accuracy," "accounts/notices accuracy," and "return preparation accuracy") clearer to indicate what is and is not included in the formula. Field Assistance implemented GAO's recommendation by updating its "data dictionary," which is used in reporting measures to all levels of the organization and specifically identifies what is or is not included in the formulas. In addition, the data dictionary now includes the following categories: operating division, measure name, type of measure, program category, related strategic goal, definition, reporting level(s), reporting data source, reports, formula/methodology, data source/measurement tools, reliability of data, frequency of data availability/reporting, purpose of measure, data limitations, calculation changes, complete description of the process, critical path, and management controls for items on critical path. The measures also include alternate sites in the definition of a Taxpayer Assistance Center, such as a mobile unit. These modifications are substantial and provide significant additional information about the measures. As a result, the measures have clarity and can be meaningful to stakeholders.

---

Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: remove automated calls from the formula for the "CSR level of service" measure.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: Initially, IRS disagreed with this recommendation, stating that the formula accurately reflected the percentage of customers that wanted to speak to a Customer Service Representative (CSR) and subsequently received service. However, automated services have increased as an alternative to CSR services and taxpayers have selected automated services before entering a CSR queue. Thus, while automated calls answered remains part of the CSR Level of Service formula, the number of these calls included in the computation is minimal, if any, according to a December 2006 interview with IRS officials. As a result, the measure's clarity has improved because the name is more closely aligned to the measure's definition since the number of automated calls is minimal. Also, transparency to users and stakeholders has improved, leading to fewer misinterpretations of results or failures to take proper action to resolve performance problems.

---

Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: revise the "CSR response level" measure to include calls from taxpayers who tried to reach a customer service representative (CSR) but did not, such as those who (1) hung-up while waiting to speak to a CSR, (2) were provided access only to automated services and hung up, and (3) received a busy signal.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: IRS initially disagreed with this recommendation because, according to officials, the Customer Service Representative (CSR) Response Level measure was in accordance with industry standards. However, IRS took alternative corrective action by eliminating the measure from Strategy and Program plan and limited its use to an internal diagnostic tool. IRS officials confirmed this change in a December 2006 interview stating that they have no plans to put it back in the Strategy and Program Plan. By removing the measure from this important document, which is presented or available to external stakeholders such as the Congress, IRS has effectively limited external reliance or possible misinterpretation of this information, while still making it available to IRS.

---

Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: analyze and use new or existing data to determine why calls are transferred and use the data to revise the "CSR services provided" measure so that it only reflects transferred calls in which the caller received help from more than one CSR (i.e., exclude calls in which a CSR simply transferred the call and did not provide service.)

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: IRS agreed with our recommendation, but instead of revising the Customer Service Representative (CSR) Services Provided measure, it took alternative corrective action by making scripting changes that have reduced transfer rates between fiscal years 2002 and 2006 by an average of almost 10 percent. IRS also better explained how transferred calls were counted in its fiscal year 2005 data dictionary. As a result, the measure is clearer and IRS has been able to reduce inefficiencies associated with transferred calls.

---

Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: either discontinue use of the "number of IRS digital daily Web site hits" measure or revise the way "hits" are calculated so that the measure more accurately reflects usage.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: IRS has implemented two new indicators to gauge Website traffic: "Number of visits" shows Website access by counting as "visits" a series of actions that begins when a visitor views his/her first page on IRS.gov and ends when the visitor leaves the site or remains idle beyond 30 minutes, while "Number of page views" counts whenever any given web page is accessed by a visitor to IRS.gov. Information on Website hits is still collected, but is used by IRS only for load balancing of servers.

---

Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: revise field assistance's "geographic coverage" measure by ensuring that the formula better reflects (1) the various types of field assistance facilities, including alternate sites and kiosks; (2) the types of services provided by each facility; and (3) the facility's operating hours.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: While IRS initially agreed that there was a need to revise this measure, the Service eventually eliminated the measure under IRS's Taxpayer Assistance Center Closure Project. The FY 2006-2007 Wage and Investment Strategy and Program Plan and 2006 data dictionary do not contain this measure. By eliminating the measure from these important external documents, IRS has effectively addressed GAO's underlying concern that it could be providing misleading information about agency performance to members of Congress or other external stakeholders.

---

Recommendation: The Commissioner of the Internal Revenue Service should modify the formulas used to compute various measures to improve clarity. If formulas cannot be implemented in time for the next issuance of the Strategy and Program Plan, the Commissioner should modify the name and definition of the following measures so it is clearer what is or is not included in the measure: revise submission processing's "productivity" measure so it provides more meaningful information to users.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: IRS agreed that the Submission Processing productivity measure needed improvement. While IRS has not revised the measure as GAO specifically recommended, Submission Processing officials reported that they have taken steps to address the underlying problems with this measure, which were that field managers did not understand it and therefore did not use it. They did this by holding multiple productivity meetings and conference calls with managers during fiscal year 2006. As a result, field managers have a better understanding of the measure and can better use it to help alert them to problems and/or respond when problems arise.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to refrain from making changes to official targets, such as electronic filing and assistance did in fiscal year 2001, unless extenuating circumstances arise. Disclose any extenuating circumstances in the Strategy and Program Plan and other key documents.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: The IRS concurred with GAO's recommendation that it should only make changes to official targets under extenuating circumstances. Further it agreed that disclosing such changes is appropriate.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to modify procedures for the toll-free customer satisfaction survey, possibly by requiring that administrators listen to the entire call, to better ensure that administrators (1) notify CSRs that their call was selected for the survey as close to the end of a call as possible and (2) can accurately answer the questions they are responsible for on the survey.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: GAO found that the customer satisfaction survey, which is the basis of the telephone customer service measure, was not administered properly. Specifically, potential bias existed because the survey administrators did not follow the sampling procedures, thus distorting the sample's randomness and the overall objectivity of the measure. The IRS has taken corrective action by (1) instructing all administering sites to use the sampling procedures and (2) revising the procedures to make them easier to use. Further, GAO recommended that the IRS modify its procedures for the toll-free customer satisfaction survey, possibly by requiring that administrators listen to the entire call, to better ensure that administrators (1) notify Customer Service Representatives that their call was selected for the survey as close to the end as possible and (2) can accurately answer the questions they are responsible for on the survey. In August 2003, the IRS notified GAO that it had instructed the administrators to listen to each call from its beginning to as close the conclusion as practical. They concurred that formalizing this practice would enable the administrators to accurately answer the survey questions.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to implement annual effectiveness studies to validate the accuracy of the data collection methods used for the five telephone measures ("toll-free tax law quality," "toll-free accounts quality," "toll-free tax law correct response rate," "toll-free account correct response rate," and "toll-free timeliness") subject to potential consistency problems. The studies could determine the extent to which variation exists in collecting data and recognize the associated impact on the affected measures. For those measures, and for the five submission processing measures that already have effectiveness studies in place ("refund timeliness individual (paper)," "notice error rate," "refund error rate-- individual (paper)," "letter error rate," and "deposit error rate"), IRS should establish goals for improving consistency, as needed.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: In 2003, IRS revised four of its five external quality measures for telephone assistance into two measures: "customer accuracy for toll-free tax law" and "customer accuracy for toll-free accounts." The fifth measure, a timeliness measure, was changed to an internal measure. At the time of these changes, the Service also put in place numerous checks and controls for the revised measures to ensure the accuracy of the data collected. Controls include daily and weekly validity checks, a "rebuttal system whereby call sites can challenge the review and coding of individual calls, and unique consistency reviews. IRS has, therefore, addressed GAO's underlying concern behind this recommendation, that there could be problems with the reliability of the data collected to report on the measures.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to ensure that plans to remove overlapping measures in telephone and field assistance are implemented.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: In fiscal year 2003, IRS officials reviewed all measures to verify that they were relevant and specific. Of the 10 overlapping measures GAO identified, four were removed from the Strategy and Program Plan, including "return preparation units," "automated completion rate," " toll-free tax law correct response rate," and "toll-free account correct response rate." Based on IRS management's judgment, measures that provided basically the same information as another measure were removed. Consequently, managers no longer have to sort through redundant, costly information that might not add value or cloud the bottom line.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: in the spirit of provisions in the Chief Financial Officers Act of 1990 and Financial Accounting Standards Number 4, develop a cost of services measure using the best information currently available for each of the four areas discussed in this report, recognizing data limitations as prescribed by GPRA. In doing so, adhere to guidance, such as Office of Management and Budget Circular A-76, and consider seeking outside counsel to determine best or industry practices.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: IRS responded that the development of cost of service measures was dependent on the Servicewide deployment of the Integrated Financial System (IFS). In 2007, IRS was still developing cost estimates for the Chief Financial Officer's IFS. However, as part of its 2007 Taxpayer Assistance Blueprint, IRS prepared unit cost per contact estimates based on fiscal year 2005 data for the major IRS taxpayer services and channels. While IRS reported that these estimates do not fully allocate all indirect overhead and support costs such as headquarters staffing, information technology modernization projects, and centralized real estate facilities procurement and employee services, they represent the best and most comprehensive estimates currently available. As a result of these initial steps, IRS has better achieved governmentwide priorities and balance of its measures.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: given the importance of automated telephone assistance, develop a customer satisfaction survey and measure for automated assistance.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: In process

Comments: IRS has taken steps to implement a customer satisfaction survey and measure for automated assistance, but it has not been implemented due to a lack of funding. During the recent prioritization effort, MITS suggested waiting until the roll-out of the Internet Customer Account Services (ICAS) project, tentatively scheduled for January 2008, to make sure telephone TRIS Automated Applications (ASSA) didn't need changes as a result of ICAS. (Historically, when moving telephone ASSAS to the web, changes to the telephone ASSAs are needed to assure customers receive the same service, whether using the phone or web.) Based on this, IRS would not consider an automated survey before July 2008, or more likely January 2009.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: put the "automated completion rate" measure back in the Strategy and Program Plan after revising the formula so that calls for recorded tax law information are not counted as completed when taxpayers hang up before receiving service.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: Although IRS did not put the measure back into the Strategy and Program Plan (SPP), it made enhancements to TeleTax, IRS's automated telephone information system, which resulted in the ability to capture the number of automated calls that were completed. IRS tracks this enhancement as part of its Automated Calls Answered measure, which is in the SPP. The fiscal year 2005 through 2006 SPP confirms that this enhancement was effective. As a result, taxpayer behavior is tracked and reported in an important document available to Congress and external stakeholders, which better gauges the automated service IRS provides to taxpayers.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: add one or more quality measures to electronic filing and assistance's suite of measures in the Strategy and Program Plan. Possible measures include "processing accuracy," "refund timeliness, electronically filed," and "number of electronic returns rejected."

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Not Implemented

Comments: IRS disagreed with this recommendation, noting that adding one or more quality measures to electronic filing and assistance's suite of measures in the Strategy and Program Plan would not enhance the electronic filing program. Accordingly, the most recent editions of Wage and Investment's Data Dictionary (FY 2006) and Strategy and Program Plan (FY 2006- FY 2007) do not contain a quality measure(s) on electronic filing (such as "processing accuracy," "refund timeliness, electronically filed," or "number of electronic returns rejected").

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: re-implement field assistance's timeliness measure.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: By the 2006 tax filing season, IRS had a system in place for monitoring and managing timeliness of customer service at walk-in sites, which provided coverage for 81 percent of all walk-in site traffic. Managers at sites equipped with the "full" Q-Matic system can monitor traffic in real-time, specifically wait-times. With this additional information and capability, IRS is better able to balance customers' need for accurate information against customers' needs for timely service at walk-in sites. IRS has, therefore, addressed GAO's underlying concern behind this recommendation, that field assistance could be emphasizing customers' need for accurate service at the expense of their need for timely service.

---

Recommendation: The Commissioner of Internal Revenue should direct the appropriate officials to include the following missing measure in the Strategy and Program Plan in order to better cover government-wide priorities and achieve balance: develop a measure that provides information about field assistance's efficiency.

Agency Affected: Department of the Treasury: Internal Revenue Service

Status: Implemented

Comments: IRS implemented a diagnostic tool to monitor Field Assistance's efficiency. We still believe that an organization performance measure that would be visible throughout IRS and to Treasury and congressional stakeholders would be preferable. However, IRS officials confirmed in September 2007 that they find the efficiency diagnostic tool useful and that they monitor frequently. As a result, we agree that IRS's implementation of an efficiency diagnostic tool is a good step for direct mangers to monitor Field Assistance's productivity and to use to identify its strengths, weakness, and areas for improvement.

---