This is the accessible text file for GAO report number GAO-09-45 entitled 'Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-Sharing Pilot Program' which was released on December 6, 2008. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Committee on Finance, U.S. Senate: United States Government Accountability Office: GAO: November 2008: Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data- Sharing Pilot Program: Tax Administration: GAO-09-45: GAO Highlights: Highlights of GAO-09-45, a report to the Committee on Finance, U.S. Senate. Why GAO Did This Study: The State Reverse File Match Initiative (SRFMI) is one of the Internal Revenue Service’s (IRS) data-sharing strategies to reduce the estimated $345 billion gross federal tax gap. SRFMI matches federal and state taxpayer data to identify noncompliant taxpayers—individuals and businesses who do not file tax returns or do not report all of their income. IRS’s document-matching program has proven to be a highly cost- effective way of identifying underreported income, thereby bringing in billions of dollars of tax revenue while boosting voluntary compliance. Based on concerns about whether IRS is fully using data from state and local governments to reduce the tax gap, GAO was asked to assess IRS’s plan for evaluating the effectiveness of the SRFMI pilot taxpayer data- sharing program. To address these objectives, GAO consulted social science and evaluation literature, published GAO guidance, and IRS guidance; reviewed and analyzed SRFMI planning and evaluation documents; and interviewed IRS officials about IRS’s plans to evaluate SRFMI. What GAO Found: IRS is developing a plan for evaluating SRFMI data but does not currently have a documented plan even as the agency enters phase III of the pilot program and is less than 1 year away from rolling out the pilot to 45 states. According to IRS officials, the SRFMI pilot includes plans to evaluate program results to make decisions about expanding data sharing with states and using compliance data to assess whether SRFMI cases are more or less productive than other cases. IRS has little documentation on its evaluation. A well-developed and documented plan can help ensure that the evaluation generates performance information needed to make effective management decisions. A sound evaluation approach should also include criteria or standards for determining pilot-program performance. However, IRS has no criteria or standards for determining where the pilot program performs adequately to be incorporated into normal IRS compliance processes. IRS officials told GAO that they plan to use research results to help formulate standards to determine pilot success but did not provide a target date when this will occur. Moreover, IRS has not completely articulated its methodology to evaluate the pilot program. IRS officials have action plans and testing sample plans for individual units. The action plans primarily contained project-management-type information such as actions or tasks to be taken by individuals, start date, completion date, and comments rather than a comprehensive description of the methodological approach for the overall pilot project. The testing sample plans were questionnaires soliciting information about compliance measures for the individual unit and the resources available for testing SRFMI data. None of the plans outline the methods, timing, or frequency of data collection. While IRS officials have begun formulating plans for developing a sampling approach and determining appropriate sample size, they encountered challenges such as delays in information-technology assistance and time limits for using taxpayer data that have impeded progress in moving forward on its evaluation methodology. The need to evaluate the program is underscored because obtaining and using SRFMI data imposes costs not only on IRS but also on the states. Without a sound assessment of pilot program results, IRS may make poor decisions about the program’s future. What GAO Recommends: GAO recommends that the Commissioner of Internal Revenue ensure that IRS develops an evaluation plan to accurately and reliably assess the SRFMI pilot program’s results. This plan should address all components of the program and include key evaluation features. IRS agreed with our recommendation. To view the full product, including the scope and methodology, click on [hyperlink, http://www.gao.gov/cgi-bin/getrpt?GAO-09-45]. For more information, contact Michael Brostek, (202) 512-9110. [End of section] Contents: Letter: Results in Brief: Background: Although IRS Has Begun Developing One, IRS Does Not Yet Have a Sound Evaluation Plan for SRFMI: Conclusions: Recommendation for Executive Action: Agency Comments: Appendix I: States Participating in Phases I, II, and III: Appendix II: Overview of Data-Matching Process and Further Examinations or Investigations under SRFMI: Appendix III: Preliminary Compliance Measures by ASFR and Field Examination Activities: Appendix IV: Comments from the Internal Revenue Service: Appendix V: GAO Contact and Staff Acknowledgments: Table: Table 1: List of States Participating in State Reverse File Match Initiative (SRFMI) Pilot Program: Figures: Figure 1: Overview of Data Matching under SRFMI Pilot Program: Figure 2: Overview of the Compliance Activities for Further Examination or Investigation: Abbreviations: ASFR: Automated Substitute for Return: ATAT: Abusive Tax Avoidance Transactions: GAO: Government Accountability Office: GLDEP: Governmental Liaison Data Exchange Program: IRS: Internal Revenue Service: QETP: Questionable Employment Tax Practice: SB/SE: Small Business/Self Employed: SB/SE Research: Small Business/Self Employed Research: SRFMI: State Reverse File Match Initiative: TIN: Taxpayer Identification Number: USCIS: United States Citizenship and Immigration: Services: United States Government Accountability Office: Washington, DC 20548: November 7, 2008: The Honorable Max Baucus: Chairman: The Honorable Charles E. Grassley: Ranking Member: Committee on Finance: United States Senate: The State Reverse File Match Initiative (SRFMI) is one of the Internal Revenue Service's (IRS) data-sharing strategies to reduce the estimated $345 billion gross federal tax gap. SRFMI matches federal and state taxpayer data to identify noncompliant federal taxpayers--individuals and businesses who do not file tax returns or do not report all of their income. Data sharing allows agencies to leverage external partnerships with other agencies and has proven to be a valuable tool to a number of federal agencies striving to improve decisions about applicants' eligibility for federal programs. As we have previously found, federal agencies are increasingly sharing taxpayer data to help verify applicant-provided information.[Footnote 1] IRS's document- matching program, for example, has proven to be a highly cost-effective way of identifying underreported income, thereby bringing in billions of dollars of tax revenue while boosting voluntary compliance.[Footnote 2] In 2007, you expressed concerns about whether IRS is fully using data from state and local governments to reduce the tax gap. Related to this interest, you requested that we assess IRS's plan for evaluating the effectiveness of the SRFMI pilot taxpayer data-sharing program. To respond to your request, we (1) consulted social science and evaluation literature, along with published GAO guidance and IRS guidance, to identify key features of an evaluation plan; (2) reviewed and analyzed SRFMI planning and evaluation documents to determine whether they contained key features of a sound evaluation plan; and (3) interviewed relevant IRS officials about IRS's plans to evaluate SRFMI, including what cost data IRS plans to use to help fully understand how productive SRFMI cases are. We conducted this performance audit from July through September 2008, in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objective. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. Results in Brief: IRS is developing a plan for evaluating SRFMI data but does not currently have a documented plan even as the agency enters phase III of the pilot program and is less than 1 year away from rolling out the pilot to 45 states. According to IRS officials, the SRFMI pilot includes plans to evaluate program results to make decisions about expanding data sharing with states and using compliance data to assess whether SRFMI cases are more or less productive than other cases. IRS has little documentation on its evaluation approach. A well-developed and documented plan can help ensure that the evaluation generates performance information needed to make effective management decisions. A sound evaluation should also include criteria or standards for determining pilot-program performance. However, IRS has no criteria or standards for determining where the pilot program performs adequately to be incorporated into normal IRS compliance processes. IRS officials told us that they plan to use research results to help formulate standards to determine pilot success but did not provide a target date when this will occur. Moreover, IRS has not completely articulated its methodology to evaluate the pilot program. IRS officials have action plans and testing sample plans for individual units. The action plans primarily contained project-management-type information such as actions or tasks to be taken by individuals, start date, completion date, and comments, rather than a comprehensive description of the methodological approach for the overall pilot project. The testing sample plans were questionnaires soliciting information about compliance measures for the individual unit and the resources available for testing SRFMI data. None of the plans outline the methods, timing, or frequency of data collection. While IRS officials have begun formulating plans for developing a sampling approach and determining appropriate sample size, IRS officials encountered challenges such as delays in information- technology assistance and time limits for using taxpayer data that have impeded progress in moving forward on developing its evaluation methodology. The need to evaluate the program is underscored because obtaining and using SRFMI data imposes costs not only on IRS but also on the states. Without a sound assessment of pilot-program results, IRS may make poor decisions about the program's future. We recommend that the Commissioner of Internal Revenue ensure that IRS develops an evaluation plan to accurately and reliably assess the SRFMI pilot program's results. This plan should address all components of the pilot program and include at a minimum key evaluation features of a sound plan: * well-defined, clear, and measurable objectives; * criteria or standards for determining pilot-program performance; * clearly articulated methodology, including sound sampling methods, determination of appropriate sample size for the evaluation design, and a strategy for comparing the pilot results with other efforts; * a clear plan that details the type and source of data necessary to evaluate the pilot, methods for data collection, and the timing and frequency of data collection; and: * a detailed data-analysis plan to track the program's performance and evaluate the final results of the project.[Footnote 3] In commenting on a draft of this report (see app. IV) on behalf of the Commissioner of Internal Revenue, the Deputy Commissioner for Services and Enforcement agreed with our recommendation to develop an evaluation plan to accurately and reliably assess the SRFMI pilot program's results. The Deputy Commissioner said that a summit will be conducted with the business unit owners and their respective research functions to further develop and enhance their existing SRFMI plans to include the key evaluation features we recommend. In addition, IRS agreed to develop an overall evaluation plan to accurately and reliably assess all components of the SRFMI pilot program's results and include the key evaluation features cited above. Background: IRS and States Have an Existing Data-Sharing Relationship: Data-sharing programs between IRS and states have been in place for many years. Historically, the flow of information was largely from the federal government to the states. IRS shares tax data with the state revenue agencies based on the states' needs, and the states use the data for tax-administration purposes. In recent years, however, IRS has taken steps to change the flow of information and pursue additional opportunities to obtain and use states' taxpayer data for tax- compliance purposes. Some examples of reciprocal data sharing between IRS and state agencies include: * Governmental Liaison Data Exchange Program (GLDEP)--IRS and state revenue agencies exchange taxpayer data for tax administration.[Footnote 4] * Questionable Employment Tax Program (QETP)--IRS shares Form 1099- MISC[Footnote 5] extract data with the state workforce agencies, and the state workforce agencies investigate whether the people on IRS's list are employees or independent contractors. * Abusive Tax Avoidance Transactions (ATAT)--IRS and states exchange information on adjustments made during tax shelter audits. According to IRS, this exchange has led to the discovery by IRS and states of tens of millions of dollars in previously unknown fraudulent tax avoidance schemes. SRFMI Pilot Program Matches State Data with IRS Taxpayer Data to Identify Nonfilers and Underreporters: In calendar year 2006, IRS saw an opportunity to expand data sharing by obtaining more information from states. Accordingly, it created the SRFMI pilot program. SRFMI has a goal to reduce the federal tax gap by improving the tax compliance of individuals and businesses who do not file (nonfilers) or who do not report all of their income (underreporters). Under SRFMI, states match IRS's return-filing information received through the existing GLDEP data-sharing arrangement against state tax data to identify individuals and businesses who filed a state return but did not file a federal return or reported income to the state but not to IRS.[Footnote 6] The pilot uses four different types of tax-return data--individual, corporate, sales, and withholding--and takes steps to validate or put state data into a format useable for compliance activities.[Footnote 7] (App. II, fig. 1 has an overview of data matching under the SRFMI program.) IRS Uses the Data-Matching Results in Several Different Compliance Activities: The result of matching federal and state tax data is a file with data on potential federal nonfilers and underreporters. IRS then filters the matched data to determine which matching results represent potential noncompliance or "leads" for compliance activities in several different organization units. These include: * Automated Substitute For Return (ASFR)--creates a substitute federal tax return where none exists and makes an assessment. * Automated Underreporter--compares filed returns with information- reporting returns to identify unreported income. For the SRFMI pilot, filed federal returns will be compared to filed state returns and not necessarily to information returns. * Field Examination--conducts face to face examinations. * Campus Examination--conducts correspondence audits generally involving basic single issues. * Specialty Tax--conducts examinations on excise taxes, employment taxes, and estate and gift taxes. * Collection--collects revenue and secures delinquent returns. * Fraud/Bank Secrecy Act--assists other functions in developing civil and criminal fraud investigations. * Criminal Investigation--conducts investigations on criminal violations of the Internal Revenue Code and related financial crimes. Each compliance activity determines which of the leads will be designated for examination, contacted by correspondence requesting additional information, or referred for further investigation for criminal activity. (App. II, fig. 2 has an overview of the distribution of SRFMI data for further examinations.) The Pilot Program Has Four Phases with Increasing Numbers of Participating States: Three states participated in phase I, 6 additional states participated in phase II, and 14 agencies from 13 states participated in phase III. (For a table showing the states participating in phases I, II, and III, see app. I.) IRS has enrolled 45 states to deliver SRFMI files in July 2009 and has the following schedule for data exchanges: * Phase I started in March 2006. * States provided SRFMI extracts for phase II in July 2007.[Footnote 8] * States provided SRFMI extracts for phase III in July 2008. * States are to provide extracts for nationwide rollout by July 15, 2009. Although IRS Has Begun Developing One, IRS Does Not Yet Have a Sound Evaluation Plan for SRFMI: According to IRS officials, the SRFMI pilot includes plans to evaluate program results to make decisions about expanding data sharing with states and using compliance data to evaluate whether SRFMI cases are more or less productive than other cases. IRS's approach to implementing the pilot is outlined in a series of briefing documents. IRS officials told us that since the program began in 2006, IRS has: * worked with states to receive state data in a uniform record layout; * validated and perfected federal and state data matches for the nine states in phases I and II;[Footnote 9] * distributed data matches to several business units; * conducted compliance activities and tabulated phase I activities for two organization units--ASFR and field examination; and: * commissioned the Small Business/Self Employed Research (SB/SE Research) unit to provide technical assistance in the areas of data analysis, methodology, and project design; to provide various tabulations of matched data and analyze the usefulness of the matches for identifying nonfilers and underreporters; to develop appropriate sample sizes for each compliance activity; and to conduct studies. Using phase I data, SB/SE Research compared preliminary results of SRFMI data used by two units in the SB/SE Division. ASFR reported on closed cases, taxpayer returns, defaults, reconsiderations, dollars assessed, and dollars assessed per return. Field examination reported on more data elements than ASFR, including no-change rate, total nonexamined cases, and average hours spent on returns. (App. III shows preliminary compliance measures used by ASFR and field examination activities.) The other compliance units have not reported results because they have not begun to work with the matched data. IRS officials told us that early results are limited and not statistically sound but said that these results helped them become familiar with how the data should be filtered and with compliance results information. SB/SE Research has provided ongoing support and consultation and has conducted some research studies to help develop an approach for evaluating SRFMI pilot results. For example, SB/SE Research matched Colorado and New Jersey tax-amnesty data to compare the potential use of resources for SRFMI cases with non-SRFMI cases and found that state tax-amnesty files cannot be used to definitively identify federal nonfilers or underreporters and that the data lacked fields needed to determine filing requirements and self-employment status.[Footnote 10] However, it found that state tax-amnesty data can be a source for potential high-income nonfiler leads and case building, with limitations. The Colorado and New Jersey data were not SRFMI data but were the only data available for a first test of the SRFMI concept. A study of phase I Arkansas, Iowa, and Massachusetts SRFMI ASFR and Field Examination cases found mixed results. SRFMI ASFR cases most clearly were less productive than regular cases, because both the dollars assessed per case and per hour were substantially lower for SRFMI than non-SRFMI cases. Field Examination SRFMI cases, on the other hand, yielded lower dollars per case but higher dollars per hour. However, IRS officials noted that these results were very preliminary and that several refinements are needed in their methodology for comparing cases. IRS SB/SE Research staff also acknowledge more data, including cost data, are needed to conduct a more conclusive cost- benefit analysis. Well-Developed Evaluation Plans Increase the Likelihood That Evaluations Will Yield Methodologically Sound Results and Support Effective Program and Policy Decisions: A well-developed and documented evaluation plan can help ensure that agency evaluations generate performance information needed to make effective program and policy decisions. Well-developed evaluation plans include, at a minimum, several key features, such as: * well-defined, clear, and measurable objectives; * criteria or standards for determining pilot-program performance; * clearly articulated methodology, including sound sampling methods, determination of appropriate sample size for the evaluation design, and a strategy for comparing the pilot results with other efforts; * a clear plan that details the type and source of data necessary to evaluate the pilot, methods for data collection, and the timing and frequency of data collection; and: * a detailed data-analysis plan to track the program's performance and evaluate the final results of the project.[Footnote 11] In addition, to ensure efficient use of its resources, IRS should include a cost-effectiveness analysis to ensure that the program produces sufficient benefits in relation to its costs. Recognizing the importance of assessing the performance of a new program, IRS requires that an evaluation plan and report be submitted with a plan for IRS reorganizations. IRS requires that the evaluation plan (1) be developed before the proposed reorganization takes place; (2) include how, when, and by whom the impact of the reorganization will be evaluated; (3) specify the measures that will be used and the standards for judging the measures; (4) state how and when the data for each measure will be obtained and the amount of change that will indicate success; (5) measure the costs and benefits, including any savings from the reorganization; (6) establish time lines for the evaluation; and (7) include the standards for deciding whether to expand, make permanent, discontinue, or modify the test organization. The reorganization evaluation report must include a statement of the business need for the reorganization, results for the measures, analysis of the results in relation to the stated business need, conclusions about the impact of the change, and recommendations.[Footnote 12] The IRS requirements are relevant, because key features of a sound evaluation plan apply to any program for which performance information is needed to make effective program and policy decisions. IRS Does Not Yet Have a Sound, Documented Evaluation Plan, Which Undermines Its Ability to Support Management Decisions about the Productivity of the SRFMI Pilot: IRS has not yet developed a methodologically sound and documented evaluation plan for the SRFMI program. IRS's current activities have a number of limitations. IRS has not defined clear, measurable objectives with specific statements of the accomplishments necessary to successfully meet the objectives. Clear and measurable objectives help ensure that the appropriate evaluation data are collected and that performance can be measured against the objectives. IRS has articulated a goal for SRFMI- -to help reduce the federal tax gap by improving tax compliance. IRS outlines its basic approach for the SRFMI pilot in a number of briefing documents and, in its SB/SE FY 2008 and 2009 Plan, alludes to SRFMI in its discussion of strategies to address the tax gap.[Footnote 13] In its briefing document for the 2008 Federation of Tax Administrators conference, IRS indicates SRFMI has potential to reduce the tax gap and improve voluntary compliance. The slides outline objectives such as: (1) secure compliance results to drive business decisions, (2) modify systems and compliance activities to further automate SRFMI, and (3) reduce the tax gap by making inroads with noncompliant business entities. These objectives are not clear. For instance, "securing compliance results to drive business decisions" does not explain what compliance results would be used or what business decisions the compliance results would drive. Nor are the objectives measurable. For example, "reduce the tax gap by making inroads with noncompliant business entities" does not specify how much of a compliance improvement would indicate success, or how many business entities or what dollar amount of new assessments would indicate that IRS is "making inroads" with noncompliant businesses. IRS officials told us that individual IRS units using the SRFMI data will have their own separate objectives for evaluating all data sources for their compliance activity and that they will measure SRFMI results by the same standards. They added that IRS is incorporating these objectives into an overall SRFMI sampling and evaluation plan. IRS has no criteria or standards for determining where the pilot program performs adequately to be incorporated into normal IRS compliance processes. IRS officials said the agency plans to use research results to help formulate standards but did not provide a target date when this will occur. IRS officials provided several projects conducted by SB/SE Research using phase I SRFMI data that will serve as the basis for developing standards. For example, they gave us tabulations of results for data matches using federal and Arkansas, Iowa, and Massachusetts state data and a preliminary analysis of the usefulness of Colorado and New Jersey tax amnesty data. Such standards are needed for IRS to determine whether the SRFMI pilot program is effective. IRS has not completely articulated its methodology to evaluate the pilot program. SRFMI methodology should include developing plans for sound sampling methods, determining appropriate sample sizes, and comparing the pilot results with IRS's similar ongoing compliance efforts to determine whether SRFMI can identify cases with higher noncompliance potential than similar ongoing cases. IRS officials provided action plans and testing sample plans for individual units when asked about their methodology. The action plans primarily contained project-management-type information such as actions or tasks to be taken by individuals, start date, completion date, and problem areas rather than a comprehensive description of the methodological approach for the overall pilot project. The testing sample plans were questionnaires soliciting information about compliance measures for the individual unit and the resources available for testing SRFMI data. None of the plans they presented outline the methods, timing, or frequency of data collection. Specifying these elements in the plan would help ensure that adequate, accurate, and timely data will be available to complete the evaluation. While IRS officials have begun developing a sampling approach and determining appropriate sample sizes, IRS officials encountered challenges that have delayed progress. First, IRS officials told us that information technology support for identifying cases with higher noncompliance potential than similar ongoing cases did not occur as planned and, as a result, two compliance activities--ASFR and Field Examination--performed manual workload selection on SRFMI cases. Second, IRS is barred by the 3-year statute of limitations from assessing tax liabilities identified by state tax data received in the earlier phases of the pilot program.[Footnote 14] IRS submitted a legislative proposal that would extend the 3-year limitation for assessments based on information obtained from state and local tax data.[Footnote 15] IRS does not have a plan that details the type and source of data necessary to evaluate the pilot program, methods for data collection, or the timing and frequency of data collection. The testing sample plans SB/SE Research has begun developing have information on the type and source of data it will collect for the four of eight compliance activities that have filled out the questionnaire. Agency officials told us that they will ask the rest of the compliance activities to fill out the same questionnaires. IRS does not have a data-analysis plan that describes how the data will be analyzed to track the program's performance and evaluate the final results. Instead, IRS has analyzed phase I data in an ad hoc manner, using data from the two compliance activities (i.e., ASFR and Field Examination) that worked the available cases. A detailed data-analysis plan is a key feature of an evaluation plan as it sets out who will do the analysis and when and how data will be analyzed to measure the pilot program's performance. IRS does not have a cost-effectiveness-analysis plan to help ensure that the program produces sufficient benefits in relation to its costs. For example, SB/SE Research was asked to conduct an analysis with available phase I matched data using federal tax returns and Arkansas, Iowa, and Massachusetts state tax-amnesty data to determine SRFMI's cost effectiveness. The results of the SB/SE Research were not conclusive, because the data they used were not a representative sample of ASFR and Field Examination SRFMI cases. Because ASFR and Field Examination did not have cost data, SB/SE Research could not do a cost- effectiveness study. Instead, for ASFR, the study compared dollars assessed per case and dollars assessed per hour worked on SRFMI cases with normal non-SRFMI cases. For Field Examination, it tracked average hours worked per return, no-change rate, cycle time, and pick-up rate. Agency officials told us that IRS has no cost data for the program and that they do not track the costs of the pilot. IRS acknowledges that more data are needed to do a complete cost assessment. Conclusions: Evaluating the SRFMI pilot can help IRS determine the benefits of using state taxpayer data for compliance purposes. IRS is exploring new territory by obtaining and using state taxpayer data to match against its own to identify taxpayers who do not file federal tax returns or underreport income on their federal tax returns. However, 2½ years after the pilot began, as it is entering phase III and is less than 1 year away from a national rollout with 45 states enrolled to deliver SRFMI files, IRS has yet to develop and document a sound evaluation plan. The need to evaluate the program is underscored because obtaining and using SRFMI data imposes costs not only on IRS but also on the states. Without a sound assessment of pilot program results, IRS may make poor decisions about the program's future. Recommendation for Executive Action: We recommend that the Commissioner of Internal Revenue ensure that IRS develops an evaluation plan to accurately and reliably assess the SRFMI pilot program's results. This plan should address all components of the pilot program and include key evaluation features of a sound plan: * well-defined, clear, and measurable objectives; * criteria or standards for determining pilot program performance; * clearly articulated methodology, including sound sampling methods, determination of appropriate sample size for the evaluation design, and a strategy for comparing the pilot results with other efforts; and: * a clear plan that details the type and source of data necessary to evaluate the pilot, methods for data collection, and the timing and frequency of data collection; and: * a detailed data-analysis plan to track the program's performance and evaluate the final results of the project. Agency Comments: On behalf of the Commissioner of Internal Revenue, the Deputy Commissioner for Services and Enforcement provided written comments on a draft of this report in a November 3, 2008, letter. The Deputy Commissioner agreed it is important to properly document and assess the SRFMI pilot program as a whole before it is expanded to additional states. The Deputy Commissioner said a summit will be conducted with the business unit owners and their respective research functions to further develop and enhance their existing SRFMI plans to include the key evaluation features we recommend. In addition, IRS agreed to develop an overall evaluation plan to accurately and reliably assess all components of the SRFMI pilot program's results and include the key evaluation features in our recommendation. As agreed with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after its date. At that time, we will send copies to the Secretary of the Treasury, the Commissioner of Internal Revenue, and other interested parties. This report will also be available at no charge on GAO's Web site at [hyperlink, http://www.gao.gov]. If you or your staff have any questions about this report, please contact me at (202) 512- 9110 or brostekm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix V. Signed by: Michael Brostek: Director, Tax Issues Strategic Issues Team: [End of section] Appendix I: States Participating in Phases I, II, and III: Below is a list of states participating in the first three phases of the pilot program. The Internal Revenue Service (IRS) expects to start a nationwide rollout in July 2009. Table 1: List of States Participating in State Reverse File Match Initiative (SRFMI) Pilot Program: Table 2: State: Arizona; Phase I: [Empty]; Phase II: [Empty]; Phase III: x. State: Arkansas; Phase I: x; Phase II: x; Phase III: x. State: California; Phase I: [Empty]; Phase II: x; Phase III: x. State: Delaware; Phase I: [Empty]; Phase II: x; Phase III: x. State: Georgia; Phase I: [Empty]; Phase II: [Empty]; Phase III: x. State: Iowa; Phase I: x; Phase II: x; Phase III: x. State: Kansas; Phase I: [Empty]; Phase II: [Empty]; Phase III: x. State: Maryland; Phase I: [Empty]; Phase II: x; Phase III: x. State: Massachusetts; Phase I: x; Phase II: x; Phase III: x. State: New Jersey; Phase I: [Empty]; Phase II: x; Phase III: x. State: New York; Phase I: [Empty]; Phase II: x; Phase III: x. State: Ohio; Phase I: [Empty]; Phase II: x; Phase III: x. State: Washington; Phase I: [Empty]; Phase II: [Empty]; Phase III: x. Source: IRS. Note: Illinois, Louisiana, Nevada, North Dakota, Oregon, Pennsylvania, South Carolina, and South Dakota may not participate in a nationwide rollout. New Jersey amnesty data used during phase I were secured outside of the SRFMI process. [End of table] [End of section] Appendix II: Overview of Data-Matching Process and Further Examinations or Investigations under SRFMI: The figure below shows an overview of how the State Reverse File Match Initiative (SRFMI) data flow between the Internal Revenue Service (IRS) and state revenue agencies. Figure 1: Overview of Data Matching under SRFMI Pilot Program: This figure is a flowchart showing an overview of data matching under SRFMI pilot program. [See PDF for image] Source: GAO analysis of IRS information. [End of figure] IRS officials told us that when IRS receives SRFMI data on potentially noncompliant business and individual taxpayers, IRS conducts data validation and perfection activities. Next, IRS plans to put the matched data through a filtering process under development to identify cases with higher potential for noncompliance and determine how data should be distributed to the individual units within the Small Business/Self Employed (SB/SE) Division that will use the data for compliance activities. IRS officials told us the SRFMI filter will be continuously refined to take into account law changes, taxpayer schemes, and other nuances that business units expect to identify as they develop a history of using the data in compliance activities. After the filtering process, the SRFMI cases will be distributed to eight different compliance activities where the business units will further screen filtered SRFMI data against their program criteria. Figure 2 shows the distribution of SRFMI data within IRS as well as the disposition of cases that use SRFMI data. Figure 2: Overview of the Compliance Activities for Further Examination or Investigation: This figure is a chart showing an overview of the compliance activities for further examination or investigation. [See PDF for image] Source: GAO analysis of IRS information. [End of figure] [End of section] Appendix III: Preliminary Compliance Measures by ASFR and Field Examination Activities: During 2008, two organization units tracked compliance activities using phase I State Reverse File Match Initiative (SRFMI) pilot data. The Automatic Substitute for Return (ASFR) program is a compliance activity that creates a substitute federal return when none exists. Following are the traditional compliance measures used by ASFR: * number of case closures, * number of taxpayer returns prepared, * number of defaults, * number of reconsiderations, * dollars assessed, and: * dollars assessed per return. Small Business/Self-Employed (SB/SE) Field Examination is a compliance activity during which agents conduct audits in an Internal Revenue Service (IRS) office or at a taxpayer's place of business. SRFMI cases are examined by either revenue agents or tax compliance officers. IRS agents conducting field examinations used federal-state matched SRFMI data. Field Examination tracked the phase I SRFMI cases it examined using the measures listed below: * total closures, * related return closures, * average hours per return, * average dollars per return, * average dollars per hour, * total dollars assessed, * no-change returns, * no-change rate, * average closed cycle days, and: * total nonexamined cases. [End of section] Appendix IV: Comments from the Internal Revenue Service: Department Of The Treasury Internal Revenue Service: Washington, D.C. 20224: Deputy Commissioner: November 3, 2008: Mr. Michael Brostek: Director, Tax Issues: U.S. Government Accountability Office: 441 G Street, N.W.: Washington, D.C. 20548: Dear Mr. Brostek: Thank you for the opportunity to review your draft report entitled "Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-Sharing Pilot Program (GAO-09-45)." We appreciate your taking the time to review the State Reverse File Matching Initiative (SRFMI) pilot program and agree it is important to properly document and assess the program as a whole before it is expanded to additional states. A comprehensive evaluation plan is being developed that will incorporate the key features you recommend. As the report acknowledges, the IRS developed action plans and testing sample plans for each of the business units participating in the SRFMI pilot program. It was necessary to focus our early efforts on adapting the wide variety of state data formats to each of our compliance functions. These formats are frequently mandated by state statutes, thus preventing the IRS from establishing uniform requirements. Partnering with state tax agencies provides the IRS with data on individual and corporate income tax, sales tax, and withholding. By matching our Master File data against the states' data, we are able to identify: * taxpayers who filed a state return but not a federal return; * differences in income reported on state and federal returns; and: * taxpayers who filed under state amnesties but failed to file a federal return. If you have any questions, please contact me or Jeff Basalla, Acting Director, Campus Compliance Services at (202) 283-7399. Sincerely, Signed by: Linda E. Stiff: Enclosure: GAO Recommendation and IRS Responses to GAO Draft Report Tax Administration: IRS Needs to Strengthen Its Approach for Evaluating the SRFMI Data-Sharing Pilot Program GAO-09-45 Recommendation: We recommend that the Commissioner of Internal Revenue ensure that IRS develops an evaluation plan to accurately and reliably assess the SRFMI pilot-program's results. This plan should address all components of the pilot program and include key evaluation features of a sound plan: * Well-defined, clear, and measurable objectives; * Criteria or standards for determining pilot program performance; * Clearly articulated methodology, including sound sampling methods, determination of appropriate sample size for the evaluation design, and a strategy for comparing the pilot results with other efforts; * A clear plan that details the type and source of data necessary to evaluate the pilot, methods for data collection, and the timing and frequency of data collection; and: * A detailed data-analysis plan to track the program's performance and evaluate the final results of the project. Comments: A summit will be conducted with the business unit owners and their respective Research functions to further develop and enhance their existing SRFMI plans to include the key evaluation features you recommend in order to commence with data output sample testing. In addition, we agree to develop an overall evaluation plan to accurately and reliably assess all components of the SRFMI pilot program's results and include the key evaluation features cited above. [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Michael Brostek, (202) 512-9110 or brostekm@gao.gov: Acknowledgments: In addition to the contact named above, Signora J. May, Assistant Director; Amy Bowser; Amy Friedlander; Cindy Gilbert; and Lou V. B. Smith made key contributions to this report. Footnotes: [1] GAO, Taxpayer Information: Options Exist to Enable Data Sharing Between IRS and USCIS but Each Presents Challenges, [hyperlink, http://www.gao.gov/products/GAO-06-100] (Washington, D.C.: Oct. 11, 2005). [2] Ibid. While IRS is generally prohibited from disclosing taxpayer information, IRS is authorized to share taxpayer data with state tax officials under 26 U.S.C. § 6103(d) for tax-administration purposes. [3] P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation: A Systematic Approach (Thousand Oaks, Calif.: 2004); GAO, Designing Evaluations, [hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.4] (Washington, D.C.: May 1991); GAO, Assessing Social Program Impact Evaluations: A Checklist Approach, [hyperlink, http://www.gao.gov/products/PAD-79-1 (Washington, D.C.: October 1978); B.R. Worthen, J.R. Sanders, and J.L. Fitzpatrick, Program Evaluation: Alternative Approaches and Practical Guidelines (New York, N.Y.: 1997). [4] For purposes of the GLDEP, a state is defined as any of the 50 states, the District of Columbia, certain U.S. possessions or territories, and any municipality with a population in excess of 250,000 that imposes a tax on income or wages (26 U.S.C. § 6103(b)(5)). [5] An IRS Form 1099-MISC is used to report payments of $600 or more for services performed for a trade or business by a nonemployee, including independent contractors. [6] In the context of SRFMI, these cases are called "matches." [7] IRS has requested that states provide individual, corporate, sales, and withholding data for SRFMI, but participating states have not always been able to provide all four. [8] A data extract is a computer-generated file that contains specific data elements. [9] Validation is the process of verifying that IRS recognizes the taxpayer using two data elements--Taxpayer Identification Number (TIN) and taxpayer name. Perfection is the process of matching the taxpayer TIN and name when the two almost match. IRS makes small adjustments to either data element to perfect the record. When IRS receives state data for SRFMI, it first checks to ensure the file structure meets SRFMI specifications and then validates and perfects the data. [10] Tax amnesty programs are held to collect taxes owed from prior years and to place those who previously avoided taxation on the tax rolls. [11] P.H. Rossi, M.W. Lipsey, and H.E. Freeman, Evaluation; [hyperlink, http://www.gao.gov/products/GAO/PEMD-10.1.4]; GAO, [hyperlink, http://www.gao.gov/products/PAD-79-1]; B.R. Worthen, J. R. Sanders, and J.L. Fitzpatrick, Program Evaluation. [12] IRS, Internal Revenue Manual 1.1.4 --Organizational Planning, Exhibit 1.1.4-6--Evaluation Plan and Report Requirements for IRS Reorganizations [hyperlink, http://www.irs.gov/irm/part1/ch01s03.html], accessed Jun. 3, 2008. [13] The SB/SE FY 2008 and 2009 Plan presents SB/SE's efforts to identify and address the most critical strategic issues facing SB/SE. [14] Section 6501(a) of the Internal Revenue Code requires IRS to assess additional federal tax liabilities in the form of tax, interest, penalties, and additions to tax within 3 years after the date a return is filed. After the required time, IRS cannot generally assess the additional liabilities. If the additional tax liabilities are assessed within the allowable time frame, IRS is then limited to 10 years to collect the amounts assessed. 26 U.S.C. § 6502(a). [15] IRS recently submitted a revenue proposal providing an exception to the 3-year statute of limitations for assessment of federal tax liability resulting from adjustments to state or local tax liability. This proposal extends the statute of limitations by 1 year from the date the taxpayer files an amended tax return with IRS reflecting adjustments to the state or local tax return or by 2 years from the date IRS receives information from the state or local revenue agency under an information-sharing agreement. U.S. Department of the Treasury, General Explanations of the Administration's Fiscal Year 2009 Revenue Proposals (Washington, D.C.: February 2008). GAO's Mission: The Government Accountability Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through GAO's Web site [hyperlink, http://www.gao.gov]. Each weekday, GAO posts newly released reports, testimony, and correspondence on its Web site. To have GAO e-mail you a list of newly posted products every afternoon, go to [hyperlink, http://www.gao.gov] and select "E-mail Updates." Order by Phone: The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s Web site, [hyperlink, http://www.gao.gov/ordering.htm]. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: [hyperlink, http://www.gao.gov/fraudnet/fraudnet.htm]: E-mail: fraudnet@gao.gov: Automated answering system: (800) 424-5454 or (202) 512-7470: Congressional Relations: Ralph Dawn, Managing Director, dawnr@gao.gov: (202) 512-4400: U.S. Government Accountability Office: 441 G Street NW, Room 7125: Washington, D.C. 20548: Public Affairs: Chuck Young, Managing Director, youngc1@gao.gov: (202) 512-4800: U.S. Government Accountability Office: 441 G Street NW, Room 7149: Washington, D.C. 20548: