Requirements
Changes and Testing Delays Have Further Increased the Costs and Delayed the
Benefits of the e-Services Project
February 2004
Reference Number: 2004-20-036
This report has cleared the Treasury
Inspector General for Tax Administration disclosure review process and
information determined to be restricted from public release has been redacted
from this document.
February
17, 2004
MEMORANDUM FOR
CHIEF INFORMATION OFFICER
FROM: Gordon C.
Milbourn III /s/ Gordon C. Milbourn III
Acting
Deputy Inspector General for Audit
SUBJECT: Final Audit Report - Requirements Changes
and Testing Delays Have Further Increased the Costs and Delayed the Benefits of
the e-Services Project (Audit # 200320023)
This report presents the results of our review of the
Internal Revenue Service’s (IRS) e-Services Release 1.1 Development. The
e-Services project will provide a set of Web-based business products as
incentives to third parties to increase electronic filing, in addition to
providing electronic customer account management capabilities to all
businesses, individuals, and other customers.
The
overall objective of this review was to determine whether the Business Systems
Modernization Office (BSMO) was providing adequate oversight on the e-Services
project in the critical areas of business case development, requirements
management, testing, and transition to support.
In
summary, although the BSMO deployed the initial release of the e-Services
system in late August 2003, the project continues to experience delays and cost
increases due to requirements changes and testing environment problems. We identified opportunities for improvement
in business case development,
requirements management, and testing oversight.
Cost estimates and planned
schedules included in previous business cases have been significantly exceeded
throughout the project’s lifecycle.
Additionally, business case information provided to gain approval for
Fiscal Year 2004 funding did not include updated cost information available at
the time of its preparation.
Management of system
requirements is critical to ensure the developed system meets user needs. We reviewed the e-Services system
requirements to determine whether requirements approved at the end of the
design phase were developed and tested.
We found that numerous requirements were deleted and documentation of
changes made to system requirements was not easily traceable from the
requirements database to approval documents.
Several hundred critical
defects were identified during acceptance testing. This testing was one of the last phases of testing and occurred
after the system had passed other testing phases. Additionally, many of these issues required software
changes. Defects identified during both
integration and acceptance testing have taken much longer to resolve than
initially planned.
To address these issues, we
recommended that the Chief Information Officer (CIO) require that system
requirements be fully developed before project development begins and that
changes after that point meet stringent criteria. Additionally, future submissions of project information should
include up-to-date cost and schedule information, and a formal lessons learned
document should be developed to fully explore e-Services project issues.
The CIO should require
management in the BSMO to ensure that the e-Services requirements database
clearly reflects changes and approval for those changes. To improve testing, the CIO should require
the BSMO to develop incentives to increase PRIME contractor accountability in
the areas of software quality and testing.
A process should also be established to review and accept test plans
before allowing the PRIME contractor to begin testing.
Management’s Response: Management
fully agreed with four of our recommendations, partially agreed to one
recommendation, and disagreed with one recommendation. IRS management plans to take the following
actions: establish a process that calls
for gathering requirements, implementing a requirements traceability
verification matrix that maps system requirements or other test components to
test cases, and identifying the verification and validation method for each
requirement or test component; and implement the recommendations from recent
Business Systems Modernization program reviews. IRS management also plans to:
continue to submit cost information to the Office of Management and
Budget (OMB) according to OMB requirements, review and guidance; develop a
thorough lessons learned document; and add additional fields to the requirements
database to ensure clear traceability and level of approval indicator.
In addition, the CIO has issued
a directive that requires fixed-price contracting for all systems development
and implementation projects. This will
require the PRIME contractor to provide its written assurance at a newly
established checkpoint that due diligence was performed in defining all
significant business requirements.
Management’s complete response to the draft report is included as
Appendix V.
Office of Audit Comment: Management
partially agreed with our recommendation to develop incentives to increase
PRIME contractor accountability in the areas of software quality and
testing. Management has included items
to increase contractor accountability in the new Systems Integration Task Order
for this fiscal year. However, they did
not include the specific incentive for the contractor for systems that
successfully pass Government acceptance testing with a minimal level of
software defects. We agree with this
corrective action because the fixed-priced contract and the written assurance
at the new checkpoint will hold the PRIME contractor accountable for systems
that do not pass Government acceptance testing due to software defects,
although it is not specifically defined in the task order.
Management disagreed with our
recommendation to establish a process to review and accept test plans before
allowing the PRIME contractor to begin testing. Rather than implement this recommendation, which would require a
formal deliverable-based solution, management has implemented an approach that
gives the development of the plan, test cases, and other test artifacts to an
integrated product team. This allows
the team to complete the plan as the required information becomes available and
firm, sometimes even into the initial days of the testing, while assuring that
all stakeholders approve of and support the plan as completed. While we agree that having an integrated
product team environment is a good approach, we believe that the plan should be
completed and approved before testing starts to allow more effective
testing. If proper planning is
implemented, then foreseeable problems can be resolved up front instead of in
the last phases of testing, reducing the number of critical defects. While we still believe our recommendation is
worthwhile, we do not intend to elevate our disagreement concerning it to the
Department of the Treasury for resolution.
Copies of this
report are also being sent to IRS managers affected by the report recommendations. Please contact me at (202) 622-6510 if you
have questions or Margaret E. Begg, Assistant Inspector General for Audit
(Information Systems Programs), at (202) 622-8510.
The Initial
Release of e-Services Was Deployed in August 2003
Previous
Business Case Estimates Have Been Significantly Inaccurate
Requirements Changes Were Not Easily Traceable
Significant Defects Identified During Late Phases of Testing Took Longer Than Expected to Resolve
Appendix I – Detailed Objective, Scope, and Methodology
Appendix II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Outcome Measures
Appendix V – Management’s Response to the Draft Report
In the Internal Revenue Service (IRS) Restructuring and Reform Act of 1998 (RRA 98), the Congress challenged the IRS to promote its electronic filing programs to the point where 80 percent of all tax returns would be filed electronically by the 2007 Filing Season. Moreover, by the 2003 Filing Season, most, if not all, returns prepared electronically should be filed electronically as well. Unfortunately, if the IRS’ current systems are not updated and refined, the aforementioned goals will not be met. Therefore, to help meet the Congressional mandate, the IRS created the e-Services project.
The e-Services project will provide a set of Web-based business products as incentives to third parties to increase electronic filing. The project focuses on fostering easy-to-use electronic products and services targeted at specific practitioner segments that inform, educate, and provide service to the taxpaying public. In addition, the e-Services project will provide electronic customer account management capabilities to all businesses, individuals, and other customers.
The e-Services project is broken down into several releases in 2003 – Releases 1.1, 1.2 and 2.0. Collectively, the releases aim to provide agency-wide registration, authorization, authentication, and application procedures to select Electronic Return Originators (ERO). In addition, the EROs will be provided with on-line power of attorney application, Taxpayer Identification Number (TIN) matching, transcript delivery, and electronic account resolution. Also, the 2003 releases will support the Modernized e-file (MeF) project.
We reviewed available documentation and interviewed PRIME contractor, IRS, and Business Systems Modernization Office (BSMO) executives, managers, and analysts located at the IRS National Headquarters and the New Carrollton, Maryland, offices. We performed the audit from November 2002 through October 2003 in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.
The IRS deployed the first release of the e-Services system, Release 1.1, in late August 2003. Although the system initially encountered various issues, including limitations on accessibility of the system to the IRS employees who will be processing activity through the system, the project team was working with the system users to correct the issues.
Launching the initial release provided the opportunity for internal and external customers to review, test, and approve specific e-Services products. This initial release allows tax professionals the ability to register online and create an electronic account, and to apply for a Preparer Tax Identification Number to use in place of their Social Security Numbers for submitting returns. In addition, this release will allow for some of the TIN matching capabilities described earlier.
Although the initial release deployment was not a total success, problems identified during the early deployment of this release are currently being resolved and plans to deploy subsequent releases are continuing. Although the deployment of the release occurred in late August 2003, the marketing of the system to external users, primarily tax practitioners, was delayed until November 2003 to ensure that any remaining issues were resolved.
Cost estimates and planned schedules
included in previous business cases have been significantly exceeded throughout
the project’s lifecycle. Additionally,
business case information provided to gain funding approval for Fiscal Year
(FY) 2004 did not include updated cost information available at the time of its
preparation.
Significant schedule delays and cost increases that
were evident during the design phase of the e-Services project are continuing
in the development and implementation phases
In our earlier e-Services audit
report, we
indicated that costs through the design phase of the e-Services project had
quadrupled from nearly $4 million to approximately $17 million, and delays of
approximately 9 months had been experienced.
Current
development, testing, and deployment phase cost projections total $119.1
million, which is an increase of $71.9 million (152 percent) from the baseline
business case projection of $47.2 million. To date, delays of 14 months have also been
experienced.
A key reason for these cost increases and schedule delays was the significant number of changes made to the e-Services system after development began. As of May 2003, we identified 56 separate change requests that were approved for the e-Services project after it started the development phase. According to BSMO management, many of these changes were necessary because of corresponding changes in the tax law. Although a change control process has been established to review all proposed changes, we believe that the process should be more stringent and better budgeting practices should be employed to enable more accurate cost estimation for future changes.
Although negotiations are underway to obtain contractor
agreement to a firm fixed-price cost for the remaining development efforts, at
least one significant item,
continuance of transition to support activities, was removed from contract
requirements during these negotiations.
These requirements will have to be included in a subsequent contract,
even though they were included in the original cost estimates for the
development tasks. Delaying these
requirements could negatively affect efforts already underway to enable the
business organization to have the capability to support the system once it is
in operation, and will likely result in extending the length of time that the
PRIME contractor is paid to maintain the system.
The schedule
slippage with Release 1.1 of the e-Services system will delay deployment of
services, such as registration of EROs, TIN matching, and secure electronic
mail (e-mail). In addition, since the
MeF project is dependent on the implementation of e-Services, such
functionalities as allowing tax professionals and businesses to register and
make application to file certain business tax forms electronically have also
been affected by delays in deploying the e-Services project.
Management
Actions: Management plans to improve its budgeting
process by increasing the amount of funding allocated to the management reserve
account. This account will cover
unplanned items, such as changes to the tax laws. In addition, an initiative has been started to better integrate
periodic tax law changes into the systems development process.
Business case information presented to justify the FY 2004 budget for the e-Services project did not include updated costs
The FY 2004 Capital Asset Plan and
Business Case (Exhibit 300) documents submitted to the Office of Management and
Budget (OMB) were not accurate at the time of submission. These documents, submitted to the OMB in the
fall of 2002, did not include updated cost information or earned value data
that was available at the time it was submitted. Specifically, data in the Alternatives Analysis Section were
obtained from the e-Services business case dated July 31, 2001, even though
earned value data clearly indicated that these estimates were not accurate and
would be greatly exceeded.
The Exhibit 300 also did not adequately reflect the earned value data showing negative performance in excess of 10 percent, which had been ongoing since early 2002. Lastly, the return on investment (ROI) calculations did not include costs incurred to design the system. Although the Enterprise Life Cycle (ELC) guidance allows these “sunk” costs to be excluded, we believe that they should be disclosed in the ROI calculations in the OMB Exhibit 300 submission to provide a more accurate estimate of project costs and benefits.
The OMB establishes policy and provides instruction on budget justification and reporting requirements for major acquisitions and major information technology systems or projects. That policy requires that all of the areas of the Exhibit 300 must be part of the agency’s planning and should be updated as soon as the information is known. While the Exhibit 300 documents are officially submitted to the OMB twice yearly, they should be used as management tools within an agency and updated as the information is available.
In addition, the OMB requires that if any of the Exhibit 300 cost, schedule, or performance variances are a negative 10 percent or more, the Exhibit must provide a complete analysis of the reasons for the variances, the corrective actions that will be taken, and the most likely estimate at completion. The earned value system should be used to identify the specific work packages where problems are occurring, and reasons for problems and corrective actions necessary to return the program, as close as feasible, to the current baseline goals should be disclosed. The rationale for the latest estimate at completion should be disclosed and discussed.
Although the IRS took corrective actions in response to the negative variances, which eventually resulted in a significant Task Order modification based on the e-Services revision of cost and schedule estimates, it did not detail the need for this action in the FY 2004 Exhibit 300. As a result, updated Net Present Value and ROI data, as well as accurate earned value data, were not reflected in the FY 2004 Exhibit 300.
The e-Services team indicated that the guidance used to prepare the FY 2004 Exhibit 300 did not clearly define what was required and has since been updated. Additionally, management stated that although earned value data had not been updated in its FY 2004 submission, the cost overruns that e-Services had experienced were clearly disclosed to representatives from the OMB and other oversight organizations in executive briefings.
Management Actions: The FY 2005 Exhibit 300 submission for the e-Services project was being prepared as we conducted our audit. Management indicated that updated cost and schedule figures were included in this submission; however, ROI data did not include “sunk” costs. Because the submission documents were still in process when we completed our fieldwork, we were unable to review them to ensure latest cost figures were included.
Additionally, new guidance has been drafted by the Information Technology Services organization to assist project teams in preparing their Exhibit 300s for the OMB. We reviewed a draft of this guidance, and the September 2003 version requires inclusion of ROI figures calculated both with and without “sunk” costs.
To better control cost increases and schedule delays, we recommend that the Chief Information Officer (CIO):
1. Establish a more formal discipline to ensure system requirements are established prior to beginning development. Strict criteria should be established to restrict changes once development begins, and any required changes that do not meet the stringent criteria should be made in a subsequent release.
Management’s Response: Management agreed to establish a process that calls for the gathering of requirements, a requirements traceability verification matrix, and the identification of the verification and validation method (e.g., test, demonstration, analysis, etc.) for each requirement or test component. They will also implement the recommendations from recent Business Systems Modernization (BSM) program reviews to further improve the requirements management area. These recommendations will be formally tracked and monitored by their executives and include: clearly define business requirements and tightly manage them to control scope; refine the change request process; and develop criteria for project decision-making including impacts and trade-offs.
2.
Ensure that the most current available project costing
information, including accurate earned value data and all costs expended to
develop the e-Services system, are included in the FY 2005 Exhibit 300
submission to the OMB.
Management’s Response: Management agreed
to continue to submit cost information to the OMB according to the OMB’s
requirements, review, and guidance.
3.
Require that a formal lessons learned document be developed
to thoroughly explore the issues experienced in the design and development of
the e-Services project so that those issues are not experienced in other
projects.
Management’s Response: On December 15,
2003, management completed a lessons learned document to address the issues
identified during Releases 1.1 and 1.2.
A thorough lessons learned document will be developed as part of the
post checkpoint activities as called for by the ELC.
Management of system requirements is critical to ensure the developed system meets user needs. We reviewed the e-Services system requirements to determine whether requirements approved at the end of the design phase were developed and tested. We found that numerous requirements were deleted and documentation of changes made to system requirements was not easily traceable to the approval documents or to the requirements database.
We selected a statistically valid sample of 68 system requirements from Release 1.1 of the e-Services project, which were approved when the e-Services project moved from the design phase to the development phase. We intended to trace these requirements through to testing to ensure that they passed the first phase of integration testing. Of the 68 we selected, 25 (37 percent) were deleted from the requirements as the system was developed. Of these 25 requirements, 20 were originally labeled as “must” requirements, which indicates that the system has to have this functionality. An additional 13 system requirements were delayed to a future release of the e-Services project. Thus, of the 68 requirements we randomly selected, over half were not included in Release 1.1 of the e-Services system.
We attempted to locate documentation of IRS approval of the deletion or deferral of these requirements. Although e-Services personnel indicated that the approvals of these deletions and deferrals were documented in the change request documents, the requirements database, which tracks all system requirements, did not clearly identify the change requests that incorporated the approvals of these deletions and deferrals. Therefore, the IRS will have difficulty ensuring that changes made to the requirements database have been approved. Without requiring a change request number documenting approval for changes to the requirements, unauthorized changes could be made. The resulting system may not meet the users needs.
The IRS indicated that a significant number of additional requirements were added to the system requirements after the e-Services project entered the development phase. Thus, numerous changes were made to the requirements approved at the end of the design phase. Many of these changes were made in meetings, but the minutes did not clearly reflect which specific requirements were affected.
A separate audit effort has been initiated to follow up on these and other system requirements issues. The recommendation in this report is focused solely on ensuring that the e-Services project requirements are adequately addressed.
To ensure that the e-Services project requirements are effectively addressed, the CIO should:
4. Require the BSMO to ensure that the e-Services requirements database clearly reflects when a requirement is changed, and which approval change request documents the change. Additionally, the change request document should indicate which specific requirements are affected by the change.
Management’s Response: Management has prepared a change request to add additional fields to the requirements database to ensure clear traceability and level of approval indicator. As mentioned in management’s response to Recommendation 1, management will be implementing the recommendations from recent BSM program reviews to further improve the requirements management area.
Several hundred critical defects were identified during acceptance testing. This testing was one of the last phases of testing and occurred after the system had passed other phases of testing. Many of these issues required software changes. Although our judgmentally selected sample indicated that these defects eventually appeared to be resolved, it took much longer to resolve them than initially planned.
There were a significant number of defects identified during acceptance testing, after the system had passed the majority of its integration testing
The IRS conducted Systems Acceptance Testing (SAT) for the
e-Services project after the PRIME contractor had passed the system through
quality testing, project testing, and the majority of its system testing. The SAT identified 362 Priority 1 and 2
issues – the two highest categories of criticality. Many of the issues identified during testing required actual
software changes or corrections before they could be resolved. Of
the 61 Priority 1 issues, 16 (26 percent) required software corrections, as did
160 (53 percent) of the 301 Priority 2 issues.
Other issues included processing problems, necessary documentation
changes, and configuration issues.
Because of the high
number of software corrections required as a result of SAT testing, the rigor
with which the quality and integration testing before SAT was conducted is
questionable. The BSMO works with the
PRIME contractor to develop and conduct test activities, but places a heavy
reliance on the PRIME contractor to ensure these activities are thorough and
complete. Even though the ELC requires
that reviews of test plans be conducted, the BSMO does not approve test plans
before the tests are conducted and does not review all defects to ensure they
have been adequately resolved before allowing the PRIME contractor to move to
the next test phase.
The BSMO had a close working
relationship with the PRIME contractor with respect to the development and
testing of the e-Services system. We
believe that this close relationship, while a positive attribute in some
instances, is a detriment with respect to review and approval of the various
testing activities. While the BSMO
seems to be closely involved in the development of the test plans, it has not
seen the need for a formal process to review and approve the test plans, and to
ensure they are conducted effectively.
In the absence of a
formal process to review and approve each phase of the test plans, there may
not be a clear understanding of the system requirements and the best way to
test each requirement. As a result,
testing could take longer than initially planned, or there may not be
sufficient test cases identified to test each requirement. Failure to test each requirement could
result in the system not performing as designed.
The e-Services
system testing took several months longer than initially planned, partially
because there was not a clear understanding of what was required to prove that
requirements were met. Additionally,
because the e-Services project has used a cost-plus fee contract, these delays
resulted in increased costs to the IRS.
Although we were unable to determine the exact amount of additional
costs to the IRS for the delays in testing, the cost projections for
development, testing, and deployment have increased nearly $72 million from
estimates made prior to beginning the development.
The majority of defects identified during integration and acceptance testing were not resolved within planned schedules
We
selected a judgmental sample of 44 defects and reviewed them for
resolution. These defects appeared to
be adequately resolved. However, we
further examined the entire population of critical and high-priority defects
identified during integration and acceptance testing for Release 1.1 of the
e-Services project and found that the vast majority of critical and
high-priority issues were not resolved within required time periods. For integration testing, resolution time
periods were 24 hours for critical issues and 3 days for high-priority
issues. For acceptance testing,
resolution time periods were 5 days for critical issues and 10 days for
high-priority issues.
Delays in resolving defects resulted in overall testing
delays. Originally, acceptance testing
was planned to be completed in February 2003.
However, this testing was not completed until June 2003. These delays were due in part to the number
of defects identified and the length of time to resolve the defects.
We believe the PRIME contractor
did not respond to and resolve defects identified during integration and
acceptance testing within expected time periods because the system was not
adequately tested for quality in the initial test phases. Defects identified in the initial phases of
system testing were not always effectively resolved when first identified. Defects become more difficult and costly to
resolve as the system moves through various phases of testing.
Not resolving defects timely becomes critical because the PRIME contractor has been working under a cost-plus fee contract. Thus, delays in resolving defects result in additional costs to the IRS. Additionally, these delays have resulted in implementation delays for the e-Services system. Lastly, the PRIME contractor has no contractual obligation to repair defects once the e-Services system goes into operation. Any repair work accomplished by the PRIME contractor once the system is operational is also performed on a cost-plus fee basis or may be negotiated at the time of discovery of the defect.
According to a National Institute of Standards and Technology (NIST) study, it becomes increasingly more expensive to resolve errors the later in the test and/or operational cycle that the errors are discovered. The NIST estimated that, relative to resolving an error during the design phase of a project, it is 90 times more expensive to resolve during the system’s testing phase, but up to 880 times more expensive to resolve after the system is operational.
To ensure that testing for the remaining releases of the e-Services system and other BSMO projects is effectively handled, we recommend that the CIO require the BSMO to:
5. Develop incentives to increase PRIME contractor accountability in the areas of software quality and testing. These incentives should reward the PRIME contractor for systems that successfully pass Government acceptance testing with a minimal level of software defects.
Management’s Response: Management partially agreed with our recommendation. Management agreed that contractor incentives
should be applied to the integration and test area. They have included items to increase contractor accountability in
the new Systems Integration Task Order for this fiscal year. However, they did not include the specific
incentive for the contractor for systems that successfully pass Government
acceptance testing with a minimal level of software defects.
The CIO issued a directive that requires fixed-price contracting
for all systems development and implementation projects. Since fixed-price contracting requires
mutually agreed specifications, management is establishing a new checkpoint, at
which point these specifications should be developed. Management will required the PRIME contractor to provide its
written assurance at this checkpoint that it performed due diligence in
defining all significant business requirements.
Office of Audit Comment: We agree with management’s corrective action to increase PRIME contractor accountability in the new Systems Integration Task Order for this fiscal year. We believe that the fixed-price contracting and the written assurance at the new checkpoint will hold the PRIME contractor accountable for systems that do not pass Government acceptance testing due to software defects, although it is not specifically defined in the task order.
6. Establish a process to review and
accept test plans before allowing the PRIME contractor to begin testing.
Management’ Response: Management disagreed
with this recommendation. Rather than
implement this recommendation, which would require a formal deliverable-based
solution, management has implemented a combined integration test approach. This approach gives the development of the
plan, test cases, and other test artifacts to a combined team, which includes
the PRIME contractor, Product Assurance, the BSMO, and business personnel, in
an integrated product team environment.
This allows the team to complete the plan as the required information
becomes available and firm, sometimes even into the initial days of the
testing, while assuring that all stakeholders approve of and support the plan
as complete.
Office of Audit Comment: We agree that having an
integrated product team environment to approve of and support the completed
plan is a good approach. However, the
plan should be completed and approved before testing commences to allow more
effective testing. If proper planning
is implemented, foreseeable problems can be resolved up front instead of in the
last phases of testing, reducing the number of critical defects.
Appendix I
Detailed Objective, Scope,
and Methodology
The overall
objective of this review was to determine whether the Business Systems
Modernization Office (BSMO) was providing adequate oversight on the e-Services
project in the critical areas of business case development, requirements
management, testing, and transition to support.
To accomplish this objective, we performed the following reviews and analyses:
I. Determined if the e-Services project business case was accurate and supportable, and provided information to justify continuing project development and maintenance throughout its life cycle.
II. Determined if requirements management activities would ensure that the e-Services project met user needs by selecting and reviewing a statistically valid sample of 68 e-Services Release 1.1 requirements from a population of 851 requirements approved when the system moved from the design phase to deployment. We intended to project the results of this sample to the population; however, because the requirements changes during system development were so extensive, we decided to simply report on the results of our sample.
III. Determined whether adequate controls were in place over Release 1.1 testing activities to ensure the system delivered would meet user requirements by selecting and reviewing a judgmental sample of 44 out of 611 critical defects identified during testing to determine whether they were adequately resolved. We used a judgmental sample because we did not plan on projecting the results.
IV. Determined whether adequate controls were in place over transition to support activities to ensure Internal Revenue Service readiness for the deployed system.
Appendix II
Major Contributors to This Report
Margaret E. Begg, Assistant Inspector General for Audit
(Information Systems Programs)
Gary V. Hinkle, Director
Scott A. Macfarlane, Director
Tammy
Whitcomb, Audit Manager
Cari Fogle, Acting Audit Manager
Glen
Rhoades, Acting Audit Manager
Jimmie Johnson, Senior Auditor
Paul Mitchell, Senior Auditor
Jacqueline Ngyuen, Senior Auditor
Esther Wilson, Senior Auditor
George L. Franklin, Auditor
Suzanne Noland, Auditor
Appendix III
Commissioner
C
Office of the Commissioner – Attn: Chief of Staff C
Deputy Commissioner for Operations Support OS
Associate Commissioner, Business Systems Modernization OS:CIO:B
Chief, Information
Technology Services OS:CIO:I
Deputy
Associate Commissioner, Business Integration
OS:CIO:B:BI
Deputy Associate
Commissioner, Program Management OS:CIO:B:PM
Deputy
Associate Commissioner, Systems Integration
OS:CIO:B:SI
Director, Tax
Administration Modernization
OS:CIO:B:PM:TAM
Acting Director, Portfolio
Management Division OS:CIO:R:PM
Chief Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis RAS:O
Office of Management Controls OS:CFO:AR:M
Audit Liaisons:
Associate
Commissioner, Business Systems Modernization
OS:CIO:B
Chief, Information Technology Services OS:CIO:I
Appendix IV
This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. This benefit will be incorporated into our Semiannual Report to the Congress.
Type and Value of Outcome Measure:
· Reliability of Information – Actual; $71.9 million increase on baseline business case projection (see page 3).
Methodology Used to Measure the Reported Benefit:
We compared the e-Services Baseline Business Case Milestone 4/5 projections of approximately $47.2 million (representing current development, testing, and deployment phase cost projections) with the revised estimates of approximately $119.1 million. There was an increase of approximately $71.9 million (152 percent).
Appendix V
Management’s Response to the Draft Report
The response was
removed due to its size. To see the
response, please go to the Adobe PDF version of the report on the TIGTA Public
Web Page.