SOCIAL SECURITY ADMINISTRATION

SOCIAL SECURITY
ADMINISTRATION'S MANAGEMENT
OF INFORMATION TECHNOLOGY
PROJECTS

July 2007

A-14-07-17099

AUDIT REPORT

Mission

By conducting independent and objective audits, evaluations and investigations, we inspire public confidence in the integrity and security of SSA's programs and operations and protect them against fraud, waste and abuse. We provide timely, useful and reliable information and advice to Administration officials, Congress and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

We strive for continual improvement in SSA's programs, operations and management by proactively seeking new ways to prevent and deter fraud, waste and abuse. We commit to integrity and excellence by supporting an environment that provides a valuable public service while encouraging employee development and retention and fostering diversity and innovation.

MEMORANDUM

Date: July 26, 2007

To: The Commissioner

From: Inspector General

Subject: Social Security Administration's Management of Information Technology Projects (A-14-07-17099)

OBJECTIVE

The objective of our review was to determine whether the Social Security Administration (SSA) receives the intended value for its Information Technology (IT) investments. Specifically, we examined whether SSA has a process in place to determine if its planned functionality and cost savings were achieved.

BACKGROUND

The Clinger-Cohen Act of 1996 requires that Federal agencies establish effective and efficient planning processes for selecting, managing, and evaluating the results of all their major investments in information systems. The Office of Management and Budget (OMB) Circular A-130, Management of Federal Information Resources, requires Federal agencies to "…conduct post-implementation reviews of information systems and information resources management processes to validate estimated benefits and costs, and document effective management practices for broader use."

SSA Information Technology Planning Process

At SSA, the Information Technology Advisory Board (ITAB) is the governing body for its IT planning process and is responsible for the development of the Agency IT Systems Plan. The Agency's ITAB is chaired by the Chief Information Officer and its membership is comprised of the Deputy Commissioner for SSA, all Deputy Commissioners for the business components, as well as other Agency executives.

The ITAB reviews a variety of SSA's IT projects categorized by investment portfolios. Each investment portfolio contains a list of IT projects, which support one of the SSA's strategic objectives documented in the Agency's Strategic Plan. Portfolio teams are led by an Agency executive who functions as the portfolio manager. The portfolio team coordinates with stakeholders to prioritize IT projects according to their importance in achieving the related strategic objective. (See Appendix C for an overview of SSA's IT planning process). The IT planning process used in prioritizing portfolio items can take as long as 6 months.

After the IT projects are prioritized and presented to the ITAB, the ITAB must decide which of the Agency's scarce resources will be assigned to the various IT projects. In making this decision, the ITAB not only considers the portfolio priorities, it also considers the related cost benefit analysis provided by the sponsoring components. Such information includes return on investment (ROI), full-time equivalent (FTE) savings, dollar savings, and cost avoidance.

Once the projects have been selected for implementation the ITAB tracks their progress. The ITAB representatives are presented with SSA's achievements and how well the Agency's major IT projects progressed in terms of cost and schedule, on a quarterly basis. However, ITAB is not informed whether the projects it approved actually achieved their estimated functionality and cost savings.

SSA's Investment Results Evaluation

SSA's evaluation of its investment results is a bifurcated process. SSA's Office of the Chief Information Officer (OCIO) has established the policy for conducting Post Implementation Reviews (PIR) to determine whether the projects' planned benefits have been achieved and the reasons for any discrepancies. This policy is documented in SSA Target Information Technology (IT) Capital Planning and Investment Control Process (CPIC) Guide. Although not intended to meet the requirements of a PIR, a separate process for evaluating the functionality of IT projects and documenting the lessons learned is conducted by SSA's Office of Systems (OS). SSA's OS conducts Post Release Reviews (PRR) to verify with the customer that the promised functionality was delivered after the completion and implementation of a project or the major release of a project.

RESULTS OF REVIEW

SSA's ITAB evaluates functionality, ROI, and cost savings information to formulate its decisions during the IT planning process. However, IT investment results are not independently verified after project completion to ensure that the functionality and cost savings were ultimately achieved. Table 1 provides four examples of IT projects that were approved by ITAB for Fiscal Year 2006 implementation. Table 1 illustrates the resources SSA estimated it needed to deliver the projects' functionality and cost savings.

TABLE 1
Table 1. Projects Proposed to the ITAB Committee for Fiscal Year 2006 Implementation
Project Name Total Costs and Staff Level Functional Description FTE Savings Net Cost Saving ROI
% Investment Results Independently Verified?
Access to information held by financial institutions $37 million 13 FTEs Collect permission to obtain financial data from all SSI beneficiaries None $ 232 million 614 No
800# Speech technology(change of address / direct deposit) $2 million
9 FTEs Increase 800# efficiency and provide more timely and accurate changes 774 $ 48 million 1,997 No
Annual wage reporting tax year new $13 million 132 FTEs Allows employers to submit annual wage reports to SSA using various methods of reporting 5,936 $ 248 million 1,897 No
WEB-based SAVE $0.8 million
3 FTE

Allows SSA to verify Alien Status with Department of Homeland Security via the Web 220 $ 14 million 1,657 No
SSA has established a PIR policy for verifying planned benefits of its IT projects that generally meets OMB's requirements. However, SSA has not been conducting PIRs to verify functionality and cost savings. PIRs would help enable ITAB to determine whether many of the IT projects it assessed and approved actually delivered the projects' functionality and cost savings as estimated. Furthermore, without the verification of functionality and cost saving information, ITAB lacks information on where dollars should be spent.

Independent of the PIRs, SSA's OS performs PRRs to verify with the customer whether the planned functionality of an IT project has been delivered. This process can be improved in a few areas as discussed below. SSA has not yet established a process to verify the estimated cost savings. For budgeting purposes, the Office of Operations conducts analysis of certain IT projects that impact on Operations FTE positions; however, the analysis is limited to Operations staff savings and therefore does not meet PIR requirements. Also, the results are only reported within the Office of Operations and to the Office of Budget.

SSA's processes for verifying functionality and cost savings could be improved if it would leverage the current OS PRR mechanism, with modifications, to meet some PIR requirements. SSA needs to address both the PIR and PRR processes.

SSA DOES NOT HAVE A PIR PROCESS

SSA did not determine whether its major IT projects have delivered the overall functionality and cost savings as required by OMB. As a result, SSA did not have an effective means to know how well its major IT investments, individually or collectively, delivered the functionality and cost savings planned to achieve the Agency's organizational goals.

OMB requires that Federal agencies conduct PIRs to validate expected functionality and cost savings after a major IT project is completed and implemented. According to SSA's OCIO, it did not have the required resources to conduct PIRs for SSA's major IT projects. In addition, related efforts of other SSA components were not coordinated or integrated to form a system that, as a whole, independently measured whether the Agency's major IT projects achieved the functionality and cost savings promised. One way to help SSA satisfy OMB's PIR requirements is by accumulating, consolidating and communicating the results of OS's current PRR activities across SSA.

To address these issues, SSA needs to coordinate its internal efforts to create a PIR process that is adequately staffed to measure whether its major IT projects deliver the expected functionality and cost savings and ensure its management and ITAB are informed of the results.
SSA's CURRENT PRR PROCESS NEEDS IMPROVEMENT

PRRs conducted by OS are the closest process within SSA to a PIR as defined by OMB. OS staff stated that its PRR procedure is an internal process to OS and was not designed to satisfy OMB requirements for a PIR. However, PIRs are required and OS's PRR process, if conducted by an independent and objective review team, can contribute to fulfilling the OMB PIR requirements. The PRR process needs to address the following issues:

Promised Functionality Was Not Systematically Verified

In 6 of the 10 OS PRR reports we evaluated (see Appendix B), the documentation did not indicate if the promised functionality was actually obtained. We reviewed these reports with the corresponding Project Scope Agreements (PSA) and other related documents. In addition to the six PRR reports, which did not state whether functionality was achieved, we also found the other four review reports did not adequately document the functionality achieved. For example, one PRR report concluded that the project met the user's expectations of functionality; however, the documentation only had support for a small portion of the total functionality documented in the PSA. As a result, SSA management could not rely on these reports to determine what functions were achieved and to what degree they were achieved.

The causes of these issues were as follows:

There was a lack of standardized methodology to ensure functionality documented in the PSAs was completely and systematically verified during the OS PRR process.

Detailed standards for reporting the degree of functionality achieved was not sufficient. Project Managers were left with discretion on the degree of details to be reported.

OS did not provide sufficient training to ensure PRRs were properly conducted.

OS Systems Process Improvement (SPI) staff stated that they were aware of and agreed with our observations in these areas. The SPI Branch is conducting a study to plan improvements for the PRR process.

To address these issues, SSA needs to:

define standards and methodology for project review teams to determine the degree of promised functionality that was delivered;
refine its guidance on functionality review reporting to ensure the degree of functionality achievements are properly reported and verified with the client; and
provide training to ensure related SSA policies and procedures are properly followed.

PRR Results Were Not Effectively Communicated To Appropriate Management

OS did not effectively communicate its functionality review results to ITAB. Results of PRRs were communicated with OS management in monthly and quarterly management meetings and were stored in a central repository database. However, there is no evidence that review results were effectively communicated outside OS to ITAB. SSA established ITAB to ensure sufficient involvement of senior SSA executives.

As a result, SSA's senior executives lacked information they could have used to evaluate the effectiveness of SSA's IT investments in achieving organizational goals and objectives. Also, without the verification of functionality and cost savings information, ITAB lacks information on where dollars should be spent.

Federal agencies are required by the Clinger-Cohen Act to develop an effective CPIC process for the selection of IT investments, the management of such investments, and the evaluation of the results of such investments.

SSA's IT decision makers and senior executives need to know whether the IT projects were completed with the promised functionality achieved. SSA needs to ensure that PRR results are reported to the ITAB and this communication is documented.

CONCLUSION AND RECOMMENDATIONS

SSA has established a policy for evaluating the results of its IT investments. However, the Agency has not implemented an effective process to determine if planned functionality and cost savings of its IT projects are actually achieved.

To make SSA's evaluation process of IT investments more effective and informative, we recommend the following:

1. Implement a PIR process as required by OMB and, to the extent possible, leverage SSA's current processes.

2. Enhance the current OS PRR process to contribute to OMB requirements by ensuring:

a. Standards and methodology are defined for the project review teams to use in evaluating the degree of promised functionality that was delivered;

b. PRR results are fully documented, reported, and communicated to appropriate SSA management consistent with established guidance; and

c. Training is provided to ensure pertinent SSA policies and procedures are properly followed when conducting PRRs.

AGENCY COMMENTS

SSA agreed with our recommendations. The text of SSA's comments is included in Appendix D.

Patrick P. O'Carroll, Jr.

Appendices
APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Agency Information Technology Planning Process Overview
APPENDIX D - Agency Comments
APPENDIX E - OIG Contacts and Staff Acknowledgments

Appendix A
Acronyms
CPIC Capital Planning and Investment Control
EGADS Electronic General Auditable Documents Store
FTE Full-Time Equivalent
IT Information Technology
ITAB Information Technology Advisory Board
OCIO Office of the Chief Information Officer
OMB Office of Management and Budget
OS Office of Systems
PIR Post Implementation Review
PRR Post Release Review
PSA Project Scope Agreement
ROI Return on Investment
SPI System Process Improvement
SSA Social Security Administration

Appendix B
Scope and Methodology
The objective of our review was to determine whether the Social Security Administration (SSA) receives the intended values for its Information Technology (IT) investments. Specifically, we examined whether SSA has a process in place to determine whether the planned functionality and cost savings were achieved.

To meet the objective of this audit, we reviewed relevant Federal laws, regulations and guidance. We reviewed SSA's IT capital planning and investment control process, with a focus on the evaluation process of IT investment results, by reviewing SSA policies, procedures and practices, and conducting interviews with relevant SSA personnel. We also examined a random sample of the Office of System's (OS) Post Release Review (PRR) reports.

We have reviewed the following Federal laws, regulations, and guidance:

Clinger Cohen Act of 1996.
Government Accountability Office, Information Technology Investment Management, A Framework for Assessing and Improving Process Maturity, dated March 2004.
Office of Management and Budget (OMB) Circular A-11, Part 7: Planning, Budgeting, and Acquisition of Capital Assets, and its supplement, Capital Programming Guide Version 2.0, dated June 2006.
OMB Circular A-130, Management of Federal Information Resources.

We have reviewed the following SSA policies, procedures, and documents:

SSA Target Information Technology (IT) Capital Planning and Investment Control Process (CPIC) Guide.
OS Procedure for Conducting a Phase I - Post Implementation Review.
SSA Information Technology Advisory Board meeting materials and minutes.

We have contacted or interviewed SSA staff in the following components:

Office of the Chief Information Officer, Office of Information Technology Systems Review;
OS, Systems Planning Staff;
Office of Operations, Office of Public Service and Operations Support, Division of Resource Management Information; and
Office of Budget, Finance and Management, Office of Budget.

We randomly selected a sample of 10 of 52 projects whose PRR reports were documented in SSA's Electronic General Auditable Documents Store database as of November 30, 2006. Our sampling frame is limited to reports documented in the database since the beginning of Calendar Year 2005 to November 2006. We reviewed these PRR reports with the corresponding Project Scope Agreements (PSA) and other related documents as needed to meet our audit objective. The 10 projects or release projects as titled on the PRR reports or PSAs were:

1. Disability Case Adjudication and Review System Release 5.0 and Electronic Disability Case Adjudication and Review System Release 3.0.
2. Social Security Number Verification Service Release 4.
3. Assignment and Correspondence Tracking System.
4. 800# Change of Address and Direct Deposit.
5. Electronic Disability Collect System / Electronic View / Electronic Interface standards, Version 8.1.
6. Windows Operating System Migration Project.
7. Software Image Distribution.
8. SSA Unified Measurement Systems Enumeration Release 1.
9. Customer Help and Information Program Service Observation Release.
10. Changes Required by New Simplification Regulations.

This audit was performed in accordance with generally accepted government auditing standards. We conducted our field work at the SSA Headquarters in Baltimore, Maryland from December 2006 until February 2007.

Appendix C

Appendix D
Agency Comments

SOCIAL SECURITY
MEMORANDUM

Date: July 5, 2007

To: Patrick P. O'Carroll, Jr.
Inspector General

From: Larry Dye

Subject: Office of the Inspector General (OIG) Draft Report, "Social Security Administration's Management of Information Technology Projects" (A-14-07-17099)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Please let me know if we can be of further assistance. Staff inquiries may be directed to
Ms. Candace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

Attachment:
SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL DRAFT REPORT, "SOCIAL SECURITY ADMINISTRATION'S MANAGEMENT OF INFORMATION TECHNOLOGY PROJECTS"(A-14-07-17099)

Thank you for the opportunity to review and comment on the draft report. We appreciate your conducting this audit of the Social Security Administration's (SSA) management of information technology (IT) projects.

Recommendation 1

SSA should implement a Post Implementation Reviews (PIR) process as required by the Office of Management and Budget (OMB) and, to the extent possible, leverage SSA's current processes.

Comment

We agree with this recommendation. SSA's Office of the Chief Information Officer (OCIO) remains committed to carrying out PIRs subject to the availability of resources. It is the intention of the OCIO to develop a PIR process that follows the IT investment throughout its life cycle, assessing the return on investment along the way, so that incremental investment decisions can be based on current assessments of resources invested and value returned, as well as better informed forecasts of future costs and benefits.

Recommendation 2

SSA should enhance the current Office of Systems' (OS) Post Release Reviews (PRR) process to contribute to OMB requirements by ensuring:

a. Standards and methodology are defined for the project review teams to use in evaluating the degree of promised functionality that was delivered;

b. PRR results are fully documented, reported, and communicated to appropriate SSA management consistent with established guidance; and

c. Training is provided to ensure pertinent SSA policies and procedures are properly followed when conducting PRRs.

Comment

We agree. We are in the process of revising the OS PRR process in line with what has been stated in the recommendation. This will go through the normal review and approval process and is anticipated for implementation in October 2007. Training will be provided as part of the Project Management Curriculum for OS project managers.

The subject report does a great service by pointing out that a number of Agency components carry out evaluations of various aspects of IT investments that may be melded into a coherent assessment of an investment's overall success in achieving its initial functional and cost-benefit expectations. SSA is fully committed to working to leverage these existing processes, along with any new processes that are required, to develop a comprehensive PIR process that will meet OMB's requirements and provide superior IT investment management information for the consideration of the Information Technology Advisory Board and other senior executives.

[In addition to the comments above, SSA provided some technical comments which have been addressed in this report.]

Appendix E
OIG Contacts and Staff Acknowledgments
OIG Contacts
Kitt Winter, Director, Data Analysis and Technical Audits Division, (410) 965-9702
Albert Darago, Audit Manager, Application Controls Branch, (410) 965-9710
Acknowledgments
In addition to those named above:
Grace Chi, Senior Auditor

For additional copies of this report, please visit our web site at www.socialsecurity.gov/oig or contact the Office of the Inspector General's Public Affairs Specialist at (410) 965-3218. Refer to Common Identification Number A-14-07-17099.

Overview of the Office of the Inspector General
The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Resource Management (ORM). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit
OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations
OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General
OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Resource Management
ORM supports OIG by providing information resource management and systems security. ORM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, ORM is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.