MEMORANDUM

Date: March 18, 2003

To: The Commissioner

From: Inspector General

Subject: Performance Indicator Audit: Paperless/Electronic Processing (A-15-02-11084)

We contracted with PricewaterhouseCoopers (PwC) to evaluate the data used to measure 18 of the Social Security Administration's (SSA) performance indicators established to comply with the Government Performance and Results Act. Attached is a final report that presents the results of three of the performance indicators PwC reviewed. The objective of this audit was to assess the reliability of performance indicators for implementing activities necessary to have the software and infrastructure in place for (1) paperless processing of Retirement and Survivors Insurance and Supplemental Security Income aged claims, (2) electronic processing of disability claims, and (3) electronic processing of hearings and appeals.

Please comment within 60 days from the date of this memorandum on corrective action taken or planned on each recommendation. If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.

James G. Huse, Jr.

OFFICE OF

THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR

AUDIT:

PAPERLESS/ELECTRONIC

PROCESSING

March 2003

A-15-02-11084

AUDIT REPORT

Mission

We improve SSA programs and operations and protect them against fraud, waste, and abuse by conducting independent and objective audits, evaluations, and investigations. We provide timely, useful, and reliable information and advice to Administration officials, the Congress, and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

By conducting independent and objective audits, investigations, and evaluations, we are agents of positive change striving for continuous improvement in the Social Security Administration's programs, operations, and management and in our own office.

MEMORANDUM

To: Office of the Inspector General

From: PricewaterhouseCoopers LLP

Date: February 24, 2003

Subject: Performance Indicator Audit: Paperless/Electronic Processing (A-15-02-11084)

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity set forth in its budget. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance. The objective of this audit was to assess the reliability of the following Fiscal Year (FY) 2002 GPRA performance indicators:

Performance Indicator
1. Implement activities necessary to have the software and infrastructure in place for paperless processing of Retirement and Survivors Insurance (RSI) and Supplemental Security Income (SSI) aged claims.
2. Implement activities necessary to have the software and infrastructure in place for electronic processing of disability claims.
3. Implement activities necessary to have the software and infrastructure in place for electronic processing of hearings and appeals.
See Appendix A for a description of the audit scope and methodology.

BACKGROUND

SSA offers retirement and long-term disability programs to the general public. Old-Age, Survivors, and Disability Insurance (OASDI) is authorized under title II of the Social Security Act (Act) and includes RSI and Disability Insurance (DI). Through the OASDI program, eligible workers and sometimes their family receive monthly benefits if they retire at an appropriate age or are found to have a disability that either prevents them from engaging in substantial gainful activity for at least 12 months or can be expected to result in death. SSI is authorized under title XVI of the Act and provides monthly payments to aged and disabled individuals based on financial need and medical requirements.

SSA's FY 2002 Annual Performance Plan (APP) includes five broad strategic objectives, the first of which is "To deliver citizen-centered world-class service." In support of this objective, SSA developed a number of goals to improve paperless and electronic processing of benefit claims and appeals. These goals will enhance SSA's service delivery and provide more efficient and effective processing of benefits. This report presents the results of our audit and SSA's completion of the following goals:

P.I./ Goal # Goal Description
1-A Accommodate dual entitlement advance file cases; automate determination of need to develop military service allegations; update the workers compensation file; and control certain exceptions via a Processing Center Action Control System (PCACS) interface. The Modernized Claims System 3.8 upgrade will allow processing of more advanced RSI and DI cases helping SSA avoid manual paper calculations.
1-B Implement Phase 2 of Attorney Fee/ Windfall Offset project. Begin analysis of additional windfall offset enhancement. An individual applying for or receiving benefits from Social Security has the right to select an attorney to represent them in pursuit of benefits from SSA. The individual may also elect to enter into a "fee agreement" with their attorney prior to a determination being made for benefits eligibility. Phase II of the Attorney Fee/Windfall Offset project will automate computations of attorney fees when there is an approved fee agreement for cases involving both title II and title XVI benefits concurrently. In addition, this process will revise how the windfall offset is calculated.
2-A Develop rules for a paperless business process, requirements/ infrastructure for the Electronic Folder (EF) and requirements to interface the EF with the legacy systems used to process disability claims. Performance Indicator #2 goals all support the Accelerated Electronic Disability (AeDib) initiative and allow SSA to have the software and infrastructure in place for electronic processing of disability claims.
2-B Develop requirements and a systems solution for the Office of Hearings and Appeals case processing system.
2-C Develop a strategy for electronic forms and integration with the EF and procure a tool.
2-D Establish policies/procedures for electronic signatures (internal and external requirements) and the policies necessary to make the EF the official Agency record.
2-E Develop infrastructure for electronic medical evidence and integration with the EF.
3-A Modernized SSI Claims System (MSSICS) will support field office entry of requests for hearings and appeals. This functionality will provide field office staff with the ability to enter requests for hearings and appeals into MSSICS.
3-B Provide web-based query access to the Consolidated Hearings Office Tracking System (HOTS) database, which includes the request for hearing. This functionality will provide all SSA employees the ability to access and view HOTS data through a web-based interface.

RESULTS OF REVIEW
The performance indicators evaluated in this report measure the completion of specific milestones for FY 2002. We performed our audit based on work completed as of December 11, 2002. We evaluated the processes and controls for the three performance indicators and determined the successful completion of each goal for FY 2002. Our audit found that four of the nine goals were not completed by fiscal year-end.

We also reviewed all completed work and associated documents for each goal. We found the results of each goal to be reasonable and adequate for their respective objectives stated in the performance plan. Where appropriate, we ensured that software releases were performed in accordance with SSA's Software Development Life Cycle and industry standard practices. We did note, however, one goal was completed prior to the start of FY 2002 and inaccurate data was reported for two goals in SSA's internal "Status Tracking Report" for the performance indicators.

We also determined if each performance indicator was an appropriate GPRA measure. We found that the performance indicators do not measure outputs or outcomes and are not qualitative or quantitative as prescribed by GPRA. However, the performance indicators address several government-wide initiatives including the Government Paperwork Elimination Act and the President's Management Agenda item for expanding electronic government. Office of Management and Budget Circular A-11 Part 6, Section 210.5 states that "Performance goals that represent milestones in achieving the general goals of a strategic plan may be appropriate." We therefore found that these performance indicators were appropriate measures for the APP.

PERFORMANCE INDICATOR #1 GOALS WERE COMPLETED FOR FY 2002

We evaluated SSA's completion of "activities necessary to have the software and infrastructure in place for paperless processing of RSI and SSI aged claims." We determined that the specific milestones listed contributed to the overall objectives of the performance indicator. Additionally, we found that these goals were completed for FY 2002 and both system upgrades were released to production. Our findings are consistent with what SSA reported in their Performance and Accountability Report for FY 2002 for this performance indicator.

PERFORMANCE INDICATOR #2 GOALS WERE PARTIALLY COMPLETED FOR FY 2002

We evaluated SSA's completion of "activities necessary to have the software and infrastructure in place for electronic processing of disability claims." We determined that the specific milestones listed contributed to the overall objectives of the performance indicator. Additionally, we found that goal 2-B was completed by fiscal year-end but that the other four goals for this performance indicator were not completed by fiscal year- end. Our findings are consistent with what SSA reported in their Performance and Accountability Report for FY 2002 for this performance indicator.

PERFORMANCE INDICATOR #3 GOALS WERE COMPLETED FOR FY 2002

We evaluated SSA's completion of "activities necessary to have the software and infrastructure in place for paperless processing of hearings and appeals." We found that both goals were completed and software updates were released to production. Our findings are consistent with what SSA reported in their Performance and Accountability Report for FY 2002 for this performance indicator.

However, we found that goal 3-A was completed in October 2000 or 1 year before the start of FY 2002 (see finding below). We also determined that the specific milestones listed contributed to the overall objectives of the performance indicator.

GOAL 3-A WAS COMPLETED PRIOR TO FY 2002

As stated above, we found that goal 3-A was completed in October 2000. We also found that initial planning for this goal began in 1996. Thus, we believe goal 3-A is an inappropriate goal for FY 2002 since it was completed 1 year prior to the start of FY 2002.

INACCURATE INTERNAL REPORTING OF PERFORMANCE INDICATOR'S PROGRESS

Our audit included a review of the internal status tracking of each performance indicator's progress. Each month project managers report their project status to the Office of Strategic Management (OSM) via e-mail. For milestone driven performance indicators such as these, project managers report the status as "on schedule", "delayed", "ahead of schedule", "behind schedule" or "completed". OSM generates its monthly status tracking report based on these e-mails. We reviewed the internal status tracking report and compared it to the actual status of the project based on interviews with the project manager and a review of documents completed as part of the project. We found that goal 3-A was consistently reported as on-schedule even though it was completed in October 2000. Additionally, we found that goal 2-E was reported as on-schedule on the internal status tracking report. However, based on our review, the goal will not be completed until FY 2003. Thus, we found that inadequate controls for the project status reporting process led to the inaccurate reporting.

OTHER MATTERS

As part of our audit, we reviewed the strategic objective for each performance indicator from the FY 2002 APP. Strategic objective A4 states that SSA seeks to "maintain accuracy, timeliness, and efficiency of service to people applying for OASDI and SSI aged benefits." While the APP had measures to determine the timeliness and accuracy of OASDI and SSI claims, we found that there were no supporting performance indicators related to efficiency. Without this performance indicator, SSA will not have baseline data to measure improvements for implementing software and infrastructure for paperless processing of OASDI and SSI aged claims.

Strategic objective A5 states that SSA seeks to "improve the efficiency of service to people applying for DI and SSI disability benefits." We found that there are no supporting performance indicators related to efficiency. SSA is making significant investments in software and infrastructure for electronic processing of disability claims and should develop indicators that report on the results of these efforts.

In conducting our audit, we also noted that the performance indicators reviewed did not include adequate descriptions of each goal. For example, goal 1-A states:

Accommodate dual entitlement advance file cases; automate determination of need to develop military service allegations; update the workers compensation file; and control certain exceptions via a PCACS interface."

The APP does not define these enhancements or explain how they relate to the underlying performance indicator. Our audit determined that this goal increases SSA's capacity to process paperless RSI claims. The APP should include plain language definitions for each goal and succinctly define how they contribute to the overall strategic objective.

CONCLUSIONS AND RECOMMENDATIONS

Our audit found that the five of the nine goals listed for the three performance indicators were completed for FY 2002. Overall, we found that the goals developed for each performance indicator contributed to the underlying strategic objective and are acceptable measures. Additionally, the goals were developed or implemented with adequate controls and an appropriate systems development methodology.

We note that four of the nine goals listed in this report were not completed by fiscal year-end. We found that goal 3-A was inappropriate since it was completed in October 2000. We also note that goal 3-A was internally reported as being on-schedule though it was completed in October 2000 and goal 2-E was reported as on-schedule even though is was not scheduled for completion until after FY 2002. Our recommendations are as follows:

1. Set goals that have not already been achieved

We recommend that SSA set goals for a fiscal year that have not already been achieved in a prior fiscal year.

2. Improve internal controls for reporting project status

We recommend that SSA improve internal controls for reporting monthly project status to OSM. Final performance indicators should be agreed on and signed off to ensure that they accurately reflect OSM and the project manager's intent. OSM should also require more detailed reporting of milestone-related performance indicators in order to accurately assess and report their status.

3. Add performance indicators that address efficiency as part of Strategic Objective A4 in the APP

We recommend that SSA add performance indicators to address efficiency as part of this Strategic Objective. This will assist SSA in measuring the results of their investments in new technology and infrastructure. The new indicators will also provide SSA the ability to track progress over time.

4. Add performance indicators that address efficiency as part of Strategic Objective A5 in the APP

We recommend that SSA add performance indicators to address efficiency as part of this strategic objective. This will assist in measuring the progress of the AeDib initiative and allow SSA to track improvements over time.

AGENCY COMMENTS

SSA agreed with our recommendations. They will remove goals that have already been completed in a prior fiscal year (Recommendation 1). SSA will also assure that components accurately report on milestones (Recommendation 2). It was noted that most milestone measures have been eliminated for FY 2003. SSA's "FY 2004 and Revised Final FY 2003 APP" will have efficiency measures for the OASDI and SSI programs (Recommendation 3 and 4). The full text of SSA's comments can be found in Appendix C.

PWC RESPONSE

The Agency plans outlined in their comments are responsive to the issues we identified and, when fully implemented, should satisfy the recommendations.

Appendices
APPENDIX A - Scope and Methodology
APPENDIX B - Acronyms
APPENDIX C - Agency Comments

Appendix A
Scope and Methodology

We conducted this audit to examine three performance indicators related to the Social Security Administration's (SSA) electronic and paperless processing initiatives. Our audit includes work completed as of December 11, 2002.

To test the completion of the goals listed in each performance indicator, we:
Reviewed all final work and related documents;
Ensured that completed work adequately addressed the stated objectives in the performance plan;
Ensured that the goals were completed by fiscal year-end;
Ensured software releases were performed in accordance with SSA's Software Development Life Cycle;
Evaluated SSA's processes and controls in completing each milestone to ensure that they conformed with industry standard practices; and
Reviewed the process for internally reporting the status of each performance indicator.
In conducting this audit, we also:
Reviewed SSA's Performance and Accountability Report for FY 2001, SSA's Annual Performance Plan for FY 2001, and SSA's Revised Performance Plan for FY 2002 to determine related milestones, definition, and data source for each performance indicator;
Reviewed Government Performance and Results Act (GPRA) and Office of Management and Budget guidance related to GPRA; and
Interviewed the respective project team for each milestone to gain an understanding of the goal, the process to complete each goal, and the outcome of the overall project.
Our audit was limited to testing at SSA's Headquarters in Woodlawn, Maryland. The procedures we performed were in accordance with the American Institute of Certified Public Accountants' Statement on Standards for Consulting Services and General Accounting Office's Government Auditing Standards ("Yellow Book") for performance audits.

Appendix B
Acronyms
Act The Social Security Act
AeDib Accelerated Electronic Disability Program
APP Annual Performance Plan
DI Disability Insurance
EF Electronic Folder
FY Fiscal Year
GPRA Government Performance and Results Act
HOTS Hearings Office Tracking System
MSSICS Modernized Supplemental Security Income Claims System
OASDI Old Age, Survivors, and Disability Insurance
OSM Office of Strategic Management
PCACS Processing Center Action Control System
RSI Retirement and Survivors Insurance
SSA Social Security Administration
SSI Supplemental Security Income

Appendix C
Agency Comments

Date: December 23, 2002

To: James G. Huse, Jr.
Inspector General

From: Larry Dye
Chief of Staff

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: Paperless/Electronic Processing" A-15-02-11084

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Staff questions may be referred to Trudy Williams on extension 50380.

SSA Response

COMMENTS OF THE SOCIAL SECURITY ADMINISTRATION (SSA) ON THE OFFICE OF THE INSPECTOR GENERAL (OIG) DRAFT REPORT, "PERFORMANCE INDICATOR AUDIT: PAPERLESS/ELECTRONIC PROCESSING" (A-15-02-11084)

Thank you for the opportunity to review and comment on the subject draft report. As requested, below are our comments.

Recommendation 1

Set goals that have not already been achieved. SSA should set goals for a fiscal year (FY) that have not already been achieved in a prior fiscal year.

Comment

We agree and will remove goals that have been completed in a prior fiscal year.

Recommendation 2

Improve internal controls for reporting projected status. SSA should improve internal controls for reporting monthly project status to the Office of Strategic Management (OSM). Final performance indicators should be agreed on and signed off to ensure that they accurately reflect OSM and the project manager's intent. OSM should also require more detailed reporting of milestone-related performance indicators in order to accurately assess and report their status.

Comment

We agree. We will assure SSA components accurately report on milestones. Since most of the milestone measures have been eliminated for FY 2003, we don't expect to encounter this problem.

Recommendation 3

Add performance indicators that address accuracy and efficiency as part of Strategic Objective A4 in the Annual Performance Plan (APP). SSA should add performance indicators to address accuracy and efficiency as part of this Strategic Objective. This will assist SSA in measuring the results of their investments in new technology and infrastructure. The new indicators will also provide SSA the ability to track progress over time.
Comment

We agree in part with this recommendation. The performance indicators for accuracy were addressed and are listed on pages 86 and 87 of the "Revised Final Performance Plan for Fiscal Year 2002." Concerning the efficiency indicator, the "FY 2004 and Revised Final FY 2003 APP" will have an efficiency measure. However, it most probably will not be part of strategic objective A4 as the agency will be refining its strategic goals and objectives.
Recommendation 4

Add performance indicators that address efficiency as part of Strategic Objective A5 in the APP. SSA should add performance indicators to address efficiency as part of this strategic objective. This will assist in measuring the progress of the AeDib initiative and allow SSA to track improvements over time.

Comment

We agree in part with this recommendation. Concerning an efficiency indicator, the "FY 2004 and Revised Final FY 2003 APP" will have an efficiency measure. However, it most probably will not be part of strategic objective A5 as the agency will be refining its strategic goals and objectives.

Other Comment

On page 4 of the report, the under the caption PERFORMANCE INDICATOR
#1 GOALS WERE COMPLETED FOR FY 2002, the first sentence reads, " We evaluated SSA's completion of "activities necessary to have the software and infrastructure in place for paperless processing or RSI and SSI aged claims." Should be revised to read, "We evaluated…infrastructure in place for electronic processing of hearings and appeals."

Overview of the Office of the Inspector General

Office of Audit

The Office of Audit (OA) conducts comprehensive financial and performance audits of the Social Security Administration's (SSA) programs and makes recommendations to ensure that program objectives are achieved effectively and efficiently. Financial audits, required by the Chief Financial Officers' Act of 1990, assess whether SSA's financial statements fairly present the Agency's financial position, results of operations and cash flow. Performance audits review the economy, efficiency and effectiveness of SSA's programs. OA also conducts short-term management and program evaluations focused on issues of concern to SSA, Congress and the general public. Evaluations often focus on identifying and recommending ways to prevent and minimize program fraud and inefficiency, rather than detecting problems after they occur.

Office of Executive Operations

The Office of Executive Operations (OEO) supports the Office of the Inspector General (OIG) by providing information resource management; systems security; and the coordination of budget, procurement, telecommunications, facilities and equipment, and human resources. In addition, this office is the focal point for the OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act. OEO is also responsible for performing internal reviews to ensure that OIG offices nationwide hold themselves to the same rigorous standards that we expect from SSA, as well as conducting investigations of OIG employees, when necessary. Finally, OEO administers OIG's public affairs, media, and interagency activities, coordinates responses to Congressional requests for information, and also communicates OIG's planned and current activities and their results to the Commissioner and Congress.

Office of Investigations

The Office of Investigations (OI) conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement of SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, physicians, interpreters, representative payees, third parties, and by SSA employees in the performance of their duties. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Counsel to the Inspector General

The Counsel to the Inspector General provides legal advice and counsel to the Inspector General on various matters, including: 1) statutes, regulations, legislation, and policy directives governing the administration of SSA's programs; 2) investigative procedures and techniques; and 3) leg l implications and conclusions to be drawn from audit and investigative material produced by the OIG. The Counsel's office also administers the civil monetary penalty program.