OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
ELECTRONIC SERVICE DELIVERY

March 2007

A-15-06-16111

AUDIT REPORT

 


Mission

By conducting independent and objective audits, evaluations and investigations, we inspire public confidence in the integrity and security of SSA's programs and operations and protect them against fraud, waste and abuse. We provide timely, useful and reliable information and advice to Administration officials, Congress and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

We strive for continual improvement in SSA's programs, operations and management by proactively seeking new ways to prevent and deter fraud, waste and abuse. We commit to integrity and excellence by supporting an environment that provides a valuable public service while encouraging employee development and retention and fostering diversity and innovation.

MEMORANDUM

Date: March 8, 2007

To: The Commissioner

From: Inspector General

Subject: Performance Indicator Audit: Electronic Service Delivery (A-15-06-16111)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 15 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. Attached is the final report presenting the results of two of the performance indicators PwC reviewed. For the performance indicators included in this audit, PwC's objectives were to:

Assess the effectiveness of internal controls and test critical controls over data generation, calculation, and reporting processes for the specific performance indicator.

Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.

Test the accuracy of results presented and disclosed in the Fiscal Year 2006 Performance and Accountability Report.

Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

This report contains the results of the audit for the following indicators:

Increase the usage of electronic entitlement and supporting actions.
Increase the percent of employee reports (W-2 forms) filed electronically.

If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.


Patrick P. O'Carroll, Jr

Attachment

MEMORANDUM

Date: March 1, 2007

To: Inspector General

From: PricewaterhouseCoopers LLP

Subject: Performance Indicator Audit: Electronic Service Delivery (A-15-06-16111)

OBJECTIVE

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

Our audit was conducted in accordance with generally accepted government auditing standards for performance audits. For the performance indicators included in this audit, our objectives were to:

1. Assess the effectiveness of internal controls and test critical controls over the data generation, calculation, and reporting processes for the specific performance indicator.

2. Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.

3. Test the accuracy of results presented and disclosed in the Fiscal Year (FY) 2006 Performance and Accountability Report (PAR).

4. Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

BACKGROUND

We audited the following performance indicators as stated in the SSA FY 2006 PAR:

Performance Indicator FY 2006 Goal FY 2006 Reported Results
Increase the usage of electronic entitlement and supporting actions 300% growth over FY 2004 baseline (2,211,200) 291.8% growth over FY 2004 baseline (2,165,865)
Increase the percent of employee reports (W-2 forms) filed electronically 70% 75%

SSA has identified four Service challenges in FY 2006. These include:

Management of the Disability Process;
Improve Programs that Provide Support for People with Disabilities;
Better Position SSA for Future Service Delivery Challenges, Including Information Technology; and
Electronic Government.

During FY 2006, SSA focused on addressing Service challenges through the use of technology. To meet these challenges, SSA provided Internet systems to the public and employers. These systems allowed the public to apply for Old-Age, Survivors, and Disability Insurance (OASDI) online, make changes to their address and direct deposit information, and request replacement cards for Medicare. Also, employers were able to submit employee wages via the Internet systems. With the onset of these Internet systems, SSA has:

Provided the public with easy and efficient access to SSA services;
Improved Agency productivity and service through increased processing efficiencies; and
Conserved OASDI Trust Funds and general revenue monies by having the public complete the significant data entry procedures for the OASDI and wage reporting processes.

RESULTS OF REVIEW

Our assessment of the two indicators included in this report did not identify any significant exceptions related to the internal controls and data reliability, accuracy of presentation or disclosure of the information related to these indicators contained in the FY 2006 PAR, or to the meaningfulness of these indicators.

Increase the usage of electronic entitlement and supporting actions

Indicator Background

This performance indicator measures the increase of the public's use of SSA's Internet services specific to applying for OASDI benefits, updating address and direct deposit information with SSA, and requesting replacement Medicare cards. There are seven systems (including three Internet Social Security Benefit Application (ISBA) Information Technology (IT) applications as noted below), that provide these Internet services to the public. They include the following:

Proof of Income - Internet and 800# voice enabled (BEVE);
Password based and Knowledge based Change of Address (iCOA);
Internet Change of Direct Deposit (iCDD);
Internet Medicare Replacement Card - Internet (iMRC) and 800# voice enabled; and
ISBA IT applications (retirement, disability, and spouse).

All applications for OASDI benefits entered into ISBA are electronically transferred to Field Offices (FO) for processing. Once received by the FOs, Claims Representatives review and authorize the ISBA transactions for processing through the Modernized Claims System (MCS). After being processed through MCS, key data elements related to the completed internet transaction applications are recorded in the Title II Operational Datastore (TII ODS). This key data is transferred to the Executive and Management Information System (EMIS) on a weekly basis for review and analysis by SSA management.

The supporting actions, including updating address and direct deposit information with SSA and requesting replacement Medicare cards, are entered into BEVE, iCOA, iCDD, and iMRC and are processed on a nightly basis. Successfully completed transactions are recorded in the Electronic Service Delivery/ Internet Management Delivery Architecture Project (EMAP) database. On a weekly basis, EMAP creates the Internet Management Information report to summarize the transactions successfully processed by these applications. This data is sent to the EMIS on a weekly basis for review and analysis by SSA management.

EMIS generates weekly reports of the aggregate count of the successful transactions processed by the seven applications for the FY. This data is stored in an Access database. The performance indicator is calculated in the following manner:

Performance Indicator Calculation

Increase the usage of electronic entitlement and supporting actions over the 2004 baseline year

*FY 06 Total Application Counts (minus) - FY 04 Total Application Counts __________________________

FY 04 Total Application Counts

* This result is multiplied by 100 to obtain a percent.

Findings

We did not identify any significant findings related to the internal controls, data reliability, accuracy of presentation, meaningfulness, or disclosure of the information related to this indicator contained in the FY 2006 PAR.

Increase the percent of employee reports (W-2 forms) filed electronically

Indicator Background

This performance indicator measures the percentage of employee reports (W-2 forms) filed electronically through:

Electronic Data Transfer (EDT);
Internet W2 Pre-Processing System (IW2PS); and
The W-2 Online application.

Employers can submit W-2s to SSA electronically via the IW2PS, W-2 Online application, or EDT. The W-2 data, submitted by employers, is converted to a common text format by the Annual Wage Reporting system (AWR), and validated against SSA records to ensure the name and Social Security Number (SSN) of the individuals are correct. If the names and SSNs match SSA records, the wage data is posted to the each individual's master earnings record in the Master Earnings File (MEF). If the name and SSN do not match SSA records, the wage data is posted to the Earnings Suspense File (ESF).

Data from the MEF and ESF is sent to the Earnings Management Information Operational Datastore (EMODS) on a daily basis. On a monthly basis, SSA management personnel produces the EMODS Report 1007 which includes the number of W-2 forms received electrically via EDT, IW2PS, and W-2 Online. These numbers are transferred into an Excel spreadsheet and totaled to obtain the "Number of W-2s filed electronically and processed to completion for a tax year" (this number is used as numerator of the indicator results).

The Office of the Actuary (OACT) estimates the total number of W-2s expected to be received and posted for the current year (this number is used as denominator of the indicator results). This estimate is recorded in the Excel spreadsheet. Refer to Appendix D for a detailed discussion of the actuarial estimate. The performance indicator is calculated in the following manner:

Performance Indicator Calculation

Number of W-2s filed electronically and processed to completion for a tax year

Sum of the 8 1/2 months (January 15, 2006 through September 30, 2006) of the W-2s filed electronically and processed to completion.

Total number of W-2s for that tax year processed to completion by the end of the processing year (mid-January)
Actuarial estimate for the total number of W-2s for the tax year 2005 processed to completion by the end of January 2007

The percentage of employee reports (W-2 forms) filed electronically

*Number of W-2s filed electronically and processed to completion for a tax year
____________________________

Total number of W-2s for that tax year processed to completion by the end of the processing year (December 31)

* This result is multiplied by 100 to obtain a percent.

Findings

We did not identify any significant findings related to the internal controls, data reliability, accuracy of presentation, meaningfulness, or disclosure of the information related to this indicator contained in the PAR.

RECOMMENDATIONS

There are no recommendations.

AGENCY COMMENTS

SSA agreed with our report. See Appendix E for the text of the Agency's comments.

Appendices

APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Statistical Methodology
APPENDIX E - Agency Comments

Appendix A
Acronyms

AWR Annual Wage Reporting system
BEVE Proof of Income
BSO Business Services Online
CES Current Employment Survey
CPS Current Population Survey
EDT Electronic Data Transfer
EMAP Electronic Service Delivery/ Internet Management Delivery Architecture Project
EMIS Electronic Management Information System
EMODS Earnings Management Information Operational Datastore
ERMS Earnings Records Maintenance System
ESF Earnings Suspense File
FO Field Office
FY Fiscal Year
GAO Government Accountability Office
GPRA Government Performance and Results Act
iCOA Internet Change of Address
iCDD Internet Change of Direct Deposit
iMRC Internet Medicare Replacement Card
ISBA Internet Social Security Benefit Application
IT Information Technology
IW2PS Internet W2 Pre-Processing System
MCS Modernized Claims System
MEF Master Earnings File
MI Management Information
OASDI Old-Age, Survivors, and Disability Insurance
OACT Office of the Actuary
OSM Office of Strategic Management
PAR Performance and Accountability Report
SSA Social Security Administration
SSN Social Security Number
TII ODS Title II Operational Datastore
T2WMI Title II Work Management Information System
U.S.C. United States Code

Appendix B
Scope and Methodology

We updated our understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing, including testing of source documentation, we performed the following:

Reviewed prior SSA, Office of the Inspector General and other reports related to SSA's GPRA performance and related information systems.
Reviewed applicable laws, regulations and SSA policy.
Met with the appropriate SSA personnel to confirm our understanding of the performance indicator.
Flowcharted the process. (See Appendix C).
Tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).
Conducted and evaluated tests of the automated and manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.
Identified attributes, rules, and assumptions for each defined data element or source document.
Recalculated the metrics or algorithm of key performance indicators to ensure mathematical accuracy.
For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators appear to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes.

We followed all performance audit standards in accordance with generally accepted government auditing standards. In addition to the steps above, we specifically performed the following to test the indicators included in this report:
INCREASE THE USAGE OF ELECTRONIC ENTITLEMENTS AND SUPPORTING ACTIONS

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following areas:
Completed application control reviews over the Internet Social Security Benefit Application (ISBA), Proof of Income - Internet and 800# voice enabled (BEVE), Password based and Knowledge based Change of Address (iCOA), Internet Change of Direct Deposit (iCDD), and Internet Medicare Replacement Card - Internet (iMRC), and Medicare Replacement Card - 800#, Title II Operational Datastore, and Electronic Service Delivery/ Internet Management Delivery Architecture Project (EMAP) database.
Recalculated the indicator for Fiscal Year (FY) 2006 and compared it to the number reported in the Performance Accountability Report.

INCREASE THE PERCENT OF EMPLOYEE REPORTS (W-2 FORMS) FILED ELECTRONCIALLY

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following areas:
Completed application control reviews over the Earnings Records Maintenance System (ERMS) and the Earnings Modernized Datastore (EMODS.)
Determined the reasonableness of the estimate used in the indicator.
Determined the adequacy of the programming logic used by SSA to calculate the increase the percent of employee reports (W-2 forms filed electronically).
Recalculated the indicator for FY 2006 and compared it to the number reported in the Performance and Accountability Report.

Appendix C
Flowchart of Increase the Usage of Electronic Entitlements and Supporting Actions

Increase the Usage of Electronic Entitlements and Supporting Actions

START
Claimant completes OASDI application or supporting action on the SSA website.
The OASDI application is input using Internet Social Security Benefit Application (ISBA). ISBA includes the following applications: retirement, spouse, and disability. The supporting actions are input using the following applications:

Proof of Income - Internet and 800# voice enabled;
Password based and Knowledge based Change of Address (iCOA);
Internet Change of Direct Deposit (iCDD); and,
Medicare Replacement Card - Internet (iMRC) and 800# voice enabled.
For ISBA transactions, the Field Office (FO) receives the Title II Work Management Information case listing that identifies Internet cases to be triggered by the Modernized Claims System (MCS).
The Internet claims data is automatically propagated into MCS when triggered by the FO action by a claims representative. (Note: There is a field that indicates the claim was received via the Internet.)
MCS processes the claim and sends the updated data to the Title II Operational Datastore (TII ODS).
Every week, the Electronic Management Information System (EMIS) Internet site is updated with the Internet workload counts.
For supporting actions, a nightly run reads the transaction data input by the public and stores it in the EMAP database.
Every week, the Internet Management information (MI) reports for iCOA, iCDD, iMRC, and Proof of Income are updated and sent to EMIS.
Every week the EMIS reports are generated and this data is transcribed into an Access database and analyzed for potential variances.
SSA Intranet site updated monthly with Increase the Usage of Electronic Entitlements and Supporting Actions to the Office of Strategic Management for reporting in the Performance and Accountability Report (PAR).

Flowchart of Increase the Percent of Employee Reports (W-2 forms) Filed Electronically

Increase the Percent of Employee Reports (W-2 forms) Filed Electronically
START
Employers send wage data to SSA via the Business Services Online (BSO) Internet application (W-2 Online and Wage File Upload) or Electronic Data Transfer (EDT).
If the wage data is sent via BSO, it will be pre-processed through the Internet W2 Pre-processing System (IW2PS).
Wage data is converted to a common format by the Annual Wage Reporting (AWR) system. Edit, validity, and balancing checks are performed on wage submissions prior to posting.
If the Name/SSN matches to a Numident record, wage data is posted to the individual's Master Earnings File (MEF) record.
If the Name/SSN does not match to a Numident record, wage data is posted to the Earnings Suspense File (ESF).
Earnings Management Information Operational Datastore (EMODS) is updated daily with earnings figures from the MEF and ESF. The EMODS Report 1007 is created.
On a monthly basis, the numbers for electronic categories (EDT, Wage File Upload, and W2 Online) are transferred to an Excel spreadsheet.
The Office of the Actuary estimates the total number of W-2s expected to be received and posted for the current year (Appendix D - Estimate Analysis) in the Progress Report on Posting Wage Items.
The number of W-2s expected to be received and posted for the current year is transferred to an Excel spreadsheet.
Excel spreadsheet formulas are used to calculate the results; increase the percent of employee reports (W-2 forms) filed electronically. The formula used is: (EDT plus IW2PS plus W2 Online) divided by the Actuarial estimate for the current tax year.
The results are reviewed and the calculations are verified by the Director of the Employer Wage Reporting and Relations Staff.
The results of Increase the Percent of Employee Reports Filed Electronically are forwarded to the Office of Chief Strategic Officer for reporting in the PAR.

Appendix D
Statistical Methodology - Share of Employee Reports Filed Electronically Calculation

SSA estimates the total number of W-2s expected to be received and posted for the current year. To calculate this figure, SSA compares the number of W-2s filed electronically by September 30th of each year to the expected number of W-2s filed over the calendar year. The calculation of the number of W-2s filed annually proceeds in four steps: 1) calculating the number of workers with W-2s posted to the Master Earnings File (MEF) for a particular calendar year, 2) calculating the number of W-2s posted to the MEF per worker for a particular calendar year, 3) calculating the number of other wage items posted to the MEF for a particular year, and 4) calculating the total number of wage items processed in a particular calendar year. These steps are described briefly below.
1. Number of Workers with W-2s Posted to MEF for a Calendar Year
The first model estimates the number of workers with W-2s for wages in a particular calendar year that get posted to the MEF by the end of the posting period for the calendar year. For instance, only workers whose 2004 wages are posted to the MEF by the end of January 2006 (the end of the posting period for calendar year 2004) would be included in this count. The model relies on employment data from two different data sets, the Current Population Survey (CPS) and the Current Employment Survey (CES), to estimate the number of workers. The relationship between the number of W-2 workers and the total employed population is estimated using each dataset. The relationship using the CPS data is estimated using a regression equation with two independent variables: change in the unemployment rate and a dummy for 1995. The relationship using the CES data is assumed to be constant.

Two sets of results are produced, one based on each data set, and the final estimate is calculated by taking a weighted average of the two. Although the CPS model does a better job matching the variation in the actual data, the CES results receive a larger weight (2/3 compared to the 1/3 CPS weight). The CES is generally seen as a more accurate measure of the labor market over short periods of time, so the Office of the Actuary places more weight on the CES results.

2. Number of W-2s Per Worker Posted to MEF for a Calendar Year
The second model estimates the number of W-2s per worker posted to the MEF for a calendar year. Similar to the example above, this figure only includes those W-2s for wages in a particular calendar year posted to the MEF by the end of the posting cycle (for example, W-2s for wages earned in 2004 that are posted by January 2006). The average number of W-2s per worker is estimated based on a regression model with the unemployment rate, illegal immigration, and a series of dummy variables as the independent variables.

3. Total Number of Wage Items Posted to the MEF
Once the average number of W-2s has been calculated, multiplying by the number of workers (from the first model) yields the total number of W-2s processed before the end of the posting period. Corrected W-2s (W-2Cs) and generated W-2s (for tips and excess wages) are calculated based on fixed relationships to the number of W-2s (based on historical values).

4. Number of W-2s Processed on a Calendar Year Basis
To calculate the total number of wage items processed on a calendar year basis, the numbers posted to the MEF for the particular calendar year must be increased by the number of records sent to the Suspense File and the wage items received for other years. Both of these are calculated assuming a constant relationship to the total number of wage items from the prior year posted to the MEF. These two items amount to approximately 5 percent of wage items posted to the MEF.

We recalculated SSA's estimate of the share of employee reports filed electronically with immaterial differences. Also, we reviewed the method for calculating employee reports filed electronically and found it reasonable.

Appendix E
Agency Comments

SOCIAL SECURITY

MEMORANDUM

Date: February 26, 2007

To: Patrick P. O'Carroll, Jr.
Inspector General

From: Larry W. Dye

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: Electronic Service Delivery" (A-15-06-16111) -- INFORMATION

We appreciate OIG's efforts in conducting this review. We are pleased there were no findings for the two performance measures audited. Successes such as this are fostered by the cooperative relationship the Agency shares with OIG. We believe that lessons learned from previous audits have resulted in improvements to performance indicators and establishing challenging, yet realistic annual targets.

Please let me know if we can be of further assistance. Staff inquiries may be directed to Candace Skurnik, Director, Audit Management and Liaison Staff on extension 54636.

Overview of the Office of the Inspector General
The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Resource Management (ORM). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit
OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations
OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General
OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Resource Management
ORM supports OIG by providing information resource management and systems security. ORM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, ORM is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.