OFFICE OF
THE INSPECTOR GENERAL

SOCIAL SECURITY ADMINISTRATION

PERFORMANCE INDICATOR AUDIT:
CLAIMS PROCESSING

March 2007

A-15-06-16109

AUDIT REPORT

Mission

By conducting independent and objective audits, evaluations and investigations, we inspire public confidence in the integrity and security of SSA's programs and operations and protect them against fraud, waste and abuse. We provide timely, useful and reliable information and advice to Administration officials, Congress and the public.

Authority

The Inspector General Act created independent audit and investigative units, called the Office of Inspector General (OIG). The mission of the OIG, as spelled out in the Act, is to:

Conduct and supervise independent and objective audits and investigations relating to agency programs and operations.
Promote economy, effectiveness, and efficiency within the agency.
Prevent and detect fraud, waste, and abuse in agency programs and operations.
Review and make recommendations regarding existing and proposed legislation and regulations relating to agency programs and operations.
Keep the agency head and the Congress fully and currently informed of problems in agency programs and operations.

To ensure objectivity, the IG Act empowers the IG with:

Independence to determine what reviews to perform.
Access to all information necessary for the reviews.
Authority to publish findings and recommendations based on the reviews.

Vision

We strive for continual improvement in SSA's programs, operations and management by proactively seeking new ways to prevent and deter fraud, waste and abuse. We commit to integrity and excellence by supporting an environment that provides a valuable public service while encouraging employee development and retention and fostering diversity and innovation.

MEMORANDUM

Date: March 16, 2007

To: The Commissioner

From: Inspector General

Subject: Performance Indicator Audit: Claims Processing (A-15-06-16109)

We contracted with PricewaterhouseCoopers, LLP (PwC) to evaluate 15 of the Social Security Administration's performance indicators established to comply with the Government Performance and Results Act. Attached is the final report presenting the results of two of the performance indicators PwC reviewed. For the performance indicators included in this audit, PwC's objectives were to:
Assess the effectiveness of internal controls and test critical controls over data generation, calculation, and reporting processes for the specific performance indicator.
Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.
Test the accuracy of results presented and disclosed in the Fiscal Year 2006 Performance and Accountability Report.
Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

This report contains the results of the audit for the following indicators:
" Average processing time for initial disability claims.
" Number of initial disability claims processed by the Disability Determination Services.

If you wish to discuss the final report, please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector General for Audit, at (410) 965-9700.

Patrick P. O'Carroll, Jr.

MEMORANDUM

Date: March 6, 2007

To: Inspector General

From: PricewaterhouseCoopers LLP

Subject: Performance Indicator Audit: Claims Processing (A-15-06-16109)

OBJECTIVE

The Government Performance and Results Act (GPRA) of 1993 requires the Social Security Administration (SSA) to develop performance indicators that assess the relevant service levels and outcomes of each program activity. GPRA also calls for a description of the means employed to verify and validate the measured values used to report on program performance.

Our audit was conducted in accordance with generally accepted government auditing standards for performance audits. For the performance indicators included in this audit, our objectives were to:

1. Assess the effectiveness of internal controls and test critical controls over the data generation, calculation, and reporting processes for the specific performance indicator.

2. Assess the overall reliability of the performance indicator's computer processed data. Data are reliable when they are complete, accurate, consistent and are not subject to inappropriate alteration.

3. Test the accuracy of results presented and disclosed in the Fiscal Year (FY) 2006 Performance and Accountability Report (PAR).

4. Assess if the performance indicator provides a meaningful measurement of the program it measures and the achievement of its stated objective.

BACKGROUND

We audited the following performance indicators as stated in the SSA FY 2006 PAR:

Performance Indicator FY 2006 Goal FY 2006 Reported Results
Average Processing Time for Initial Disability Claims 93 days 88 days
Number of Initial Disability Claims Processed by the Disability Determination Services (DDS) 2,663,000 2,532,264

SSA administers the Old-Age and Survivors Insurance (OASI), Disability Insurance (DI) and Supplemental Security Income (SSI) programs. The OASI program, authorized by Title II of the Social Security Act, provides income for eligible workers and for eligible members of their families and survivors. The DI program, also authorized by Title II of the Social Security Act, provides income for eligible workers who have qualifying disabilities and for eligible members of their families before those workers reach retirement age. The SSI Program, authorized by Title XVI of the Social Security Act, was designed as a needs-based program to provide or supplement the income of aged, blind, and/or disabled individuals with limited income and resources.

To determine eligibility for both Title II and Title XVI programs, applicants must first file a claim with SSA. This is typically accomplished through an appointment in one of SSA's approximately 1,300 field offices (FO), through the SSA telephone network, or online via the Internet Social Security Benefit Application. Interviews with the applicants are conducted by FO personnel via the telephone or in person to determine the applicants' non-medical eligibility. If the applicant is filing for benefits based on disability, basic medical information concerning the disability, medical treatments, and identification of treating sources (e.g. a Doctor's office) is obtained.

FO personnel input the applicant's information into the Modernized Claims System (MCS) for OASI and DI claims or the Modernized SSI Claims System (MSSICS) for SSI claims. This establishes the application and/or protective filing date of the claim. A relatively minor number of OASI and DI claims are input through the SSA Claims Control System (SSACCS). SSACCS is used to process claims that cannot be fully processed through MCS. For example, when a Title II record is established, the MCS application allows for entry of up to 11 claimants on the specific record. Additional claimants to a single MCS record would need to be recorded on SSACCS. DI and SSI disability claims are sent to the State DDS office for review of medical information and determination of the receipt of benefits. The State DDS offices input case determinations into the National Disability Determination Services System (NDDSS).

RESULTS OF REVIEW

Our assessment of the two indicators included in this report did not identify any significant exceptions related to the accuracy of presentation or disclosure of the information related to these indicators in the FY 2006 PAR or to the meaningfulness of these indicators.

Our assessment of the two indicators included in this report identified an issue with internal controls that affected the data reliability. Specifically, for both indicators included in this report, we noted that SSA systems personnel had direct data access, therefore the data used to calculate the performance indicator could be inappropriately modified which could impact the results of the performance indicator.

For the indicator "Average Processing Time for Initial Disability Claims," weaknesses were found in the configuration of the UNIX operating system and Oracle database that contains information used to calculate the performance indicator results. The weaknesses noted do not impact the internal controls over this performance indicator, but rather are noted as instances of non-compliance with SSA's Risk model configuration standards.

For both indicators included in this report, we noted that an audit trail for transactions processed through the SSACCS application did not exist.

Average Processing Time for Initial Disability Claims

Indicator Background

When determinations are made for DI claims, SSA personnel update the corresponding MCS records and the Work Management System (WMS). Claims data for those claims that cannot be processed through MCS is maintained in SSACCS. After the award or denial has been processed, both WMS and SSACCS transfer this claims data to the Title II Operational Datastore (TII ODS). The data is then sent to the Social Security Unified Measurement System (SUMS) Data Warehouse, and stored in the Title II Processing Time (TIIPT) module.

When determinations are made for SSI claims, SSA personnel update the Supplemental Security Record (SSR), and claims data is forwarded to the SSI Exception Control System. This system ensures the claims data, either the award or denial, is complete before the data is sent to the Title XVI Operational Datastore (TXVI ODS). The data is then sent to the SUMS Data Warehouse, and stored in the SSI Processing Time (SSIPT) module.

Queries are used to obtain the processing time data for both Title II and Title XVI disability claims, on a monthly and fiscal year-to-date basis. The results of these queries are combined to determine the monthly and fiscal year-to-date average processing time for all disability claims (Title II and Title XVI). These figures are then posted to the SSA Intranet. Refer to the following formula.

Performance Indicator Calculation

Average Processing Time for Initial Disability Claims

Total Processing Time for DI and SSI Disability Claims
____________________________

Total Claims Processed for DI and SSI Disability

Further, processing time is measured from the application date or protective filing date to either the date of the denial notice or the date the system completes processing and awards the payments.

Findings

Internal Controls and Data Reliability

We found 56 systems personnel had the "All" access designation within the Top Secret security software to the NDDSS datasets used to calculate the indicator results. This level of access allows users to create, delete and modify any of the data (or datasets) contained within the datasets we reviewed. Therefore, the data used to calculate the performance indicator could be inappropriately modified and could impact the results of this performance indicator. This level of access prevents SSA from ensuring the integrity of this production data. By allowing systems personnel to have the "All" access designation, SSA is not conforming to the revised Office of Management and Budget (OMB) Circular A-130, Management of Federal Information Resources, Appendix III, Security of Federal Automated Information Resources, principles of "least privileged access" or segregation of duties. It should be noted that this access was removed during the course of the audit. While we were able to recalculate the interim and year-end indicator results, as a result of this issue, we could not consider the data to be reliable.

We also found an audit trail for transactions processed through the SSACCS was not created or reviewed. This could prevent management from reviewing and identifying inappropriate or unauthorized transactions being processed through SSACCS.

Finally, our review of the SUMS Data Warehouse UNIX system and Oracle database identified 10 security and compliance issues. This review was conducted against the SSA developed UNIX Risk Model configuration standard, National Institute of Standards and Technology (NIST) guidelines and the Defense Information Security Agency (DISA) Security Technical Implementation Guides (STIGS). We identified nine exceptions to the requirements of the SSA UNIX Risk Model. During our review of the Oracle database, we were informed that SSA management has not formally approved the configuration standard (risk model) for the Oracle database environment.

We did not identify any significant exceptions related to the disclosure of the information related to this indicator contained in the PAR, or to the meaningfulness of this indicator.

Number of Initial Disability Claims Processed by the Disability Determination Services (DDS)

Indicator Background

The performance indicator measures the number of DI and SSI disability initial claims that have been reviewed by DDS personnel. DDS personnel are responsible for determining claimants' disability and ensuring that adequate evidence is available to support the determination. Upon determining an applicant's non-medical eligibility status, SSA sends the DI and SSI initial claims file to the DDS. When a disability determination is made by DDS personnel, the decision is entered into the NDDSS and the case is noted as closed. The data within NDDSS is automatically transferred to the Disability Operational Datastore (DIODS). The total number of processed (or cleared) initial disability claims are reported as of September 29, 2006 on the State Agency Operations Report (SAOR), generated from DIODS. Refer to the formula below.

Performance Indicator Calculation

Total Claims Processed for Title II and Title XVI

Total Workloads of Initial Claims Clearances as of September 29, 2006

Findings

Internal Controls and Data Reliability

The DIODS data used to classify the disability claims as clearances was not archived and maintained in accordance with revised OMB Circular A-123, Management's Responsibility for Internal Control, Appendix A: Internal Control Over Financial Reporting. OMB Circular A-123 requires agencies to ensure that documentation of significant events is readily available for examination. SSA management stated that the detailed data was not maintained due to limited data storage space and lack of personnel resources. Therefore, we performed alternative testing procedures to assess the reliability of the indicator data presented in the PAR.

SSA was able to provide a copy of the code used to generate the indicator results for our review. We concluded that the code was designed to calculate the indicator results as described by SSA management. In addition, we selected numerous cases from DIODS, and compared the case information to the corresponding records in the SSR and MBR. This testing was performed to ensure the accuracy of the data when it was transferred from the MBR or SSR to DIODS. Also, this testing did not result in any exceptions. Lastly, we were able to observe the final calculation of this indicator on a real-time basis. We compared the final reported results of this indicator as reported in the PAR with the final data recorded on the SAOR report (which includes final indicator results). We identified no exceptions with this testing.

As a result of these tests, we are reasonably comfortable that the data reported in the PAR for this indicator are complete, accurate, and consistent. However, the data cannot be considered reliable as the potential for inappropriate alteration existed during the timeframe that systems personnel had update access to NDDSS datasets.

We noted that an audit trail for transactions processed through the SSACCS application did not exist and SSA systems personnel had direct data access to NDDSS that would allow them to update production performance indicator data. The details of these findings are discussed in the findings section of the indicator "Average Processing Time of Initial Disability Claims."

We did not identify any significant exceptions related to the meaningfulness of this indicator or disclosure of the information related to this indicator contained in the PAR.

RECOMMENDATIONS

We recommend SSA:

1. Ensure personnel do not have the ability to directly modify, create or delete the datasets used to calculate the results of these indicators.

2. Maintain an audit trail for SSACCS that captures the user identification, terminal, date and time the transaction was processed. Policies and procedures should be implemented requiring a review of the audit trail for inappropriate access or processing of transactions. In lieu of making these changes to SSACCS, SSA should ensure that the SSACCS replacement system is configured with the appropriate audit trail controls.

Specific to the performance indicator, "Average Processing Time for Initial Disability Claims" we recommend SSA:

3. Ensure that the SUMS Data Warehouse UNIX system is configured to be in compliance with the SSA UNIX Risk Model and Government guidelines from NIST and DISA. SSA should also formalize the configuration standard for the Oracle database environment and ensure that this standard complies with the SSA Security Handbook, Government guidelines, and is officially approved by SSA management.

Specific to the performance indicator, "Number of Initial Disability Claims Processed by the DDS" we recommend SSA:

4. Maintain the detailed data used to calculate the performance indicator results that are reported in the PAR.

AGENCY COMMENTS

The Agency agreed with two of our recommendations (numbers 1 and 3) and disagreed with two recommendations (numbers 2 and 4). With regard to recommendation 2, SSA stated that since SSACCS will be phased out, it is cost-prohibitive to maintain an audit trail for this system's transactions. Also, based on the Agency's comments, we are withdrawing recommendation number 4. See Appendix D for the full text of the Agency's comments.

PWC RESPONSE

We appreciate the Agency's comments and consideration of our recommendations. In regard to recommendation number 2, PwC was not provided any documentation detailing the timeframe for the "phase out" of SSACCS. As such, PwC continues to recommend that SSA maintain an audit trail for SSACCS since this data is used for calculation of the indicator results. However, in lieu of making these changes to SSACCS, SSA should ensure that the SSACCS replacement system is configured with the appropriate audit trail controls.

Appendices

APPENDIX A - Acronyms
APPENDIX B - Scope and Methodology
APPENDIX C - Process Flowcharts
APPENDIX D - Agency Comments

Appendix A
Acronyms

DDS Disability Determination Service
DI Disability Insurance
DIODS Disability Operational Datastore
DISA Defense Information Security Agency
FO Field Office
FY Fiscal Year
GPRA Government Performance and Results Act
ISBA Internet Social Security Benefit Application
MBR Master Beneficiary Record
MCS Modernized Claims System
MSSICS Modernized Supplemental Security Income System
NDDSS National Disability Determination Services System
NIST National Institute of Standards and Technology
OASI Old-Age and Survivors Insurance
PAR Performance and Accountability Report
SAOR State Agency Operations Report
SSA Social Security Administration
SSACCS Social Security Administration Claims Control System
SSI Supplemental Security Income
SSIPT Supplemental Security Income Processing Time
SSR Supplemental Security Record
STIGS Security Technical Implementation Guides
SUMS Social Security Unified Measurement System
TII ODS Title II Operational Datastore
TIIPT Title II Processing Time
TSC Teleservice Center
TXVI ODS Title XVI Operational Datastore
U.S.C. United States Code
WMS Work Measurement System

Appendix B
Scope and Methodology

We updated our understanding of the Social Security Administration's (SSA) Government Performance and Results Act (GPRA) processes. This was completed through research and inquiry of SSA management. We also requested SSA to provide various documents regarding the specific programs being measured as well as the specific measurement used to assess the effectiveness and efficiency of the related program.

Through inquiry, observation, and other substantive testing, including testing of source documentation, we performed the following:

Reviewed prior SSA, Government Accountability Office, Office of the Inspector General and other reports related to SSA's GPRA performance and related information systems.
Reviewed applicable laws, regulations and SSA policy.
Met with the appropriate SSA personnel to confirm our understanding of the performance indicator.
Flowcharted the process. (See Appendix C).
Tested key controls related to manual or basic computerized processes (e.g., spreadsheets, databases, etc.).
Conducted and evaluated tests of the manual controls within and surrounding each of the critical applications to determine whether the tested controls were adequate to provide and maintain reliable data to be used when measuring the specific indicator.
Identified attributes, rules, and assumptions for each defined data element or source document.
Recalculated the metrics of key performance indicators to ensure mathematical accuracy.
For those indicators with results that SSA determined using computerized data, we assessed the completeness and accuracy of that data to determine the data's reliability as it pertains to the objectives of the audit.

As part of this audit, we documented our understanding, as conveyed to us by Agency personnel, of the alignment of the Agency's mission, goals, objectives, processes, and related performance indicators. We analyzed how these processes interacted with related processes within SSA and the existing measurement systems. Our understanding of the Agency's mission, goals, objectives, and processes were used to determine if the performance indicators appear to be valid and appropriate given our understanding of SSA's mission, goals, objectives and processes.

AVERAGE PROCESSING TIME FOR INITIAL DISABILITY CLAIMS

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following areas:
Completed application control reviews over the National Disability Determination Services System (NDDSS), Title II Operational Datastore, Title XVI Operational Datastore, and the SSA Unified Measurement System (SUMS) Data Warehouse. An application control review includes testing access controls, data input, data output, data rejection, and data processing, as applicable.
Completed reviews for the SUMS Data Warehouse UNIX system and Oracle database.
Determined the adequacy of the programming logic used by SSA to calculate the average processing time for initial disability claims.
Recalculated the indicator for Fiscal Year 2006 and compared it to the number reported in the Performance and Accountability Report (PAR).

NUMBER OF INITIAL DISABILITY CLAIMS PROCESSED BY THE DISABILITY DETERMINATION SERVICES (DDS)

Audited the design and effectiveness of the SSA internal controls and the accuracy and completeness of the data related to the following areas:
Completed application control reviews over the NDDSS and the Disability Operational Datastore (DIODS). An application control review includes testing access controls, data input, data output, data rejection, and data processing, as applicable.
Performed a comparison of data from the summary data from the State Agency Operations Report and the Master Beneficiary Record (MBR) and Supplemental Security Record (SSR) to ensure accuracy and completion of the transfer of files from the MBR and SSR, through NDDSS and into DIODS.
Determined the adequacy of the programming logic used by SSA to calculate the initial disability claims processed.
Traced data from supporting reports to the indicator calculation total included on the PAR.

Appendix C
Flowchart of Average Processing Time for Initial Disability Claims

Flowchart of Average Processing Time for Initial Disability Claims Cont.

Average Processing Time for Initial Disability Claims
Claimant contacts SSA via Field Office (FO) visit, mail, phone call to FO or Tele-Service Center (TSC), or online via the Internet Social Security Benefit Application (ISBA).
Can the FO personnel interview the claimant today?
o No - Set up a teleclaim or in-office appointment.
o Yes - FO interviews claimant via teleclaim or in-office appointment, first verifying non-medical issues.
Is claimant potentially eligible for Title II and/or Title XVI?
o No - Does claimant insist on filing?
o No - STOP
o Yes - Establish Disability Insurance (DI) application using Modernized Claims System (MCS) or Modernized Supplemental Security Income Claims System (MSSICS) or SSA Claims Control System (SSACCS). This is the application date or start date.
Review non-medical issues.
Determine effective filing date. This may also be the start date if it is earlier than the application date.
If possible, make and enter non-medical decision into MCS, MSSICS or SSACCS.
Is this a non-medical denial?
o No - Create medical folder with Form SSA-831.
o Yes - A. Adjudicate the non-medical portion of the claim via MCS or MSSICS.
Send folder to the Disability Determination Service (DDS).
DDS inputs receipt of case in National Disability Determination Services System (NDDSS).
NDDSS receives claimant information from MCS, MSSICS or SSACCS.
DDS gathers and reviews medical evidence to make a medical determination.
If medical information is not sufficient, a consultative examination is scheduled.
DDS makes a decision and enters the medical information in NDDSS.
DDS inputs medical decision as reported on Form SSA-831.
Claim is approved or denied. Medical portion of the decision is adjudicated. This is the end date.
Case is closed in NDDSS, which interfaces with MCS, MSSICS, and SSACCS.
Folder is sent back to FO.
Did FO input the non-medical determination prior to sending folder to DDS?
o No - A - Adjudicate non-medical portion of claim via MCS or MSSICS.
o Yes - Folder is filed.
B (Title II)
C (Title XVI)
B (Title II) - MCS updates Workload Management System (WMS). SSA-1418 (screen) updates SSACCS with claim information.
Data is fed to the Title II Operational Datastore (TII ODS).
Data flows to the Social Security Unified Measurement System (SUMS).
Title II Processing Time (TIIPT) is stored in the SUMS Data Warehouse.
A query is used to access ad hoc reporting via the TIIPT template and rules.
TIIPT Monthly and FYTD Reports are generated.
C (Title XVI) - Supplemental Security Record (SSR) is updated with an initial determination date and claim data is routed to SSI Exception Control System.
Data is fed to the Title XVI Operational Datastore (TXVI ODS).
Data flows to the Social Security Unified Measurement System (SUMS).
SSI Processing Time (SSIPT) is stored in the SUMS Data Warehouse.
Brio is used to access ad hoc reporting via the SSIPT template and rules.
SSIPT Monthly and FYTD Reports are generated.
TIIPT and SSIPT are combined to calculate the initial disability claims processing time (days).
SSA Intranet site updated monthly with Average processing time for initial disability claims to be reported in the Performance and Accountability Report (PAR).

Flowchart of Number of Initial Disability Claims Processed by the Disability Determination Services

Number of Initial Disability Claims Processed by the Disability Determination Services (DDS)
Claimant contacts SSA via Field Office (FO) visit, mail, phone call to FO or Tele-Service Center (TSC), or online via the Internet Social Security Benefit Application (ISBA).
Can the FO personnel interview the claimant today?
o No - Set up a teleclaim or in-office appointment.
o Yes - FO interviews claimant via teleclaim or in-office appointment, first verifying non-medical issues.
Is claimant potentially eligible for Title II and/or Title XVI?
o No - Does claimant insist on filing?
o No - STOP
o Yes - Establish Disability Insurance (DI) application using Modernized Claims System (MCS) or Modernized Supplemental Security Income Claims System (MSSICS) or SSA Claims Control System (SSACCS).
Review non-medical issues.
Determine effective filing date.
If possible, make and enter non-medical decision into MCS, MSSICS or SSACCS.
Is this a non-medical denial?
o Yes - Claim is denied.
o No - Create medical folder with Form SSA-831.
Send folder to the Disability Determination Service (DDS).
DDS inputs receipt of case in National Disability Determination Services System (NDDSS).
NDDSS receives claimant information from MCS, MSSICS or SSACCS.
DDS gathers and reviews medical evidence to make a medical determination.
If medical information is not sufficient, a Consultative Examination is scheduled.
DDS inputs medical decision as reported on Form SSA-831.
Claim is approved or denied. Medical portion of the decision is adjudicated.
Case is closed in NDDSS.
NDDSS interfaces with Disability Operational Datastore (DIODS) to provide processed claims data.
DIODS counts the number of processed initial disability claims by DDS on a weekly basis for reporting on the State Agency Operations Report (SAOR).
Staff reviews weekly SAOR report to identify anomalies and corrects errors.
Year-end SAOR report number is recorded in the PAR for the performance indicator Number of initial disability claims processed by the DDS.

Appendix D
Agency Comments

SOCIAL SECURITY

MEMORANDUM

Date: February 16, 2007

To: Patrick P. O'Carroll, Jr.
Inspector General

From: Larry W. Dye

Subject: Office of the Inspector General (OIG) Draft Report, "Performance Indicator Audit: Claims Processing" (A-15-06-16109)--INFORMATION

We appreciate OIG's efforts in conducting this review. Our comments on the draft report content and recommendations are attached.

Please let me know if we can be of further assistance. Staff inquiries may be directed to Ms. Candace Skurnik, Director, Audit Management and Liaison Staff, at extension 54636.

SSA Response

COMMENTS ON THE OFFICE OF THE INSPECTOR GENERAL DRAFT REPORT "PERFORMANCE INDICATOR AUDIT: CLAIMS PROCESSING" (A-15-06-16109)

Thank you for the opportunity to review and comment on the draft report. We appreciate OIG's efforts to assist us in evaluating our performance data and we are committed to ensuring that our performance data are accurate and reliable. We are disappointed with the way this report characterizes the reliability and accuracy of these two indicators given the fact that in a "real time" environment Pricewaterhouse Coopers (PwC) was able to witness the calculations and validate their accuracy.

As a mutually agreed upon compromise and solution to not maintaining data, the Social Security Administration (SSA) and PwC agreed to the "real-time" audit process for some fiscal year 2006 reviews. We successfully worked with PwC to develop a process by which they recalculated and validated the data used to report on these measures and determined it was accurate. However, on page 3, while summarizing the results of the review, the report focuses on issues related to internal controls, data reliability, audit trail and personnel access to data. It is silent on the fact that the auditors were able to calculate or recalculate the indicator results and found no "exceptions" related to the disclosure of information or meaningfulness of the indicator. While the findings are reported later under each specific indicator measured, omitting that information from the "summary" gives the reader the impression that there is something wrong with our data, which is clearly not the case since PwC was able to recalculate the data.

Specifically, the Office of Management and Budget's (OMB) Circular A-11, section 230.2e states "Performance data need not be perfect to be reliable, particularly if the cost and effort to secure the best performance data will exceed the value of any data so obtained." The report states that "the DIODS data used to classify the disability claims as clearances was not archived and maintained in accordance with revised OMB Circular A-123, Management's Responsibility for Internal Control, Appendix A: Internal Control Over Financial Reporting." We disagree with this statement. OMB Circular A-123 states that, "Effective internal control over financial reporting provides reasonable assurance that misstatements, losses, or noncompliance with applicable laws and regulations, material in relation to financial reports, would be prevented or detected." In numerous other references throughout Circular A-123, the objective of internal control over financial reporting is to provide "reasonable assurance." The mutually agreed upon methodology for the real time audit resulted in PwC being able to "gain comfort around the accuracy of the reported results." The comfort PwC was able to gain should provide reasonable assurance. As indicated in our response below, the specific recommendations in the OMB Circular A-11 directive apply to both of these indicators. The Circular A-123 guidance for reasonable assurance and the Circular A-11 guidance for cost-effective performance data are complimentary and when considered together with PwC's comfort around the accuracy of the reported results, should eliminate the inclusion of any reference to detailed data not being maintained.

Our responses to the specific recommendations are provided below.

Recommendation 1

SSA should ensure personnel do not have the ability to directly modify, create or delete the datasets used to calculate the results of these indicators.

Response

We agree. We have already taken steps to address the issue. The draft report accurately states that this access was removed during the course of the audit.

Recommendation 2

SSA should maintain an audit trail for the Social Security Administration Claims Control System (SSACCS) that captures the user ID, terminal, date and time the transaction was processed. Policies and procedures should be implemented requiring a review of the audit trail for inappropriate access or processing of transactions.

Response

We disagree. SSACCS will be phased out; therefore it is cost-prohibitive to maintain an audit trail for this system's transactions. The OMB's Circular A-11, section 230.2e states, "Performance data need not be perfect to be reliable, particularly if the cost and effort to secure the best performance data will exceed the value of any data so obtained." We believe this directive applies in this situation.

Recommendation 3

Specific to the performance indicator, "Average Processing Time for Initial Disability Claims," that SSA should ensure that the Social Security Unified Measurement System (SUMS) Data Warehouse UNIX system is configured to be in compliance with the SSA UNIX Risk Model and Government guidelines from National Institute of Standards and Technology (NIST) and Defense Information Security Agency (DISA). SSA should also formalize the configuration standard for the Oracle database environment and ensure that this standard complies with the SSA Security Handbook, Government guidelines, and is officially approved by SSA management.

Response

We agree. We have taken steps to correct the SUMS UNIX issues. Concerning the Oracle database environment, a new Oracle Risk Model has been developed using DISA Security Technical Implementation Guides. A monitoring process is being developed.

Recommendation 4

Specific to the performance indicator, "Number of Initial Disability Claims Processed by the Disability Determination Service," that SSA maintain the detailed data used to calculate the performance indicator results that are reported in the Accountability Report.

Response

We disagree. As noted in the report, there are capacity issues that prevent SSA from archiving all detail data in the Disability Operational Datastore. It is also cost-prohibitive to maintain the detail-level data required to recalculate performance results for a full year for this measure. As noted above, SSA and PwC agreed to the "real-time" audit process as a solution to not maintaining this data. In addition, SSA has complied with guidance in OMB Circulars A-11 and 123 that address management's responsibility for internal control. In summary, since this audit was conducted in the "real-time" format, this should not be a finding as we successfully worked with PwC to develop a process by which they could and did, conclude that the summary data were accurate.

Overview of the Office of the Inspector General
The Office of the Inspector General (OIG) is comprised of our Office of Investigations (OI), Office of Audit (OA), Office of the Chief Counsel to the Inspector General (OCCIG), and Office of Resource Management (ORM). To ensure compliance with policies and procedures, internal controls, and professional standards, we also have a comprehensive Professional Responsibility and Quality Assurance program.

Office of Audit
OA conducts and/or supervises financial and performance audits of the Social Security Administration's (SSA) programs and operations and makes recommendations to ensure program objectives are achieved effectively and efficiently. Financial audits assess whether SSA's financial statements fairly present SSA's financial position, results of operations, and cash flow. Performance audits review the economy, efficiency, and effectiveness of SSA's programs and operations. OA also conducts short-term management and program evaluations and projects on issues of concern to SSA, Congress, and the general public.

Office of Investigations
OI conducts and coordinates investigative activity related to fraud, waste, abuse, and mismanagement in SSA programs and operations. This includes wrongdoing by applicants, beneficiaries, contractors, third parties, or SSA employees performing their official duties. This office serves as OIG liaison to the Department of Justice on all matters relating to the investigations of SSA programs and personnel. OI also conducts joint investigations with other Federal, State, and local law enforcement agencies.

Office of the Chief Counsel to the Inspector General
OCCIG provides independent legal advice and counsel to the IG on various matters, including statutes, regulations, legislation, and policy directives. OCCIG also advises the IG on investigative procedures and techniques, as well as on legal implications and conclusions to be drawn from audit and investigative material. Finally, OCCIG administers the Civil Monetary Penalty program.

Office of Resource Management
ORM supports OIG by providing information resource management and systems security. ORM also coordinates OIG's budget, procurement, telecommunications, facilities, and human resources. In addition, ORM is the focal point for OIG's strategic planning function and the development and implementation of performance measures required by the Government Performance and Results Act of 1993.