Office
of the Inspector General
Jo Anne B.
Barnhart
Commissioner
Inspector General
Performance Measure Review: Reliability of the Data Used to Measure Anti-Fraud Performance (A-02-01-11013)
We recently completed our first 3-year cycle of reviewing the Social Security Administration's performance measures. In conducting this work, we used the services of an outside contractor, PricewaterhouseCoopers, LLP, (PwC) to assist us in our efforts.
For this report, we used PwC to conduct the review of four of the Agency’s performance indicators related to anti-fraud activities. The objective of the review was to assess the reliability of the data used to measure the Agency’s anti-fraud efforts. These anti-fraud indicators measure the results of the work conducted by the Office of Investigations within the OIG. To ensure an independent review of these indicators, we relied on our contractor to assess the reliability of the data behind these measures.
The attached
final report presents the results of the contractor’s work and its recommendations
for improvement. We are generally in agreement with all of the contractor’s
recommendations focusing on OIG operations. My office will track the individual
recommendations and we will periodically provide our status in implementing
these recommendations. We plan to work closely with the Agency on those recommendations
where coordination would be appropriate. If you wish to discuss the final report,
please call me or have your staff contact Steven L. Schaeffer, Assistant Inspector
General for Audit, at (410) 965-9700.
James
G. Huse, Jr.
OFFICE OF
THE INSPECTOR GENERAL
SOCIAL SECURITY ADMINISTRATION
PERFORMANCE MEASURE REVIEW:
RELIABILITY OF THE DATA USED TO
MEASURE ANTI-FRAUD PERFORMANCE
January
2002
A-02-01-11013
EVALUATION REPORT
Evaluation of Selected Performance Measures of the Social Security Administration:
Reliability of the Data Used to Measure
Anti-Fraud Performance
Office
of the Inspector General
Social Security Administration
INTRODUCTION
This report is one of five separate stand-alone reports, corresponding to the following Social Security Administration (SSA) process and performance measures (PM):
Fiscal Year (FY) 2000 Goal: 7,600
FY 2000 Goal: $40 million
FY 2000 Goal: $80 million
FY 2000 Goal: 1,800
This report reflects our understanding and evaluation of the process related to PMs #5 through #8. To achieve its strategic goal "To make SSA program management the best-in-business, with zero tolerance for fraud and abuse," SSA has developed several strategic objectives. One of these objectives is "To aggressively deter, identify, and resolve fraud." SSA’s FY 2001 Annual Performance Plan (APP) contains four performance indicators developed to meet this objective as follows:
We performed our testing from September 21, 2000 through February 15, 2001. Our engagement was limited to testing at SSA’s headquarters in Woodlawn, Maryland and the OIG office in Philadelphia, Pennsylvania. The procedures that we performed were in accordance with the American Institute of Certified Public Accountants’ Statement on Standards for Consulting Services, and are consistent with appropriate standards for performance audit engagements in Government Auditing Standards (Yellow Book, 1994 version). However, we were not engaged to and did not conduct an audit, the objective of which would be the expression of an opinion on the reliability or accuracy of the reported results of the performance measures evaluated. Accordingly, we do not express such an opinion.
BACKGROUND
SSA has been engaged in an aggressive program to deter, detect, investigate and prosecute fraud. To carry out this effort, SSA and the OIG have cooperated in developing a comprehensive anti-fraud plan. The SSA OIG, Office of Investigations (OI) is responsible for investigating allegations of fraud, waste and abuse. The four indicators evaluated focus on the OIG’s output efforts to achieve improvements in deterring, identifying and resolving fraud.
The first performance measure, "Number of investigations conducted," represents the number of investigations "conducted" by OI resulting from allegations that have sufficient information or potential risk to warrant further review or action by a criminal investigator. Allegations may be received from Congress, SSA employees, or from the public by mail, e-mail, fax, Internet, or telephone (i.e., 800 Hotline). Investigations are counted as "conducted" when all OI actions have been completed or the investigator has determined that further action is not warranted due to lack of investigative leads. The objective is to raise the number of investigations conducted (i.e., closed), which relates to the strategic goal to aggressively deter, identify and resolve fraud. This performance measure is presented as a workload count, and includes every closed case, including cases for which it was determined that no fraud was involved.
The second and third performance measures, "OASDI dollar amounts reported from investigate activities" and "SSI dollar amounts reported from investigative activities," represent amounts reported by the OI from fines/penalties, assessments, savings, recoveries and restitution/judgments related to investigative activities. The components of investigative activities, or "monetary achievement" are defined as follows:
The objective is to report the dollar values of the OI efforts/activities to identify and resolve fraud, which is directly related to the strategic goal "To aggressively deter, identify and resolve fraud."
The last performance measure, "Number of Criminal Convictions," represents the number of criminal convictions as a result of OI activities. This performance measure is presented as a workload count of all cases concluding in a criminal conviction. In addition to purposes served by formally charging a person with the commission of a crime, the criminal prosecution process has an impact, which may deter others from committing violations, and therefore is directly related to the strategic goal "To aggressively deter, identify and resolve fraud."
Allegation and Case Investigative System (ACIS)
Information concerning potential wrongdoing involving SSA programs, employees, or operations which are received by an SSA OIG component are accounted for in ACIS. The system encompasses the initial receipt of an allegation, all steps taken throughout the investigative process, and the final outcome of all investigations. The system resides in a database (ADABAS) application, at the Center for Information Technology (CIT) of the National Institutes of Health (NIH).
The ACIS process begins with the receipt of an allegation. The Allegation Management Division (AMD) staff records these allegations in the allegation management module of ACIS. They are then electronically forwarded to either an OI field division (FD), a SSA field component, another agency, or are closed. Allegations may be received from Congress or from the public via mail, e-mail, fax, Internet, or telephone (i.e., 800 Hotline). FDs may also receive allegations directly, and are also required to enter the allegation information into ACIS. If an investigation is warranted, all information and supporting documentation will be forwarded to the appropriate individuals within the FD.
When an investigation is opened, ACIS generates an OI-1, ACIS Case Opening Report. To complete and document an investigation, the Special Agent (SA) is required to complete a series of forms. Once the case is taken to court, if the court orders a restitution or judgment, the agent completes the OI-68, Report of Court Ordered Restitution/Judgment. The OI-67, Monetary Achievement Worksheet, may also accompany an OI-68. The OI-67 is the form used to document any civil monies recovered during the investigation, as well as to assist in calculating the monetary achievement documented on the OI-68. Once all aspects of the investigation are completed the SA completes an OI-9, ACIS Criminal and Administrative Disposition Form. This form includes the judicial and criminal disposition data, as well as criminal and administrative monetary achievements, and is used by the administrative personnel, SAs, Assistant Special Agents-in-Charge (ASAC) or Special Agents-in-Charge (SAC) to input the monetary achievements into ACIS.
The OI-68 is mailed to OI Manpower and Administration Division (MAD) in Woodlawn, Maryland. It is then reviewed against the data, which is entered into ACIS for accuracy and distributed to SSA’s Debt Management Section for notification of the court-ordered restitution. In addition, the OI-68 and the payments are sent to the Mid-Atlantic Program Service Center (MATPSC) to be processed. The MATPSC cross-references the checks to the OI-68 and posts them to the correct Social Security numbers (SSN).
RESULTS OF EVALUATION
During the period of September 21, 2000 to February 15, 2001, we evaluated the current processes, systems and controls, which support the FY 2000 SSA performance measurement process. In addition, we determined the accuracy of the underlying performance measure data. Our evaluation of the information provided by SSA management as of February 15, 2001, allowed us to determine that the reported FY 2000 results of the four performance measures tested (as itemized below) were reasonably stated based on the methodology used by SSA.
Performance Measure |
Reported Result
|
|
8,051 |
|
$46 million |
|
$128 million |
|
2,604 |
However, we noted certain deficiencies in SSA’s methodology used to analyze the performance measures and certain limitations with ACIS. Although we do not consider any of the deficiencies noted during our evaluation to be significant, consideration should be given to the following recommendations, as we believe there are opportunities for improvement, thereby increasing the value of the performance measures as management tools.
These items were noted as a result of our testing the underlying performance measure data, as well as the Electronic Data Processing (EDP) and manual controls of the systems generating the performance measure data, and are discussed in detail below.
1. ACIS has some data integrity deficiencies.
To ensure that the amounts reported in the SSA Government Performance and Results Act of 1993 (GPRA) section of the Performance and Accountability Report were reasonably accurate and reliable, we obtained a random sample of 45 OASDI, 45 SSI and 20 criminal conviction cases from the ACIS system and requested supporting documentation for each of the cases. In addition, we judgmentally selected 10 cases from the closed-case cabinet at the Philadelphia sub-office, traced them to ACIS and ensured the cases were properly documented.
The results of our evaluation of the cases are as follows:
We noted that 5 of the 45 OASDI cases evaluated had double-counting errors totaling $235,911 for OASDI amounts reported from investigative activities. This resulted from dollar amounts in both the scheduled recovery and restitution fields in ACIS. Per the Special Agent Handbook (SAH), these two fields are mutually exclusive, and therefore a case should not have the same dollars entered in both fields. These two fields are then combined with savings, judgments, settlements, fines, assessments, and penalties to arrive at the OASDI monies from investigative activities.
In our evaluation of the 120 ACIS cases selected, we found 2 cases in which the procedures used to calculate savings were not consistent with those procedures outlined in the SAH. The first was a fugitive felon case, for which the amount of savings was calculated over 3 years. Per the SAH, Chapter 3, Section 3-75-C8 "Program savings for fugitive felon cases will be calculated on a 2-year projection." In the second case OI found an individual embezzling benefit checks from an entitled beneficiary. A savings amount was calculated for this case, even though in a similar case, savings were not calculated.
Our evaluation of the cases also noted that 36 out of the 120 ACIS cases selected contained data anomalies. The data anomalies represent instances where ACIS data did not match the data reported in the supporting documentation (OI-68, OI-9, OI-4, or OI-1). For example, in several cases the fraud loss entered into ACIS was higher than the fraud loss reported in the supporting documentation. In some cases, the recovery or assessment included in the supporting documentation was not reported in ACIS. These errors could have been caused by either 1) a data entry error made by administrative personnel, SACs, ASACs or SAs, or 2) by documentation errors within the case file, made by SAs.
OI has in place a supervisory review of case files at the field division level. However, the supervisory review, Form OI-20, includes a review of the status of the case, not a review of the forms within the case file to ensure that the forms are completed appropriately. The SAH, Chapter 3, Section 3-60 B, states the following: "While the review of documents and evidence in the case file is important, the case review process should not be viewed as merely a review of documents. Rather it is an evaluation of the investigative progress and potential of the SA’s cases." In addition, the SAH does not include procedures to ensure that the data entered in ACIS by the OI field divisions is correct.
Although the OI MAD has a Quality Review process, this process is limited to a comparison of Form OI-68 "Report of Court-Ordered Restitution/Judgment" to the restitution report produced at MAD, and the restitution amounts reported in ACIS. In addition, since not all cases require an OI-68, not all cases are reviewed. The lack of supervisory review of the forms and limited review by OI Headquarters has led to errors going undetected, thus diminishing the accuracy of the number reported by OI as part of the GPRA section of SSA’s Performance and Accountability Report.
ACIS is an antiquated system that lacks sufficient capabilities to continue to meet OIG’s needs. The OIG stated that the system has been "frozen" and that it is in the process of evaluating commercial off the shelf software packages as an alternative system. These data integrity opportunities for improvement should be considered when selecting and implementing the new system.
A program saving is an integral part of the dollars reported from the investigative
activities calculation. For disability cases, the cooperative disability investigation
(CDI) teams claim a set amount of savings of $66,500 per case, for applicants
who are denied eligibility due to the findings of the investigation. SSA arrived
at this figure in 1997 by applying the amount of an average lifetime benefit
figure to an average Disability Determination Services claim that is denied.
This amount has not been adjusted since 1997 to reflect the yearly increase
of SSI benefit payments. OI, in conjunction with SSA’s Office of the Actuary,
is currently developing a method to calculate the savings figure for each case
based on a claimant’s age, gender, life expectancy and amount of benefit received.
The new method will be applied to all CDI cases beginning in FY 2002.
There is a lack of documentation for some ACIS processes in SSA OIG operations,
as follows:
Without formally documented processes, management cannot be assured that SSA OIG personnel understand all of the requirements for successful change control, user access recertifications, and systems design and development processes.
When processes are not documented, such as formal processes for change control, user access recertification, and ACIS system design and development, there is no accountability that the procedures were followed. When there is no accountability for the procedures associated with these processes, it becomes difficult to resolve any problems or issues. This lack of assurance may negatively impact SSA OI operations.
Both of the OIG personnel that are responsible for programming ACIS changes
have the ability to promote program changes to production. This increases the
risk that inappropriate or incorrect changes could be placed into production,
thereby compromising the functionality of ACIS.
Office of Management and Budget (OMB) Circular A-130, Appendix III, Security of Federal Automated Information Resources, states that agencies are required to establish controls to assure adequate security for all information processed, transmitted, or stored in Federal automated information systems. This appendix stresses the importance of management controls affecting individual users of information technology. The appendix states: "Technical and operational controls support management controls. To be effective, all must interrelate."
In addition, Federal Information Processing Standards (FIPS) Publication 73 entitled Guidelines for Security of Computer Applications, Section 6.3.4 highlights the increased risk associated with programmers having access to program code once an application is operational. This increases the possibility for unauthorized changes to existing code that could benefit the programmer without being detected.
The program category field within ACIS is used to identify which program (OASDI or SSI) the fraud was committed against. The possibility exists that an investigation may involve fraud against both OASDI and SSI concurrently. While agents have the ability to enter in more than one program category per case into ACIS, the associated Monetary Statistics report only pulls the information from the first program category in each case. Therefore, in the case of concurrent fraud, agents are instructed to enter the dollar amounts into the program category where the greater amount of fraud occurred. This results in an overstatement of dollar amounts in the program category that the case was entered into, and an understatement of dollar amounts in the other program category.
CONCLUSIONS AND RECOMMENDATIONS
Throughout our evaluation of the four performance measures, we noted the strong commitment of SSA's OIG staff to correctly implement GPRA. Our evaluation found that the FY 2000 results of the four performance measures tested were reasonably stated. However, our evaluation noted that: 1) ACIS has some data integrity deficiencies; 2) the savings calculation is not adequately supported; 3) SSA OIG lacks sufficient documentation for several key ACIS control areas; 4) SSA OIG lacks appropriate supervisory review in the change control process; and 5) OASDI and SSI dollar amounts from investigations may be over or understated. We recommend that the OIG take the following corrective actions:
ACIS has a number of data integrity deficiencies.
To ensure that the OI offices across the country are following the procedures outlined in the SAH, that data entry and documentation errors do not go undetected, and to correct internal control issues found during our case testing we recommend that OI:
These procedures will prevent erroneous data being entered into ACIS in the future, and therefore prevent inaccurate GPRA reporting.
The savings calculation is not adequately supported.
SSA OIG
lacks sufficient documentation for several key ACIS control areas.
To ensure
that the SSA OIG has proper controls in place for change control, user access
recertification, and system design and development, we recommend that the
Office of Executive Operations (OEO):
SSA OIG lacks of appropriate supervisory review in the change control process.
OASDI and SSI dollar amounts from investigations may be over or understated.
APPROPRIATENESS OF THE PERFORMANCE MEASURES
As part of this engagement, we evaluated the appropriateness of each of the performance measures with respect to GPRA compliance and SSA’s APP. We determined whether the specific indicators and goals corresponded to the strategic goals identified in SSA’s APP, determined whether each of these indicators accurately measure performance, and determined their compliance with GPRA requirements.
The relationships between PMs #5 through 8 and the applicable SSA Strategic Goal is depicted in the following figure:
The SSA mission is supported by five strategic goals, including Goal 3, "To make SSA program management the best-in-business, with zero tolerance for fraud and abuse." Goal 3, in turn, is supported by several strategic objectives, including the relevant objective "to aggressively deter, identify, and resolve fraud." Performance Measures #5 through #8 address the OIG’s OI work related to SSA programmatic fraud. Assuming that the metrics are reliable, the diagram indicates that PMs #5 - #8 logically align with SSA’s strategic planning process.
Based on the taxonomy of performance measures included in Appendix F, PMs #5 through #8 are measures of accomplishment because they report on a result achieved with SSA resources. They are further categorized as output measures because they indicate the accomplishments or results that occur because of the SSA services provided. As shown in Appendix F, output measures include the number of investigations conducted.
Within the framework of GPRA, Performance Measures #5 through #8 fall within the intent of an output measure because they provide, "…a description of the level of activity or effort that will be produced." Just as with counts of workload, dollar amounts of workload are also a "level of activity required" or "workload" measurement. In addition, one output can be the workload driver for another output (e.g., the number of investigations conducted is a factor that drives the number of criminal convictions). Therefore, all four measures are considered output measures. The intent of these four performance measurements is to address the "zero tolerance for fraud" strategic goal. QAThey can all be useful to management and external stakeholders, as encouraged by OMB Circular A-11, which provides guidance on the creation of an agency’s performance measures. However, if corrective actions are not implemented to correct the issues identified as part of this report, potential errors in the data may in the future produce unreliable results.
OTHER
MATTERS
As part of this evaluation, we identified several points that the OIG should consider when designing the new AMS, as well as other less significant matters that are peripheral to this engagement. The points are discussed below.
There
is a need for Service Level Agreements.
In the current organizational structure of the SSA OIG, the personnel responsible for supporting ACIS are part of the OIG’s OEO. The personnel who use ACIS for their day to day job functions reside in the OI. During the course of our evaluation, we discovered that there was not any formal agreement between the OEO and the OI, which details the expectations and responsibilities of both offices with respect to ACIS and its support.
Control Objectives for Information and related Technology (COBIT) developed as generally applicable and accepted standard for good Information Technology (IT) security and control practices, states the following:
"Users and the IT function should have a written agreement which describes the service level in qualitative and quantitative terms. The agreement defines the responsibilities of both parties. The IT function must offer the agreed quality and quantity of service and the users must constrain the demands they place upon the service within the agreed limits."
The lack of such an agreement can lead to miscommunication and unfulfilled expectations, both of which could hamper the ability of affected OIG staff to perform their job functions effectively and efficiently. An agreement between both sides, such as a Service Level Agreement (SLA), could detail each office’s expectations and associated job duties, and provide accountability for their performance.
Recommendation
There
is no correlation between dollars reported from investigations and dollars
collected by SSA.
Both OASDI and SSI dollars reported from investigative activities are recorded into ACIS. These figures are collected by case number, and the SSN upon which the fraud was committed. However, it does not reference the SSN of the individual who committed the fraud. ACIS does not provide for accounting of the dollars actually collected and the tracking ceases with the closing of a case.
The accounting for the dollars, which are actually collected by SSA, is performed by SSA’s Debt Management System (DMS). The DMS contains information on the SSN to which the payment is posted. This may not always be the SSN that the fraud was committed against, but may be the SSN of the individual making the payment. Because the SSNs cannot always be cross referenced, neither the OIG nor DMS can categorize dollars collected as dollars associated with OIG cases.
Recommendation
The title
"Criminal Convictions" for PM #8 is misleading.
The category
"Criminal Convictions" contains several types of case resolutions.
In addition to criminal convictions, the category also includes work with
immigration and deportation cases, the satisfaction of a fugitive warrant,
and civil judgments. While there is a footnote explaining that the category
contains more than criminal convictions, it does not specifically state what
is included in the total.
Based on our recommendation, OI has changed the name of this performance measure
in the draft SSA’s FY 2003 APP to "Number of Judicial Actions."
In addition, SSA’s FY 2003 APP includes the following definition for Judicial
Actions: A judicial action is any event during the criminal justice process
that causes an individual suspected of committing a crime to be arrested for
the crime, or to appear before a judge to enter a plea of guilty, or to face
trial before a judge or jury.
There
is a lack of disaster recovery documentation.
ACIS, used by the SSA OIG to maintain its investigation information, resides at the NIH CIT. As such, the SSA OIG is considered to be a user bureau by NIH, and is occasionally asked by NIH to participate in its annual disaster recovery tests.
The ACIS systems administrator also performs the duties of a security coordinator at NIH. As such, NIH notifies him when a disaster occurs and is at that time given his instructions. However, a formal copy of the NIH disaster recovery plan is not kept by the SSA OIG.
Recommendation
APPENDICES
APPENDIX A – Scope and Methodology
APPENDIX B – Acronyms
APPENDIX C – Performance Measure Summary Sheets
APPENDIX D – Performance Measure Process Maps
APPENDIX E – Performance Measure Taxonomy
Scope and Methodology
The Social Security Administration’s (SSA) Office of the Inspector General (OIG) contracted PricewaterhouseCoopers, LLP (PwC) to evaluate 11 SSA performance indicators identified in its Fiscal Year (FY) 2001 Annual Performance Plan (APP). This report reflects our understanding and evaluation of the process related to PMs #5 through #8. We performed our testing from September 21, 2000 through February 15, 2001. Since FY 2001 performance results were not yet available as of the date of our evaluation, we performed tests of the performance data and related internal controls surrounding the maintenance and reporting of the results for FY 2000. Specifically, we performed the following:
Our engagement was limited to testing at SSA’s headquarters in Woodlawn, Maryland and the OIG office in Philadelphia, Pennsylvania. The procedures that we performed were in accordance with the American Institute of Certified Public Accountants’ Statement on Standards for Consulting Services, and are consistent with appropriate standards for performance audit engagements in Government Auditing Standards (Yellow Book, 1994 version). However, we were not engaged to and did not conduct an audit, the objective of which would be the expression of an opinion on the reliability or accuracy of the reported results of the performance measures evaluated. Accordingly, we do not express such an opinion.
Obtained an understanding and reviewed the current ACIS data flows and processes.
We obtained an understanding of the underlying processes and operating procedures surrounding ACIS and the generation of performance measures through interviews and meetings with the appropriate SSA OIG personnel and by reviewing the following documentation:
Identified and tested critical controls (both EDP and manual) of ACIS.
Based on the understanding we obtained during the planning part of this engagement, a review of related prior-year audit work at SSA, and our understanding of the Federal Information Systems Controls Audit Manual (FISCAM) methodology, we developed and performed tests of internal controls (both general and application) related to ACIS, for the following areas:
Tested the accuracy of the underlying FY 2000 data for each of the specified performance measures.
To verify, validate, and test the accuracy of the FY 2000 data we performed the following:
We requested copies of the OI-1 "Case Opening Report," OI-9 "ACIS Criminal & Administrative Disposition Form" and the OI-68 "Court-Ordered Restitution/Judgment Form" and performed the following:
Ensured the following documents were included in the file:
For cases involving restitution we ensured that in addition to the above documents the case file included the following:
Recalculated each specific FY 2000 measure to ascertain its mathematical accuracy
Determined whether performance measures were meaningful and in compliance with GPRA
As part of this engagement, we evaluated the appropriateness of each of the performance measures with respect to GPRA compliance and SSA’s APP. We determined whether the specific indicators and goals corresponded to the strategic goals identified in SSA’s APP, determined whether each of these indicators accurately measure performance, and determined their compliance with GPRA requirements.
ACRONYMS
ACIS Allegation
and Case Investigative System
ADABAS A
Database
AMD Allegation
Management Division
AMS Allegation
Management System
APP Annual
Performance Plan
ASAC Assistant
Special Agent-in-Charge
CAATs Computer
Assisted Audit Techniques
CIT Center
for Information Technology
DI Disability
Insurance
DMS Debt
Management System
EDP Electronic
Data Processing
FD Field
Division
FIPS Federal
Information Processing Standards
FISCAM Federal
Information System Controls Audit Manual
FY Fiscal
Year
GPRA Government
Performance and Results Act
IT Information
Technology
J&C Judgment
and Commitment Order
MAD Manpower
and Administration Division
MATPSC Mid-Atlantic
Program Service Center
MBR Master
Beneficiary Record
NIH National
Institutes of Health
OASDI Old-Age
and Survivors and Disability Insurance
OEO Office
of Executive Operations
OI Office
of Investigations
OIG Office
of the Inspector General
OMB Office
of Management and Budget
PM Performance
Measure
PwC PricewaterhouseCoopers
LLP
SA Special
Agent
SAC Special
Agent-in-Charge
SAH Special
Agent Handbook
SAS Statements
on Auditing Standards
SLA Service
Level Agreement
SSA Social
Security Administration
SSI Supplemental
Security Income
SSN Social
Security number
SSR Supplemental
Security Record
Performance Measure Summary Sheets
Name of Measure |
Measure Type |
Strategic Goal/Objective |
||
5) Number of investigations conducted (i.e., closed) |
Workload |
Goal: To make SSA program management the best-in-business, with zero tolerance for fraud and abuse. Objective: To aggressively deter, identify and resolve fraud. |
||
Purpose |
Report Frequency |
|||
To increase the number of investigations conducted resulting from allegations that have sufficient information or potential risk to warrant further review or action by a criminal investigator. |
Semiannual |
|||
How Computed |
Data Source |
Data Availability |
Data Quality |
|
Investigations are counted as "conducted" when all OIG actions have been completed, i.e., the investigator has presented the facts of the case to a prosecutor or has determined that further action is not warranted due to lack of investigative leads. |
ACIS |
Adequate |
Adequate |
|
Target Goal |
Division |
Designated Staff Members |
||
7,600 |
OIG |
Mike Arbuco, Amy Shemenski, Dennis Fabel, Dawn Zgorski, Oliver Webb |
||
EDP Controls Testing and Results |
||||
We performed tests of internal controls (both general and application) related to ACIS, for the following areas:
Refer to "Results of Evaluation" for a description of the findings. |
||||
CAATs Testing and Results |
||||
We obtained the FY 2000 ACIS data and performed CAATs using ACL to accumulate counts for this measure, and then compared those results to those reported for GPRA. We then reconciled any differences through data analysis and subsequent discussions with SSA OIG personnel. Refer to "Results of Evaluation" for a description of the findings. |
||||
Data and Manual Controls Testing and Results |
||||
We verified the completeness and accuracy of the ACIS data by judgmentally selecting a sample of 10 case files from the closed cases cabinet in the Philadelphia OI office and performed the following:
Refer to "Results of Evaluation" for a description of the findings. |
Name of Measure |
Measure Type |
Strategic Goal/Objective |
||
6) OASDI dollar amounts reported from investigative activities |
Workload |
Goal: To make SSA program management the best-in-business, with zero tolerance for fraud and abuse. Objective: To aggressively deter, identify and resolve fraud. |
||
Purpose |
Report Frequency |
|||
To report OASDI dollars from penalties, assessments, savings, recoveries and restitutions related to investigative activities. |
Semi-Annual |
|||
How Computed |
Data Source |
Data Availability |
Data Quality |
|
OASDI dollars from penalties, assessments, savings recoveries, and restitutions related to investigative activities that are reported by OIG filed divisions and included in OIG semi-annual reports. |
ACIS |
Adequate |
Adequate, except for the data integrity anomalies found as part of our evaluation. Refer to "Results of Evaluation" for a description of the findings. |
|
Target Goal |
Division |
Designated Staff Members |
||
$ 40 Million |
OIG |
Mike Arbuco, Amy Shemenski, Dennis Fabel, Dawn Zgorski, Oliver Webb |
||
EDP Controls Testing and Results |
||||
We performed tests of internal controls (both general and application) related to ACIS, for the following areas:
Refer to "Results of Evaluation" for a description of the findings. |
||||
CAATs Testing and Results |
||||
We obtained the FY 2000 ACIS data and performed CAATs using ACL to accumulate counts for this measure, and then compared those results to those reported for GPRA. We then reconciled any differences through data analysis and subsequent discussions with SSA OIG personnel. Refer to "Results of Evaluation" for a description of the findings. |
||||
Data and Manual Controls Testing and Results |
||||
To ensure that accuracy of the reported information we selected a judgmental sample of 45 OASDI cases from ACIS and performed the following:
Refer to "Results of Evaluation" for a description of the findings. |
Name of Measure |
Measure Type |
Strategic Goal/Objective |
||
7) SSI dollar amounts reported from investigative activities |
Workload |
Goal: To make SSA program management the best-in-business, with zero tolerance for fraud and abuse. Objective: To aggressively deter, identify and resolve fraud. |
||
Purpose |
Report Frequency |
|||
To report SSI dollars from penalties, assessments, savings, recoveries and restitutions related to investigative activities. |
Semi-annual |
|||
How Computed |
Data Source |
Data Availability |
Data Quality |
|
SSI dollars from penalties, assessments, savings recoveries, and restitutions related to investigative activities that are reported by OIG filed divisions and included in OIG semi-annual reports. |
ACIS |
Adequate |
Adequate, except for the data integrity anomalies found as part of our evaluation. Refer to "Results of Evaluation" for a description of the findings. |
|
Target Goal |
Division |
Designated Staff Members |
||
$ 80 Million |
OIG |
Mike Arbuco, Amy Shemenski, Dennis Fabel, Dawn Zgorski, Oliver Webb |
||
EDP Controls Testing and Results |
||||
We performed tests of internal controls (both general and application) related to ACIS, for the following areas:
Refer to "Results of Evaluation" for a description of the findings. |
||||
CAATs Testing and Results |
||||
We obtained the FY 2000 ACIS data and performed CAATs using ACL to accumulate counts for this measure, and then compared those results to those reported for GPRA. We then reconciled any differences through data analysis and subsequent discussions with SSA OIG personnel. Refer to "Results of Evaluation" for a description of the findings. |
||||
Data and Manual Controls Testing and Results |
||||
To ensure that accuracy of the reported information we selected a judgmental sample of 45 SSI cases from ACIS and performed the following:
Refer to "Results of Evaluation" for a description of the findings. |
Name of Measure |
Measure Type |
Strategic Goal/Objective |
||
8) Number of Criminal Convictions |
Workload |
Goal: To make SSA program management the best-in-business, with zero tolerance for fraud and abuse. Objective: To aggressively deter, identify and resolve fraud. |
||
Purpose |
Report Frequency |
|||
To report the number of criminal convictions as related to SSA/OIG investigative activities. |
Semi-Annual |
|||
How Computed |
Data Source |
Data Availability |
Data Quality |
|
Number of criminal convictions as related to SSA/OIG investigative activities. |
ACIS |
Adequate |
Adequate |
|
Target Goal |
Division |
Designated Staff Members |
||
1,800 |
OIG |
Mike Arbuco, Amy Shemenski, Dennis Fabel, Dawn Zgorski, Oliver Webb |
||
EDP Controls Testing and Results |
||||
We performed tests of internal controls (both general and application) related to ACIS, for the following areas:
Refer to "Results of Evaluation" for a description of the findings |
||||
CAATs Testing and Results |
||||
We obtained the FY 2000 ACIS data and performed CAATs using ACL to accumulate counts for this measure, and then compared those results to those reported for GPRA. We then reconciled any differences through data analysis and subsequent discussions with SSA OIG personnel. Refer to "Results of Evaluation" for a description of the findings |
||||
Data and Manual Controls Testing and Results |
||||
To ensure that accuracy of the reported information we selected a judgmental sample of 20 criminal convictions from ACIS and performed the following:
Refer to "Results of Evaluation" for a description of the findings |
Performance Measure Process Maps
PM 5-8 Investigation Process
Investigation Process cont.
Investigation Process cont.
Investigation Process cont.
Payment Process
Performance Measure Taxonomy