I. INTRODUCTION
Food and Drug Administration (FDA) Compliance Policy Guide 7150-09 establishes the
agency's policy on, "Fraud, Untrue Statements of Material Facts, Bribery, and Illegal
Gratuities" (fraud policy). The fraud policy sets forth FDA's regulatory approach
regarding applicants who seek to subvert the agency's review and approval processes for
premarket applications. The policy also outlines recommended corrective actions by which
applicants may seek to restore FDA's confidence in their integrity and permit the agency
to proceed with substantive scientific review of their premarket applications.
This "POINTS TO CONSIDER FOR INTERNAL REVIEWS AND CORRECTIVE ACTION OPERATING
PLANS" (document) describes actions applicants may take to affirm the validity of
data that have been called into question by FDA. This document is intended to permit
flexibility of application to the wide range of scientific activities, products, and
processes associated with applications submitted to FDA. This document does not establish
criteria, standards, or requirements that bind FDA or any applicant. This document also
does not create or confer any rights, privileges, or benefits on or for any person.
As discussed in Compliance Policy Guide 7150.09, FDA intends to conduct validity
assessments when the reliability of data in an application has been called into question
because of wrongful acts. As part of the validity assessment, the applicant should
undertake a comprehensive effort to determine the scope of the wrongful acts and their
effects on the reliability of data submitted to FDA.
When an applicant finds it necessary to take comprehensive corrective actions based on
notification by FDA of questions regarding the validity of data in one or more
applications, the applicant should, after conferring with the agency, provide FDA with a
written commitment that describes the corrective actions to be taken. As noted in the
fraud policy, the corrective actions generally should include full cooperation with
ongoing federal investigations, identification of all individuals who were or may have
been associated with or involved in the wrongful acts, a credible internal review, and
plans to develop and implement an effective corrective action operating plan. The
objective of the internal review described in this document is to identify all instances
of fraud and of untrue statements of material facts associated with applications submitted
to FDA. However, as a matter of policy, the FDA will not rely on the applicant's internal
review as a substitute for its own investigations, but will use the applicant's findings
to supplement the agency's own investigations and audits. The objectives of the applicant
in implementing a corrective action operating plan should be to ensure the reliability of
data in applications, to ensure the safety, effectiveness, and quality of marketed
products, and to take steps to preclude future instances of wrongful acts and
noncompliance with regulatory requirements for approved applications.
Depending on the circumstances, FDA may request different or additional measures. In
acting to remedy violations of law and establish reliability of data in applications
submitted to FDA, applicants may find it appropriate to take measures other than those
described herein. Applicants are encouraged to discuss their corrective action operating
plan with the agency to prevent expenditure of money and effort on activities that may
later be determined to be unacceptable by FDA.
II. CONDUCT OF THE INTERNAL REVIEW
The fraud policy encourages applicants to conduct a credible internal review, which
involves an outside consultant, in order to identify all instances of wrongful acts
associated with applications submitted to FDA, including any discrepancies between
conditions identified in approved applications and manufacturing conditions during actual
production.
Adequate internal reviews are crucial to assisting FDA in identifying all instances of
wrongful acts in a reasonable period of time and to the development by the applicant of a
useful corrective action operating plan. In general, internal reviews should include: (1)
an analysis of the problems identified by FDA in its notice to the applicant, (2) a
suitable and specific audit plan, (3) audits conducted in accord with the audit plan, and
(4) audit reports that accurately reflect the audit findings. Each of these steps is
described below:
A. Analysis of Problems Identified by FDA - Upon receipt by an applicant of FDA's
notification that the agency has reason to believe that the applicant or its employees
have engaged in wrongful acts that may undermine the integrity of the applicant's product
applications, the applicant should examine each of the problems and deficiencies
identified by FDA to determine the cause and scope of the wrongful acts and to establish
the best approach to conducting an audit that will identify all instances of wrongful
acts. In determining the cause and scope of the wrongful acts, the applicant should
examine its organizational structure, personnel and their responsibilities, the nature and
extent of managerial or other personnel's participation in activities related to the
development or approval of the implicated product applications, the applicant's operating
procedures, and any contractual agreements with third parties such as outside testing
facilities or clinical investigators. The circumstances surrounding instances of wrongful
acts may involve more than one person. For example, if data falsification were attributed
to one employee in the Quality Control Laboratory, the applicant should determine whether
the employee's supervisor and coworkers were involved in the falsification. In addition,
all work conducted by employees involved in the wrongful acts and associated with any
other FDA regulated products or applications should be examined.
B. The Audit Plan
1. Purpose - The audit plan describes the strategies and procedures to be used to
identify all instances of wrongful acts associated with applications submitted to FDA,
including any discrepancies between manufacturing conditions identified in approved
applications and manufacturing conditions during actual productions. This includes all
data or other information generated in support of an application, those submitted with the
application, and those generated subsequent to approval.
2. Who Develops the Audit Plan - The applicant and/or the outside consultant may
develop the audit plan. If the applicant or the outside consultant independently develop
the audit plan, both the applicant and the outside consultant should agree to the plan
before an audit commences.
3. Content - The audit plan should include a detailed description of the strategies and
procedures that will be used to conduct the audit. The audit plan should be sufficiently
flexible to permit the consultant to conduct all verifications necessary to assure
validity of the data or other information associated with the application. The audit plan
also should be sufficiently flexible to permit the consultant to revise the strategy and
procedures for the audit if information obtained during the audit indicates there is a
need to broaden or shift the focus of the audit. The plan should be comprehensive,
thorough, and complete, including an identification of all records, applications, and
other documentation that will be examined and all personnel (past and present) that will
be interviewed. The audit plan also should include a diagram of the applicant's major
organizational structure with key positions associated with the development and submission
of an application highlighted. Additional information about audit processes may be found
in the references listed under section IV of this document.
FDA recommends that the applicant submit the audit plan to the agency for review prior
to implementation. Generally, the following information should be included in the plan:
a. Scope of the Audit - The audit process seeks to validate events that occurred in the
past. Accordingly, validating information may be obtained through interviews with
personnel who were involved with developing the data included in an application and
through examination of documentation, specimens, and samples that were prepared at the
time of the past events. Therefore, the scope of the audit should encompass not only those
activities, applications, and personnel identified by the agency in its notification to
the applicant, but also those activities, applications, and personnel identified by the
applicant in its preliminary analysis of the problems identified by FDA, described at
section II.A. Therefore, the audit plan should identify all personnel to be interviewed,
the functions, specific activities, and products that may have been affected, and all
associated data, records, samples, specimens, and other documentation that could reveal
information about the wrongful acts.
b. Controls to Assure Impartiality - The audit plan should provide a complete
description of all controls that will be used to assure impartiality of the audit process.
For example, FDA recommends that the applicant specifically hire an outside consultant to
conduct the audit and to lead the audit team, if a team is necessary. In addition, the
consultant should be provided complete freedom and authority to conduct a thorough and
meaningful audit. The consultant's reports (including draft reports) should not be altered
by the applicant and recommendations should be considered and implemented as soon as
possible. Applicants should voluntarily cooperate in the consultant's investigations and
promptly provide all pertinent and requested documents.
c. Qualifications of the Consultant - The audit plan should include the name and
qualifications of the consultant and of each member of the audit team, if a team is used.
The qualifications required to assure that a consultant is appropriate for a specific
audit will depend on the nature of the investigation, the products involved, and other
circumstances in the case. The consultant generally should be familiar with the product
and operations to be audited, should possess the proper mix of education, training, and
experience to conduct an appropriate audit and should be free of any past involvement in
the wrongful acts or other activities being audited. The background normally associated
with consultants includes work experience in quality assurance, in regulatory affairs,
and/or with regulatory agencies. Quality control, laboratory, scientific, and
manufacturing personnel who have auditing experience may also qualify. Generally, a
criminal investigator, or a
Certified Public Accountant or an attorney may not be qualified to conduct an audit
independently, but may possess specific skills necessary to assist an audit team.
d. Projected Time Frames for Conducting the Audit - The audit plan should include a
timetable for the initiation, conduct, and completion of the audit. The timetable should
identify interim deliverables that the auditor will furnish to the applicant at specific
times documenting that progress is being made to comply with the plan.
e. Specifications for the Final Audit Report - The audit plan should include
specifications for the content and format of the final report of the audit, including a
scheme for analyzing, describing, and presenting the audit findings. Because the audit
report will be used by the applicant to develop the corrective action operating plan, the
audit plan may instruct the consultant to include in the audit report recommendations for
necessary corrective actions. The plan should state that a complete audit report,
identical to the report submitted to the applicant, should be submitted to FDA.
C. Conduct of the Audit - The audit should be conducted in accordance with the audit
plan. When the elements described below are identified in the plan for auditing, the
following guidance may be useful to the consultant examining an application.
1. Nonclinical and Clinical Studies
a. Protocol - The consultant should scientifically evaluate the study protocol and any
amendments to the protocol and determine their adequacy for achieving the study
objectives. Flaws should be identified. In addition, the consultant should interview
scientific, supervisory, and support personnel involved in the study to determine that
they fulfilled their respective roles in the study and to confirm that study operations
were conducted as specified in the protocol and in accordance with standard operating
procedures and applicable FDA regulations.
b. Study Report - The consultant should compare the protocol and its amendments, as
well as any applicable standard operating procedures, to the methods and procedures used
for conducting the study as described in the study report to determine if all of the study
objectives were properly attained and that there was no evidence of wrongful acts. A check
of study data and documentation for completeness and consistency is a measure that
suggests the study objectives were achieved. However, an independent collaboration should
be obtained to demonstrate that the study data and associated documents were not a part of
any wrongful act. An evaluation of the equipment as well as the capabilities of the staff
also should be made to determine that, in fact, the methods and procedures described in
the protocol and standard operating procedures could have been followed.
The consultant should compare the final study report with the raw data and other study
records to confirm that the conclusions reached in the final report are fully and
accurately substantiated by the study records. The consultant also should assure that none
of the. information was derived through wrongful acts. Accountability should be determined
for all data derived from the experimental units. Further, when the tabular conclusions
are the result of calculations applied to raw data, the appropriateness and accuracy of
the calculations should be determined and verified. If study data are generated by
automated data collection systems, the consultant should verify that computer hardware and
software were properly validated and are being properly maintained.
c. Specimen Examination - The consultant should randomly examine selected and
representative specimens generated as a result of a study and required to be retained for
confirmation of the number and kind of specimens referred to in the final study report. In
addition, pathology data should be reviewed for total animal and tissue accountability
including a reconciliation of tissues lost to histological processing. Summary tables and
narrative reports should be examined for consistency in terminology and in the appropriate
use of topographies and morphologies. All gross observations recorded at necropsy or on
individual animal necropsy records should be compared with histopathological diagnoses.
All wet tissues should be inventoried. Slides and blocks from all tissues should be
compared and matched. If neoplasms and target tissues need to be re-examined
microscopically, an independent pathology consultant may be necessary. Differences in
diagnoses between the original and consulting pathologists should be resolved.
2. Manufacturing Processes - When indicated by the audit plan, the consultant should
review the manufacturing processes for the product that is the subject of an application,
and such review should include, but not be limited to, the following elements:
a. Personnel - The consultant should interview employees to verify that actual work was
performed and to confirm that procedures were followed. Employees should be requested to
examine signatures/initials on documents, to verify their work, and to identify their
signature or initials to be their own. To further verify an employee's participation, the
consultant should examine employee time cards, vacation records, employment history and
the like. The consultant or a member of the consultant team who performs employee
interviews should be well versed in identifying and recognizing criminal and/or other
wrongful acts.
b. Raw Materials, Components, and Ingredients - The consultant should review and verify
receiving records for raw materials, components, and ingredients to confirm that the items
were received, the date received, and the amount received and to verify that they were
received from approved sources. Lot and serial numbers should be used to document the use
of raw materials, components, and ingredients. Raw material verification should be cross
referenced with raw material test records. The consultant should conduct interviews with
suppliers of raw materials, components, and ingredients to determine the validity of
records, chain of custody, and other pertinent information.
c. Test Records - The consultant should review test records for raw materials,
components, and ingredients to verify that the item was received, the date received, and
the amount received and to verify that they were received from approved sources. The
consultant also should review in-process and finished product test records to assess the
validity of the product being manufactured. Stability test records, reserve sample
records, and other records of in-house procedures may provide further information
pertinent to production verifications and lot number traceability. All of these records
should be thoroughly reviewed to determine that no wrongful acts were committed.
Analytical testing records, work books, and charts should be examined. The consultant
should also determine if any unofficial or private records were (or are) kept by any past
or present individuals and identify any discrepancies between the unofficial records and
the official records.
d. Production and Process Records - The consultant should review the master batch
records, production history, and batch records for inconsistencies, determine if these
records accurately reflect the FDA approved procedures and practices for the specific
product, and review appropriate records to verify all aspects of the manufacturing
process. All of the records also should be reviewed to assure that appropriate FDA
approvals were received for any changes or alterations made in the manufacturing
procedures and practices. Further, all these records should be reviewed to assure that no
wrongful acts were committed by past or present individuals. The consultant also should
determine if any private or unofficial records were (or are) being kept. The consultant
should review packaging and shipping records to provide assurance that the amount of
product manufactured is documented and can be verified by the batch record. Interviews
should be conducted with appropriate vendors and shippers to determine the validity of
records, chain of custody, and other pertinent information.
e. Equipment - The consultant should verify the use and existence of all equipment
(manufacturing, packaging, laboratory, etc.) by examining equipment use logs, cleaning
records, calibration records, servicing records, lease records, etc. Equipment purchase
documents, for example, can be useful in verifying the existence of the equipment
identified in manufacturing records. The capacity, speed, and function of the equipment
should be consistent with the manufacturing records. This information should be used to
verify that the amount of finished product manufactured is consistent with the equipment
use. For example, certain equipment is needed to manufacture a drug product with a wet
granulation as opposed to one with a dry granulation. Equipment use logs, calibration
records, servicing records, cleaning records, and similar other records should be
thoroughly reviewed for down-time, individual use time, etc., to assure the validity of
records and equipment. The consultant also should determine if any private or unofficial
records were (or are) being kept.
3. In Vivo Bioequivalence and Bioavailability Studies - The consultant should assure
that the batches being tested were legitimate, manufactured, and tested in accordance with
the application. Further, the consultant should audit the records as recommended in
2(a),(b),(c),(d), and (e). If the product is subject to an in vivo bioequivalence or
bioavailability study, the consultant should verify the accuracy of the data provided to
the applicant by the test facility. The verification should extend to both the clinical
and analytical aspects of the study.
D. Audit Report The consultant should prepare a report of the audit findings in
accordance with the audit plan, submit the audit report to the applicant, and submit a
copy of the audit report to FDA. The audit report should properly identify all the members
of the audit team along with their qualifications and the specific audits they
participated in. The audit team should be signatory to the final report. The consultant's
report
should also indicate the number of draft reports prepared before the final report and
where they are available.
III. THE CORRECTIVE ACTION OPERATING PLAN
The corrective action operating plan is the essential instruction by which applicants
plan to achieve correction of violative, unacceptable, and improper practices and to
preclude future occurrences of wrongful acts. The operating plan is derived from the
information gathered during the internal review. FDA recommends that the applicant submit
the corrective action operating plan to the agency for review prior to implementation.
A. Content of the Corrective Action Operating Plan - The corrective action operating
plan should include: (1) an analysis of the audit findings, including an analysis of-all
identified instances of wrongful acts associated with applications submitted to FDA,
identification all individuals who were or may have been associated with or involved in
the wrongful acts, and identification of defective practices, procedures, products, and
applications; (2) the disposition of any recommendations made by the consultant; (3) a
description of the actions taken and to be taken to achieve correction of fraud and other
wrongful acts or deficiencies identified by FDA and discovered by the internal review; (4)
a timetable for implementation of the corrective actions; (5) identification of the
persons responsible for taking and assuring the satisfactory completion of each of the
corrective actions according to timeframes; (6) a comprehensive ethics program that
describes standards for employees and procedures for educating employees about the program
and for enforcing the program; and (7) the procedures for monitoring the effectiveness of
the operating plan and to assure that the applicant can be expected to manufacture
products in compliance with current good manufacturing practices and application
requirements.
In addition, the corrective action operating plan should establish a mechanism for
assuring that supervisory, technical, and scientific employees are made aware of the
requirements of the Federal Food, Drug, and Cosmetic Act in their area of responsibility,
as well as the requirements under 18 USC 1001.
B. Implementation of Corrective Actions - The effectiveness of the applicant's
corrective actions will depend on effective management control of their implementation and
the exercise of in-process quality assurance practices (monitoring) during the corrective
actions process. Management should ensure that any new personnel hiring and training
practices, standard operating procedures, documentation of activity changes,
organizational rearrangements, and other major shifts in operations are done properly.
Adherence to the timetable for implementing corrective actions should be enforced and
monitored by responsible personnel.
C. Verification of the Adequacy of the Corrective Actions - Applicants should not
assume that the implementation of the operating plan has cured all problems. applicants
should establish an on-going program to monitor all activities related to the development
and submission of applications to FDA to assure that the applicant's records are reliable
and that the applicant can manufacture products in compliance with current good
manufacturing practices and application requirements.
IV. References
Baldwin, J.K. and B.K. Hoover. Data Audit as a Management Tool. Accountabil. Res.: Pol
& Qual. Assur. 1 71 (1989).
Carson, P.A. and N.J. Dent. Good Laboratory and Clinical Practices. Techniques for the
Quality Assurance Professional. Neinemanne Newnes, Publisher. Oxford, England (1990).
Cocchetto, D.M. et al. Accuracy of Data Collection in a Clinical Trail with Guidelines
for Case Report Auditing. Jour. Clin. Res. Drug Devel. 1 187 (1987).
Food and Drug Administration. Compliance Program Guidance Manual Program 7438.811.
Clinical Investigators (1989).
Food and Drug Administration. Compliance Program Guidance Manual Program 7438.808. Good
Laboratory Practice (Nonclinical Laboratories) (1989).
Hoover, B.K. and J.K. Baldwin. Meeting the QA Challenges of the 1980's: Team Auditing
by Toxicologist and QA Professionals. Jour. Amer. Col. of Tox. 3 129 (1984).
Loeb, S.E. and A.E. Shamoo. Data Audit: Its Place in Auditing. Accountabil. Res.: Pol.
& Qual. Assur. 1 22 (1989)
Pincus, K.V. Financial Auditing and Fraud Detection: Implications for Scientific Data
Audit. Accountabil. Res.: Pol. & Qual. Assur. 1 53 (1989).