Skip directly to search Skip directly to A to Z list Skip directly to navigation Skip directly to site content Skip directly to page options
CDC Home

Investigating Scientific Misconduct

This page contains several links to PDF files which may require a browser plug-in to view correctly. If you do not have the most recent version of Adobe Acrobat Reader, or are having difficulty viewing the PDF, download the plug-in here.

Manual GUIDE: General Administration, CDC-GA-2002-08 (Formerly CDC-89)
Date of Issue: 9/29/00 UPDATED: 10/23/02
CIO Contact: Office of the Director, Office of the Chief Science Officer
Material Superseded: None

  1. INTRODUCTION AND APPLICABILITY
  2. DEFINITIONS
  3. GENERAL POLICIES AND PRINCIPLES
    1. Allegations of Scientific Misconduct
    2. Protection from Liability
    3. Protection Against Retaliation
    4. Right to Representation
    5. Confidentiality
    6. Conflict of Interest
    7. Restoration of Reputation
  4. RESPONSIBILITIES
    1. Deciding Official
    2. ARILO
    3. CIO Director
    4. CIO ADS
    5. Inquiry Committee
    6. Investigation Committee
    7. Office of General Counsel
    8. Atlanta Human Resources Center, DHHS
    9. Subcommittee on Scientific Conduct and Ethics
  5. ASSESSMENT OF ALLEGATIONS OF SCIENTIFIC MISCONDUCT
    1. Allegation of Scientific Misconduct
    2. Assessment of Allegation of Scientific Misconduct and Other Information
    3. Outcome of Assessment
    4. Allegations of Misconduct other than Scientific
  6. CONDUCTING THE INQUIRY
    1. Appointment of Inquiry Committee
    2. Notification of Respondent(s)
    3. Notification of Supervisors
    4. Sequestration of Records
    5. Committee Meetings and Interviews
    6. Inquiry Report
    7. Review of Report
    8. Decision and Notification
    9. Time Limit for Completing Report
    10. Termination of the Inquiry
  7. CONDUCTING THE INVESTIGATION
    1. Sequestration of Additional Records
    2. Notification of Respondent(s)
    3. Notification of Supervisors
    4. Recording Admissions
    5. Appointment of Interview Committee
    6. Objection to Committee or Experts
    7. Charge to the Committee and 1st Meeting
    8. Developing the Investigation Plan
    9. Reviewing the Evidence
    10. Committee Deliberations
    11. The Investigation Report
    12. Comments on Draft Report
    13. Transmittal of Final Report
    14. Time Limit for Completing Report
    15. Review and Decision
    16. Termination of the Investigation
  8. ADMINISTRATIVE ACTIONS
    1. Recommendations for Sanctions
    2. Possible Sanctions
    3. Protections/Appeals
    4. Possible Actions when a Respondent(s) is found not guilty
  9. REFERENCES
  10. MODEL INVESTIGATION REPORT

I.INTRODUCTION AND APPLICABILITY

The Centers for Disease Control and Prevention and the Agency for Toxic Substances and Disease Registries (CDC/ATSDR) expect adherence to exemplary standards of intellectual honesty in the formulation, conduct, and reporting of scientific work. Allegations of scientific misconduct are taken seriously by CDC/ATSDR.

The policy and procedures that follow enable allegations of scientific misconduct to be processed promptly, confidentially, and fairly. Investigations of allegations of scientific misconduct must balance the equal concerns for protecting the integrity of research as well as the careers and reputations of researchers. A prompt response minimizes harm to the public that could result if misconduct is found and allows those who are incorrectly accused to quickly clear their reputations. The policy establishes a set of recommended maximum time limits for evaluating scientific misconduct. Actual time frames may vary from case to case because of complexity, collection of evidence, or other factors, and so this policy does not propose adoption of a binding, universal set of time constraints.

Allegations of misconduct that prove to be untrue, even if they were made in good faith, can damage careers and have an adverse effect on research. Confidentiality helps protect innocent people incorrectly or unjustly accused as well as those who bring the allegations. Fairness allows all who become involved in scientific misconduct cases to have the opportunity to participate appropriately in addressing the issue and seeks to protect innocent participants from adverse consequences.

The CDC/ATSDR scientific misconduct policy and procedures apply to all scientific activities (e.g.; human subject research, non-human subject research, technical assistance, emergency response, surveillance, screening, etc) conducted, or proposed to be conducted by any CDC/ATSDR employee or trainee as part of his or her official duties or training. This policy and related policies of other agencies apply to CDC employees and trainees assigned outside of CDC by detail, inter-personnel agreement (IPA), or long-term training.

Activities funded by CDC/ATSDR conducted by persons other than CDC employees or trainees are not covered by this policy unless an allegation includes a CDC employee or trainee. Contract employees are employees of the contractor and not covered by this policy. Contractors and institutions/agencies/organizations funded by CDC are responsible for preventing scientific misconduct by developing related policies. A CDC inquiry or investigation may include an interview with a non-CDC employee or trainee upon notification and approval of the employer.

The procedures described in this document do not create any right or benefit, substantive or procedural, enforceable at law by a party against CDC, its agencies, officers, or employees.


II. DEFINITIONS

A. Agency Research Integrity Liaison Officer (ARILO) - the official designated by the CDC/ATSDR Director to be responsible for all matters related to research integrity programs - the CDC Associate Director for Science (ADS) or the Acting Associate Director for Science.

B. Allegation - any written statement, having a self-identified author, describing possible scientific misconduct and given to the CDC ADS. A Good Faith Allegation is an allegation made with the honest belief that scientific misconduct may have occurred. A Bad Faith Allegation is an allegation by a complainant(s) who knows, or through reasonable inquiry could have known, that the allegation is untrue or frivolous. (See Section V).

C. Complainant(s) - a person who makes an allegation of scientific misconduct.

D. CDC/ATSDR Counsel - counsel from the Office of the General Counsel (OGC).

E. CDC Deciding Official - the CDC official who makes the agency’s final determination on findings of scientific misconduct - the Deputy Director for Science and Public Health.

F. Inquiry - follows the initial assessment of allegation and precedes the investigation. The inquiry is the process of gathering information and initial fact-finding to determine whether an allegation or apparent instance of scientific misconduct warrants an investigation. (See section VI).

G. Investigation - follows the inquiry. The investigation is the formal examination and evaluation of all relevant facts to determine if scientific misconduct has occurred, and, if so, to determine the person(s) who committed it and the seriousness of the misconduct. (See section VII)

H. The Office of Research Integrity (ORI), the office within the U.S. Department of Health and Human Services (DHHS) that is responsible for review of scientific misconduct and research integrity activities.

I. Research Record -

  • Evidence - any physical or electronic data or results that embody the facts resulting from scientific inquiry. Includes, but is not limited to: scientific records, transcripts or recordings of interviews, committee correspondence, administrative records, grant applications and awards, manuscripts, publications, expert analyses, progress reports, abstracts, theses, oral presentations, internal reports, and journal articles.
  • E-mail - CDC/ATSDR e-mail messages (including attachments) that are evidence of the activities at CDC/ATSDR or have informational value are considered Federal records and therefore Government property in accordance with CDC policy “Record Keeping Procedures for Managing E-mails and Attachments That Qualify as Federal Records”.

J. Respondent(s) - the person(s) against whom an allegation of scientific misconduct is directed or the person whose actions are the subject of the inquiry or investigation.

K. Retaliation - any action taken by the CDC/ATSDR or its staff that adversely effects the employment status or reputation of an individual because the individual has, in good faith, made an allegation of scientific misconduct, or of inadequate institutional response thereto, or has cooperated in good faith with an inquiry or investigation of such allegation.

L. Scientific misconduct - Scientific misconduct is defined as fabrication, falsification, or plagiarism in proposing, performing, or reviewing scientific activities, or in reporting scientific results.

  • Fabrication is inventing data or results.
  • Falsification is manipulating materials, equipment, or processes, or changing or omitting data or results so that the scientific record is inaccurate.
  • Plagiarism is the appropriation of ideas, processes, results, or words of another person without giving appropriate credit, including ideas, processes, results, or words obtained through confidential review of scientific proposals and manuscripts.
  • A finding of scientific misconduct implies that activities occurred that represent a significant departure from common practice. These activities must have been committed knowingly or in reckless disregard. Scientific misconduct must be proven by a preponderance of evidence [evidence which is of greater weight or more convincing than the evidence which is offered in opposition to it; that is, evidence which as a whole shows that the fact sought to be proved is more probable than not].
  • Scientific misconduct does not include honest error or honest difference of opinion. Research misconduct not covered by this guidance includes failure to follow the procedures described in the CDC Human Subjects Manual, publishing without following the Authorship of CDC or ATSDR Publications or the Clearance Procedures for Scientific and Technical Documents, making a bad faith allegation, or noncompliance with CDC scientific policies.
  • Research misconduct not covered by this policy can be assessed using the CDC/ATSDR Guide for Investigating Misconduct and Processing Disciplinary/Adverse Actions located in the AHRC Supervisor's Human Resources Reference Guide.

M. The Subcommittee on Scientific Conduct and Ethics (SSCE) - a subcommittee of the Committee of Excellence in Science including all CIO Associate Directors for Science.


III. GENERAL POLICIES AND PRINCIPLES

A. Allegation of Misconduct - Individuals, believing they have evidence of scientific misconduct, should report such evidence to the CDC Associate Director for Science. Individuals unfamiliar with the principles of scientific misconduct are encouraged to first discuss their concerns with, or seek advice from, individuals they trust and are familiar with this policy, before bringing a formal complaint. An individual may seek the advice of a CIO ADS, who should be familiar with this policy and its underlying principles. The mere filing of an allegation should not bring science to a halt or be a basis for other disciplinary or adverse action absent other compelling reasons.

B. Protection from Liability - All CDC/ATSDR representatives and Committee members involved in evaluating scientific misconduct will be represented by the Office of General Counsel for all official actions associated with the Inquiry. The Office of the General Counsel does not represent the complainant(s) or the respondent. If the claimant or respondents are agency employees, they must seek private representation during inquiries/investigations of scientific misconduct.

C. Protection against Retaliation - No CDC/ATSDR employee shall subject anyone to harassment, nor take any action against any individual as a reprisal for making a good faith allegation or providing any information pursuant to this policy. Administrative sanctions may be made against a complainant who makes an allegation in bad faith.

Individuals covered by this policy, who believe that they have been subjected to retaliation, may contact the ARILO and have the option of filing a grievance under the administrative grievance procedure or applicable negotiated grievance procedure, or filing a complaint with the U.S. Office of Special Counsel under the Whistleblower Protection Act, 5 U.S.C. 2302.

D. Right to Representation or Interpretation - The respondent(s), witness(es), and complainant(s) may bring counsel or an advisor to interviews by the Inquiry or Investigation Committee. However, that person may participate only by advising the respondent, not by asking or answering questions of the Committee. Any individual questioned by the committees who is a Federal employee included in a bargaining unit represented by a labor union has the right to have a union representative present during the interview (5 U.S.C. 7114(a)(2)(B). Any witness may bring an interpreter to an interview if necessary.

E. Confidentiality - A statement of confidentiality must be signed by anyone who receives information about the allegations, including ARILO, Deciding Official, members of the Inquiry Committee, Investigation Committee, witnesses, the complainant, the respondent, laboratory members, and those involved in notification and sequestration. Members of the committees and experts will agree in writing to observe the confidentiality of the proceedings and any information or documents reviewed as part of the Inquiry or Investigation. Outside of the official proceedings of the committees, they may not discuss the proceedings with the respondent, complainant, witnesses, or anyone not authorized by the ARILO to have knowledge of the Investigation.

If the complainant requests anonymity, CDC/ATSDR will make an effort to honor the request during the allegation assessment or inquiry within applicable policies and regulations and state and local laws, if any. This complainant will be advised that if the matter is referred to an investigation committee and the complainant’s testimony is required, anonymity may no longer be guaranteed.

F. Conflict of Interest - the ARILO, Deciding Official, voting and non-voting members of the Inquiry and Investigation Committees, and experts should not have a real or apparent personal or professional conflict of interest in the case. They must function impartially, and have appropriate expertise to evaluate the evidence and issues related to the allegations. Inquiry and Investigation committee members must sign a statement indicating that no personal or professional conflicts of interest exist with respect to the respondent, complainant, or the case in question. The ARILO will develop this statement.

G. Restoration of Reputation - If an Inquiry or Investigation concludes that the allegation is without basis or cannot be sufficiently evaluated, reasonable steps will be taken by the CDC/ATSDR to restore the reputation of the respondent(s). At a minimum, the respondent(s) should receive a written statement summarizing the conclusions of the committee. CDC/ATSDR will distribute additional copies of the final conclusions at the request of the respondent(s).


IV. RESPONSIBILITIES

A. Deciding Official - makes the final determination within CDC/ATSDR to accept the recommendation of the ARILO. If this determination differs from the ARILO, the Deciding Official will explain in writing the basis for rendering a different decision. Reasons why the Deciding Official may arrive at a different decision include, but are not limited to: fairness, consistency, and sufficiency of evidence. The Deciding Official may return the report to the ARILO with a request for further fact-finding or analysis before reaching a decision. If an allegation is made against the ARILO, the Deputy Director of Science and Public Health or his/her designee will assume the ARILO’s role. If a situation arises wherein the Deputy Director must take on the role of the ARILO, he/she cannot /should not also be the deciding official as that could give rise to claims of bias, conflict of interest, etc. Therefore, should the Deputy Director have to take on the ARILO role, the Director would then become the deciding official.

B. ARILO - Oversees and coordinates the CDC/ATSDR's activities and policies related to scientific integrity.

Represents the CDC/ATSDR on matters of scientific integrity policy through membership on the DHHS Agency Research Integrity Liaison Group.

  1. Initiates and carries out, or supervises the carrying out of, assessments, inquiries, and investigations of suspected scientific misconduct, whether based on allegations or other evidence.
  2. Ensures prompt reporting of possible scientific misconduct to the Deciding Official.
  3. Works closely with the Inquiry and Investigation Committees to identify relevant issues to be addressed, assist in organization, and answer any questions raised by Committee members.
  4. Evaluates the Inquiry Report and makes final determination whether the evidence is sufficient to support possible findings of scientific misconduct and thereby justify conducting an Investigation.
  5. Evaluates the Investigation Report and makes initial determination to accept the recommendation of the Investigation Committee. If this determination differs from that of the Investigation Committee, the ARILO will explain in writing the basis for rendering a decision different from the committee. Reasons why the ARILO may arrive at a different determination include, but are not limited to: fairness, consistency, and sufficiency of evidence. The ARILO may return the report to the Investigation Committee with a request for further fact-finding or analysis.
  6. Is authorized to carry out the above activities including specifically the authority to secure scientific records and materials related to suspected or alleged misconduct.
  7. Provides Inquiry and Investigation Reports to the Deciding Official.

C. CIO Director - will cooperate with any Inquiry or Investigation conducted and implement appropriate administrative actions based upon the final case decision by the Deciding Official.

D. CIO Associate Directors for Science (CIO ADS) – are responsible for the integrity of scientific and public health practice activities in their CIO. They are consulted, as appropriate, in the initial handling of allegations of possible scientific misconduct in their CIO and will cooperate with the inquiries and investigations. A CIO ADS cannot be a member of an inquiry or investigation committee(s) examining allegations of misconduct in their own CIO, or participate as a member(s) of the SSCE. An individual may seek the advice of a CIO ADS, who should be familiar with this policy and its underlying principles.

E. Inquiry Committee - determines whether sufficient credible evidence of possible scientific misconduct exists to warrant a formal Investigation. The Inquiry does not determine whether scientific misconduct has occurred or who is responsible. The Inquiry Committee collects, reviews, and reports available evidence including the testimony of the complainant(s), respondent(s), and other designated key witnesses.

F. Investigation Committee - determines whether misconduct has been committed, by whom, and to what extent. The Investigation will determine whether there are circumstances that would justify adding additional charges of misconduct and identify these circumstances. The Investigation Committee accomplishes its task by exploring in detail the allegations and examining the evidence in depth. The findings of the Investigation will be set forth in an Investigation Report.

G. Office of the General Counsel (OGC) – - represents CDC/ATSDR during the scientific misconduct inquiry and investigation. The OGC advises the ARILO, the Inquiry and Investigation Committees, and the Deciding Official on relevant legal issues. The CDC/OD maintains a chain of custody for all records sequestered during the evaluation of scientific misconduct. The OGC represents only CDC/ATSDR staff responsible for managing or conducting the institutional scientific misconduct process as part of their official duties. The OGC does not represent the complainant(s) or the respondent or any other person participating during the inquiry, investigation, or any follow-up action, except CDC/ATSDR staff responsible for managing or conducting the institutional scientific misconduct processes as part of their official duties. As available, the OGC is represented at all meetings of Inquiry and Investigation Committees. OGC representatives do not vote in deliberations.

H. Atlanta Human Resources Center, DHHS (AHRC) - advises the ARILO and the Inquiry and Investigation Committees concerning employee misconduct other than scientific misconduct and in administrating possible adverse actions. An AHRC representative may attend meetings at the request of the ARILO, Inquiry Committee, and/or Investigation Committee. The representative serves as a consultant, non-voting member.

I. Subcommittee on Scientific Conduct and Ethics (SSCE) - works with the ARILO to establish CDC/ATSDR scientific misconduct policy and to appoint the Inquiry Committee and Investigation Committee. Members may serve on the Inquiry or Investigation Committees.


V. ASSESSMENT OF ALLEGATIONS OF SCIENTIFIC MISCONDUCT

A.   Allegations of Scientific Misconduct. Allegations of scientific misconduct must be made in a written statement by a self-identified author, describing possible scientific misconduct. The allegation must include sufficient detail to make clear the nature of the activity which is regarded as the alleged scientific misconduct together with a description of facts, events, and circumstances which led to the allegations. The allegation should be received by the CDC ADS (ARILO). The ARILO may make an allegation of scientific misconduct based upon information known to him or her.

B. Assessment of Allegations and/or Other Information - An assessment of the allegations will be made by the ARILO to determine whether the allegations, if true, would constitute scientific misconduct as defined and whether the information is sufficiently specific to warrant and enable an Inquiry. No evaluation of the facts themselves will occur at this stage. The assessment will be completed within seven (7) calendar days of receipt of the report of alleged scientific misconduct.

C. Outcome of the Assessment - An assessment may conclude that scientific misconduct may have occurred in which case an inquiry is warranted. In the case of multiple respondents the ARILO will determine the number and nature of the inquiry or inquiries. An assessment may also lead to the conclusion that the allegation is unfounded or insufficient in which case no inquiry is warranted. Upon completion of the assessment, the ARILO will notify in writing the complainant, the Deciding Official, and anyone else who became aware of the allegations. If the decision is not to proceed, the matter shall be closed and the records will be retained for no fewer than five (5) years by the ARILO.

D. Allegations Other than Scientific Misconduct - The ARILO will consult with, and promptly report to, the appropriate CDC/ATSDR official allegations describing events or conduct that might be a threat to human or animal scientific subjects, a violation of safety regulations, financial irregularities, discrimination, sexual harassment, criminal activity, or other violations of Federal law, regulation, or policy.


VI. CONDUCTING AN INQUIRY

For the purpose of this policy, the following responsibilities apply:

A. Appointment of an Inquiry Committee - The ARILO, in consultation with the SSCE, will appoint an Inquiry Committee and Committee Chair prior to notification of the respondent and within seven (7) days of establishing the need for an Inquiry. The Committee will consist of 3 voting members. At least two voting members will be senior scientists familiar with this policy. In general, it is desirable to include among the voting members a person of similar professional designation as the respondent(s) (e.g., another postdoctoral fellow if the respondent(s) is a postdoctoral fellow) and/or a person of similar training (e.g., epidemiologist, microbiologist, statistician). The committee should reflect a racial and gender balance whenever possible. Committee members must sign a statement that they have no conflict of interest in the case. A representative of AHRC and of OGC, may attend attend the meetings at the request of the committee. The committee may identify technical experts to provide subject matter expertise to the committee. Technical experts are not voting members of the committee.

B. Notification of the Respondent(s) - The committee chair will lead the notification process, and shall arrange that this process be performed in a private place in the least disruptive manner possible in order to minimize disturbance and embarrassment to the respondent. If needed, a language interpreter should be present.

The committee chair will explain the mechanism of the Inquiry into misconduct in science. A notification memo signed by the ARILO will be provided and explained to the respondent(s) at this time. The nature of the allegations will be described in the memo. The respondent(s) will receive copies of this policy. The committee chair will emphasize that the respondent(s) is considered innocent of scientific misconduct until proven otherwise.

The Committee’s members should be made known to the respondent(s) at the time the respondent(s) is notified of the suspected misconduct and the initiation of the inquiry. If the respondent(s) objects to the Committee's membership, a written objection providing reason(s)/justification(s) may be submitted to the ARILO within three (3) working days of the announcement of the Committee's membership. The ARILO will then evaluate whether that person should be replaced.

C. Notification of Supervisors - The immediate supervisors of the respondent, complainant, and expert witnesses will be notified that an allegation of scientific misconduct has been made in which the employee will be called upon to meet with an inquiry committee. The supervisor will not be told the exact nature of the employee’s role in the evaluation of the allegation. The supervisor will assure that the employee has sufficient time to participate and that the employee’s performance evaluation is not adversely affected.

D. Sequestration of Scientific Records – The committee chair, OGC as available, and the CIO ADS or CIO representative as appropriate will accompany the respondent(s) to the worksite, and to any other location as appropriate for sequestration of the relevant scientific records.”

The committee chair, CDC/ATSDR Office of the General Counsel, and the CIO ADS or CIO representative as appropriate, will accompany the respondent(s) to the worksite, and to any other location as appropriate, for sequestration of the relevant scientific records. If the respondent(s) is not available, sequestration from any CDC/ATSDR facility may begin in his/her absence. Materials to be sequestered may include notebooks, data records, documents, manuscripts, computer files, and other relevant materials.

An inventory list will be prepared and signed by both the committee chair and the respondent. A copy will be provided to the respondent(s) at the time of sequestration. A chain of custody will be maintained by the Office of the Director, including locked storage, for all sequestered materials to ensure that the originals are kept intact and unmodified. This policy protects all parties from concerns about subsequent modification of records. In order to provide the respondent(s) full opportunity to respond to the allegations, copies of sequestered materials will be made available to the respondent(s) as soon as is practical upon request. In addition, the respondent(s) will be permitted to view the original sequestered materials under the supervision of the committee chair or the ARILO.

E. Inquiry Committee Meetings and Interviews - The ARILO will describe the allegations to the Committee and work with them to identify a set of issues to be addressed during the Inquiry process.

The Inquiry Committee will interview the complainant(s), the respondent(s), and other individuals likely to have knowledge relevant to the allegations. Both the complainant(s) and respondent(s) may suggest others to be interviewed or brought in as witnesses, but the committee is responsible for deciding who will be interviewed. The Inquiry Committee will conduct all interviews in a professional and objective manner.

All interviews will be recorded, for later transcription as necessary, and summarized. A transcript or summary of their interview will be provided to each witness for review and correction of errors. Changes to the transcript or summary will only be made to correct errors in transcription. Witnesses may add comments or additional information as an addendum.

The Committee will examine relevant scientific records and materials. The respondent(s) will be provided by the Committee updates regarding any new allegations or new evidence, so that the respondent(s) can prepare a sufficiently complete response. The Committee may request to re-interview the respondent, or the respondent(s) may request another session if issues arise for which additional responses and clarifications are necessary.

After evaluating the evidence, the Inquiry Committee will determine whether it suggests that scientific misconduct may have occurred and warrants a formal, full-scale Investigation. The findings and recommendations of the Inquiry Committee are described in the draft Inquiry Report, which should be completed within 53 calendar days of the first meeting of the Inquiry Committee. If, during evaluation of the evidence, reasonable indication of possible criminal violations is found, ORI must be notified within 24 hours. ORI should also be notified about immediate health hazards resulting from the misconduct. If there is a concern that the incident will be publicly reported, ORI should be informed.

F. The Inquiry Report - A written Inquiry Report must be prepared that includes the names, titles, and affiliations of the Committee members and expert consultants, if any; the nature of the alleged misconduct; a summary of the Inquiry process used; a list of the scientific records reviewed; summaries of any interviews; a description of the evidence in sufficient detail to demonstrate whether an Investigation is warranted; and the Committee's decision as to whether an Investigation is recommended and whether any other actions should be taken if an Investigation is not recommended. All relevant dates should be included in the Report. CDC/ATSDR counsel will review the Report.

G. Review of the Inquiry Report - The ARILO will provide the respondent(s) with a copy of the draft Inquiry Report for comment and rebuttal. The complainant(s) and witnesses will be provided with those portions of the draft Inquiry Report that address their role or statements in the Inquiry. The ARILO will establish appropriate conditions for review to protect the confidentiality of the draft Report.

The complainant(s), respondent(s), and witnesses will provide their comments, if any, to the Inquiry Committee within seven (7)calendar days of their receipt of the draft Report. These reviewers may seek an extension in writing of this deadline for good cause. Any comments that the complainant(s) or respondent(s) submits on the draft Report will become part of the final Inquiry Report and record. Based on the comments, the Inquiry Committee may revise the Report as appropriate.

H. Decision and Notification - The ARILO will make the determination, within seven (7) calendar days of receipt of the Inquiry Committee Report, of whether the evidence is sufficient to support findings of scientific misconduct that justify conducting an Investigation. If his/her determination differs from that of the Inquiry Committee, the ARILO will provide a detailed explanation on the basis for rendering this decision to the Deciding Official. The ARILO’s explanation should be consistent with the ORI definition of scientific misconduct, the CDC/ATSDR's policies and procedures, and the evidence reviewed and analyzed by the Inquiry Committee. The ARILO may return the report to the Inquiry Committee with a request for further fact-finding or analysis. The Inquiry is completed when the ARILO determines that an Investigation will be conducted or that the Inquiry is complete and does not justify an Investigation. If findings from the Inquiry do indicate that an investigation is necessary, this investigation will be initiated within 30 days after completion of the Inquiry.

The ARILO will notify the complainant(s), the respondent(s), Deciding Official, and appropriate CDC/ATSDR officials as to the decision to proceed or not to an Investigation and will remind them of their obligation to cooperate in the event that an Investigation is opened. The ARILO will also notify ORI if the decision is to go forward with an Investigation before the Investigation begins.

I. Time Limit for Completing the Inquiry Report - It is recommended that the Inquiry Committee complete and submit its report in writing to the ARILO within 60 calendar days following its first meeting. The ARILO may approve an extension for good cause. If the ARILO approves an extension, the reason for the extension will be entered into the records of the case and the Report. The complainant(s) and respondent(s) will be notified of the extension. All records of the Inquiry process will be retained for five (5) years by the Office of the Associate Director for Science, CDC/ATSDR.

J. Termination - If the institution plans to terminate the inquiry for any reason without completing all relevant requirements under [50.103(d)], a report of such planned termination, including a description of the reasons for such termination, shall be made to ORI.


VII. CONDUCTING THE INVESTIGATION

A. Sequestration of Additional Records - The ARILO will immediately sequester any additional pertinent scientific records that were not previously sequestered during the Inquiry. The institution may decide to investigate additional allegations not considered during the Inquiry stage. The procedures to be followed for sequestration during the Investigation are the same procedures that apply during the Inquiry (See section VI.C). The institution should protect Federal funds or equipment and individuals affected by the investigation. The appropriate interim administrative actions in order to protect Federal funds and to ensure that the purposes of the Federal financial assistance are being carried out should be taken.

B. Notification of the Respondent(s) - The ARILO will notify the respondent(s) immediately (within 24 hours if possible) after the determination is made to open an Investigation. The notification includes: a copy of the Inquiry Report; the specific charges of scientific misconduct; the definition of scientific misconduct; the procedures to be followed in the Investigation, including the appointment of the Investigation Committee and experts; the opportunity for the respondent(s) to be interviewed, to provide information, to be assisted by counsel or by a union representative if appropriate, to challenge the membership of the Committee and experts based on bias or conflict of interest, and to comment on the draft Report; the fact that ORI will perform an oversight review of the report; and an explanation of the respondent's right to request a hearing before the DHHS Departmental Appeals Board if a finding of scientific misconduct is made.

C. Notification of Supervisors - The immediate supervisors of the respondent, complainant, and expert witnesses will be notified that an allegation of scientific misconduct has been made in which the employee will be called upon to meet with an investigation committee. The supervisor will not be told the exact nature of the employee’s role in the evaluation of the allegation. The supervisor will assure that the employee has sufficient time to participate that does not affect any performance evaluation.

D. Recording Admissions - If the respondent(s) admits to the misconduct, he or she should write and sign a statement attesting to the occurrence and extent of the misconduct. The respondent(s) should acknowledge that the statement was voluntary. The admission may not be used as a basis for closing the Investigation unless the committee has determined the extent and significance of the misconduct and all procedural steps for completion of the Investigation have been met. The committee may consult with the ARILO or CDC/ATSDR counsel when deciding whether an admission has adequately addressed all of the relevant issues such that the Investigation can be considered completed. The Investigation should not be closed unless the respondent(s) has been appropriately notified and given an opportunity to comment on the Investigation Report. If the case is considered complete, it should be forwarded to the ARILO with recommendations for appropriate CDC/ATSDR actions and then to ORI for review.

E. Appointment of the Investigation Committee - The ARILO, in consultation with the SSCE will appoint the Investigation Committee and the Committee Chair within seven (7) days of establishing the need for an investigation. The Investigation Committee should consist of at least five voting members who have the necessary expertise and training to evaluate the evidence and issues related to the allegations, interview the principals and key witnesses, and conduct the investigation. These individuals may be scientists, administrators, subject matter experts, lawyers, or other qualified persons. The committee should reflect a racial and gender balance whenever possible. All committee members must sign a statement indicating that no personal or professional conflicts of interest exist with respect to the respondent, complainant, or the case in question. One member of the Investigation Committee will be a person of similar professional designation as the respondent (e.g., another postdoctoral fellow if the respondent(s) is a postdoctoral fellow). None of the voting members will have served on the Inquiry Committee, but the Chair of the Inquiry Committee will be available for consultation. A representative of OGC will also attend the meetings. A representative of AHRC may attend at the request of the ARILO or the committee.

Experts may be appointed (or carried over from the Inquiry if no objection is raised) to advise the committee on scientific or other issues. The respondent(s) may suggest other experts. Experts cannot vote within the committee.

F. Objection to Committee or Experts by Respondent(s) - The ARILO will notify the respondent(s) of the proposed committee membership as soon as it has been established. If the respondent(s) objects to the Committee’s suggested membership, a written objection listing specific reasons, must be submitted to the ARILO within three (3) working days. The ARILO will then determine whether there is any bias or conflict of interest that would necessitate replacing the challenged member with a qualified substitute.

G. Charge to the Committee and the First Meeting – The ARILO will send a written charge to the Committee that describes the allegations and related issues identified during the Inquiry, defines scientific misconduct, and identifies the name of the respondent. The charge will state that the committee is to evaluate the evidence and testimony of the respondent, complainant, and key witnesses to determine whether, based on a preponderance of the evidence, scientific misconduct occurred and, if it did, its seriousness and who was responsible.

During the Investigation, if additional information becomes available that changes the subject matter of the Investigation or would suggest additional respondents, the committee will notify the ARILO. The ARILO will notify the respondent(s), including the additional respondents, of the new information and, after consultation with the committee and obtaining the views of all the respondents, make the decision as to whether the new information justifies inclusion in the current Investigation or initiation of a new Inquiry. Additional respondents, believing that inclusion in an ongoing investigation limits their access to the full process established by this policy, may provide the ARILO with a written request for a new Inquiry.

The ARILO must be advised by the committee of any necessary interim actions needed to protect human or animal subjects, or other steps required by regulation or policy.

The ARILO and CDC/ATSDR counsel should be consulted throughout the Investigation on compliance with these procedures and HHS regulations, appropriate investigatory and interviewing methods and strategies, legal issues, and the standard of proof. OGC, as available, will be present at all interviews and meetings throughout the Investigation to advise the committee.

The ARILO, with the assistance of CDC/ATSDR counsel and AHRC, will convene the first meeting of the Investigation Committee to review the charge, the Inquiry Report, and the prescribed procedures and standards for the conduct of the Investigation, including the necessity for confidentiality and for developing a specific investigation plan. Members of the committee and experts will agree in writing to protect the confidentiality of the proceedings and any information or documents reviewed as part of the Investigation. Outside of the official proceedings of the committee, they may not discuss the proceedings with the respondent, complainant, witnesses, or anyone not authorized by the ARILO to have knowledge of the Investigation. The Investigation Committee will be provided with a copy of the CDC/ATSDR Policies and Procedures for Investigating Scientific Misconduct

H. Developing an Investigation Plan - At the initial meeting, the Committee will begin development of its Investigation Plan and complete it as soon as reasonably possible. The Investigation Plan will include, but is not limited to: an inventory of all previously secured evidence and testimony; a determination of whether additional evidence needs to be secured, what witnesses need to be interviewed, including the complainant(s) and respondent, and other witnesses with knowledge of the scientific or events in question; a proposed schedule of meetings, briefing of experts, and interviews; anticipated analyses of evidence (scientific, forensic, or other); and a plan for the Investigation Report.

I. Reviewing the Evidence - The Investigation Committee will obtain and review all relevant documentation and perform or arrange to have carried out necessary analyses of the evidence, including scientific, forensic, statistical, or other analyses. The Investigation Committee will conform to the following guidelines:

The Investigation Committee will prepare carefully for each interview. All relevant documents and scientific data should be reviewed in advance and specific questions or issues that the committee wants to cover during the interview should be identified. The committee may appoint a committee member to take the lead on each interview. If significant questions or issues arise during an interview that require committee deliberation, the committee should take a short recess to discuss the issues. Committee deliberations must never be held in the presence of the interviewee.

At the Investigation stage, interviews should be in-depth and all significant witnesses should be interviewed. Each witness should have the opportunity to respond to inconsistencies between his or her testimony and the evidence or other testimony, subject to the need to take reasonable steps to maintain the confidentiality of the testimony. The Investigation Committee will conduct all interviews in a professional and objective manner. All interviews will be recorded, for later transcription as necessary, and summarized. A transcript or summary of the interview will be provided to each witness for review and correction of errors. Witnesses may add comments or additional information, but changes to the transcript or summary will only be made to correct errors in transcription.

During the course of the investigation, ORI should promptly be advised of any developments which disclose facts that may affect current or potential DHHS funding for individual(s) under investigation or that the PHS needs to know to ensure appropriate use of Federal Funds and otherwise protect the public interest. If there are immediate health hazards or if information suggests that the incident will be publicly reported, notification of ORI is necessary. If reasonable indication of possible criminal violations is found, ORI must be notified within 24 hours.

J. Committee Deliberations - In reaching a conclusion on whether there was scientific misconduct and who committed it, the burden of proof is on the institution to support its conclusions and findings by a preponderance of the evidence. The committee will consider whether there is sufficient evidence that the institution can meet its burden of proving misconduct by a preponderance of the evidence. The committee will also consider whether there is evidence of honest error or honest differences in interpretations or judgments of data, such that scientific misconduct cannot be proven by a preponderance of the evidence.

K. The Investigation Report - see Appendix B.

L. Comments on the Draft Report - The Chair of the Investigation Committee will provide the respondent(s) with a copy of the draft Investigation Report for comment. The respondent(s) will be allowed fourteen (14) calendar days to review and comment on the draft report. The findings of the final Report should take into account the respondent's comments in addition to all the other evidence. The respondent's comments will be attached to the final Report.

The Chair of the Investigation Committee will provide the complainant(s) with those portions of the draft Investigation Report that address the complainant's role and opinions in the Investigation. The complainant(s) will be allowed 14 calendar days to review and comment on the draft report. The complainant's comments will be added to the final Report. The Chair of Investigation Committee will inform the recipients of the confidentiality under which the draft Report is made available and establish appropriate conditions to ensure such confidentiality.

The ARILO may grant an extension for comments on the draft report based on a written request from the respondent(s) demonstrating good cause.

M. Transmittal of the Final Report - After comments have been received and the necessary changes have been made to the draft Report, the Investigation Committee will transmit the final Report with attachments, including the respondent's and complainant's comments, to the ARILO.

N. Time Limit for Completing the Report - The final Investigation Report will be submitted to the ARILO within 120 days (106 days for draft report plus 14 days for comments and revisions) of the first meeting of the Investigation Committee, unless the Investigation Committee submits a written request for extension and the ARILO grants the extension. A request for extension must also be submitted to ORI. This request should include an explanation for the delay, an interim report on the progress to date, an outline of what remains to be done, and an estimated date of completion.

O. CDC/ATSDR Review and Decision - The ARILO will review the Investigation report and issue a proposed determination whether to accept the Investigation Report, its findings, and the recommended CDC/ATSDR actions within seven (7) days of receiving it from the Investigation Committee. If this determination differs from that of the Investigation Committee, the ARILO will explain in detail the basis for rendering a decision different from that of the Investigation Committee. The ARILO's explanation should be consistent with the ORI definition of scientific misconduct, CDC/ATSDR's policies and procedures, and the evidence reviewed and analyzed by the Investigation Committee. The ARILO may also return the report to the Investigation Committee with a request for further fact-finding or analysis. The ARILO's determination, together with the Investigation Committee's Report, constitutes the final Investigation Report.

The ARILO will submit the final Investigation report to the Deciding Official who will review the report and make CDC/ATSDR’s final determination within fourteen (14) days. If this determination differs from that of the ARILO, the Deciding Official will explain in detail the basis for rendering a decision different from that of the ARILO.

When a final decision on the case has been reached by the Deciding Official, the ARILO will notify both the complainant(s) and the respondent(s) in writing. In addition, the ARILO, in consultation with appropriate CDC/ATSDR offices, will determine whether law enforcement agencies, professional societies, professional licensing boards, editors of journals in which falsified reports were published, collaborators of the respondent(s) in the work, or other relevant parties should be notified of the outcome of the case.

After completion of a case and all ensuing related actions, the ARILO will prepare a complete file, including the records of any Inquiry or Investigation and copies of all documents and other materials furnished to the ARILO or committees. The ARILO will keep the file for five years after completion of the case to permit later assessment of the case. Authorized DHHS personnel will be given access to the records upon request.

The ARILO will submit a report on the Investigation to ORI for review. ORI will have 240 days to review the report to determine whether the investigation was fair, objective, and competent. If a finding of misconduct was made, ORI will also review the finding of misconduct and supporting evidence, as well as any sanctions recommended. ORI will then forward its recommendations for sanctions to the HHS Assistant Secretary of Health (ASH), who will issue a final decision regarding the proposed sanctions within 60 days.

If the ASH makes a finding of scientific misconduct, the respondent(s) may request, within 30 days of receipt of the notification of findings, a hearing before the HHS Department Appeals Board.


VIII. ADMINISTRATIVE ACTIONS

A. Recommendations for Sanctions - If the Investigation Committee, in its report to the ARILO, includes a determination that scientific misconduct has occurred, the Investigation Committee will include recommendations to the ARILO, the appropriate CIO Director, and the Division of Commissioned Personnel for sanctions to be applied. A finding that a bad faith allegation has been made will be referred to the appropriate CIO Director and processed using the CDC/ATSDR Guide for Investigating Misconduct and Processing Disciplinary/Adverse Actions.

Although sanctions on a particular individual may have consequences that are much broader; i.e. members of a team may be indirectly or directly affected as well, sanctions imposed must be consistent with the nature of the violation and should not be weakened because of their impact on other individuals. There should be a logical correspondence between the nature and severity of the misconduct and the sanctions imposed. Consistency and fairness across CDC/ATSDR should be sought. Under no circumstances should a finding of scientific misconduct be treated as negotiable, or weakened because of the impact on other individuals or other matters not relevant to the finding.

B. Possible Sanctions - A recommendation for formal disciplinary action against a Federal employee for conducting scientific misconduct must be forwarded to the appropriate C/I/O director who is the official delegated authority to initiate discipline. The list of possible sanctions provides examples of possible sanctions that could be recommended by the Investigation Committee following a finding of scientific misconduct. The list is not comprehensive.

  • removal from a particular project
  • letter of reprimand
  • closer monitoring of work
  • probation
  • suspension without pay
  • denial of a raise in salary or salary/rank reduction
  • termination of employment

C. Protections/Appeals - Any individual subject to a sanction will be afforded all procedures that would ordinarily apply before that sanction goes into effect.

For Federal employees, some of these sanctions must be implemented under 5 C.F.R. Part 752. This regulation provides for formal procedures, such as advance written notice, right to reply, and right to representation before the agency takes adverse actions such as suspension, removal, or reduction in grade/pay. Adverse decisions may be grievable under the appropriate grievance procedure, or appealable to the U.S. Merit Systems Protection Board.

The Division of Commissioned Personnel (DCP) will be notified of any Public Health Service Commissioned Corps Officer identified as having committed scientific misconduct. Review and actions based on the CDC findings will be determined by DCP

For fellows and students, DHHS regulations governing CDC/ATSDR trainees state that CDC may terminate a trainee at any time if the trainee materially fails to comply with the terms and conditions of the training award. Scientific misconduct is a basis for termination. CDC/ATSDR policies for training positions generally require that these individuals must be given notice and an opportunity to comment prior to termination.

D. Possible Actions when a respondent(s) is Found Not Guilty - All individuals who knew of the allegation because they were involved in or knew of the Inquiry and Investigation (e.g., witnesses, attorneys, supervisors, lab members and colleagues of the defendant, journal editors who were complained to, etc.) should be formally notified of the exoneration. The exoneration could be publicized more broadly (without repeating the initial allegation), but only upon request of the respondent(s) (since publicity might do more harm than good from the respondent's perspective). The CIO should ensure that the respondent's position and responsibilities have, to the extent possible, been unaffected by the Investigation.


IX. REFERENCES

A. Final Rule, Public Health Service Policies on Research Misconduct – 42 CFR Part 93 – June 2005

B. Office of Research Integrity, 42USC289(b)

C. Policies of General Applicability, 42CFR Part 50

D. Record Keeping Procedures for Managing E-mails and Attachments That Qualify as Federal Records, Information Resources CDC-IR-2000-01

E. Authorship Policy, General Administration CDC-GA-2005-08

F. Clearance Of Information Products Distributed Outside CDC For Public Use, General Administration CDC-GA-2005-06


APPENDIX A

Flow Chart for Investigating Scientific Misconduct

Recommended Maximum Time Activity Persons Involved
     
7 Days Allegation  
     
Allegations of Scientific Misconduct Complainant
Assessment of Allegations ARILO
Outcome of Assessment ARILO
 
Inquiry
 
7 days Appointment of Inquiry Committee ARILO, SSCE
1 day Notification of respondent and sequestration of records ARILO, respondent, supervisor, CIO rep
60 days Inquiry meetings and interviews (53 days) ARILO, respondent, CDC Office of the General Counsel, AHRC rep, experts, witnesses, complainant
7 days Review of Inquiry Report (7 days) ARILO, CDC counsel, CIO rep, respondent, witnesses, complainant
7 days Decision and Notification ARILO
 
Time Activity Persons Involved
 
Investigation
 
1 day Notification of respondent and sequestration of additional records ARILO, respondent, supervisor, CIO rep
7 days Appointment of Investigation Committee ARILO, SSCE
120 days Investigation Committee meetings and Report (106 days) Investigation Comm., ARILO, CDC Office of the General Counsel, respondent, witnesses, experts, complainant
120 days Comments on Draft Report (14 days) Chair of Investigation Comm., respondent, complainant
120 days Transmittal of final Report Investigation Comm.
7 days ARILO Review ARILO
14 days CDC Decision Deciding Official
 
ORI Review, Department Decision
 
240 days Review of Investigation report ORI
60 days Issuance of Final Decision on Sanctions HHS Asst. Sec. Hlth.
 
Appeal
 
30 days Request for Appeals Hearing Respondent, HHS Appeals Board
 


APPENDIX B

Model Investigative Report

The Committee shall assemble a Report in the following fashion. A dissenting minority may submit a parallel report outlining the reasons for dissent. The following annotated outline may prove useful in preparing the required Investigation Report, except when special factors suggest a different approach.

I.   Background

Include sufficient background information to ensure a full understanding of the issues that concern the PHS under its definition of scientific misconduct. This section should detail the facts leading to the CDC/ATSDR Inquiry, including a description of the science at issue, the persons involved in the alleged misconduct, the role of the complainant, and any associated public health issues. All relevant dates should be included.

II. Allegations

List the allegations of scientific misconduct that were investigated and any additional scientific misconduct allegations that arose during the Inquiry and Investigation. The source and basis for each allegation should be cited. The allegations identified in this section will form the structure or context in which the subsequent analysis and findings are presented.

III. Inquiry: Process and Recommendations

A copy of the Inquiry Report is attached.

IV. Investigation: Process

Summarize the Investigation process, including the composition of the committee (names, degrees, departmental affiliation, and expertise), and the charge to the committee. List the persons interviewed, any additional evidence secured and reviewed and the measures taken to ensure its security, the policies and procedures used and any other factors that may have influenced the proceedings.

V. Investigation: Analysis of Each Allegation

Describe the particular matter in which the alleged misconduct occurred and why and how the issue came to be under investigation.

The analysis should take into account all the relevant statements, claims (e.g., a claim of a significant positive result in an experiment), rebuttals, documents, and other evidence, including circumstantial evidence, related to the issue. The source of each statement, claim, or other evidence should be cited (e.g., notebook with page and date, medical chart documents and dates, relevant manuscripts, transcripts of interviews, etc.).

Any use of additional expert analysis should be noted (informatics, forensic, statistical, or special analysis of the physical evidence, such as similarity of features or background in contested figures).

Relevant statements, including rebuttals, made by the complainant, respondent, and other pertinent witnesses should be summarized or quoted and the appropriate individuals referenced/cited. Each argument that the respondent(s) raised in his or her defense against the scientific misconduct allegation should be summarized and the source of each argument cited. Inconsistencies in the statements of the respondent, the complainant, other witnesses, or experts, should be noted.

The analysis should be consistent with the PHS definition of scientific misconduct. It should describe the relative weight given to the various witnesses and pieces of evidence, noting inconsistencies, credibility, and persuasiveness.

Any evidence that shows that the respondent(s) acted with intent, that is, any evidence that the respondent(s) knowingly engaged in the alleged falsification, fabrication, or plagiarism, should be described.

Similarly, the evidence supporting the possibility that honest error or differences of scientific opinion occurred with respect to the issue should be described.

VI. Conclusions/Findings of Misconduct or No Misconduct

The Investigation Committee's finding for each identified issue should be concisely stated. The Investigation Report should make separate findings as to whether or not each issue constitutes scientific misconduct, using the PHS definition.

A finding of scientific misconduct should be supported by a preponderance of the evidence. If the Investigation Committee finds scientific misconduct on one or more issues, the report should identify the type of misconduct for each issue (fabrication, falsification or plagiarism). The report should indicate the extent and seriousness of the fabrication, falsification, or plagiarism, including its effect on scientific findings, publications, scientific subjects, and the organizational unit or project in which the misconduct occurred.

VII. Recommended CDC/ATSDR Actions

Based on its findings, the Investigation Committee should recommend administrative actions that it believes the CDC/ATSDR should take consistent with its policies and procedures, including appropriate actions against the respondent, such as a letter of reprimand, special supervision, probation, termination, etc. (See Section VII. C.). The Investigation Committee should also identify any published scientific reports or other sources of scientific information (such as databases) that should be retracted or corrected.

If the Investigation Committee determines that the complainant(s) raised bad faith allegations, the Committee will recommend appropriate administrative actions that CDC/ATSDR should take consistent with its policies and procedures.

VIII. Documenting the Investigation File

The purpose of the documentation is to substantiate the Investigation's findings.

The Investigation Committee, with the assistance of the Office of General Counsel, should maintain an index of all the relevant evidence it secured or examined in conducting the Investigation, including any evidence that may support or contradict the report's conclusions. Evidence includes, but is not limited to, scientific records, transcripts or recordings of interviews, committee correspondence, administrative records, grant applications and awards, manuscripts, publications, and expert analyses.

IX. Attachments

Copies of all significant documentary evidence that is referenced in the report should be appended to the report, if possible (relevant notebook pages or other scientific records, relevant committee or expert analyses of data, transcripts or summary of each interview, complainant(s) and respondent(s) responses to the draft report(s), manuscripts, publications or other documents, including grant progress reports and applications, etc.). A "List of Attachments" should be included.

Allegedly false statements, misrepresentations in figures or parts of figures, areas of plagiarism, etc., on a copy of the page or section of the questioned document (e.g., a page from a scientific notebook) should be identified. A side-by-side comparison with the actual data or material that is alleged to have been plagiarized should be included.

 

  • Page last reviewed: April 15, 2009
  • Page last updated: April 13, 2009
  • Content source: Office of the Chief Science Officer
Contact Us:
  • Centers for Disease Control and Prevention
    1600 Clifton Rd
    Atlanta, GA 30333
  • 800-CDC-INFO
    (800-232-4636)
    TTY: (888) 232-6348
    24 Hours/Every Day
  • cdcinfo@cdc.gov
USA.gov: The U.S. Government's Official Web PortalDepartment of Health and Human Services
Centers for Disease Control and Prevention   1600 Clifton Rd. Atlanta, GA 30333, USA
800-CDC-INFO (800-232-4636) TTY: (888) 232-6348, 24 Hours/Every Day - cdcinfo@cdc.gov

A-Z Index

  1. A
  2. B
  3. C
  4. D
  5. E
  6. F
  7. G
  8. H
  9. I
  10. J
  11. K
  12. L
  13. M
  14. N
  15. O
  16. P
  17. Q
  18. R
  19. S
  20. T
  21. U
  22. V
  23. W
  24. X
  25. Y
  26. Z
  27. #