Guidelines for Ensuring the Quality of Information Disseminated to the Public
D. � Centers for Disease Control and Prevention and Agency for Toxic Substances and Disease Registry
I. � Agency Mission
The Centers for Disease Control and Prevention
(CDC) and the Agency for Toxic Substances and Disease Registry (ATSDR) are
two of the operating components of the HHS. CDC has remained at the forefront
of public health efforts to prevent and control infectious and chronic diseases,
injuries, workplace hazards, disabilities, and environmental and occupational
health threats for more than 50 years. CDC is the lead federal agency for
protecting the health and safety of people -- at home and abroad, providing
credible information to enhance health decisions, and promoting health through
strong partnerships.
CDC seeks to accomplish its mission by working with partners throughout
the nation and world to monitor health, detect and investigate health problems,
conduct research to enhance prevention, develop and advocate sound public
health policies, implement prevention strategies and programs, promote healthy
behaviors, foster safe and healthful environments, and provide leadership
and training. CDC has developed and sustained many vital partnerships with public
and private entities that improve service to the American people. In FY 2000,
the workforce of CDC comprised approximately 8,500 FTE in 170 disciplines
with a public health focus. Although CDC's national headquarters is in Atlanta,
Georgia, more than 2,000 CDC employees work at other locations nationwide
including virtually all States. Approximately 160 are assigned overseas in
45 countries. In addition, CDC is comprised of 12 Centers, Institutes, and
Offices (CIOs). These organizational components, listed below, respond individually
in their areas of expertise and pool their resources and expertise on cross-cutting
issues and specific health threats.
- National Center on Birth Defects and Developmental Disabilities
- National Center for Chronic Disease Prevention and Health Promotion
- National Center for Environmental Health
- National Center for Health Statistics
- National Center for HIV, STD, and TB Prevention
- National Center for Infectious Diseases
- National Center for Injury Prevention and Control
- National Immunization Program
- National Institute for Occupational Safety and Health
- Epidemiology Program Office
- Public Health Practice Program Office
- Office of the Director
ATSDR was established in 1980 by the Comprehensive Environmental Response,
Compensation, and Liability Act, also known as Superfund. ATSDR works to
prevent exposures to hazardous wastes and to environmental releases of hazardous
substances. Working with States and other Federal agencies, ATSDR seeks to
prevent exposure and adverse health effects associated with exposure to hazardous
substances from waste sites. The agency conducts public health assessments,
health studies, surveillance activities and health education training in
communities around waste sites or exposed to environmental releases. ATSDR
also develops toxicological profiles of hazardous chemicals found at these
sites. The agency has 10 regional offices and an office in Washington, DC,
and a staff of about 400 persons. Although CDC and ATSDR are separate agencies, both strive to protect
and improve the health of the American public. The Director of CDC also serves
as the Administrator of ATSDR. Unless otherwise specified, all subsequent references to CDC also
include ATSDR and all practices and procedures described in this document
apply to both agencies.
II. � Scope and Applicability of Guidelines for CDC
CDC will ensure that disseminated information meets
the standards of quality set forth in the OMB, HHS and CDC guidelines. It is
CDC's policy to ensure and maximize the quality, objectivity, utility, and integrity
of information that it disseminates to the public. We strive to provide information
that is accurate, reliable, clear, complete, unbiased, and useful. We are committed
to integrating the principle of information quality into every phase of information
development, including creation, collection, maintenance, and dissemination.
CDC guidelines do not apply to the National Center for Health Statistics (NCHS). While NCHS is a component
of CDC, NCHS is the nation's principal health statistics agency and as such
has separate guidelines.
The pre-dissemination review described in the guidelines only applies
to information disseminated on or after October 1, 2002. The administrative
mechanism for correction applies to information that the agency disseminates
on or after October 1, 2002, regardless of when the agency first disseminated
the information.
The guidelines apply to information in all media -- print, electronic,
audiovisual, and oral. They apply to substantive information, such as studies
and reports, rather than to information pertaining to basic agency operations.
Information that is disseminated at the request of CDC or with specific CDC
approval through a contract, a grant, or a cooperative agreement is subject
to these guidelines.
Examples are provided below of the types of information that the CDC considers within and outside the scope of the guidelines.
- Covered Information
Scientific research papers, books, journal articles, reports, and similar
materials, unless they have disclaimers to distinguish the research from
CDC views and positions;
Other official reports, brochures, documents, newsletters, and audiovisual products;
Oral information, including speeches, interviews, expert opinions
only if representing CDC's views, official positions, or policies;
Statistical information - statistical analyses, aggregated information by programs.
- Information Not Covered
Documents not authored by CDC (either directly or by contract) and
not representing official views, including research and science supported
by CDC funding;
Opinions where the presentation makes it clear that what is being offered is personal opinion rather than fact or CDC's views;
Archival information disseminated by CDC (for example, Internet distribution of published articles);
Information dissemination limited to government employees or agency contractors or grantees;
Information intended solely for intra- or inter-agency use or sharing
of government information, such as evaluation of a specific public health
program to assess the success in achieving its objectives, technical assistance
reports, training materials, manuals;
Information intended to be limited to public filings, subpoenas, or adjudicative processes;
Press releases that support the announcement or give public notice of information that CDC has disseminated elsewhere.
III. � Types of Information Disseminated by CDC to the Public
Annually,
CDC produces hundreds of publications of various types and provides over
100,000 pages of Web content for access by the public. All publications that
carry the CDC logo are considered official publications or releases, and
must follow CDC policy and procedures for preparation, review, approval,
and distribution (www.cdc.gov/od/foia/policies/clearance.htm).
Examples of the types of information disseminated by CDC to the public
are listed below. Some types fit into more than one category and are mentioned
in each.
- Scientific research studies.
CDC encourages professional dissemination of scientific research by employees
and those funded by CDC to conduct research. These research studies may be
published by CDC, such as the Morbidity and Mortality Weekly Report
(MMWR) or non-CDC publications including journals, books, chapters, editorials,
reviews, proceedings or abstracts. These are usually authored by or co-authored
by CDC staff scientists as part of their official duties or may be authored
by CDC partners, CDC advisory committees, or working groups convened by CDC.
- Statistical products
CDC releases data sets and disseminates statistical reports produced
by its data collection programs. These include vital statistics, population-based
health surveys, and surveys of health care providers.
- Programmatic and administrative information.
CDC disseminates community health assessments and information in
connection with and as a byproduct of the administration of programs, such
as Program-in-Brief documents, At-A-Glance documents, and program brochures.
- Authoritative health, medical and human services information aimed at consumers and health and human services professionals.
CDC publishes the MMWR which includes Recommendations and Reports.
CDC generates Health Alerts, Public Health Advisories, and guidelines for
dealing with specific public health threats. CDC also provides the website
Travelers' Health, which publishes guidelines for international travelers
including the "Yellow Book" and official expert opinions. CDC produces and
broadcasts science educational materials and training modules, including
Public Health Grand Rounds Satellite broadcasts, Web-assisted Audio Conferences
for State and Local Health Policymakers, and the Health Training Network
Satellite Broadcast.
- Public heath surveillance, and epidemiology information.
CDC publishes the MMWR Summary of Notifiable Diseases and CDC Surveillance Summaries,
and other surveillance summaries on a variety of infectious diseases such
as HIV/AIDS and tuberculosis, as well as other non-infectious conditions
such as Birth Defects Surveillance, National Oral Health Surveillance, Pediatric
Nutrition Surveillance, Pregnancy Nutrition Surveillance, Hazardous Substance
Release/Health Effects Database, Flu Bulletin, Influenza Season Reports and
Occupational Morbidity and Mortality Surveillance, Adult Blood Lead Epidemiology
and Surveillance, Coal Workers X-ray Surveillance Program, National Surveillance
System of Pneumoconiosis Mortality, National Traumatic Occupational Fatalities
Surveillance System. In addition CDC publishes outbreak investigations or
other items reported in the MMWR that are not authoritative or urgent. ATSDR
disseminates information products including Public Health Assessments, Public
Health Consultations, Fact Sheets, health study reports, Toxicological Profiles,
Case Studies in Environmental Medicine, and Hazardous Substances and Public
Health (newsletter).
IV. � Types of Dissemination Methods
CDC disseminates information through a wide range of methods, often using more than one medium for the same information.
- Print,
including publications in peer-reviewed literature, published reports, periodicals, brochures, books, and correspondence;
- Electronic,
such as the CDC Website, CD -ROM, Listserv, e-mail, automated voice and fax systems, hotlines and clearinghouses;
- Audiovisual,
broadcast scripts, audio or videotapes, and
videocasting. CDC's Public Health Training Network makes satellite broadcasts
and Webcasts available nationally.
- Oral,
formal speeches, oral presentations, and interviews, or commentaries for publication or broadcast.
V. � Agency Quality Assurance Policies, Standards and Processes for Ensuring
the Quality of Information Dissemination to the Public.
- Overview
CDC's policies and procedures are designed to ensure and maximize the
quality of its information products with regard to their utility, objectivity,
and integrity. The agency's quality assurance process begins at the inception
of the information development process. CDC has guidelines to address the
general principles concerning the responsibilities of the CDC staff in the
collection and recording of data, publication practices, authorship determination,
peer review, confidentiality of information, collaborations, and human subjects
research. Authorship issues and review and clearance procedures are set forth
in the "Authorship of CDC Publications and the Clearance Procedures for Scientific
and Technical Documents" (www.cdc.gov/od/foia/policies/clearance.htm).
CDC reviews the quality (including the objectivity, utility, and
integrity) of information before it is disseminated and treats information
quality as integral to every step of the development of information, including
its creation, collection, maintenance and dissemination. Further, CDC is
committed to demonstrating in its Paperwork Reduction Act clearance packages
that each draft information collection will result in information that will
be collected, maintained, and used in a way that is consistent with OMB,
HHS and CDC information quality guidelines. The individual CIO Associate
Directors for Science (ADS) or designee are responsible for assuring the
quality of information disseminated by CDC and that the quality assurance
methods and procedures described in Overview of Quality Assurance Policies
and Practices in HHS are met. To meet the standards for external merit review
of research and scientific studies and intramural research programs, CDC
policy is to peer review extramural research and intramural research studies
and programs.
The CIO ADS or designee are responsible for clearance of documents
originating in that CIO before dissemination and for ensuring that the necessary
clearances are obtained and that written material distributed is appropriate
and consistent with HHS policy. While each CIO can determine preparation,
review and approval procedures, all must meet standards provided by the ADS,
CDC and those provided in the HHS Part I Overview D.4.d.
- Utility
CDC addresses utility, a measure of the usefulness of information products
to its intended users, by staying informed of user needs through information
product research and user needs assessment, user feedback, consultation with
advisory committees, and conference participation.
- Objectivity
CDC provides assurance that information is accurate, reliable, and unbiased.
Objectivity is achieved through existing review and clearance procedures
and, in many cases, the peer review of disseminated information.
- Integrity
CDC assures the integrity of its data and information products through
the enforcement of rigorous controls that protect against unauthorized access,
revision, or corruption. Some of the controls used at CDC include access
control, user authentication, encryption, access monitoring, provision of
unalterable electronic content, and audit trails.
- CDC Information Review and Approval Policies and Procedures by Type of Information
- Health and Public Health Information
- Scientific research studies
CDC encourages professional dissemination of scientific research and
other information by its employees. Publications or presentations by CDC
employees are expected to meet high standards of quality, make a substantial
contribution to the field, and contain sufficient information for the informed
audience to assess its validity. Publication of scientific information by
individual employees must undergo a formal review and clearance process by
the CIO ADS or designee before dissemination. This review includes the evaluation
of data collection measures for completeness, accuracy and timeliness, data
management and analysis, clarity and accuracy of presentation, and validity
of interpretation of findings.
Oral presentations undergo appropriate supervisory review. Laboratory
data are reviewed to assure that good laboratory data practice was followed
for sampling, methodology, instrumentation and analysis.
Intramural research programs will be subject to review and monitoring
by external, objective peer review through an advisory committee or board
of scientific counselors. Scientific research studies submitted to journals
are subject to peer review of methods and findings by the journal prior to
publication. ATSDR has a mandated policy for external peer review of all
intramural and extramural research study protocols and findings prior to
public dissemination.
- Authoritative health, medical and human services information aimed at consumers and health and human services professionals
CDC disseminates authoritative health and medical information routinely
as part of its mission. As an example, articles or reports for publication
in the MMWR are subject to routine CDC review and approval procedures in
the originating CIO. Because information disseminated in the MMWR often has
impact on the practice of public health, the CDC ADS must also review and
approve it. Health Alerts related to bioterrorism that are disseminated by
CDC are also reviewed and approved at the CDC ADS level prior to release.
- Public heath surveillance and epidemiology information
CDC often obtains surveillance information from third parties, such
as States, grantees, or community-based organizations. Reliance on third
parties places limits on CDC's quality assurance, although the accuracy,
completeness and timeliness of the information are subject to sample audits,
site visits, and an evaluation for completeness and consistency with trends
and external controls. The MMWR Summary of Notifiable Diseases,
for example, depends on data reported from States. CDC conducts audits and
checks for consistency for trends before reporting these data. ATSDR produces
Toxicological Profiles for hazardous substances found at National Priorities
List sites as well as other documents that undergo public comment periods
before being finalized and distributed. The Toxicological Profiles and other
ATSDR documents are first produced as drafts and are then subject to public
comments following announcement in the Federal Register and using other means.
Only after considering the comments, the profiles and documents are finalized
and then distributed to the public.
ATSDR has a government to government policy on Tribal Nations that
specifies how the agency works with and respects Tribal rights, sovereignty,
and culture. Data or information collected from American Indian/Alaska Native
communities requires approval from the Tribal government and direct involvement
in the research or study from concept to completion. The Tribe reserves the
right to review and critique the design and findings. Issues of release and
ownership of data, information or other products must be agreed to by the
Tribal government. Close collaboration and involvement of the Tribe is essential
to ensure quality, utility, objectivity and integrity of information prior
to being disseminated.
- Statistical products
CDC routinely employs a number of widely accepted methods and procedures
for ensuring quality, including independent assessments of statistical methodologies,
peer reviews, and observance of professional standards. To insure the utility
of CDC statistical and analytic information products, CDC conducts independent
research and consults experts in areas such as data collection, data analysis
and a variety of substantive topics and areas. Additionally, CDC maintains
ongoing contact with users, and participates in conferences, and workshops
in order to objectively assess and identify the current and future data needs
of CDC's constituents. Further, CDC employs a wide variety of dissemination
mechanisms to make its statistical and analytic information products widely
available and broadly accessible.
To assure that statistical and analytic information products are
accurate, reliable, and unbiased, CDC obtains these data through generally
accepted statistical theory and practice. Dissemination of data also follows
generally recognized guidelines in terms of defining acceptable standards
regarding minimum response rates, maximum standard errors, cell size suppression,
quality of coding and other processing operations. CDC also maintains staff
expertise in areas such as concept development, survey planning and design,
data collection, data processing and editing, data analysis, evaluation procedures,
and methods of data dissemination.
All CDC statistical and analytic information products undergo a
formal clearance process before dissemination. Publications and reports,
whether in electronic or paper form, are reviewed by a CIO ADS or designee.
These reviews cover the clarity of descriptive text, the appropriateness
of the methodology, the soundness of the analysis, the adherence to confidentiality
and disclosure avoidance restrictions, the readability of tabular and graphic
presentations of data. Finally, all products undergo editorial review, (e.g.,
formatting, proofreading, spell checks, proper punctuation). Oral presentations
undergo appropriate supervisory review. The CIO ADS or designee may also
review for programmatic and policy implications on behalf of and in consultation
with other division or senior staff. In addition, all public-use tapes are
reviewed by the CIO ADS or designee for accuracy and appropriate confidentiality
protections.
CDC statistical and analytic information products are derived using
generally acceptable statistical practices and methodologies which are clearly
documented and available to the public. These procedures enable responsible
statisticians and analysts outside of CDC to replicate CDC's statistical
methods and obtain results consistent with those obtained by CDC.
VI. � Agency Administrative Complaint Procedures
CDC has developed
administrative mechanisms to allow affected persons to seek and obtain correction
of disseminated information that does not comply with OMB, HHS and CDC guidelines.
CDC will establish a Website to advise information consumers of the
agency's information quality guidelines, the process to submit a complaint,
information needed by the complainant, and a description of the complaint
adjudication process. CDC will centralize the initial receipt, logging, and
tracking of all complaints received under this provision in the Management
Analysis and Services Office (MASO), Office of Program Services. Complaints
will be forwarded to the office that has subject matter responsibility for
the information product in question.
VII. � Influential Scientific, Financial and Statistical Information
CDC considers the information disseminated in the MMWR Recommendations and Reports,
the Hazardous Substance Release/Health Effects Database, Toxicological Profiles,
ATSDR Public Health Assessments, and Federal Register publications related
to science as influential scientific information.
Risk Assessment
Some of the influential information that we disseminate is based
on an analysis of the risks to the public of certain actions or exposures
to hazardous substances. For purposes of this guidance, we are defining risk
as the likelihood that injury or damage is or can be caused by a substance,
technology, or activity. We use risk analysis (the integration of risk assessment
with risk management and risk communication) as a tool to enhance the scientific
basis for all of our regulatory decisions.
The OMB Guidelines provide special considerations that must be taken
into account in certain risk assessments, those that provide the basis for
the dissemination of influential information. The guidelines state that "With
regard to analysis of risks to human health, safety, and the environment
maintained or disseminated by the agencies, agencies shall either adopt or
adapt the quality principles applied by Congress to risk information used
and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996
(SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B))."
The SDWA risk assessment principles are as follows:
- To the degree that the agency action is based on science, the agency shall use
- the best available, peer-reviewed science and supporting studies conducted
in accordance with sound and objective scientific practices
- data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
- In the dissemination of public information about risks, the
agency shall ensure that the presentation of information about risk effects
is comprehensive, informative, and understandable.
- In a document made available to the public in support of a regulation, the agency shall specify, to the extent practicable
- Each population addressed by any estimate of applicable risk effects
- The expected risk or central estimate of risk for the specific populations affected
- Each appropriate upper-bound or lower-bound estimate of risk
- Each significant uncertainty identified in the process of the
assessment of risk effects and the studies that would assist in resolving
the uncertainty and
- Peer-reviewed studies known to the agency that support, are
directly relevant to, or fail to support any estimate of risk effects and
the methodology used to reconcile the inconsistencies in the scientific data
Many of our actions are based on scientific experts' judgments using
available data, are essentially qualitative and do not lend themselves to
the types of quantitative risk assessments contemplated by the SDWA principles.
As a result, we have adapted the general principles for risk assessments
from the SDWA to fit these situations.
- The agency will use
- the best available science and supporting studies conducted in accordance
with sound and objective scientific practices, including peer-reviewed science
and supporting studies when available
- data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
- In the dissemination of public information about risks, the
agency will ensure that the presentation of information about risk effects
is comprehensive, informative, and understandable.
In situations where a quantitative risk assessment is conducted, we generally
follow basic risk assessment principles in the NAS paradigm of 1983. Our
needs for quantitative risk assessments range over a wide variety of hazards
including physical hazards encountered during exposure to toxic substances
and antimicrobial resistance to antibiotic therapy. Thus, we also ascribe
to the statement from NAS when it revisited the risk assessment process in
1994 (Science and Judgment in Risk Assessment,
NAS 1994): "Risk assessment is not a single process, but a systematic approach
to organizing and analyzing scientific knowledge and information." In each
of the areas we regulate, we apply risk assessment practices to the specific
task that are widely accepted among relevant domestic and international public
health agencies. For quantitative risk assessments in support of the dissemination
of influential information, CDC intends to apply the following principles:
- The agency will use
- the best available science and supporting studies conducted in accordance
with sound and objective scientific practices, including peer-reviewed science
and supporting studies when available.
- data collected by accepted methods (if reliability of the method and the nature of the decision justifies use of the data).
- In the dissemination of public information about health risks,
the agency shall ensure that the presentation of information is comprehensive,
informative, and understandable, within the context of its intended purpose.
- In a document made available to the public, the agency shall specify, to the extent practicable-
- Each population addressed by any estimate of applicable effects;
- The expected or central estimate of risk for the specific populations affected;
- Each appropriate upper-bound and/or lower-bound risk estimates;
- Data gaps and other significant uncertainties identified in
the process of the risk assessment and the studies that would assist in reducing
the uncertaintaies; and
- Additional studies not used in the risk assessment that support
or fail to support the findings of the assessment and the rationale of why
they were not used.
VIII. � Special Considerations for Agency Dissemination
Special
consideration also applies to information products that are urgent in nature
and because of the potential risk to human health and safety, certain information
products may be disseminated in an expedited manner without having fully
complied with all normal quality guidelines; however, basic quality principles
and processes will still apply and be followed.
Last revised: November 12, 2003
|