Guidelines for Ensuring the Quality of Information Disseminated to the Public
D. Centers for Disease Control and Prevention and
Agency for Toxic Substances and Disease Registry
[ Full Contents of Guidelines ]
-
Agency Mission
-
Scope and Applicability of Guidelines for CDC
-
Covered Information
-
Information Not Covered
-
Types of Information Disseminated by CDC to the Public
-
Scientific research studies
-
Statistical Products
-
Programmatic and administrative information
-
Authoritative health, medical and human services information
aimed at consumers and health and human services professionals
-
Public heath surveillance and epidemiology information
-
Types of Dissemination Methods
-
Print
-
Electronic
-
Audiovisual
-
Oral
-
Agency Quality Assurance Policies, Standards and Processes for
Ensuring the Quality of Information Dissemination to the Public
-
Overview
-
CDC Information Review and Approval Policies and Procedures
by Type of Information
-
Agency Administrative Complaint Procedures
-
Responsibility of the Complainant
-
CDC/ATSDR Responsibility
-
Appeals
-
Influential Scientific, Financial and Statistical
Information
-
Special Considerations for Agency Dissemination
The Centers for Disease Control and Prevention (CDC) and the Agency for Toxic
Substances and Disease Registry (ATSDR) are two of the operating components
of the HHS. CDC has remained at the forefront of public health efforts to
prevent and control infectious and chronic diseases, injuries, workplace
hazards, disabilities, and environmental and occupational health threats
for more than 50 years. CDC is the lead federal agency for protecting the
health and safety of people at home and abroad, providing credible
information to enhance health decisions, and promoting health through strong
partnerships.
CDC seeks to accomplish its mission by working with partners throughout the
nation and world to monitor health, detect and investigate health problems,
conduct research to enhance prevention, develop and advocate sound public
health policies, implement prevention strategies and programs, promote healthy
behaviors, foster safe and healthful environments, and provide leadership
and training.
CDC has developed and sustained many vital partnerships with public and private
entities that improve service to the American people. In FY 2000, the workforce
of CDC comprised approximately 8,500 FTE in 170 disciplines with a public
health focus. Although CDC's national headquarters is in Atlanta, Georgia,
more than 2,000 CDC employees work at other locations nationwide including
virtually all States. Approximately 160 are assigned overseas in 45 countries.
In addition, CDC is comprised of 12 Centers, Institutes, and Offices (CIOs).
These organizational components, listed below, respond individually in their
areas of expertise and pool their resources and expertise on cross-cutting
issues and specific health threats.
-
National Center on Birth Defects and Developmental Disabilities
-
National Center for Chronic Disease Prevention and Health Promotion
-
National Center for Environmental Health
-
National Center for Health Statistics
-
National Center for HIV, STD, and TB Prevention
-
National Center for Infectious Diseases
-
National Center for Injury Prevention and Control
-
National Immunization Program
-
National Institute for Occupational Safety and Health
-
Epidemiology Program Office
-
Public Health Practice Program Office
-
Office of the Director
ATSDR was established in 1980 by the Comprehensive Environmental Response,
Compensation, and Liability Act, also known as Superfund. ATSDR works to
prevent exposures to hazardous wastes and to environmental releases of hazardous
substances. Working with States and other Federal agencies, ATSDR seeks to
prevent exposure and adverse health effects associated with exposure to hazardous
substances from waste sites. The agency conducts public health assessments,
health studies, surveillance activities and health education training in
communities around waste sites or exposed to environmental releases. ATSDR
also develops toxicological profiles of hazardous chemicals found at these
sites. The agency has 10 regional offices and an office in Washington, DC,
and a staff of about 400 persons.
Although CDC and ATSDR are separate agencies, both strive to protect and
improve the health of the American public. The Director of CDC also serves
as the Administrator of ATSDR.
Unless otherwise specified, all subsequent references to CDC also include
ATSDR and all practices and procedures described in this document apply to
both agencies.
[ Go to Contents ]
CDC will ensure that disseminated information meets the standards of quality
set forth in the OMB, HHS and CDC guidelines. It is CDC's policy to ensure
and maximize the quality, objectivity, utility, and integrity of information
that it disseminates to the public. We strive to provide information that
is accurate, reliable, clear, complete, unbiased, and useful. We are committed
to integrating the principle of information quality into every phase of
information development, including creation, collection, maintenance, and
dissemination. CDC guidelines do not apply to the National Center for Health
Statistics (NCHS). While NCHS is a component of CDC, NCHS is the nation's
principal health statistics agency and as such has separate guidelines.
The pre-dissemination review described in the guidelines only applies to
information disseminated on or after October 1, 2002. The administrative
mechanism for correction applies to information that the agency disseminates
on or after October 1, 2002, regardless of when the agency first disseminated
the information.
The guidelines apply to information in all media print, electronic,
audiovisual, and oral. They apply to substantive information, such as studies
and reports, rather than to information pertaining to basic agency operations.
Information that is disseminated at the request of CDC or with specific CDC
approval through a contract, a grant, or a cooperative agreement is subject
to these guidelines.
Examples are provided below of the types of information that the CDC considers
within and outside the scope of the guidelines.
-
Covered Information
-
Scientific research papers, books, journal articles, reports, and similar
materials, unless they have disclaimers to distinguish the research from
CDC views and positions;
-
Other official reports, brochures, documents, newsletters, and audiovisual
products;
-
Oral information, including speeches, interviews, expert opinions only if
representing CDC's views, official positions, or policies;
-
Statistical information - statistical analyses, aggregated information by
programs.
-
Information Not Covered
-
Documents not authored by CDC (either directly or by contract) and not
representing official views, including research and science supported by
CDC funding;
-
Opinions where the presentation makes it clear that what is being offered
is personal opinion rather than fact or CDC's views;
-
Archival information disseminated by CDC (for example, Internet distribution
of published articles);
-
Information dissemination limited to government employees or agency contractors
or grantees;
-
Information intended solely for intra- or inter-agency use or sharing of
government information, such as evaluation of a specific public health program
to assess the success in achieving its objectives, technical assistance reports,
training materials, manuals;
-
Information intended to be limited to public filings, subpoenas, or adjudicative
processes;
-
Press releases that support the announcement or give public notice of information
that CDC has disseminated elsewhere.
[ Go to Contents ]
Annually, CDC produces hundreds of publications of various types and provides
over 100,000 pages of Web content for access by the public. All publications
that carry the CDC logo are considered official publications or releases,
and must follow CDC policy and procedures for preparation, review, approval,
and distribution
(www.cdc.gov/od/foia/policies/clearance.htm).
Examples of the types of information disseminated by CDC to the public are
listed below. Some types fit into more than one category and are mentioned
in each.
-
Scientific research studies.
CDC encourages professional dissemination of scientific research by employees
and those funded by CDC to conduct research. These research studies may be
published by CDC, such as the Morbidity and Mortality Weekly Report
(MMWR) or non-CDC publications including journals, books, chapters, editorials,
reviews, proceedings or abstracts. These are usually authored by or co-authored
by CDC staff scientists as part of their official duties or may be authored
by CDC partners, CDC advisory committees, or working groups convened by CDC.
-
Statistical products
CDC releases data sets and disseminates statistical reports produced by its
data collection programs. These include vital statistics, population-based
health surveys, and surveys of health care providers.
-
Programmatic and administrative information.
CDC disseminates community health assessments and information in connection
with and as a byproduct of the administration of programs, such as
Program-in-Brief documents, At-A-Glance documents, and program brochures.
-
Authoritative health, medical and human services information aimed at
consumers and health and human services professionals.
CDC publishes the MMWR which includes Recommendations and
Reports. CDC generates Health Alerts, Public Health Advisories, and
guidelines for dealing with specific public health threats. CDC also provides
the website Travelers' Health, which publishes guidelines for
international travelers including the "Yellow Book" and official expert opinions.
CDC produces and broadcasts science educational materials and training modules,
including Public Health Grand Rounds Satellite broadcasts, Web-assisted Audio
Conferences for State and Local Health Policymakers, and the Health Training
Network Satellite Broadcast.
-
Public heath surveillance, and epidemiology information.
CDC publishes the MMWR Summary of Notifiable Diseases and CDC
Surveillance Summaries, and other surveillance summaries on a variety
of infectious diseases such as HIV/AIDS and tuberculosis, as well as other
non-infectious conditions such as Birth Defects Surveillance, National Oral
Health Surveillance, Pediatric Nutrition Surveillance, Pregnancy Nutrition
Surveillance, Hazardous Substance Release/Health Effects Database, Flu Bulletin,
Influenza Season Reports and Occupational Morbidity and Mortality Surveillance,
Adult Blood Lead Epidemiology and Surveillance, Coal Workers X-ray Surveillance
Program, National Surveillance System of Pneumoconiosis Mortality, National
Traumatic Occupational Fatalities Surveillance System. In addition CDC publishes
outbreak investigations or other items reported in the MMWR that are not
authoritative or urgent. ATSDR disseminates information products including
Public Health Assessments, Public Health Consultations, Fact Sheets, health
study reports, Toxicological Profiles, Case Studies in Environmental Medicine,
and Hazardous Substances and Public Health (newsletter).
[ Go to Contents ]
CDC disseminates information through a wide range of methods, often using
more than one medium for the same information.
-
Print
,
including publications in peer-reviewed literature, published reports,
periodicals, brochures, books, and correspondence;
-
Electronic
,
such as the CDC Website, CD -ROM, Listserv, e-mail, automated voice and fax
systems, hotlines and clearinghouses;
-
Audiovisual
,
broadcast scripts, audio or videotapes, and videocasting. CDC's Public Health
Training Network makes satellite broadcasts and Webcasts available nationally.
-
Oral
,
formal speeches, oral presentations, and interviews, or commentaries for
publication or broadcast.
[ Go to Contents ]
-
Overview
CDC's policies and procedures are designed to ensure and maximize the quality
of its information products with regard to their utility, objectivity, and
integrity. The agency's quality assurance process begins at the inception
of the information development process. CDC has guidelines to address the
general principles concerning the responsibilities of the CDC staff in the
collection and recording of data, publication practices, authorship
determination, peer review, confidentiality of information, collaborations,
and human subjects research. Authorship issues and review and clearance
procedures are set forth in the "Authorship of CDC Publications and the Clearance
Procedures for Scientific and Technical Documents"
(www.cdc.gov/od/foia/policies/clearance.htm).
CDC reviews the quality (including the objectivity, utility, and integrity)
of information before it is disseminated and treats information quality as
integral to every step of the development of information, including its creation,
collection, maintenance and dissemination. Further, CDC is committed to
demonstrating in its Paperwork Reduction Act clearance packages that each
draft information collection will result in information that will be collected,
maintained, and used in a way that is consistent with OMB, HHS and CDC
information quality guidelines. The individual CIO Associate Directors for
Science (ADS) or designee are responsible for assuring the quality of information
disseminated by CDC and that the quality assurance methods and procedures
described in Overview of Quality Assurance Policies and Practices in HHS
are met. To meet the standards for external merit review of research and
scientific studies and intramural research programs, CDC policy is to peer
review extramural research and intramural research studies and programs.
The CIO ADS or designee are responsible for clearance of documents originating
in that CIO before dissemination and for ensuring that the necessary clearances
are obtained and that written material distributed is appropriate and consistent
with HHS policy. While each CIO can determine preparation, review and approval
procedures, all must meet standards provided by the ADS, CDC and those provided
in the HHS Part I Overview D.4.d.
-
Utility CDC addresses utility, a measure of the usefulness of information
products to its intended users, by staying informed of user needs through
information product research and user needs assessment, user feedback,
consultation with advisory committees, and conference participation.
-
Objectivity CDC provides assurance that information is accurate, reliable,
and unbiased. Objectivity is achieved through existing review and clearance
procedures and, in many cases, the peer review of disseminated information.
-
Integrity CDC assures the integrity of its data and information products
through the enforcement of rigorous controls that protect against unauthorized
access, revision, or corruption. Some of the controls used at CDC include
access control, user authentication, encryption, access monitoring, provision
of unalterable electronic content, and audit trails.
-
CDC Information Review and Approval Policies and Procedures
by Type of Information
-
Health and Public Health Information
-
Scientific research studies
CDC encourages professional dissemination of scientific research and other
information by its employees. Publications or presentations by CDC employees
are expected to meet high standards of quality, make a substantial contribution
to the field, and contain sufficient information for the informed audience
to assess its validity. Publication of scientific information by individual
employees must undergo a formal review and clearance process by the CIO ADS
or designee before dissemination. This review includes the evaluation of
data collection measures for completeness, accuracy and timeliness, data
management and analysis, clarity and accuracy of presentation, and validity
of interpretation of findings.
Oral presentations undergo appropriate supervisory review. Laboratory data
are reviewed to assure that good laboratory data practice was followed for
sampling, methodology, instrumentation and analysis.
Intramural research programs will be subject to review and monitoring by
external, objective peer review through an advisory committee or board of
scientific counselors. Scientific research studies submitted to journals
are subject to peer review of methods and findings by the journal prior to
publication. ATSDR has a mandated policy for external peer review of all
intramural and extramural research study protocols and findings prior to
public dissemination.
-
Authoritative health, medical and human services information aimed at
consumers and health and human services professionals
CDC disseminates authoritative health and medical information routinely as
part of its mission. As an example, articles or reports for publication in
the MMWR are subject to routine CDC review and approval procedures in the
originating CIO. Because information disseminated in the MMWR often has impact
on the practice of public health, the CDC ADS must also review and approve
it. Health Alerts related to bioterrorism that are disseminated by CDC are
also reviewed and approved at the CDC ADS level prior to release.
-
Public heath surveillance and epidemiology information
CDC often obtains surveillance information from third parties, such as States,
grantees, or community-based organizations. Reliance on third parties places
limits on CDC's quality assurance, although the accuracy, completeness and
timeliness of the information are subject to sample audits, site visits,
and an evaluation for completeness and consistency with trends and external
controls. The MMWR Summary of Notifiable Diseases, for example, depends
on data reported from States. CDC conducts audits and checks for consistency
for trends before reporting these data. ATSDR produces Toxicological Profiles
for hazardous substances found at National Priorities List sites as well
as other documents that undergo public comment periods before being finalized
and distributed. The Toxicological Profiles and other ATSDR documents are
first produced as drafts and are then subject to public comments following
announcement in the Federal Register and using other means. Only after
considering the comments, the profiles and documents are finalized and then
distributed to the public.
ATSDR has a government to government policy on Tribal Nations that specifies
how the agency works with and respects Tribal rights, sovereignty, and culture.
Data or information collected from American Indian/Alaska Native communities
requires approval from the Tribal government and direct involvement in the
research or study from concept to completion. The Tribe reserves the right
to review and critique the design and findings. Issues of release and ownership
of data, information or other products must be agreed to by the Tribal
government. Close collaboration and involvement of the Tribe is essential
to ensure quality, utility, objectivity and integrity of information prior
to being disseminated.
-
Statistical products
CDC routinely employs a number of widely accepted methods and procedures
for ensuring quality, including independent assessments of statistical
methodologies, peer reviews, and observance of professional standards. To
insure the utility of CDC statistical and analytic information products,
CDC conducts independent research and consults experts in areas such as data
collection, data analysis and a variety of substantive topics and areas.
Additionally, CDC maintains ongoing contact with users, and participates
in conferences, and workshops in order to objectively assess and identify
the current and future data needs of CDC's constituents. Further, CDC employs
a wide variety of dissemination mechanisms to make its statistical and analytic
information products widely available and broadly accessible.
To assure that statistical and analytic information products are accurate,
reliable, and unbiased, CDC obtains these data through generally accepted
statistical theory and practice. Dissemination of data also follows generally
recognized guidelines in terms of defining acceptable standards regarding
minimum response rates, maximum standard errors, cell size suppression, quality
of coding and other processing operations. CDC also maintains staff expertise
in areas such as concept development, survey planning and design, data
collection, data processing and editing, data analysis, evaluation procedures,
and methods of data dissemination.
All CDC statistical and analytic information products undergo a formal clearance
process before dissemination. Publications and reports, whether in electronic
or paper form, are reviewed by a CIO ADS or designee. These reviews cover
the clarity of descriptive text, the appropriateness of the methodology,
the soundness of the analysis, the adherence to confidentiality and disclosure
avoidance restrictions, the readability of tabular and graphic presentations
of data. Finally, all products undergo editorial review, (e.g., formatting,
proofreading, spell checks, proper punctuation). Oral presentations undergo
appropriate supervisory review. The CIO ADS or designee may also review for
programmatic and policy implications on behalf of and in consultation with
other division or senior staff. In addition, all public-use tapes are reviewed
by the CIO ADS or designee for accuracy and appropriate confidentiality
protections.
CDC statistical and analytic information products are derived using generally
acceptable statistical practices and methodologies which are clearly documented
and available to the public. These procedures enable responsible statisticians
and analysts outside of CDC to replicate CDC's statistical methods and obtain
results consistent with those obtained by CDC.
[ Go to Contents ]
CDC has developed administrative mechanisms to allow affected persons to
seek and obtain correction of disseminated information that does not comply
with OMB, HHS and CDC guidelines.
CDC will establish a Website to advise information consumers of the agency's
information quality guidelines, the process to submit a complaint, information
needed by the complainant, and a description of the complaint adjudication
process. CDC will centralize the initial receipt, logging, and tracking of
all complaints received under this provision in the Management Analysis and
Services Office (MASO), Office of Program Services. Complaints will be forwarded
to the office that has subject matter responsibility for the information
product in question.
-
Responsibility of the Complainant
To seek a correction of information disseminated by the agency, individuals
must follow the procedures described below:
-
complaints or requests for review and correction of information must be in
written (hard copy or electronic) form;
-
requests shall be sent to CDC by mail at CDC/ATSDR, Attn: MASO, MS-E11, 1600
Clifton Road, N.E.; Atlanta, GA 30333 or by e-mail at:
InfoQuality@cdc.gov; and
-
requests shall state that an information quality request for correction is
being submitted.
The complaint must contain:
-
a detailed description of the specific information that needs to be corrected
including where the information is located, i.e. the publication title, date,
and publication number, if any, or the Website and Web page address (url),
or the speech title, presenter, date and place of delivery;
-
the specific reasons for believing the information does not comply with OMB,
HHS or CDC guidelines and is in error and supporting documentation, if any;
-
the specific recommendations for correcting the information;
-
a description of how the person submitting the complaint is affected by the
information error; and
-
the name, mailing address, telephone number, e-mail address, and organizational
affiliation, if any, of the individual making the complaint.
Complainants should be aware that they bear the 'burden of proof' with respect
to the necessity for correction as well as with respect to the type of correction
they seek.
-
CDC/ATSDR Responsibility
CDC will respond to all requests for correction within 60 calendar days of
receipt. If the request requires more than 60 calendar days to resolve, the
requestor will be informed that more time is required, notified of the reason
why, and provided an estimated decision date. Based on a review of the
information provided, the agency will determine whether a correction is warranted
and, if so, what action to take. CDC will respond to the requestor by letter
or e-mail, explaining the findings of the review and the actions that the
agency will take, if any. The response will consider the nature and timeliness
of the information involved and such factors as the significance of the
correction on the use of the information and the magnitude of the correction.
The response will describe how the complainant may request reconsideration
of the CDC decision.
-
Appeals
If the individual submitting the complaint does not agree with CDC's decision
(including the corrective action, if any), the complainant may send a written
hard copy or electronic request for reconsideration within 30 days of receipt
of the agency's decision. The appeal must state the reasons why the agency
response is insufficient or inadequate. Complainants must attach a copy of
their original request and the agency's response to it. Clearly mark the
appeal with the words, "Information Quality Appeal," and send the appeal
by mail to CDC/ATSDR, Attn: MASO, MS-E11; 1600 Clifton Road, N.E., Atlanta,
GA 30333 or by e-mail to
InfoQuality@cdc.gov.
The agency will respond to all requests for appeals within 60 calendar days
of receipt. If the request requires more than 60 calendar days to resolve,
the agency will inform the complainant that more time is required and indicate
the reason why and an estimated decision date.
The agency official who resolved the original complaint will not have
responsibility for the appeal. MASO will direct all appeals to an appropriate
CDC official in the Office of the Director based on the nature of the information
product and complaint.
[ Go to Contents ]
CDC considers the information disseminated in the MMWR Recommendations
and Reports, the Hazardous Substance Release/Health Effects Database,
Toxicological Profiles, ATSDR Public Health Assessments, and Federal Register
publications related to science as influential scientific information.
Some of the influential information that we disseminate is based on an analysis
of the risks to the public of certain actions or exposures to hazardous
substances. For purposes of this guidance, we are defining risk as the likelihood
that injury or damage is or can be caused by a substance, technology, or
activity. We use risk analysis (the integration of risk assessment with risk
management and risk communication) as a tool to enhance the scientific basis
for all of our regulatory decisions.
The OMB Guidelines provide special considerations that must be taken into
account in certain risk assessments, those that provide the basis for the
dissemination of influential information. The guidelines state that "With
regard to analysis of risks to human health, safety, and the environment
maintained or disseminated by the agencies, agencies shall either adopt or
adapt the quality principles applied by Congress to risk information used
and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996
(SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B))."
The SDWA risk assessment principles are as follows:
-
To the degree that the agency action is based on science, the agency shall
use
-
the best available, peer-reviewed science and supporting studies conducted
in accordance with sound and objective scientific practices
-
data collected by accepted methods (if reliability of the method and the
nature of the decision justify use of the data)
-
In the dissemination of public information about risks, the agency shall
ensure that the presentation of information about risk effects is comprehensive,
informative, and understandable.
-
In a document made available to the public in support of a regulation, the
agency shall specify, to the extent practicable
-
Each population addressed by any estimate of applicable risk effects
-
The expected risk or central estimate of risk for the specific populations
affected
-
Each appropriate upper-bound or lower-bound estimate of risk
-
Each significant uncertainty identified in the process of the assessment
of risk effects and the studies that would assist in resolving the uncertainty
and
-
Peer-reviewed studies known to the agency that support, are directly relevant
to, or fail to support any estimate of risk effects and the methodology used
to reconcile the inconsistencies in the scientific data
Many of our actions are based on scientific experts' judgments using available
data, are essentially qualitative and do not lend themselves to the types
of quantitative risk assessments contemplated by the SDWA principles. As
a result, we have adapted the general principles for risk assessments from
the SDWA to fit these situations.
-
The agency will use
-
the best available science and supporting studies conducted in accordance
with sound and objective scientific practices, including peer-reviewed science
and supporting studies when available
-
data collected by accepted methods (if reliability of the method and the
nature of the decision justify use of the data)
-
In the dissemination of public information about risks, the agency will ensure
that the presentation of information about risk effects is comprehensive,
informative, and understandable.
In situations where a quantitative risk assessment is conducted, we generally
follow basic risk assessment principles in the NAS paradigm of 1983. Our
needs for quantitative risk assessments range over a wide variety of hazards
including physical hazards encountered during exposure to toxic substances
and antimicrobial resistance to antibiotic therapy. Thus, we also ascribe
to the statement from NAS when it revisited the risk assessment process in
1994 (Science and Judgment in Risk Assessment, NAS 1994): "Risk assessment
is not a single process, but a systematic approach to organizing and analyzing
scientific knowledge and information." In each of the areas we regulate,
we apply risk assessment practices to the specific task that are widely accepted
among relevant domestic and international public health agencies.
For quantitative risk assessments in support of the dissemination of influential
information, CDC intends to apply the following principles:
-
The agency will use
-
the best available science and supporting studies conducted in accordance
with sound and objective scientific practices, including peer-reviewed science
and supporting studies when available.
-
data collected by accepted methods (if reliability of the method and the
nature of the decision justifies use of the data).
-
In the dissemination of public information about health risks, the agency
shall ensure that the presentation of information is comprehensive, informative,
and understandable, within the context of its intended purpose.
-
In a document made available to the public, the agency shall specify, to
the extent practicable-
-
Each population addressed by any estimate of applicable effects;
-
The expected or central estimate of risk for the specific populations affected;
-
Each appropriate upper-bound and/or lower-bound risk estimates;
-
Data gaps and other significant uncertainties identified in the process of
the risk assessment and the studies that would assist in reducing the
uncertaintaies; and
-
Additional studies not used in the risk assessment that support or fail to
support the findings of the assessment and the rationale of why they were
not used.
[ Go to Contents ]
Special consideration also applies to information products that are urgent
in nature and because of the potential risk to human health and safety, certain
information products may be disseminated in an expedited manner without having
fully complied with all normal quality guidelines; however, basic quality
principles and processes will still apply and be followed.
Last revised: December 13, 2006