Guidelines for Ensuring the Quality of Information Disseminated to the Public
I. � National Institutes of Health
These guidelines were developed to implement the Office of Management and Budget (OMB) Guidelines and the Department of Health and Human Services (HHS) guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies, hereafter referred to as the OMB Information Quality Guidelines and the HHS Part I: Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated to the Public. This report provides the information quality guidelines for the National Institutes of Health (NIH), and explains how these guidelines will ensure and maximize the quality, objectivity, utility, and integrity of information, including statistical information, disseminated by NIH to the public. This report also details our administrative mechanisms for allowing affected persons to seek and obtain appropriate correction of information maintained and disseminated by the NIH.
The OMB Information Quality Guidelines can be found in the Federal Register, September 28, 2001 The guidelines apply primarily to the dissemination of substantive information (e.g., scientific reports, articles, studies, summaries, speeches, official expert opinions, brochures, statistical information, or compendiums) rather than information pertaining to basic agency operations. Such information can be in any media -- printed, electronic, audiovisual, and the like -- and must be authored or issued by the agency or its contractors, and represent our view.
NIH will ensure that disseminated information meets the standards set forth in the OMB, HHS, and NIH guidelines. It is NIH's policy to ensure and maximize the quality, objectivity, utility, and integrity of the information it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. The quality assurance process begins at the inception of the information development process. NIH is committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination. Each federal agency is already required to demonstrate the "practical utility" of a proposed collection of information in its Paperwork Reduction Act (PRA) submission, i.e., for draft information collections designed to gather information that the agency plans to disseminate. NIH will demonstrate in its PRA clearance packages that each such draft information collection will result in information that will be collected, maintained, and used in a way consistent with OMB, HHS, and NIH information quality guidelines. NIH intends to make use of the PRA clearance process to help improve the quality of information that we collect and disseminate, and to ensure that it complies with all applicable guidelines. The standards to which NIH shall adhere include the following: .
I. � Agency Mission
Founded in 1887, today NIH is one of the world's foremost medical research centers, and the Federal focal point for medical research in the U.S. NIH, comprised of 27 separate Institutes and Centers, is 1 of 8 health agencies of the Public Health Service, which, in turn, is part of the U.S. Department of Health and Human Services (HHS).
Simply described, the goal of NIH research is to acquire new knowledge to help prevent, detect, diagnose, and treat disease and disability, from the most rare genetic disorder to the common cold. The NIH mission is to uncover new knowledge that will lead to better health for everyone. NIH works toward that mission by: Conducting research in its own laboratories; supporting the research of non-Federal scientists in universities, medical schools, hospitals, and research institutions throughout the country and abroad; assisting in the training of research investigators; and promoting communication of medical and health sciences information.
In FY 2002, NIH received $23.5 billion in support of its mission. Of that amount, nearly 84 percent supports non-Federal researchers working in universities, medical centers, hospitals, and research institutions throughout the country and abroad (collectively referred to as extramural research), and about 10 percent is allocated to in-house research laboratories located on the NIH campus and several off-campus sites (referred to as intramural research). NIH has over 17,000 employees, approximately 3,000 of which have doctoral or medical degrees.
It is NIH policy to make available to the public the results and accomplishments derived from the activities that it funds. Therefore, NIH-funded intramural and extramural investigators are expected to make the results and accomplishments of their activities available to the research community and to the public at large, and to effect their timely transfer to industry for commercialization.
NIH is organized into Institutes and Centers (ICs), each with its own mission and functions, separate appropriations, and statutory authorities. Although these ICs may have different administrative procedures in place, they operate under the same general NIH policies and requirements. The NIH Policy Manual System is the formal mechanism for issuing NIH policy. The system is comprised of a series of NIH Manual Chapters. It provides an organized, central repository of information that is accessible to all NIH employees. Individual IC's have the flexibility to incorporate the quality and accountability requirements of Federal and NIH guidelines into their own information resource management and administrative practices in the most applicable manner.
At NIH, the Associate Director for Communications, who is also the
Director of the Office of Communications and Public Liaison (OCPL) in the
Office of the Director, will have overall responsibility for implementing
NIH Information Quality Guidelines, and will work collaboratively with the
ICs. OCPL is the central office for communications at NIH. As such, OCPL
takes the lead across the NIH for setting communications policy, and for
communicating information about NIH programs, issues, and accomplishments
to the public and public interest groups and, to a lesser extent, to the
scientific community and the medical professions. The office is the communications
link between the ICs and the Office of the Assistant Secretary for Public
Affairs in the HHS, and serves as the coordinating office or central source
for NIH IC matters related to publications, including printing, HHS/PHS/NIH
clearance and review procedures, Joint Committee on Printing, U.S. Congress,
and Government Printing Office printing and binding regulations, and copyright
rules. Among its many activities, the office produces and distributes a number
of publications that highlight NIH research results and scientific advances;
provides print, radio, and TV coverage of NIH news and activities; produces
the NIH Record; and publishes consumer health information, primarily in a
newsletter for the press and public entitled The NIH Word on Health. OCPL also supports and coordinates the principal NIH Web site (www.nih.gov)
with direct responsibility for several major areas of the NIH home page that
address the special needs of healthcare professionals, patients, members
of the press, the public, and employees. It manages the NIH Web Coordinating
Committee that provides leadership for the design and content of the NIH
Web site, including reviewing new Web sites before they are integrated into
the structure of the NIH home page; works with other relevant offices and
committees in establishing operational standards and guidelines for Web sites
at NIH; and manages the responses to electronic mail sent to the NIH home
As the lead office for NIH Information Quality, OCPL responsibilities include:
II. � Scope of Applicability of Guidelines for Agency
The OMB Information Quality Guidelines require NIH to evaluate and identify the types of NIH information that will be subject to the Guidelines. This section identifies the types of information covered by the Guidelines, and also lists the types of information that are exempt. The NIH Office of the General Counsel originally reviewed this information on November 27, 2001 and considered NIH's interpretation to be consistent with the intent of the law.
The pre-dissemination review described in the guidelines only applies to official information (with the NIH imprimatur) that is released on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information. They apply to information in all media — printed, electronic, audiovisual, verbal, and other. The Guidelines focus primarily on the dissemination of substantive information (i.e., reports, studies, summaries) rather than information pertaining to basic agency operations. Information that is disseminated at the request of NIH or with specific NIH approval through a contract or a grant is subject to these Guidelines. Examples are provided below of the kinds of information that the NIH considers to be covered and not covered by the OMB Information Quality Guidelines. Although information that is not covered by the OMB Guidelines are not subject to the new administrative complaint procedures, the information is still subject to the usual NIH internal review procedures for accuracy and quality.
Other official reports, brochures, documents, newsletters, electronic documents, and audiovisual productions (i.e., a unified presentation, developed according to a plan or script, containing visual imagery, sound, or both, and used to convey information)
Editorials, commentaries, letters-to-the-editor, only if NIH staff representing official NIH viewpoints provides them
Oral information, including speeches, interviews, expert opinions, only if representing NIH's views, official positions, or policies
Statistical information -- statistical analyses, aggregated information by program, IC, or for NIH, including funding information and histories (by disease, funding mechanism, dollars, and other criteria) -- prepared for public dissemination
Consensus panel reports and open meetings' proceedings and minutes
Documents not authored by the agency and not representing the agency's views, including information authored and distributed by NIH grantees1
Information that is limited in dissemination to Government employees or agency contractors or grantees
Information pertaining to basic agency operations, e.g., information about agency authorities, activities, programs, along with contact information for the public, organizational charts, NIH or IC Directors' Status Reports, solicitations [program announcements (PAs)/requests for applications (RFAs)], receipt and review materials (e.g., summary statements, information for advisory councils or advisory committee members)
Information intended solely for intra- or interagency use or sharing of Government information
Responses to requests for agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or other similar laws
Information relating solely to correspondence with individuals or persons
Press releases that support the announcement or give public notice of information that NIH has disseminated elsewhere
Information intended for public filings, subpoenas, or adjudicative processes involving specific parties (There are well-established procedural safeguards and rights to address the quality of adjudicatory decisions and to provide persons with an opportunity to contest decisions. These guidelines do not impose any additional requirements on agencies during adjudicative proceedings and do not provide parties to such adjudicative proceedings any additional rights of challenge or appeal.)
Opinions where the agency's presentation makes it clear that what is being offered is personal opinion rather than fact or the agency's views
III. � Types of Information Disseminated by NIH to the Public
Each year, NIH components produce 400 or more publications of various types, and about 140,000 static Web pages. All publications that carry the NIH imprimatur, i.e. are considered official NIH publications or releases, must follow NIH policy and procedures for preparation, review, approval, and distribution (see Section V). The types of information disseminated by NIH to the public include the following, however, the OMB guidelines are not directly applicable to all of the information in these categories (See Section II):
IV. � Types of Dissemination Methods
NIH information is disseminated in many mediums, with the following four being most common:
publications, books, newsletters, brochures, booklets, pamphlets, and reports.
formal speeches, oral presentations, interviews, or commentaries for publication or broadcast; letters-to-the-editor or correspondence likely to result in similar publications.
broadcast scripts, audio or videotapes, and videocasting. The Center for Information Technology (CIT) makes special NIH events, seminars, and lectures available to viewers on the NIH network and the Internet from the VideoCast Web site.
The NIH Web site is the most popular Government Web site after the Internal Revenue Service, and has about 3 million unique visitors per month. The NIH Web site is recognized as one of the most respected and trusted sources for authoritative health information (Consumer Reports, January 2002; Forbes.com review, September 10, 2001; Business Wire, January 29, 2001). The NIH Web site is not just one site, but also a large collection of sites residing on over 150 servers with over 140,000 static pages that are crawled and indexed on public servers. Some areas are updated daily, while others may not be updated for weeks or months.
V. � Agency Quality Assurance Policies, Standards, and Processes for Ensuring the Quality of Information Disseminated to the Public
All NIH documents and audiovisuals must be prepared in accordance with professional and ethical standards, as well as generally accepted standards of good taste. They must be appropriate for dissemination by this Federal agency, and must undergo appropriate review and approval prior to release. NIH must adhere to the laws and regulations applying to publications and audiovisuals, including OMB Information Quality Guidelines, the HHS Printing Handbook, and relevant NIH Manual chapters. NIH efforts to ensure and maximize information quality begin at the preparation stage, and continue through the review and approval stages. Existing NIH policies developed in concert with Federal computer security laws provide appropriate security safeguards to ensure integrity of NIH documents, i.e., ensure that the information is protected from unauthorized access, revision, corruption, or falsification.
The NIH has many quality control measures embedded in the scientific process to ensure that the information disseminated is of the highest quality. NIH grant applications undergo rigorous scientific review. Scientific journals do not publish articles until they have gone through a similar peer review process. There is a tension inherent in biomedical research between releasing information in a timely fashion and waiting for the peer review process to result in a published article. Sometimes the NIH provides "late breaking news" to the public on research findings prior to publication in scientific journals and prior to peer review by the journals. However, when it does so, there is an internal review process that routinely draws upon external expertise and monitoring/advisory review boards to ensure that information disseminated to the public summarizes the facts as they are currently known, and that appropriate disclaimers are attached.
The policies and procedures to be followed in the preparation, review, approval, and distribution of NIH information materials, including scientific and professional materials, can be found in NIH Manual Chapter 1183: NIH Publications and Audiovisuals: Preparation, Review, Approval, and Distribution and NIH Manual Chapter 1184: Scientific and Professional Information Presented By NIH Employees: Review, Approval, and Distribution. The procedures currently in place were developed to be sensible, workable, flexible, and timely, and were updated in February 2002 to better articulate OMB, HHS, and NIH information quality guidelines. In the scientific and research context, technical information that has been subjected to formal, independent, external peer review is generally presumed to be of reasonable quality.
The general principles concerning the responsibilities of the NIH research staff in the collection and recording of data, publication practices, authorship determination, peer review, confidentiality of information, collaborations, human subjects research, and financial conflicts of interest are exemplified in the "Guidelines for the Conduct of Research in the Intramural Research Programs at NIH." These guidelines can be found on the Web (www.nih.gov/news/irnews/guidelines.htm). NIH recognizes the scientific need for replication of findings, and encourages data sharing as appropriate. After publication, the research data, any unique reagents, and any supporting data that form the basis of the communication in question should be made available promptly and completely to all responsible scientists seeking further information. Exceptions may be necessary to maintain the confidentiality of clinical data or if unique materials were obtained under agreements that preclude their dissemination. Investigators should retain research data long enough to allow replication of study results -- in general, 5 to 7 years.
The review, approval, and dissemination of substantive scientific information by NIH and/or its ICs require adherence to appropriate clearance procedures set forth in NIH Manual Chapters, internal Web sites, or memos, and are consistent with HHS and OMB guidelines. The originating office is responsible for obtaining the necessary clearances for reproduction and distribution of printed materials and should ensure that written material distributed is appropriate and consistent with HHS policy.4
A document that has obtained publication clearance for paper printing is often posted on the sponsoring IC's Web page for greater accessibility. NIH/IC Web documents derived from IC-approved printed publications should not need additional approvals. NIH/IC Web documents with no print counterpart require content clearance by the appropriate IC office or contact person to ensure that the information observes all applicable requirements governing information for release to the public. These include the requirements provided in NIH Manual Chapter 1183. When IC Web pages are related to more than one IC (e.g., trans-IC publications, special interest groups), the appropriate IC office or contact person for the primary IC responsible for creating the Web page should be notified regarding clearance requirements.
This section describes NIH procedures and practices in place for review and approval of substantive scientific information that is meant for dissemination primarily to the public, and that NIH considers being subject to the OMB Information Quality Guidelines (see Section II above).
NIH encourages professional dissemination of scientific research and other information on behalf of public health by its employees. Professional and scholarly writing, lecturing, editing, and publishing are an essential part of research, are in the public interest, and bring credit and distinction to NIH and to the employees themselves. In assisting employees to share information about their official and professional activities, NIH seeks to advance scientific knowledge and contribute to professional education. Ordinarily first report of any scientific research results or other professional findings is made by publication in a scientific or professional journal or presentation at a meeting of a professional organization. The choice of the journal or meeting to which reports are offered is the prerogative of the author(s).
There are many quality control measures embedded in the scientific process to ensure that the information disseminated by NIH employees is of the highest quality. Publications or presentations by NIH employees are expected to meet high standards of quality, make a substantial contribution to the field, and contain sufficient information for the informed audience to assess its validity.
To ensure and maximize the quality of information disseminated by NIH employees, any non-extemporaneous presentation (written or electronic) by an NIH employee on a subject related to his/her NIH duties must be reviewed and approved through an internal NIH process prior to submitting for publication consideration. With few exceptions, non-extemporaneous oral presentations on health policy or practice, or presentations with policy implications, must also be cleared in advance. Manuscripts intended for publication are customarily subjected to an external peer review process directed by the interested publisher, volume editor, or journal editor.
Publication or oral presentation of scientific and professional information by individual employees must conform to applicable laws and regulations, including OMB Information Quality Guidelines, and the HHS Standards of Conduct Regulations. Customary professional practices impose certain constraints on the degree to which NIH employees may be identified with the results of research and development work, including that obtained in collaboration with extramural grantees.5
All Institutes have either formal or informal internal operating procedures for identifying printing requirements and tracking publications. These procedures come in a variety of forms such as policy issuances, internal Web sites, memos, or annual requests for printing requirements. In addition, the concept clearance process for new publications is often the vehicle that Institutes use to track the development of a new publication and to identify its attendant printing requirements. The requirements for clearance of prospective publications are contained in NIH Manual Chapter 1183. These requirements state that each prospective publication must be cleared through the Communications Office within the originating NIH component and then be approved by the Office of Communications and Public Liaison, OD/NIH and the Office of the Assistant Secretary for Public Affairs (OASPA), HHS.
In brief, the NIH Manual Chapter 1183 requires that any official
publication (including book, bibliography, chapter of a book or textbook,
booklet, brochure, collection of abstracts, fact sheet, house organ, index,
leaflet, manual, monograph, newsletter, pamphlet, review, periodical, proceeding,
recurring report, statistical compendium, Internet document, audiovisual,
or the like), prepared by any NIH component directly or through a contract
must be sent for HHS clearance through the Editorial Operations Branch, using
form HHS-615, Publication Planning and Clearance Request. This clearance
requirement does not apply to publication of articles in journals. The authority
to permit the initial publication of articles written by NIH employees in
privately published journals, encyclopedias, and textbooks can be delegated
according to NIH Manual Chapter 1130 (Delegation of Authority).
Statistical compendiums, including statistical analyses, aggregated information by program, IC, or for NIH, including funding information and histories (by disease, funding mechanism, dollars, or other criteria), do not require approval by the Office of the Director, NIH. However, the Director of the originating IC is required to determine that the data conform to the accepted quality standards, and if applicable, that the reported statistics be substantially reproducible (see NIH Manual Chapter 1183).
In general, any writing by an NIH employee on a work-related subject, whether intended for electronic or print publication, or for oral delivery, must be prepared according to accepted NIH standards of quality, reviewed for substantive content, and administratively approved. The purpose of the NIH clearance process is to improve the quality of information, and to ensure the accuracy, objectivity, utility, and validity of information. NIH Manual Chapter 1184, states the policy and procedures to be followed in the review, publication, and distribution of scientific, technical, and other professional manuscripts and speeches by NIH employees. IC Directors (or their delegates) are responsible for establishing and maintaining controls to ensure competent and timely clearance of professional writing and presentations by developing procedures appropriate to the type of information. They are also responsible for maintaining files of requests for approval and actions taken. Individual ICs may determine how best to meet these requirements.
Written presentations by intramural scientists are reviewed and approved by Laboratory/Branch Chiefs and sometimes by Scientific Directors. The intramural approval process also ascertains that all animal, human subjects, and technology transfer requirements are met, that major press and policy implications are noted, and that at least one supervisory scientist finds the work to be of merit. (See the Intramural Research Sourcebook at www1.od.nih.gov/oir/sourcebook/oversight/pub-clear.htm).
Materials requiring review in the Office of the Director, NIH, should be approved by a designated review officer within the originating IC, or by a person in a supervisory relationship to the author, prior to submission to the Office of the Director, NIH. No such preliminary review is required for writing by an IC Director. Any statement, commentary, or discussion of Federal policies or practices related to the employee's position or duties that might be construed as reflecting an official position by NIH, HHS, or the Federal Government must be approved in the Office of the Director, NIH.
For scientific and technical documents, the scientific community recognizes peer review as the primary means of quality control. According to OMB Information Quality Guidelines, material subjected to formal, independent, external peer review may generally be considered to be of acceptable objectivity. However, this presumption of objectivity is refutable based on a persuasive showing to the contrary by a complainant in the particular instance. The single most important determinant of a scientific review group's competence and credibility is its members. Reviewers must have scientific excellence (as demonstrated by their grant and publication records, and academic degrees and honors), and must merit respect in the scientific community. They must possess a wide breadth of expertise, be fair and objective, and should not be influenced by inappropriate personal interests (competition, scientific bias, personal antagonisms, and other irrelevant factors.). Reviewers should review materials for propriety, accuracy, completeness and quality (including objectivity, utility, and integrity).
Consistent with HHS Standards of Conduct (73.735-705 Writing and Editing), employees are encouraged to engage in outside writing and editing when such activity is not otherwise prohibited. If the writing or editing activity is related to the employee's official duties or other responsibilities and programs of the Federal Government, the employee must (i) make no mention of his or her official title or affiliation with the Department, or (ii) use his or her official title or affiliation with the Department and a disclaimer, or (iii) submit the material for clearance within the operating component, under procedures established by the component. When clearance is denied at any lower level, the employee shall have recourse for review up to the head of the principal operating component. This clearance will show there are no official objections to the activity and the employee may then use his or her official title or affiliation usually without a disclaimer. Except where the requirement for disclaimer is waived as a result of official clearance, disclaimers shall be used in all writing and editing related to the employee's official duties or other responsibilities and programs of the Federal Government: (i) in which the employee identifies himself or herself by official title or affiliation with the Department, or (ii) when the prominence of the employee or the employee's position might lead the public to associate him or her with the Department, even without identification other than name. Disclaimers shall read as follows unless a different wording is approved by the Assistant General Counsel, Business and Administrative Law Division, Office of the General Counsel: "This (article, book, etc.) was (written, edited) by (employee's name) in (his or her) private capacity. No official support or endorsement by (name of operating component or of Department) is intended or should be inferred."
Normally, the need for a disclaimer is eliminated through the clearance
process. However, a disclaimer may still be needed even after official clearance
to clarify that the presentation should not be construed as necessarily representing
NIH views, and/or to distinguish the status of information (e.g., preliminary,
based on partial data set). The Department's regulations (Standards of Conduct)
to which the NIH subscribes, require that disclaimers be used in all unofficial
writing and editing related to the employee's official duties and/or affiliation
with programs of the Federal Government in which the employee's identification
with NIH is to be shown, can be inferred, or is well-known.
Any statements, comments, or discussion of Federal policies or practices that are relevant to the employee's position or duties, draw conclusions, advocate or oppose professional practices or positions on subjects related to NIH duties, or might otherwise be construed as reflecting an official position by NIH, HHS, or the Federal Government, are covered by the OMB Guidelines, and must be approved in the Office of the Director, NIH.
An NIH employee may respond orally to questions and requests for information from any source, including the news media, without prior review and approval but must adhere to internal IC guidelines for informing the IC Information Officer, Congressional Liaison Officer, or other appropriate official about the nature of the information to be discussed. An employee may appear as a member of a discussion panel or seminar and on radio, television, and Web broadcasts without prior approval if the appearance does not require a manuscript or written text or statement, and if there is no conflict with NIH Policy as provided in Manual Chapter 1184. An employee should limit his/her statements and responses to subjects about which he/she has official knowledge and should present only official HHS and NIH positions in discussion of policy matters.
No review or approval is required for nonofficial and private writing, speaking, and publishing by an employee unless his/her NIH employment is likely to be regarded as influencing the content.
NIH employees are responsible for the statements they make, regardless of whether they have been cleared. If one presents material that requires clearance but that has not been cleared prior to presentation, then the employee must inform the audience of the personal or unofficial nature of his or her views. An example of an appropriate disclaimer follows:
"This material is presented from my own perspective, and should not be taken as representing the viewpoint of the Department, NIH, or [IC]."
NIH employees shall not identify themselves as NIH employees in unofficial materials prepared for dissemination to nonprofessional audiences, such as a letter-to-the-editor. These materials must be reviewed prior to presentation in the Office of the Director, NIH, if an employee's identification with NIH is to be shown, can be inferred, or is well known.
The NIH Office of Medical Applications of Research (OMAR) manages the NIH Consensus Development Conference (CDC) Program, the focal point for evidence-based assessments of medical practice and state of the science on behalf of the medical community and the public. Under this program, OMAR organizes major conferences that produce Consensus Statements and State of the Science Statements on controversial issues in medicine important to healthcare providers, patients, and the general public. NIH Consensus Statements and State of the Science Statements are disseminated widely, and more than 120 NIH Consensus Statements and State of the Science Statements have been issued since the program's inception in 1977. Organizationally, OMAR is under the Associate Director for Disease Prevention in the Office of the Director, NIH, and works closely with NIH Institutes, Centers, and Offices to assess, translate, and disseminate the results of biomedical research that can be used in the delivery of important health services to the public.
An NIH Consensus Statement is a report evaluating scientific information on a given biomedical or public health intervention with the purpose of resolving a particular controversial issue in clinical practice. Each NIH Consensus Statement answers a series of four to six questions concerning efficacy, risk, and clinical applications, and recommends directions for future research, and is the product of an NIH Consensus Development Conference. NIH Consensus Statements synthesize new information, largely from recent or ongoing medical research, that has implications for reevaluation of routine medical practices. They do not give specific algorithms or guidelines for practice.
NIH Consensus Statements are written by broad-based, independent panels of non-Federal, non-advocate individuals knowledgeable in the field of medical or public health science under consideration. The makeup of each panel represents various sectors of professional and community life and typically includes research investigators, healthcare providers, methodologists, and a public representative.
Following circulation of the draft statement to the conference audience for comment, the panel resolves any conflicting recommendations and releases a revised statement at the end of the conference. The Web site for the Consensus Development program can be found at: consensus.nih.gov.
If a suggested topic does not have an adequately defined and available base of scientific information, conference planning and implementation may still proceed. However, rather than being designated a Consensus Development Conference, the conference will be designated as a State of the Science Conference. NIH State of the Science Conferences and Workshops generally adhere to the NIH CDC format because the process is useful for evaluating complex issues. Usually, speakers present findings or perspectives on the issue. The public is invited to address questions to the speakers, and policy implications may be discussed. A report of the findings can emerge in one of a variety of formats including publication in a clinical or scientific journal. The Web site for the State of the Science Statements can be found at: odp.od.nih.gov/consensus/ta/talist.htm.
Although it is difficult to quantify their impact, the NIH Consensus Statements and State of the Science Statements are intended to influence important public health discussions on topics affecting or broadly applying to a significant number of people. The severity of the problem (morbidity and mortality) and the feasibility of intervention are key considerations. For example, the program has had measured success in influencing reimbursement policy and specialty organization policy, thereby indirectly affecting physician behavior. Each conference is jointly sponsored and administered by one or more ICs of NIH and by OMAR. Depending on the topic, other Federal agencies with biomedical components may join in sponsoring a CDC. In conjunction with each conference, the Agency for Health Care Research and Quality (AHRQ) provides a systematic review of the literature on the conference topic through one of its Evidence-Based Practice Centers.
Both the Consensus Statements and the State of the Science Statements are independent reports of the convened panel; none are policy statements of the NIH or the Federal Government. However, NIH funds and disseminates the Consensus Reports, and considers these Statements to be subject to OMB's higher standard of substantial reproducibility. These Statements meet the OMB's quality standards because of the balanced, rigorous, and systematic procedures that OMAR has in place to develop them. The panel makes available the evidence on which the Consensus Statement is based. If consensus cannot be achieved, minority or alternative views are included. The systematic literature review conducted by AHRQ is published with their explicit methods. The reports are peer-reviewed by expert panels, and posted on the Internet for at least one month for public comment.
To make environmental health research findings more applicable to human risk assessment, NIH works in partnership with the CDC and the EPA to develop better ways to monitor and assess human exposure to specific chemicals. One of our most visible publications is the Report on Carcinogens (RoC), a congressionally mandated document that lists agents, substances, mixtures or exposure circumstances that are known or reasonably anticipated to be human carcinogens, and to which a significant number of persons residing in the United States are exposed. Responsibility for producing this report has been delegated to the National Toxicology Program (NTP) and the Director NTP also serves as the Director, NIEHS, NIH.6 The RoC is a composite of Summary Profiles that describes the carcinogenicity, exposure, and regulatory information for each listing with relevant tables and appendices. Substances, agents, mixtures or exposure circumstances that are being considered by the NTP for possible listing or de-listing in the RoC (referred to as nominations) are evaluated by a review process consisting of sequential reviews by distinct scientific review committees -- two Federal scientific review groups and one nongovernmental scientific peer-review body. External peer review of the nominations is performed by a subcommittee of the NTP Board of Scientific Counselors in open, public meetings. Publicly available, peer-reviewed technical reports are the primary sources of data used in the preparation of the Background Documents for each specific nomination.
Continuing opportunities for public comment and participation are an integral part of the process. The recommendations of the three (3) scientific review groups are published in the Federal Register, NTP newsletters and web pages and other appropriate publications to solicit final public comment and input for nominations. The recommendations and all public comments are provided to the NTP Executive Committee, who reviews this information and provides the Director, NTP with their recommendations. All recommendations and public comments are then reviewed by the Director, NTP, who forwards the final draft of the Report that contains his recommendations to the Secretary, HHS for the listings or de-listings in the RoC. Upon review and approval by the Secretary, HHS, and submission to Congress, a notice of the RoC publication, indicating all newly listed or de-listed agents, substances, mixtures or exposure circumstances is published in the Federal Register, NTP newsletters and web pages and other appropriate publications.
It is important to note that the RoC does not present assessments
of carcinogenic risks. Listing of substances in this Report, therefore, does
not establish that such substances present carcinogenic risks to individuals
in their daily lives. Such formal risk assessments are the purview of the
appropriate Federal, State, and local health regulatory and research agencies.
However, for each effluent, ambient, or exposure standard established by
a Federal agency with respect to a listed substance, the RoC is required
to state the extent to which, on the basis of available medical, scientific,
or other data, the implementation of such standard decreases the risk to
public health from exposure to the substance. This requires quantified information
on the extent of protection from cancer that the public receives from established
Federal standards. Only in a few instances, where studies of long-term human
exposures and cancer incidence in restricted environments are available,
can risk be estimated with complete confidence.
The NIH Manual chapter 1183 requires that official materials or information prepared by any NIH component directly or through a contract must be sent for HHS clearance through the Editorial Operations Branch, using form HHS-615, Publication Planning and Clearance Request. This clearance requirement does not apply to publication of articles in journals. Information developed by a clearinghouse for an NIH IC is subject to the OMB Guidelines. See NIH Manual Chapter 1183 for further information regarding this requirement.
At NIH, there are essentially three types of materials being disseminated through information clearinghouses to the public: (1) Materials produced by NIH staff or contractors that undergo usual NIH review and approval processes; (2) materials produced by NIH grantees that are subject to policies and procedures in the Public Health Service (PHS) Grants Policy Statement; and (3) other materials not produced by NIH but available through libraries, whether in print or in electronic format, with appropriate disclaimers attached. Virtually all NIH ICs direct their clearinghouses to distribute only materials produced by the IC or other NIH ICs or Federal agencies. Non-Federal materials typically undergo careful IC scientific review before they are authorized for dissemination by the clearinghouse, and those materials are accompanied by appropriate disclaimers.
NIH has developed World Wide Web (WWW) Guidance (April 15, 1998), which is available on the Internet (irm.cit.nih.gov/policy/guideli2.html).
Each IC has a designated IC contact/reviewer for information and approvals
related to developing Web pages and operating a new Web server. The list
of IC contacts/reviewers is available on the Web (www.nih.gov/employee/weblist.htm)
and was last updated on March 6, 2002. NIH/IC Web page creators periodically
review material on the Web page to determine whether it is accurate and up
to date. Information, particularly time-sensitive information, should be
posted as soon as possible. Web page creators are expected to promptly update
or remove out-of-date information.
NIH Web managers are urged to exercise caution when linking to non-NIH, external websites. Professional judgment should be used to weigh the benefits against the possible risks of linking to other resources. In particular, links to sites providing medical and scientific information needs to be on par with the standards used at NIH to ensure the credibility of the information offered there. Steps should be taken to ensure that such links do not give the impression of endorsing the organizations we link to. NIH/IC Web pages containing links to external Web pages not located on NIH servers should include a link to a statement that releases NIH from responsibility for the material included in the external Web page. Again, it is important to avoid giving a user the impression that NIH is endorsing information or a commercial product described in an external site. Disclaimers on copyright, endorsement (general and external links), liability, and medical information are also used, as appropriate, for individual IC Web sites.
The IC designates a main office or contact person for information and approvals related to Web pages and the operation of Web servers. NIH personnel, contractors, and other authorized users of NIH networks must notify this office or contact person prior to setting up a Web server. Information about appropriate security measures regarding Web Servers is available at: irm.cit.nih.gov/nihsecurity/NIHWebServPol.doc. The IC or the OD Information Office can provide detailed information on required approvals including both NIH and IC-specific policies relating to publication of documents. NIH/IC documents derived from IC-approved printed publications should not need additional approvals (see Section V.2).
The NIH Center for Information Technology (CIT) is charged with providing,
coordinating, and managing information technology for NIH, and with advancing
computational science. In terms of computer security, CIT has three distinct
objectives: Confidentiality -- ensuring that there is no deliberate or accidental
improper disclosure of sensitive automated information; integrity -- protecting
against deliberate or accidental corruption of automated information; and
availability -- protecting against deliberate or accidental actions that
cause automated information resources to be unavailable to users when needed.
Information is accorded protection against disclosure, alteration, loss,
or destruction based on the degree of sensitivity.
CIT has instituted a structured management control review process that applies throughout the system life cycle. Risk analyses are conducted to strike a balance between an acceptable level of risk and the costs and inconvenience associated with safeguards. A system recertification/accreditation must be conducted at least once every 3 years. Additional information about NIH computer security measures can be found in The Computer Security Handbook of CIT (www.cit.nih.gov/security/handbook.html).
VI. � Agency Administrative Complaint Procedures
NIH has developed administrative procedures to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS, and NIH guidelines (See NIH Manual Chapter 1185, forthcoming). Additional guidance on appropriate responses can be obtained from OCPL/OD/NIH, particularly if the complaint involves a policy statement or official position. NIH will establish a website to advise information consumers of the agency's information quality guidelines, the process to submit a request for correction, information needed by the requestor, and a description of the complaint adjudication process.
The resolution process addresses the valid needs of the complainant without disrupting NIH processes. Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction, as well as with respect to the type of correction they seek. In making a determination of whether or not to correct information, NIH may reject claims made in bad faith or without justification, and is required to undertake only the degree of correction that is appropriate for the nature and timeliness of the information involved.
Office of the Director
National Institutes of Health
Building 1, Room 344
9000 Rockville Pike
Bethesda, MD 20892
The request should state that an information quality request for correction is being submitted, and should provide the following information:
A complaint about information originating from a division or office within the Office of the Director (OD), NIH, should be addressed by the director of the division or office, with input from the Office of Communications and Public Liaison (OCPL), OD, NIH.
If more than one IC was involved in releasing the information, the IC of the lead NIH author should take primary responsibility for coordinating a response.
The response to the complainant will be by letter or email, and should explain the findings of the NIH review of the merits of the complaint, and the actions to be taken, if any. The response should consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response should also describe how the complainant may request reconsideration. NIH will respond to all requests for corrections within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the responsible official will inform the complainant that more time is required and indicate the reason for the delay and an estimated decision date.
Whether or not corrective action is warranted, the response to the complainant should describe the preparation, clearance, review, and approval process prior to the information being disseminated, including any specifics about the peer review process to support the rigor and objectivity of the review, e.g., the breadth and depth of experts, community input, and message testing. The response should also reiterate any disclaimers that accompanied the information when it was first released.
Additional guidance on appropriate responses can be sought from OCPL, OD, NIH, particularly if the complaint involves a policy statement or official position.
If the IC determines that corrective action is warranted, the IC shall forward the following information to OCPL, OD, NIH without delay:
OCPL responsibilities include:
Office of the Director
National Institutes of Health
Building 1, Room 344
9000 Rockville Pike
Bethesda, MD 20892
Any office that originally disseminated the information and/or responded to the original complaint should not have responsibility for the resolution of the appeal. If the information in dispute was originally disseminated by the OCPL/NIH, then an appeal should be addressed to the NIH Director at the address listed above, or sent electronically to InfoQuality@od.nih.gov.
NIH will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason for the delay and an estimated decision date.
VII. � Influential Scientific, Financial, and Statistical Information
The OMB Information Quality Guidelines require that "influential" scientific, financial, or statistical information in official Government documents must be based on studies that can be substantially reproduced if the original or supporting data were to be independently reanalyzed using the same methods. "Influential" when used in the phrase "influential scientific, financial, or statistical information" means that the NIH can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions, or will have important consequences for specific health practices, technologies, substances, products, or firms." NIH is committed to applying rigorous scientific standards to ensure the accuracy, reliability, and reproducibility of research results.
The reproducibility standard applies to analytic results and not necessarily to the original and supporting data used to produce the analytic results. To facilitate the replication of scientific and other influential information by qualified third parties, NIH continues to encourage the sharing of original data and methods where practicable. After publication, the research data, any unique reagents, and any supporting data that form the basis of any research communication should be made available promptly and completely to any person who seeks further information.
Since the influence and implications of NIH-disseminated information cannot always be fully anticipated, all NIH scientific reports are expected to state clearly how analytic results are generated -- the specific data used, various assumptions, specific analytic methods, statistical procedures, sources of error -- making the analysis sufficiently transparent so as to be capable of being reproduced. NIH advocates the archiving of data where feasible to facilitate the reproducibility of influential information. Exceptions may be necessary to maintain the confidentiality of clinical data or if unique materials were developed or obtained under agreements that preclude their dissemination. Investigators should retain research data long enough to allow replication of study results -- in general, 5 to 7 years. In situations where public access to underlying data is not practicable, NIH shall apply especially rigorous robustness checks to analytic results and document what checks were undertaken.
Examples of the types of information disseminated by NIH that have the potential of being considered influential and that fall within the scope of the OMB Guidelines include:
For scientific and technical documents, the scientific community recognizes peer review as the primary means of quality control. NIH routinely seeks input from qualified peer reviewers of influential materials for propriety, accuracy, completeness, and quality (including objectivity, utility, and integrity) prior to dissemination. Although concerted efforts are made to ensure that influential information be subjected to rigorous peer review and reproducibility specifications, standard operating procedures may be temporarily disrupted under urgent situations, such as when an imminent threat to public health or homeland security is identified.
With respect to health, safety, and environmental information, NIH does not have a mandate to conduct formal risk assessments, which are the purview of the appropriate Federal, State, and local health regulatory and research agencies (see Section V.2.iv). NIH makes every effort to ensure that the presentation and dissemination of information about environmental health is comprehensive, informative, and understandable, and that scientific conclusions are based on: (1) The best available science and supporting studies, particularly peer-reviewed studies, conducted in accordance with sound and objective scientific practices; and (2) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).
VIII. � Special Considerations for Agency Dissemination
Sometimes NIH provides "late breaking news" of urgent import to the public on research findings prior to publication in scientific journals and prior to peer review by journals. Under such special circumstances, NIH may temporarily waive adherence to information quality guidelines. However, when it does so, there is an internal review process that routinely draws upon external expertise and relevant monitoring or advisory boards to ensure that information disseminated to the public summarizes the facts as NIH currently knows them, and that appropriate disclaimers are attached.
IX. � References
HHS Standards of Conduct Regulations (45 CFR 73.735-705). Updated October 1, 2000.
Guidelines for the Conduct of Research in the Intramural Research Programs at the National Institutes of Health, 3rd edition, January 1997 (for a discussion of publication practices and authorship issues.) www.nih.gov/news/irnews/guidelines.htm
The HHS Printing Handbook, September 1998.
NIH Grants Policy Statement, March 1, 2001. grants.nih.gov/grants/policy/nihgps_2001
NIH Instruction and Information Memorandum OER 90-8. NIH Staff (Co-)Authorship of Publications Resulting from NIH Extramural Awards (1184). December 7, 1990.
NIH Manual Chapter 1130 Delegation of Authority, Program: General No. 3, Publish Articles and Results of Scientific Research and No. 4, Availability of Records for Examination or Copying (June 12, 1985) www1.od.nih.gov/oma/manualchapters
NIH Manual Chapter 1183 -- NIH Publications and Audiovisuals: Preparation, Review, Approval, and Distribution, February 27, 2002. www1.od.nih.gov/oma/manualchapters/management/1183
NIH Manual Chapter 1184 -- Scientific and Professional Information Presented By NIH Employees: Review, Approval, and Distribution, February 27, 2002. www1.od.nih.gov/oma/manualchapters/management/1184
NIH Manual Chapter 1185 -- Complaints about NIH Information Quality, forthcoming.
NIH Manual Chapter 6308 -- Acquisition of Printing Requirements at the NIH, February 2, 2002. www1.od.nih.gov/oma/manualchapters/contracts/6308
Office of Management and Budget Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies. Final Guidelines. February 22, 2002 www.whitehouse.gov/omb/fedreg/reproducible2.pdf
OMB, Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies. September 28, 2001. http://www.whitehouse.gov/omb/fedreg/final_information_quality_guidelines.html
OMB, Circular No. A-130, Revised (Transmittal Memorandum No. 4). Management of Federal Information Resources. November 30, 2000.
World Wide Web NIH Guidance. April 15, 1998. irm.cit.nih.gov/policy/guideli2.html
Last revised: November 12, 2003