Guidelines for Ensuring the Quality of Information Disseminated to the Public
Part I -- HHS Overview
A. � Summary
The U.S. Department of Health and Human Services (HHS) developed these guidelines to implement Office of Management and Budget (OMB) January 2002 requirement that all federal agencies issue guidelines for ensuring the quality of the information that they disseminate to the public. The Department of Health and Human Services is the United States government's principal agency for protecting the health of all Americans and providing essential human services, especially to those who are least able to help themselves. The Department includes more than 300 programs, covering a wide spectrum of activities.
In the course of carrying out program missions, agencies and staff offices within HHS disseminate a wide variety of information to the public, ranging from research and statistical reports to authoritative health and medical information. Many of these information dissemination activities and products rank among the highest quality scientific, statistical and programmatic information among federal agencies, and in many cases set the national and international standard for quality.
HHS is committed to disseminating information that meets the standards of quality set forth in OMB and in the guidelines discussed in this document. It is HHS's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination. The guidelines that follow describe the quality assurance policies and practices that support information dissemination activities in HHS.
Following an overview of the OMB Guidelines and HHS guidelines, the quality assurance policies of each of the major operating agencies and staff offices of HHS are described. Each set of agency guidelines includes a description of a) the mission of the agency, b) the scope and applicability of the guidelines within the agency, c) the types of information that the agency disseminates, d) the dissemination methods employed by the agency, e) the policies, standards and practices that the agency employs to ensure the quality of the information it disseminates, and f) an administrative mechanism and contact points for each agency so that individuals may seek correction of any information that is believed not to meet the OMB, HHS, or agency-specific guidelines along with an administrative appeals process.
B. � OMB Guidelines
On September 28, 2001, and as amended on February 22, 2002, OMB issued final Guidelines to implement section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554). The statute directs OMB to "issue government wide guidelines that provide policy and procedural guidance to federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by federal agencies." By October 1, 2002, agencies must issue their own implementing guidelines. The guidelines only apply to information that is disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.
In general, the OMB Guidelines require agencies to adopt a basic standard of quality as a performance goal and take appropriate steps to incorporate information quality criteria into agency information dissemination practices. Quality is to be ensured and established at levels appropriate to the nature and timeliness of the information to be disseminated, and specific standards may be adopted that are appropriate to the various categories of information that is disseminated. Agencies are to develop a process for reviewing the quality of information before it is disseminated. Further, information quality is to be treated as an integral step in every aspect of the information development process.
In issuing the Guidelines, OMB outlined several guiding principles. First, OMB designed the Guidelines to apply to a wide variety of government dissemination activities that may range in importance and scope. OMB also designed the Guidelines to be generic enough to fit all media, whether printed or electronic. OMB specifically sought to avoid the problems inherent in developing detailed, prescriptive, "one size fits all" guidelines that would artificially require all types of dissemination activities to be treated in the same manner. Second, OMB designed the Guidelines so that agencies will meet basic information quality standards. The Guidelines recognize that some government information may need to meet higher or more specific standards than others, depending on their purpose and scope. The more important the information, the higher the quality standards to which it might be held, for example, "influential scientific, financial or statistical information" described below. At the same time, OMB recognizes that information quality comes at a cost. Accordingly, agencies are encouraged to weigh the costs and benefits of higher information quality in the development of information, and the level of quality to which it will be held.
Third, OMB designed the Guidelines so that agencies can apply them in a common sense, workable manner. OMB expects agencies to use existing processes rather than create new and potentially duplicative or contradictory processes. Finally, OMB recognizes that the Guidelines cannot be implemented in the same way by all agencies. While the implementation may differ, the essence of the Guidelines will apply. The agencies must make their methods transparent by providing documentation, ensure quality by reviewing the underlying methods used, by consulting as needed with both experts and users, and by keeping users notified about corrections and revisions. These underlying principles apply equally well across the diversity of HHS agencies and information dissemination activities, and they have been adopted in the approach to the HHS Guidelines described below.
C. � HHS Responsibilities
In accordance with the OMB Guidelines, agencies subject to the Paperwork Reduction Act (44USC 3502(1)) are required to:
Issue their own information quality guidelines ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by the agency not later than one year after the issuance of the OMB Guidelines;
Establish administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency that does not comply with the guidelines. Agencies also are to specify appropriate time periods for agency decisions on whether and how to correct the information, and are to notify the affected persons of the action taken. If the person who requested the correction does not agree with the agency's decision (including the corrective action, if any), that person may file for reconsideration within the agency. The agency is to establish an administrative appeal process to review the initial decision, and specify appropriate time limits in which to resolve such requests for reconsideration.
Report periodically to the OMB Director on the number and nature of complaints received by the agency regarding the accuracy of information disseminated by the agency and how such complaints were resolved by the agency.
The HHS guidelines described in this plan incorporate the underlying principles that OMB used in designing their government-wide guidelines. First, the HHS guidelines apply to a wide range of government information dissemination activities across HHS and are generic enough to fit all types of media, including print, electronic, and other forms within HHS. Second, the HHS guidelines are intended to assure that all the information that is disseminated meets a basic level of quality and that more important information meets a more rigorous quality standard. Third, the HHS guidelines explicitly recognize the very different types of information that various HHS agencies disseminate depending on their missions, including the need for flexibility in implementation and avoidance of a "one size fits all" approach. Fourth, the statement of HHS information quality policies and procedures are issued in the form of guidelines and not a regulation.
HHS itself encompasses a broad and diverse range of health and human services programs which, while unified in their pursuit of broad goals, are themselves very diverse, encompassing the nation's largest health insurance plan, the nation's preeminent biomedical research agency, as well as most of the nation's federal capacity for public health protection and preparedness and income assistance to needy families. Accordingly, the HHS approach to implementation of the OMB Guidelines is designed to allow HHS agencies and offices to use existing agency quality assurance mechanisms, and apply the guidelines in a flexible manner that recognizes the mission of the agency, the wide range of data that is disseminated and the frequent reliance on third party sources.
D. � Framework for HHS Guidelines
HHS views the guidelines as an evolving document and process. HHS will continually review the performance of the guidelines in the context of agency statutes and missions and will make revisions to the guidelines as necessary.
In carrying out their diverse statutes and missions, HHS agencies develop and support the dissemination of the following types of substantive information:
Utility involves the usefulness of the information to its intended users. Utility is achieved by staying informed of information needs and developing new data, and information products where appropriate. Based on internal analyses of information requirements, convening and attending conferences, working with advisory committees and stakeholders, sponsoring outreach activities, and where appropriate, testing publications with targeted audiences to ensure relevance, clarity, and comprehensiveness, HHS agencies keeps abreast of information needs.
Integrity refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. HHS agencies have in place rigorous controls to ensure the integrity of its administrative information. Three distinct objectives are pursued in protecting the integrity of information: ensuring that there is no deliberate or accidental improper disclosure of sensitive automated information; protecting against deliberate or accidental corruption of automated information; and protecting against deliberate or accidental actions that cause automated information resources to be unavailable to users when needed. Information is accorded protection against disclosure, alteration, loss, or destruction based on the degree of sensitivity.
In addition, HHS agencies use appropriate safeguards to protect data from improper disclosure by backing up critical data periodically, and, if a security incident occurs, by following proper incident response procedures. Managers are responsible for ensuring that employees, both Government and contractors, observe all security requirements, and that employees receive appropriate security training. HHS also is instituting an enterprise-wide structured management control review process that applies throughout the system life cycle. As part of this process, risk analyses are conducted to establish a balance between an acceptable level of risk and the costs associated with safeguards.
In addition, HHS is subject to a number of statutory requirements that protect the sensitive information it gathers and maintains on individuals. Among these are:
Further, when HHS agencies prepare a Paperwork Reduction Act (PRA) clearance submission, they strive to engage in a data development effort that will result in information that will be collected maintained, and used in a way that is consistent with OMB, HHS, and agency-specific information quality guidelines. As a general policy, HHS views data and information quality as a continuing process that begins at the inception of the information development process with project conceptualization and carries through all phases of data planning, design and execution, including information dissemination activities. Further, HHS agencies that support or sponsor research and statistical activities are encouraged to not only describe the methods and data sources in a clear and transparent manner in the reports they release, but also to make the data used for the report available to the public through public use data files, restricted access research files, research data centers, data archives and other mechanisms consistent with confidentiality, legal and proprietary restrictions. In addition, HHS agencies review the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treat information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. The quality assurance processes that are used to ensure the quality of specific categories of information are described below.
Statistical activities of HHS agencies are based on reliable data sources and are carried out in accordance with modern statistical theory and practice, including scientific sampling, statistical inference and analytical techniques and practices. All statistical programs employ or have access to experts in statistics and research design. HHS houses the National Center for Health Statistics, the federal government's designated general purpose statistical agency for health statistics, as well as programmatic and special purpose statistical activities. All proposals for original data collection activities in HHS undergo a rigorous and exacting review process in connection with the Paperwork Reduction Act, which also provides opportunity for public comment in the design of the information collection. Frequent meetings with user groups are common, and individual surveys and statistical systems often employ project specific technical advisory groups. Substantive reports from HHS statistical activities undergo a quality review process within their organizations before they are released, including expert review by supervisors, internal peer review by qualified scientists and statisticians, and in some cases external peer review as well as expert review by other offices prior to dissemination.
Programmatic and administrative information -- A significant amount of substantive information is disseminated by HHS agencies in connection with and as a byproduct of the administration of programs. Often the programmatic and administrative information disseminated is obtained from third parties, such as States, grantees, health plan contractors or intermediaries, or community-based organizations. In their stewardship function, agencies often collect, compile, standardize, analyze and disseminate such programmatic information. While the reliance on third parties places limits on the federal quality assurance authorities, a variety of techniques are employed to promote the accuracy, completeness and timeliness of the information. These include use of generally accepted accounting and financial management procedures and principles, internal controls, legal certifications and assurances on the part of the organizations supplying the information, audited financial reports and statements, as well as sample audits and site visits, and checks for completeness and consistency with trends and external controls. Programmatic reports are typically subject to supervisory review before release.
Program evaluation studies are often undertaken by HHS agencies to assess program functioning and identify opportunities for improvement. Agencies employ quality assurance procedures in the choice and development of evaluation projects. Proposals for evaluation activities in HHS agencies are usually reviewed by agency management for suitability, utility and methodology in accordance with sound evaluation design and standards of evaluation practice. Many evaluation projects have specific technical advisory committees that oversee the design and conduct of the evaluation in accordance with standard evaluation theory and practice, and they often provide an expert review of the final report. Results of evaluation activities are released to the public only after agency management has completed a review of the quality, accuracy and completeness of the report.
Regulatory information -- A variety of information is used in support of regulatory development and decision-making. Regulatory activity undertaken by HHS agencies closely tracks statutory authorities and program responsibilities. Scientific, financial, and statistical information used in support of regulatory decision-making is subject to a quality review process within the agency involving appropriate experts depending upon the nature of the information. In addition, all significant proposed regulations are reviewed by OMB prior to issuance for public comment, and all proposed regulatory actions provide for extensive public comment.
Authoritative health, scientific and consumer information -- Several HHS agencies develop and disseminate authoritative health and human services information intended for consumers and the professional community. In some instances, the agency simply provides a link to information developed by other authoritative organizations. In other instances, the agency develops its own consumer and professional practice information. In the latter case, the information is reviewed for scientific and medical accuracy and completeness by experts within the agency before it is disseminated. In a number of instances the information also is reviewed by scientific and medical advisory bodies before dissemination as well, depending on the nature of the information.
Public health surveillance and epidemiological information -- Several HHS agencies compile, analyze and disseminate information from public health surveillance systems and epidemiological activities. In many surveillance systems, the primary information is developed by State and local government agencies, clinical laboratories and other health care entities and reported to CDC for national aggregation and analysis. Data quality is assured through use of reliable data sources, appropriate statistical techniques, agreement on national reporting standards, quality control procedures, standard case definitions and reports, adherence to professional practices and standards for public health reporting in the U.S., and frequent consultation with the user community. Before such information is disseminated, it is reviewed for medical, scientific and public health accuracy, soundness and utility by agency experts. Comment and feedback is encouraged on such information, and HHS agencies work closely with the relevant professional and public health organizations.
Because of confidentiality, ethical and feasibility constraints and legal obligations to third parties supplying the information, there may be instances where original or supporting data may not be available to the public, but HHS agencies typically will work with qualified third parties to facilitate understanding, and transparency in data sources and methods will be emphasized in the report or in reference documents. In the case of analytical studies, HHS agencies will make provisions for sufficient transparency about data and methods so that an independent reanalysis could be undertaken by a qualified member of the public.
E. � Overview of HHS Agency Complaint Procedures
The OMB Guidelines require all agencies to establish administrative mechanisms allowing affected persons to seek and obtain correction of information disseminated by the agency that does not comply with OMB, HHS or agency-specific guidelines. Agencies also are to specify appropriate time periods for agency decisions on whether and how to correct the information, and are to notify the affected persons of the action taken. If the person who requested the correction does not agree with the agency's decision (including the corrective action, if any), that person may file for reconsideration within the agency. The agency is to establish an administrative appeal process to review the initial decision, and specify appropriate time limits in which to resolve such requests for reconsideration. The agency need not respond substantively to information not covered by the guidelines or to frivolous or repetitive requests for correction.
Requests for correction of information quality pursuant to Section 515 are to be directed to the respective agency. A common format for complaints has been developed across HHS. The approach is described below. To accompany the actual implementation of the Guidelines in October 2002, HHS has created a department-wide website describing in user friendly terms the procedures and contact persons for submitting requests for corrections. The web site can be accessed at www.hhs.gov/infoquality.
In general, to seek an information quality request for correction of
information disseminated by any HHS agency, individuals should follow the
procedures described below. Requests for correction that are specific and
provide evidence to support the need for correction will enable the agency
to provide a satisfactory response. Complainants should be aware that they
bear the "burden of proof" with respect to the necessity for correction as
well as with respect to the type of correction they seek.
Responsibility of the Agency
Based on a review of the information provided, the agency will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information, the magnitude of the correction and the resource requirements for the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
Appeals for Reconsideration
If the individual submitting the complaint does not agree with the agency's decision (including the corrective action), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to the specific agency appeals address.
The agency official who handles the original complaint will not have responsibility for resolving the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
Rulemakings and Other Public Comment Procedures
Existing public comment procedures for rule-makings and other formal agency actions already provide well established procedural safeguards that allow affected persons to raise information quality issues on a timely basis. Accordingly, agencies will use these existing procedures to respond to information quality complaints that arise in this process.
In cases where the agency disseminates a study, analysis, or other information prior to the final agency action or information product, requests for correction will be considered prior to the final agency action or information product in those cases where in the agency's judgment issuing an earlier response would not unduly delay issuance of the agency action or information product and the complainant has shown a reasonable likelihood of suffering actual harm from the agency's dissemination if the agency does not resolve the complaint prior to the final agency action or information product.
Last revised: November 13, 2003