Guidelines for Ensuring the Quality of Information Disseminated to the Public
E. � Centers for Medicare & Medicaid Services
I. � Agency Mission
The Centers for Medicare & Medicaid Services (CMS) administers the Medicare program, providing health care security and choice for aged and disabled people in this country. Jointly with the State governments, CMS administers the Medicaid program and the State Children's Health Insurance Program (SCHIP). CMS is the largest purchaser of health care in the United States -- our programs account for more than one third of the dollars spent on health care in the U.S. economy. In 2001, CMS programs provided health care coverage for 70 million people -- nearly one out of every four Americans.
In administering the Medicare, Medicaid and SCHIP, CMS activities are performed with one thing in mind: a commitment to the people who rely upon our programs for access to and quality of the health care they need. CMS' mission, though complex in execution, is simply stated: We assure health care security for beneficiaries. And our vision is no less ambitious: In the stewardship of our programs, we lead the Nation's health care system toward improved health for all.
CMS could not accomplish its mission by working alone. We work in partnership with many other organizations and individuals: other Federal and State agencies, and Territorial and Tribal governments; private-sector contractors; health plans, practitioners, and health care facilities; professional organizations, academia, and advocacy groups; and beneficiaries and their families.
II. � Scope and Applicability of Guidelines
CMS will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and CMS guidelines. It is CMS's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.
The pre-dissemination review described in the guidelines only applies to substantive information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.
To correct information that is subject to the Information Quality Guidelines, please follow the procedure for correcting CMS information outlined in Section VI of this notice.
III. � Types of Information Disseminated
Generally speaking, information released by CMS falls into one of two broad categories: Medicare consumer information, and program information and evaluation. Typically, CMS does not provide detailed consumer information about Medicaid or SCHIP because these functions are carried out by individual State governments.
IV. � Types of Dissemination Methods
CMS uses four modes to disseminate information.
V. � Agency Quality Assurance Policies, Standards and Processes
At CMS, the quality assurance process begins at the inception of the information development process. Information released by CMS is developed from reliable data sources using accepted methods for data collection and analysis, and is based on thoroughly reviewed analyses and models. Further, CMS reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.� The guidelines below describe procedures that CMS employs to assure the quality of its information products, including their utility, objectivity, integrity, transparency, and reproducibility.
Based on internal analyses of information requirements, convening and attending conferences, working with advisory committees, and sponsoring outreach activities, CMS keeps abreast of information needs with respect to the analysis of CMS programs. To the extent possible, we base our analysis on CMS administrative data and surveys by other federal agencies or established survey organizations. When major needs for data related to CMS populations are identified, CMS conducts special-purpose surveys to address these needs. CMS's ongoing publication series and other information products are reviewed to ensure that they remain relevant and address current information needs. Based on internal product reviews, consultation with users, and in response to changing needs and emphases, content of ongoing information products is changed, new products are introduced and others discontinued. CMS prepares special reports and topical studies that address emerging information needs stemming from proposed changes in the law and related policy debates. Where appropriate, contact information is available on each publication (and in some cases on each table of a publication) to allow feedback and questions by users.
New and revised information products are tested with focus groups of intended recipients. In many cases, the structure of the content itself is a collaborative process involving providers, consumers, academicians, and policy analysts.
Using reliable data sources. Much of the information disseminated by CMS is based on administrative data files. These files contain information used to manage CMS programs, including data to determine benefit entitlement, and to compute and pay benefits. CMS staff conduct ongoing reviews of claims information in CMS data systems to ensure its accuracy. CMS administrative data are also covered under CMS's Financial Management Systems and conform to the high standards of financial accountability demanded by these Systems. These financial management systems are mandated by the Office of Management and Budget and are designed to provide complete, reliable, consistent, timely and useful management information to enable agencies to carry out their fiduciary responsibilities.
CMS-sponsored surveys are conducted using methodologies that are consistent with generally accepted professional standards for all aspects of survey development, including sample frame development, statistical design of the survey sample, questionnaire design and testing, data collection, sampling and coverage errors, nonresponse analysis, imputation of missing data, weights and variance estimates. CMS surveys follow guidelines and policies set forth in the Paperwork Reduction Act (PRA) and other regulations related to the conduct of government surveys. Through the PRA process, CMS ensures that information will be collected, maintained, and used in a way that is consistent with OMB, HHS and CMS information quality guidelines. CMS also prepares information products using data produced or maintained by other federal agencies and established survey organizations. Where possible, external data used by CMS are produced using generally accepted methodologies.
Where samples from administrative data files are employed for analysis, sound statistical methods are employed to develop samples. Staff involved in producing statistical publications are knowledgeable about the content, structure and limitations of the administrative data files and maintain working relations with staff who create, update and maintain these files.
Where administrative files are linked to surveys for analysis, sound procedures for extracting and linking data from external sources are employed, based on a thorough understanding of the relevant components of the data sources. Computer programs for sampling, extracting and linking external data files are developed and tested by technically qualified staff. Samples from administrative files are evaluated to ensure that samples are representative of the underlying administrative data files. Estimates prepared from external data sources are reviewed to ensure that the data extraction and linkage processes were implemented correctly.
Using sound analytic techniques. Analytical reports are prepared using a variety of analytical techniques. These techniques range from simple tabulations and descriptive summary statistics to multivariate statistical methods and econometric models. Analytical techniques are reviewed for their appropriateness to the data and the analysis being conducted and are clearly identified in reports. Actuarial estimates are subject to the standards of that profession and are certified by the Chief Actuary.
Preparing statistical data products. Estimates in statistical data products are prepared from representative random samples of CMS administrative data files and from reliable external data sources. Procedures for sampling from administrative files and linking external data files to administrative data files are prepared using accepted statistical methods.
Output is reviewed by knowledgeable staff within the originating component and across components as appropriate. Estimates are compared to prior year estimates and estimates from other sources to ensure consistency, reasonableness and reliability.
All data sources used in producing statistical data products are identified, either for the publication as a whole or for individual tables. Documentation includes specification of variables used, definitions of variables when appropriate, sampling errors and disclosure avoidance rules or techniques.
Preparing analytical reports and policy studies. Information contained in analytical reports and policy studies is based on estimates derived from reliable administrative data files and external data sources. Analysts apply sound statistical and analytical techniques and are knowledgeable about the data sources and models being used.
All data sources are identified. When analyses are based on simulation model projections, the assumptions used to produce the projections are also identified as well as the rationale for the assumptions used and the impact of using alternative assumptions.
All analytic reports and policy studies are reviewed by technically qualified staff to ensure that analysis is valid, complete, unbiased, objective and relevant. Analytic reports and policy studies that are considered to be more technically complex are also reviewed by subject matter experts outside of the originating component to provide additional perspective and expertise.
Establishing editorial review for accuracy and clarity of information in publications. All information products are edited and proofread before release to ensure clarity and coherence of the final report. Text is edited to ensure that the report is easy to read and grammatically correct, that thoughts and arguments flow logically, and that information is worded concisely and lucidly. Tables and charts are edited to ensure that they clearly and accurately illustrate and support points made in the text, and include concise but descriptive titles. Tables and charts clearly indicate the unit of measure and the universe being examined and all internal labels (column heads, row stubs, and panel headings) accurately describe the information they contain. All changes made to a manuscript during the editing process are checked by a proofreader and reviewed and approved by the author.
CMS is highly protective of the confidentiality of information it holds through its policies and practices. CMS administers social insurance programs that -- directly or indirectly -- touch the lives of almost every American. CMS has in place programs and policies for securing CMS resources as required by the Government Information Security Reform Act (P.L.106-398, Title X, Subtitle G). These security procedures address all major components of information security.
CMS is subject to a number of statutory requirements that protect the sensitive information it gathers and maintains on individuals. Among these are:
CMS's guidelines call for identification and documentation of data sets used in producing estimates and projections, and for clear descriptions of the methods used. Many estimates and projections included in CMS information products are not directly reproducible by the public because the underlying data sets used to produce them are confidential. However, some statistical publications that are based on publicly available data and whose programs are made available on request are fully reproducible by the public. Further, some CMS data is available to people outside the agency through adherence to a data use agreement, provided that the use of the data is consistent with the appropriate systems of records notice. Where estimates and projections may not be easily reproduced by third parties due to the complexity and detail of the methods and data, greater emphasis is placed on periodic review by outside panels of technical experts.
CMS also achieves transparency through wide dissemination of its information. Many reports and other data products are available both as printed and electronic documents. New documents posted on our Web sites are section 508 compliant, making information available to an audience that includes persons who have a visual impairment and read online using assistive technology. (Older documents are being converted to meet section 508 standards.)
VI. � Agency Administrative Complaint Procedures
CMS has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and CMS guidelines.
Existing public comment procedures for rule-makings and other formal agency actions already provide well established procedural safeguards that allow affected persons to raise information quality issues on a timely basis. Accordingly, CMS will use these existing procedures to respond to information quality complaints that arise in this process.
In cases where agency disseminates a study, analysis, or other information prior to the final agency action or information product, requests for correction will be considered prior to the final agency action or information product in those cases where in the agency's judgment issuing an earlier response would not unduly delay issuance of the agency action or information product and the complainant has shown a reasonable likelihood of suffering actual harm from the agency's dissemination if the agency does not resolve the complaint prior to the final agency action or information product.
The complaint shall contain
Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek.
The agency official who resolved the original complaint will not have responsibility for the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
Centers for Medicare & Medicaid Services
7500 Security Boulevard
Baltimore, MD 21244-1850
Alternatively, the complaint can be emailed to email@example.com
Appeals filed under this guideline should be mailed or emailed to the same address, and should contain the word "appeal" on the envelope or in the subject line.
VII. � Influential Scientific, Financial, and Statistical Information
The OMB Information Quality Guidelines require that "influential" scientific, financial, or statistical information in official Government documents must be based on data or studies that could be substantially reproduced if the original or supporting data were to be independently reanalyzed using the same methods. "Influential" means that CMS can reasonably determine that dissemination of the information will have a substantial impact on important public policies or important private sector decisions or will have important consequences for specific health practices, technologies, substances, produces, or firms. Examples of the types of information disseminated by CMS that have the potential to be influential include the annual reports of the Medicare Board of Trustees and annual publication of provider payment rates.
CMS is committed to applying rigorous scientific standards to ensure the accuracy and reliability of program evaluation results. The scientific/research, financial, and statistical community recognizes peer review as the primary means of quality control. CMS routinely seeks input from qualified peer reviewers, inside and outside the Federal government prior to dissemination of this type of information.
Last revised: November 12, 2003