Guidelines for Ensuring the Quality of Information Disseminated to the Public
E. Centers for Medicare & Medicaid Services
[ Full Contents of Guidelines ]
-
Agency Mission
-
Scope and Applicability of Guidelines
-
Types of Information Disseminated
-
Types of Dissemination Methods
-
CMS Policies, Standards and Processes for Ensuring the Quality
of Information Disseminated to the Public
-
Agency Administrative Complaint Process
-
Influential Scientific, Financial and Statistical
Information
The Centers for Medicare & Medicaid Services (CMS) administers the Medicare
program, providing health care security and choice for aged and disabled
people in this country. Jointly with the State governments, CMS administers
the Medicaid program and the State Children's Health Insurance Program (SCHIP).
CMS is the largest purchaser of health care in the United States
our programs account for more than one third of the dollars spent on health
care in the U.S. economy. In 2001, CMS programs provided health care coverage
for 70 million people nearly one out of every four Americans.
In administering the Medicare, Medicaid and SCHIP, CMS activities are performed
with one thing in mind: a commitment to the people who rely upon our programs
for access to and quality of the health care they need. CMS' mission, though
complex in execution, is simply stated: We assure health care security for
beneficiaries. And our vision is no less ambitious: In the stewardship of
our programs, we lead the Nation's health care system toward improved health
for all.
CMS could not accomplish its mission by working alone. We work in partnership
with many other organizations and individuals: other Federal and State agencies,
and Territorial and Tribal governments; private-sector contractors; health
plans, practitioners, and health care facilities; professional organizations,
academia, and advocacy groups; and beneficiaries and their families.
[ Go to Contents ]
CMS will ensure that disseminated information meets the standards of quality
set forth in the OMB, HHS and CMS guidelines. It is CMS's goal to ensure
and maximize the quality, objectivity, utility, and integrity of information
that it disseminates to the public. We strive to provide information that
is accurate, reliable, clear, complete, unbiased, and useful. We are committed
to integrating the principle of information quality into every phase of
information development, including creation, collection, maintenance, and
dissemination.
The pre-dissemination review described in the guidelines only applies to
substantive information disseminated on or after October 1, 2002. The
administrative mechanism for correction applies to information that the agency
disseminates on or after October 1, 2002, regardless of when the agency first
disseminated the information.
-
Information Covered
Information that IS subject to the Information Quality Guidelines
includes:
-
statistics and information designed for public dissemination to help CMS
program beneficiaries make informed choices
-
statistical or actuarial information
-
studies and summaries prepared for public dissemination to inform the public
about the impact of CMS programs
-
studies and summaries prepared for use in formulating broad program policy.
To correct information that is subject to the Information Quality Guidelines,
please follow the procedure for correcting CMS information outlined in
Section VI of this notice.
-
Information Not Covered
Information that is NOT subject to the Information Quality Guidelines
includes:
-
personal information maintained for purposes of enrollment or adjudicative
decisions
-
data used to verify and pay individual claims
-
information pertaining to basic agency operations, such as the Chief Financial
Officer's Report
-
procedural and policy manuals
-
management information that is produced primarily for internal use
-
information which is disseminated only to CMS employees or agency contractors
-
opinions where the agency's presentation makes it clear that what is being
offered is personal opinion rather than fact or the agency views
-
press releases that support the announcement or give public notice of information
that CMS has disseminated elsewhere.
[ Go to Contents ]
Generally speaking, information released by CMS falls into one of two broad
categories: Medicare consumer information, and program information and
evaluation. Typically, CMS does not provide detailed consumer information
about Medicaid or SCHIP because these functions are carried out by individual
State governments.
-
Medicare consumer information
This information is designed to help people enrolled in Medicare make informed
decisions about the types of insurance and service providers they use. For
example,
-
Medicare & You is an annual handbook mailed to Medicare enrollees.
The book provides basic benefit information, and also provides information
about managed care plans and other types of health insurance available to
enrollees.
-
Nursing Home Compare is a web site containing information about long-term
care facilities. In combination with other factors, this information can
be used by people who are contemplating nursing home care for themselves
or others, to decide which facility is best for them.
-
Program information and evaluation
This type of information is designed to help improve the performance of CMS
programs. Examples include:
-
Program information provides summary statistics describing use of
CMS program services, the number of people using those services, expenditure
for those services, etc. For example, CMS publishes an annual Medicare
and Medicaid Statistical Supplement to the Health Care Financing
Review.
-
Statistical data sets are claims-level, person-level, or provider-level
data used for program evaluation, health services research, or policy
development. For example, CMS prepares Standard Analytic Files (SAFs) extracting
Medicare hospital inpatient claims that can be used by outside researchers
who abide by a data use agreement. The agency also releases information from
the Medicare Current Beneficiary Survey (MCBS), a survey of a random sample
of Medicare enrollees that collects social, demographic, and economic information
about those enrollees.
-
Research and evaluation reports focus on issues germane to CMS programs
and policy. Typically, each innovation in CMS programs must be evaluated;
often, such innovations are preceded by research performed by CMS staff or
by outside researchers under contract to the agency. Many of these evaluations
are released through Reports to the Congress.
-
Technical reports summarize specific aspects of CMS and its program
activities. For example, each year CMS actuaries produce the data underlying
the report of Medicare's trustees on the financial health of the program.
-
Payment updates establish the amount Medicare will pay for particular
services or for capitated care of beneficiaries. For example, each year the
agency publishes a fee schedule update that determines payments for physician
services.
[ Go to Contents ]
CMS uses four modes to disseminate information.
-
Internet
The agency maintains two web sites. One of them
www.medicare.gov is devoted
to consumer information. It is intended for use by program beneficiaries,
their families and care-givers, and their advocates. The second
site
www.cms.hhs.gov is designed
to serve researchers, policymakers, and health care providers. The sites
are cross-linked for ease of access.
-
Printed matter
CMS produces a wide variety of pamphlets, handbooks, and directories related
to our programs. For example, CMS publishes the Health Care Financing
Review, containing scholarly articles relevant to CMS programs as well
as data highlights from those programs. In addition to the Medicare &
You handbook described above, smaller leaflets are prepared to describe
program eligibility, enrollment, and benefits.
-
Storage Media
Data from CMS operations are available in a variety of storage media upon
request for use at non-CMS computing facilities. Aside from download files
available through the Internet, this form of information dissemination is
ad hoc in nature.
-
Telephone
CMS maintains a national toll-free telephone number, 1-800-MEDICARE, to process
questions and provide information about the Medicare program.
-
Multi-media
CMS is engaged in an ongoing multimedia effort to provide information about
the Medicare program to its enrollees. The National Medicare Education Program
(NMEP) is unique among the agency's information products in that it is
intentionally designed to incorporate Internet, printed, and telephone
information seamlessly.
[ Go to Contents ]
At CMS, the quality assurance process begins at the inception of the information
development process. Information released by CMS is developed from reliable
data sources using accepted methods for data collection and analysis, and
is based on thoroughly reviewed analyses and models. Further, CMS reviews
the quality (including the objectivity, utility, and integrity) of information
before it is disseminated and treats information quality as integral to every
step of the development of information, including its creation, collection,
maintenance and dissemination. The guidelines below describe procedures
that CMS employs to assure the quality of its information products, including
their utility, objectivity, integrity, transparency, and reproducibility.
-
Utility
Utility involves the usefulness of the information to its intended users.
Utility is achieved by staying informed of information needs and developing
new data, models, and information products where appropriate.
Based on internal analyses of information requirements, convening and attending
conferences, working with advisory committees, and sponsoring outreach
activities, CMS keeps abreast of information needs with respect to the analysis
of CMS programs. To the extent possible, we base our analysis on CMS
administrative data and surveys by other federal agencies or established
survey organizations. When major needs for data related to CMS populations
are identified, CMS conducts special-purpose surveys to address these needs.
CMS's ongoing publication series and other information products are reviewed
to ensure that they remain relevant and address current information needs.
Based on internal product reviews, consultation with users, and in response
to changing needs and emphases, content of ongoing information products is
changed, new products are introduced and others discontinued. CMS prepares
special reports and topical studies that address emerging information needs
stemming from proposed changes in the law and related policy debates. Where
appropriate, contact information is available on each publication (and in
some cases on each table of a publication) to allow feedback and questions
by users.
New and revised information products are tested with focus groups of intended
recipients. In many cases, the structure of the content itself is a collaborative
process involving providers, consumers, academicians, and policy analysts.
-
Objectivity
Objectivity involves a focus on ensuring that information is accurate, reliable
and unbiased and that information products are presented in an accurate,
clear, complete and unbiased manner. Objectivity is achieved by using reliable
data sources and sound analytical techniques, and carefully reviewing information
products prepared by qualified people using proven methods.
Using reliable data sources. Much of the information disseminated
by CMS is based on administrative data files. These files contain information
used to manage CMS programs, including data to determine benefit entitlement,
and to compute and pay benefits. CMS staff conduct ongoing reviews of claims
information in CMS data systems to ensure its accuracy. CMS administrative
data are also covered under CMS's Financial Management Systems and conform
to the high standards of financial accountability demanded by these Systems.
These financial management systems are mandated by the Office of Management
and Budget and are designed to provide complete, reliable, consistent, timely
and useful management information to enable agencies to carry out their fiduciary
responsibilities.
CMS-sponsored surveys are conducted using methodologies that are consistent
with generally accepted professional standards for all aspects of survey
development, including sample frame development, statistical design of the
survey sample, questionnaire design and testing, data collection, sampling
and coverage errors, nonresponse analysis, imputation of missing data, weights
and variance estimates. CMS surveys follow guidelines and policies set forth
in the Paperwork Reduction Act (PRA) and other regulations related to the
conduct of government surveys. Through the PRA process, CMS ensures that
information will be collected, maintained, and used in a way that is consistent
with OMB, HHS and CMS information quality guidelines. CMS also prepares
information products using data produced or maintained by other federal agencies
and established survey organizations. Where possible, external data used
by CMS are produced using generally accepted methodologies.
Where samples from administrative data files are employed for analysis, sound
statistical methods are employed to develop samples. Staff involved in producing
statistical publications are knowledgeable about the content, structure and
limitations of the administrative data files and maintain working relations
with staff who create, update and maintain these files.
Where administrative files are linked to surveys for analysis, sound procedures
for extracting and linking data from external sources are employed, based
on a thorough understanding of the relevant components of the data sources.
Computer programs for sampling, extracting and linking external data files
are developed and tested by technically qualified staff. Samples from
administrative files are evaluated to ensure that samples are representative
of the underlying administrative data files. Estimates prepared from external
data sources are reviewed to ensure that the data extraction and linkage
processes were implemented correctly.
Using sound analytic techniques. Analytical reports are prepared
using a variety of analytical techniques. These techniques range from simple
tabulations and descriptive summary statistics to multivariate statistical
methods and econometric models. Analytical techniques are reviewed for their
appropriateness to the data and the analysis being conducted and are clearly
identified in reports. Actuarial estimates are subject to the standards of
that profession and are certified by the Chief Actuary.
Preparing statistical data products. Estimates in statistical
data products are prepared from representative random samples of CMS
administrative data files and from reliable external data sources. Procedures
for sampling from administrative files and linking external data files to
administrative data files are prepared using accepted statistical methods.
Output is reviewed by knowledgeable staff within the originating component
and across components as appropriate. Estimates are compared to prior year
estimates and estimates from other sources to ensure consistency, reasonableness
and reliability.
All data sources used in producing statistical data products are identified,
either for the publication as a whole or for individual tables. Documentation
includes specification of variables used, definitions of variables when
appropriate, sampling errors and disclosure avoidance rules or techniques.
Preparing analytical reports and policy studies. Information
contained in analytical reports and policy studies is based on estimates
derived from reliable administrative data files and external data sources.
Analysts apply sound statistical and analytical techniques and are knowledgeable
about the data sources and models being used.
All data sources are identified. When analyses are based on simulation model
projections, the assumptions used to produce the projections are also identified
as well as the rationale for the assumptions used and the impact of using
alternative assumptions.
All analytic reports and policy studies are reviewed by technically qualified
staff to ensure that analysis is valid, complete, unbiased, objective and
relevant. Analytic reports and policy studies that are considered to be more
technically complex are also reviewed by subject matter experts outside of
the originating component to provide additional perspective and expertise.
Establishing editorial review for accuracy and clarity of information
in publications. All information products are edited and proofread
before release to ensure clarity and coherence of the final report. Text
is edited to ensure that the report is easy to read and grammatically correct,
that thoughts and arguments flow logically, and that information is worded
concisely and lucidly. Tables and charts are edited to ensure that they clearly
and accurately illustrate and support points made in the text, and include
concise but descriptive titles. Tables and charts clearly indicate the unit
of measure and the universe being examined and all internal labels (column
heads, row stubs, and panel headings) accurately describe the information
they contain. All changes made to a manuscript during the editing process
are checked by a proofreader and reviewed and approved by the author.
-
Integrity
Integrity refers to the security of information from unauthorized access
or revision to ensure that the information is not compromised through corruption
or falsification. CMS has in place rigorous controls to ensure the integrity
of its administrative information.
CMS is highly protective of the confidentiality of information it holds through
its policies and practices. CMS administers social insurance programs
that directly or indirectly touch the lives of almost
every American. CMS has in place programs and policies for securing CMS resources
as required by the Government Information Security Reform Act (P.L.106-398,
Title X, Subtitle G). These security procedures address all major components
of information security.
CMS is subject to a number of statutory requirements that protect the sensitive
information it gathers and maintains on individuals. Among these are:
-
Health Insurance Portability and Accountability Act of 1996
-
Privacy Act of 1974
-
Computer Security Act of 1987
-
Office of Management and Budget (OMB) Circulars A-123, A-127, and A-130
-
Government Information Security Reform Act
-
Federal Managers' Financial Integrity Act (FMFIA) of 1982
-
Transparency and Reproducibility
If an agency is responsible for disseminating "influential" information,
guidelines for dissemination should include a high degree of transparency
about data and methods to facilitate its reproducibility by qualified third
parties. Information is considered influential if it will have a substantial
impact on important public policies or important private sector decisions.
CMS's guidelines call for identification and documentation of data sets used
in producing estimates and projections, and for clear descriptions of the
methods used. Many estimates and projections included in CMS information
products are not directly reproducible by the public because the underlying
data sets used to produce them are confidential. However, some statistical
publications that are based on publicly available data and whose programs
are made available on request are fully reproducible by the public. Further,
some CMS data is available to people outside the agency through adherence
to a data use agreement, provided that the use of the data is consistent
with the appropriate systems of records notice. Where estimates and projections
may not be easily reproduced by third parties due to the complexity and detail
of the methods and data, greater emphasis is placed on periodic review by
outside panels of technical experts.
CMS also achieves transparency through wide dissemination of its information.
Many reports and other data products are available both as printed and electronic
documents. New documents posted on our Web sites are section 508 compliant,
making information available to an audience that includes persons who have
a visual impairment and read online using assistive technology. (Older documents
are being converted to meet section 508 standards.)
[ Go to Contents ]
CMS has developed administrative mechanisms to allow affected persons to
seek and obtain correction of disseminated information that does not comply
with OMB, HHS and CMS guidelines.
Existing public comment procedures for rule-makings and other formal agency
actions already provide well established procedural safeguards that allow
affected persons to raise information quality issues on a timely basis.
Accordingly, CMS will use these existing procedures to respond to information
quality complaints that arise in this process.
In cases where agency disseminates a study, analysis, or other information
prior to the final agency action or information product, requests for correction
will be considered prior to the final agency action or information product
in those cases where in the agency's judgment issuing an earlier response
would not unduly delay issuance of the agency action or information product
and the complainant has shown a reasonable likelihood of suffering actual
harm from the agency's dissemination if the agency does not resolve the complaint
prior to the final agency action or information product.
-
Responsibility of the Complainant
To seek a correction of information disseminated by the agency, individuals
shall follow the procedures described below.
-
A complaint or request for review and correction of information shall be
in written hard copy or electronic form;
-
it shall be sent to the agency by mail or electronic-mail (e-mail) at the
address below; and
-
it shall state that an information quality request for correction is being
submitted.
The complaint shall contain
-
a detailed description of the specific material that needs to be corrected
including where the material is located, i.e. the publication title, date,
and publication number, if any, or the web site and web page address (url),
or the speech title, presenter, date and place of delivery;
-
the specific reasons for believing the information does not comply with OMB,
HHS, or CMS guidelines and is in error and supporting documentation, if any;
-
the specific recommendations for correcting the information;
-
a description of how the person submitting the complaint is affected by the
information error; and
-
the name, mailing address, telephone number, e-mail address, and organizational
affiliation, if any, of the individual making the complaint.
Complainants should be aware that they bear the "burden of proof" with respect
to the necessity for correction as well as with respect to the type of correction
they seek.
-
Responsibility of the Agency
Based on a review of the information provided, the agency will determine
whether a correction is warranted and if, so what action to take. The agency
will respond to the requestor by letter or e-mail. The agency's response
will explain the findings of the review and the actions that the agency will
take, if any. The response will consider the nature and timeliness of the
information involved and such factors as the significance of the correction
on the use of the information and the magnitude of the correction. The response
will describe how the complainant may request reconsideration. The agency
will respond to all requests for correction within 60 calendar days of receipt.
If the request requires more than 60 calendar days to resolve, the agency
will inform the complainant that more time is required and indicate the reason
why and an estimated decision date.
-
Appeals
If the individual submitting the complaint does not agree with the agency's
decision (including the corrective action, if any), the complainant may send
a written hard copy or electronic request for reconsideration within 30 days
of receipt of the agency's decision. The appeal shall state the reasons why
the agency response is insufficient or inadequate. Complainants shall attach
a copy of their original request and the agency response to it, clearly mark
the appeal with the words, "Information Quality Appeal," and send the appeal
to the specific agency appeals address.
The agency official who resolved the original complaint will not have
responsibility for the appeal. The agency will respond to all requests for
appeals within 60 calendar days of receipt. If the request requires more
than 60 calendar days to resolve, the agency will inform the complainant
that more time is required and indicate the reason why and an estimated decision
date.
-
Contact
Complaints filed under this guideline shall be mailed to:
Information Quality
Centers for Medicare & Medicaid Services
7500 Security Boulevard
Baltimore, MD 21244-1850
Alternatively, the complaint can be emailed to
infoquality@cms.hhs.gov
Appeals filed under this guideline should be mailed or emailed to the same
address, and should contain the word "appeal" on the envelope or in the subject
line.
[ Go to Contents ]
The OMB Information Quality Guidelines require that "influential" scientific,
financial, or statistical information in official Government documents must
be based on data or studies that could be substantially reproduced if the
original or supporting data were to be independently reanalyzed using the
same methods. "Influential" means that CMS can reasonably determine that
dissemination of the information will have a substantial impact on important
public policies or important private sector decisions or will have important
consequences for specific health practices, technologies, substances, produces,
or firms. Examples of the types of information disseminated by CMS that have
the potential to be influential include the annual reports of the Medicare
Board of Trustees and annual publication of provider payment rates.
CMS is committed to applying rigorous scientific standards to ensure the
accuracy and reliability of program evaluation results. The scientific/research,
financial, and statistical community recognizes peer review as the primary
means of quality control. CMS routinely seeks input from qualified peer
reviewers, inside and outside the Federal government prior to dissemination
of this type of information.
Last revised: December 13, 2006