The Office
of Management and Budget (OMB) has issued
government-wide information quality guidelines
under
section 515 of the Treasury and General Government Appropriations Act for Fiscal
Year 2001. The OMB section 515 guidelines have been prepared to ensure and
maximize the quality, utility, objectivity, and integrity of information
disseminated by federal agencies. The guidelines direct each federal agency to
issue its own section 515 guidelines. In response to the OMB directive, the
guidelines presented here (highlighted in bold) describe procedures the Railroad
Retirement Board (RRB) will employ to ensure the quality of its information
products, including their utility,
objectivity,
integrity,
transparency, and
reproducibility. The accompanying
text describes steps that RRB is currently taking that conform to these
guidelines.
RRB provides information to the public about the
current experience of its programs, the projected scope and impact of those
programs in the future, and the effect of proposed changes to the Railroad
Retirement system. Information products describe the impact of RRB programs on
our economy, society, and beneficiary populations and provide detailed
demographic and economic information on beneficiaries and covered workers. Those
products are used by government planners and policymakers as well as by
actuaries, economists and other social scientists, the media, and the public to
analyze Railroad Retirement programs and their impact on the nation.
Utility
Utility involves the usefulness
and availability of the information to its intended audience. To produce
useful, policy-relevant information, agencies must stay informed of
information needs and develop new data, models, and information products
where appropriate.
RRB will keep
informed of information needs through active and ongoing contact with the
user community and will provide vehicles for user input into our information
programs. RRB keeps abreast of information needs with respect to the
analysis of RRB programs through a variety of means, including carrying out
internal analyses of information requirements, convening and attending
conferences, conducting user surveys, working with advisory committees, and
sponsoring outreach activities. Contact information is available, where
appropriate, on a variety of information products to allow for questions,
comments, and suggestions from users.
RRB's analytical
and statistical publications and other information products will be reviewed
to ensure that they remain relevant and timely and that they address current
information needs. On the basis of internal product reviews and
consultation with users, and in response to changing needs and emphases, the
content of ongoing information products is changed, new products are
introduced, and some products are discontinued. RRB prepares special reports
and topical studies that address emerging information needs stemming from
proposed changes in the law and related policy debates. RRB identifies
requirements for simulation models to support the preparation of analytical
reports and policy studies and modifies its current models or develops new
models accordingly. When major gaps in data related to RRB populations are
identified, RRB may conduct special-purpose surveys to address those needs.
RRB's information
dissemination process will make information products widely available and
broadly accessible. Most reports and other data products are
available both as printed and electronic documents. They are announced on
the RRB Web site, and most electronic versions can be accessed and
downloaded directly. All documents posted on our Web site since June 21,
2001, are in compliance with section 508 and are therefore available to an
audience that includes persons who have a visual impairment and read online
using assistive technology.
Objectivity
Objectivity
involves a focus on ensuring that information and its presentation are
accurate, reliable, and unbiased and that information products are presented
in an accurate, clear, complete, unbiased and well-documented manner.
Objectivity is achieved by using reliable data sources and sound analytical
techniques, by having information products prepared by qualified people
using proven methods, and by carefully reviewing the content of all
information products.
Use
of reliable data sources
Information products disseminated by RRB will be
based on reliable, accurate data that have been validated. Much of the
information disseminated by RRB is based on Railroad Retirement
administrative data files. Those files contain information used to manage
RRB programs, including data to determine benefit entitlement, and to
compute and pay benefits. RRB conducts ongoing quality assurance reviews of
claims and postentitlement information in its data systems to ensure their
accuracy. RRB administrative data are also covered under RRB's financial
management systems and conform to their high standards of financial
accountability. Those systems are mandated by the Office of Management and
Budget and are designed to provide complete, reliable, consistent, timely,
and useful management information to enable agencies to carry out their
fiduciary responsibilities. In addition, RRB information products employ
reliable data from external sources; in particular, administrative data and
survey data from other federal agencies as well as survey data from
established survey organizations.
Surveys sponsored by RRB will be conducted using
methodologies that are consistent with generally accepted professional
standards for all aspects of survey design and implementation. RRB employs
and documents accepted professional standards and practices for all major
survey activities, including sample frame development, sample design,
questionnaire design and testing, data collection, analysis of sampling and
coverage errors, nonresponse analysis, imputation of missing data,
weighting, and variance estimation. RRB surveys follow guidelines and
policies set forth in the Paperwork Reduction Act and other regulations
related to the conduct of government surveys. PRA submissions for RRB
statistical surveys shall state that the survey will be conducted using
methodologies that are consistent with generally accepted professional
standards for all aspects of survey design and implementation and that
survey data quality will conform to RRB section 515 information quality
standards.
All data employed in the preparation of
information products will be compiled using statistically sound procedures
implemented by qualified professional staff. When analysis requires using
samples from administrative data files, RRB employs statistically acceptable
methods to design and select the samples. Data samples are designed and
compiled by staff knowledgeable about the content, structure, and
limitations of the administrative data files employed. In addition, those
staff members maintain working relations with agency personnel who create,
update, and maintain those files to ensure that their understanding of files
is current and complete. When information products require administrative
files linked to external data sources, RRB employs sound procedures for
extracting and linking data from external sources based on a thorough
understanding of the relevant components of the data sources.
Preparation of statistical estimates
All estimation and sampling procedures will be
prepared using statistically sound procedures designed by qualified
professional staff. Samples from administrative files are evaluated to
ensure that the samples are representative and subject specialists compare
estimates with comparable information from prior years and from other
sources to ensure reasonableness and reliability. Computer programs for
sampling from administrative files and using data from external data files
are reviewed and tested by technical staff. Data files incorporating
external data sources are reviewed to ensure that extraction and linkage
processes have been implemented correctly.
Data sources, sampling errors, and disclosure
limitation methods will be documented in publications, either for the
publication as a whole or for individual tables. Documentation in RRB
publications contains information on data sources including definitions and
specifications of variables. Report documentation also includes, where
appropriate, information on sampling errors and a description of rules or
techniques for avoiding disclosure of confidential information.
Preparation of simulation models
RRB uses several simulation models to estimate the
effects of demographic and economic trends and legislative and policy
options on Railroad Retirement programs and beneficiary populations now and
in the future. Models are based on RRB's best judgments of current and
future behavioral relationships and methods of projecting key program
outcomes.
For all simulation models, implementation
procedures will be tested, and the models' performance will be thoroughly
evaluated. All simulation models are extensively tested and reviewed within
RRB to verify that the computer programs that were developed to implement
models conform to the stated objectives. Where appropriate, historical
simulations are developed to evaluate the success of a model in producing
reasonable projections. Where appropriate, based on a model's complexity and
scope of application, RRB convenes technical committees to review a model's
performance and evaluate whether it meets its objectives. Models are
periodically updated to reflect input from internal and external reviews and
research findings on behavioral relationships.
Documentation will be available for all simulation
models. Documentation for simulation models developed within RRB will be
available upon request.
Preparation of analytical reports and policy studies
Analytic reports will be prepared by subject
specialists who use sound statistical and analytic methods and are
knowledgeable about the data sources and models being used. Reports are
prepared by staff using a variety of analytical techniques ranging from
simple tabulations and descriptive summary statistics to multivariate
statistical methods and econometric models. Staff preparing analytic reports
and policy studies are expert in their use of relevant administrative data
files, external data sources, and projections from simulation models.
Analytical techniques will be reviewed to ensure
that they are appropriate for the data and the analysis to which they are
applied and they will be documented. All analytical reports and policy
studies are reviewed by qualified staff to ensure that the analysis is
valid, complete, unbiased, objective, and relevant. Reports and studies that
are considered to be more technically complex and are likely to have a
greater impact are also reviewed by external technical committees to provide
additional perspective and expertise. Analytic techniques are clearly
described in reports and data sources are identified in RRB publications.
When analyses are based on projections from simulation models, the
assumptions used to produce the projections are identified, as well as the
rationale for the assumptions used and the impact of using alternative
assumptions.
Editorial review for accuracy and clarity of information in publications
All information products will be edited and
proofread before release to ensure clarity and coherence of the final
report. Text is edited to ensure that the report is easy to read and
grammatically correct, thoughts and arguments flow logically, and
information is worded concisely and lucidly. Tables and charts are edited to
ensure that they clearly and accurately illustrate and support points made
in the text and include concise but descriptive titles. Tables and charts
indicate the unit of measure and the universe being examined, and all
internal labels (column headings, row stubs, and panel headings) accurately
describe the information they contain. All changes made to a manuscript
during the editing process are checked by a proofreader and reviewed and
approved by the author.
Policy for correcting errors
A comprehensive errata policy will inform users of
both printed and Web-based publications when an error has been found and
corrected. If an error is detected before an initial mailing, RRB includes
an errata notice with the mailing. If the mailing has been sent out, an
errata sheet is issued with all subsequent publications that are
disseminated and, where appropriate, the errata sheet is sent to all those
who received the initial mailing. Errata notices are placed on the first
page of the Web version to inform both new and repeat site visitors about
the mistake, and the corrected version of the document is posted on the Web.
Integrity
Integrity, as used in the OMB
quality guidelines, refers to the security of information from unauthorized
access or revision to ensure that the information is not compromised through
corruption or falsification. RRB administers social insurance programs for
railroad workers and their families. The agency is highly protective of the
confidentiality of information it holds through its policies and practices.
To ensure the
integrity of its administrative information, RRB will employ rigorous
controls that have been identified as representing sound security practices.
RRB has in place programs and policies for securing its resources as
required by the Government Information Security Reform Act (P.L. 106-398,
title X, subtitle G). Those security procedures address all major components
of information security and apply to all RRB operating components. In
addition, RRB is subject to statutory requirements to protect the sensitive
information it gathers and maintains on individuals. Those requirements are
contained in the following documents:
- Privacy Act of 1974
- Computer Security Act of 1987
- Office of Management and Budget (OMB) Circulars
A-123, A-127, and A-130
- Government Information Security Reform Act
- Federal Managers' Financial Integrity Act (FMFIA)
of 1982
- Section 6103 of the Internal Revenue Code
- IRS Tax Information Security Guidelines for
Federal, State, and Local Agencies
Transparency and
Reproducibility
Information products that are
subject to these RRB section 515 information quality guidelines include
statistical and actuarial information. The guidelines focus on reports,
studies, and summaries prepared for public dissemination to inform the
public about the impact of Railroad Retirement programs and to use in
formulating broad program policy.
For the purpose of these guidelines, transparency
refers to a clear description of the methods, data sources, assumptions,
outcomes, and related information that will allow a data user to understand
how an information product was designed and produced. Reproducibility of
information refers to the ability, in principle, for a qualified individual
to use the documentation of methods, assumptions, and data sources to
achieve comparable findings. In practice, opportunities for direct
reproducibility are often limited by restrictions on access to confidential
information.
RRB will make the
information it disseminates and the methods used to produce this information
as transparent as possible so that they can, in principle, be reproducible
by qualified individuals. RRB guidelines call for clear
documentation of data and methods used in producing estimates and
projections. Their implementation will ensure the transparency and
reproducibility of our disseminated information. Some statistical
publications are based on publicly available data, and the computer programs
that produce the statistics can be made available on request; accordingly,
information in those publications is fully reproducible by the public.
Many estimates and projections included in RRB's
information products are not directly reproducible by the public because the
underlying data sets used to produce them are confidential. Others may not
be directly reproducible because of the complexity and detail of the methods
and data. In those cases, greater emphasis is placed on periodic review by
outside panels of technical experts and on documentation of methods,
assumptions, data sources and related information.
Much of RRB's disseminated actuarial, statistical,
and analytical information is potentially influential because it has an
impact on important public policies or important private-sector decisions
relating to the Railroad Retirement program. Information products that are
deemed to have a greater impact on public policies are subject to more
extensive internal review and, where appropriate, review by external
technical panels prior to release. |