Guidelines for Ensuring the Quality of Information Disseminated to the Public
G. � Health Resources and Services Administration
I. � Agency Mission
The Health Resources and Services Administration
(HRSA) is the lead Federal agency in promoting access to health care services
that improve the Nation's health. With a statutory emphasis on special needs,
underserved, and vulnerable populations, HRSA mobilizes its bureaus, programs,
staff, and partners to assure access to quality health care. The four main
strategies to achieve this goal are to: eliminate barriers to care; eliminate
health disparities; assure quality of care; and, improve public health and
health care systems. To fulfill HRSA's mission, its programmatic portfolio
includes a range of programs or initiatives designed to increase access to
care, improve quality, and safeguard the health and well-being of the Nation's
most vulnerable populations.
Collectively, HRSA programs work to improve access to care for the more
than 38.7 million Americans who are uninsured and the 40 million who live
in medically undeserved areas. HRSA supports over 700 community health centers;
funds services for people living with HIV/AIDS through the Ryan White CARE
(Comprehensive AIDS Resources Emergency) Act; assists States and health care
organizations in improving services to mothers and children; oversees the
National system that allocates organs, tissue, and bone marrow for transplantation;
and works with academic health centers and other training programs to enhance
the diversity and distribution of the Nation's health care workforce.
The primary operating units in HRSA each contribute to the overall mission and major goals and objectives:
- The Bureau of Health Professions: provides national leadership to assure a health professions workforce that meets the health care needs of the public.
- The HIV/AIDS Bureau: provides leadership in the delivery of high quality HIV primary care and supporting services for uninsured and underinsured individuals and families affected by HIV/AIDS.
- The Maternal and Child Health Bureau: works on behalf of America's mothers, children, and families in ways that will assure continued improvement in their health, safety, and well-being.
- The Office of Rural Health Policy: serves as the leading Federal proponent for better rural health care services.
- The Office of Special Programs: ensures access and capacity to scarce resources including assurance of access to organ and bone marrow transplantation programs.
- The Bureau of Primary Health Care: increases access to comprehensive primary and preventive health care and works to improve the health status of underserved and vulnerable families and individuals.
More information about HRSA, its programs and activities is available on the HRSA web site www.hrsa.gov.
II. � Scope and Applicability of Guidelines for Agency/Office
HRSA will ensure that disseminated information meets the standards of quality
set forth in the OMB, HHS and HRSA guidelines.� It is HRSA's policy to ensure
and maximize the quality, objectivity, utility, and integrity of information
that it disseminates to the public. We strive to provide information that
is accurate, reliable, clear, complete, unbiased, and useful. We are committed
to integrating the principle of information quality into every phase of information
development, including creation, collection, maintenance, and dissemination.
The guidelines apply to substantive information that is disseminated
by Federal agencies subject to the Paper Reduction Act (PRA), excluding information
that is someone's opinion rather than fact or the agency's views. More specifically,
it applies to agency initiated or sponsored information. The HRSA information
to which these guidelines would apply are aggregate statistical types of
information that are abstracted from grantee applications, policy and program
information (i.e., data and state profile information, community information,
official reports), as well as other electronic documents, newsletters, and
brochures that are provided to the public. For example, this includes the
HIV/AIDS State Profiles, the Title V Information System web site, the current
findings from the National Sample Survey of Registered Nurses, the Area Resource
File, and the HRSA State Profiles. The pre-dissemination review described in the guidelines only applies
to information first disseminated on or after October 1, 2002. The administrative
mechanism for correction applies to information that the agency disseminates
on or after October 1, 2002, regardless of when the agency first disseminated
the information. The guidelines would not include data provided directly to HRSA
by grantees, such as individual grantee application forms, reporting requirements,
performance plans, etc., but would apply to information that is disseminated
from such collections of information. Examples of other types of information
collections to which these guidelines would not apply is the information
that is collected for the National Practitioner Data Bank (NPDB), since this
information is not disseminated to the public but is restricted in access
to those entities and persons who, under specified conditions may request
information from the NPDB. Other information that is not covered includes
papers, books, journal articles, and other documents that have the disclaimer
that the information contained therein does not represent the agency's views.
Information that is not covered also includes press releases, correspondence
with individual persons, archival records, or opinions.� In addition, the
guidelines do not apply to press releases that support the announcement or
give public notice of information that HRSA has disseminated elsewhere.
III. � Types of Information Disseminated by the Agency to the Public
HRSA disseminates a range of information to the public via the HRSA web
site, the HRSA Information Center, and the Office of Communications. The
HRSA Information Center is a gateway to information about HRSA programs.
All information supplied through this center has been cleared through HRSA
and ASPA. Through the Information Center, health care professionals, policymakers,
researchers, and members of the public can obtain material on HRSA-supported
health programs. The HRSA website also provides information, data, and reports
on community health resources, health professions training, HIV/AIDS, managed
care, maternal and child health, minority health, oral health, primary health
care, rural health, and women's health. Within the HRSA web site there is
a wealth of public information. Users of this site may review information
on-line or download it for future use.
- Examples of Program, Policy, and Administrative data:
- The HRSA State Profiles features resources granted by the Agency
within each state targeted to serve low-income, uninsured, and medically
undeserved populations. To provide an understanding of each state's unique
environment, the Profiles also include demographic data and health status
indicators. Each State Profile features:
- An overview of funds granted within a state
- Selected demographic data
- Health status and health care access indicators
- Health care provider resources
- Highlights of HRSA resources targeted in the areas of primary health
care, health care provider resources, maternal and child health, public health,
rural health, and HIV/AIDS.
The State Profiles are six-page reports highlighting the health status
of each State's population and HRSA's investment of health resources within
each State and the District of Columbia for the years 1997 through 1999.
Two categories of data are provided: program data describing resources provided
to States by HRSA, and demographic and health status indicator data. For
each data element, there are data definitions, source information, types
of calculations and derivations, value ranges, and a description of the HRSA
Program from which the data were collected. Users can create customized reports
by selecting specific data elements, and can generate data to be imported
into a spreadsheet or database.
- The HRSA HIV/AIDS Bureau State Profiles describe spending
and service information of the Ryan White CARE Act programs. Information
is provided on location of grantee, clients served, grantee accomplishments,
and the characteristics of the HIV epidemic in the state (e.g., co-morbidities
and other funding sources such as Medicaid, the largest payer of HIV/AIDS
services in the nation), reporting requirements and progress reports for
the Title I, II, III, and IV grantees; CDC Surveillance Reports; and other
sources including Census and GAO.
- The Title V Information System (IS) provides information
on the status of maternal and child health in the United States. Data are
compiled from annual Title V Block Grant applications and reports submitted
by all U.S. States, Territories, and Jurisdictions. Information is provided
on key measures of maternal and child health, budgets and expenditures, program
data, performance measures, and summary data. Users may download prepared
tables, graphs, brochures, fact sheets, and reports on maternal and child
health, as well as the Electronic Reporting Package (ERP) that States use
to report their data.
- The HRSA Preview: the HRSA Preview provides the general
public with a single source of program and application information related
to the Agency's competitive grant offerings. It contains a description of
competitive and other grant programs scheduled for awards in Fiscal Year
2002, and includes instructions on how to contact the Agency for information
and receive application kits for all programs. The following specific information
is included in the HRSA Preview: (1) program title; (2) legislative authority;
(3) purpose; (4) eligibility; (5) funding priorities and/or preferences;
(6) estimated dollar amount of competition; (7) estimated number of awards;
(8) estimated project period; (9) Catalog of Federal Domestic Assistance
(CFDA) identification number; (10) application availability date; (11) letter
of intent deadline (if any); (12) application deadline; (13) projected award
date; (14) programmatic content, with telephone and e-mail addresses. Certain
other information, including how to obtain and use the HRSA Preview and grant
terminology, can also be found in the HRSA Preview.
- HRSA's Bureau of Health Professions Area Resource File (ARF):
the ARF is a county level data set with information on health professions,
health facilities, utilization, expenditures, population characteristics,
and geographic environment. The data set includes county level estimates
for the entire U.S. of the number of physicians, including specialties and
limited demographic information, dentists, dental hygienists, optometrists,
pharmacists, podiatrists, nurses, physician assistants, and other health
professions. Information is available on hospitals, nursing homes, HMOs,
inpatient days and other utilization measures, hospital and Medicare expenditures,
and various selected variables containing information on the resident population
and environmental characteristics. The ARF utilizes secondary data obtained
from Census, the American Medical Association, National Center for Health
Statistics, and others and compiles a wide array of data sources. As such,
the limitations of these data sources all apply, as do any cross source comparability
issues.
- Example of statistical data:
The National Sample Survey of Registered Nurses (NSSRN): The Seventh
NSSRN was conducted in 2000 and is the nation's most extensive and comprehensive
source of statistics on all those with current licenses to practice in the
United States. It provides information on the number of registered nurses,
their educational background and specialty areas; their employment settings,
position levels, and salaries; their geographic distribution; their personal
characteristics including gender, racial/ethnic background, age, family status,
and satisfaction with their job. The substantial database resulting from
the 2000 study may be used for many different types of analyses concerning
a variety of subjects. The report presents an overview of the personal, professional,
and employment characteristics of the almost 2.7 million registered nurses
in the country as of March 2000. A summary of the findings from the study
and some comparisons to the findings of prior studies in this series, are
presented in the report. Appendix A contains a series of tables summarizing
the data. A review of the survey methodology and the statistical techniques
used in sample selection, response weighting, and identification of sampling
errors are found in Appendix B. The survey instrument is included in Appendix
C of the report.
IV. � Types of Dissemination Methods
Information is disseminated
by HRSA by a variety of mechanisms: the HRSA Office of Communications (OC)
serves as the Agency point of contact for clearing and producing HRSA information
products. Information is available from the HRSA Information Center in the
form of publications, brochures, fact sheets, and other resources on health
care services. Through the Information Center, health care professionals,
policymakers, researchers, and members of the public can obtain material
on HRSA-supported health programs. Information on HRSA publications is provided
in the Information Center catalog, and may also be ordered directly from
the HRSA web site. For each of the earlier examples given above of information disseminated
by the Agency to the public, the following dissemination methods are used:
- The HRSA State Profiles are available through the HRSA web site on stateprofiles.hrsa.gov and the information can be viewed and downloaded using Adobe Acrobat and provides a formatted six page profile for each state.
- The HIV/AIDS Bureau State Profiles can be accessed at hab.hrsa.gov.
The State Profiles can be viewed or downloaded in Adobe Acrobat. The site
has a phone number and email contact information for further requests and
feedback.
- The Title V Information System is available on the HRSA Bureau of Maternal and Child Health web site at www.mchdata.net.
Program materials can be downloaded to obtain prepared tables and graphs,
brochures, fact sheets, annual reports, and more. Hard copy versions are
available from the Information Center. The web site also includes specific
contact information for questions regarding the content of the site and questions,
problems or comments regarding the functionality of the site.
- The HRSA Preview was published in the Federal Register on 8/9/2001 and is also available for downloading on the web at www.hrsa.gov/grants.htm. Additional information is available at:
HRSA Grants Application Center
Attention: Grants Management Officer
901 Russell Avenue, Suite 450
Gaithersburg, MD 20879
or by calling the HRSA Grants Application Center at 1-877-477-2123.
- The Area Resource File is available in electronic format only.
Contact information is provided on the HRSA web site and includes an address,
phone number, fax number, email, and web site specific to ARF.
- Findings from the 2000 National Sample Survey are provided on the HRSA web site at bhpr.hrsa.gov/healthworkforce/rnsurvey
and can be downloaded in Adobe Acrobat. Contact information for the Nursing
Data and Analysis Staff in the Bureau of Health Professions, HRSA, is also
provided. Preliminary findings were published in a report and are available
from the HRSA Information Center.
IV. � Agency Quality Assurance Policies, Standards and Processes for
Ensuring the Quality of Information Disseminated to the Public
HRSA
reviews the quality (including the objectivity, utility, and integrity) of
information before its is disseminated and treats information quality as
integral to every step of the development of information, including its creation,
collection, maintenance and dissemination.
- Program information:
Program information provided by grantees in the form of application data,
reporting requirements, performance reports and progress reports, receives
a series of reviews prior to any dissemination to the public. Grantees conduct
internal reviews in order to provide data that are accurate, consistent,
and complete. The HRSA Office or Bureau conducts a standard review to ensure
data quality, completeness, and reliability.
- Reports, publications, and other products:
HRSA publications, audiovisual products, and exhibits are required
to be consistent with government-wide and HHS public affairs policies. HRSA
has published guidelines which closely follow the regulations described in
the HHS Public Affairs Management Manual and Government Printing and Binding
Regulations to assist program staff in obtaining appropriate review and clearance
of information products. To produce a publication, a complete HHS-615 (Publication
Planning and Clearance Request) form must be submitted to, and approved by,
ASPA.
Publications to be produced at HRSA's request under contract are
subject to HHS clearance and the Joint Committee on Printing (JCP) printing
procedures. An HHS-524 clearance is required for publications to be developed
as part of a larger public affairs services contract. The HHS-524 must be
approved before a Request for Proposals (RFP) can be issued.
- Statistical data disseminated to the public:
At HRSA, the quality assurance process begins at the inception of the information development process.
For information disseminated from sample surveys and other research
related or evaluative activities, the widely accepted standards of technical
and scientific review are utilized to ensure data quality. These standards
include, where appropriate, peer review, internal expert review, Institutional
Review Board (IRB) review and OMB review.
For each of the examples provided earlier, the following quality assurance procedures are employed.
- The HRSA State Profiles are reviewed by program staff from each Bureau
providing input into the profile prior to dissemination. This is an interval
review by each Office and Bureau from which data are obtained. The originating
office is responsible for ensuring that all necessary internal review, approval,
and clearance is obtained prior to dissemination. Where indicators are taken
from other sources, that source is clearly indicated and the limitations
of the data are described. For the demographic and other data derived from
other sources, HRSA provides an explicit list of the source information,
the year the data were obtained, citations, and related URLs. For example,
for health indicators such as percent of people insured in the State who
had private insurance, the source listed is the U.S. Census Bureau, Health
Insurance Statistical Tables, and the citation is Table HI-4, Health Insurance
Coverage Status and Type of Coverage by State-All Persons, 1998. Each data
element contains such a source listing along with definitions.
- The HIV/AIDS Bureau uses CAREWare, a software package for use
by the HIV/AIDS Bureau providers to track clients and services. The software
and manual are available online, and the system captures all the data items
required for the reports needed. CAREWare contains consistency and edit checks
on inputted data for quality assurance. This system was designed specifically
to collect the exact data elements and generate complete reports. The HIV/AIDS
program staff review all data for verification prior to dissemination. The
CARE Act State Profiles data provides, for each element, a source list and
year of data, as well as notes relating to limitations, rounding, and restrictions,
where appropriate. For example, for the Title II Profile Report, Clients
Served by gender, age, race/ethnicity, exposure category, the source is listed
as the Annual Administrative Report to HRSA, with a note that Accounts are
rounded to the nearest 10. Data are not reported from cells of fewer than
six clients.
- The Maternal and Child Health Bureau Title V annual report uses
an electronic reporting package that was developed in collaboration with
the states in order to improve accuracy and completeness of the report. This
reporting package provides for automatic calculations of ratios, rates, and
percentages, and carries data over from year to year, and assures that data
used in multiple tables is entered only once. This system has in place a
mechanism that warns states when conflicting or unacceptable figures appear
as a quality check on the final data. Knowledgeable MCHB program staff monitor
and review all information submitted in the grant annual report for completeness,
accuracy, and reliability.
- The HRSA Preview undergoes appropriate internal Division, Bureau,
and Office review and approval prior to dissemination. HRSA adheres to all
accepted standards applying to the publication of information in the Federal
Register. The HRSA Preview undergoes extensive Office/Bureau, and Agency
wide review to ensure accuracy, reliability, and comprehensiveness prior
to dissemination to the pubic.
- The Area Resource File provides county level data from a variety
of data sources. As a result the data have a number of limitations and users
of the file are provided with a disclaimer as to the applicability and detail
of the data. The ARF utilizes secondary data only, and the years and detail
for which all county level data are available are considerably limited. Each
data source conducts its own review prior to the release of information.
Knowledgeable staff within the originating office and across offices conduct
standard reviews for completeness and reliability of the data.
- The National Sample Survey of Registered Nurses received extensive
review during each phase of operation: survey design, sample selection, fielding
of the survey, data editing, and analysis. Clearance was required of all
proposed sampling methods, survey procedures, the data collection instrument,
and analytic plans by HRSA, HHS, and OMB. This clearance requires a series
of expert review. A scientific review, an internal review, and IRB review
were conducted to ensure that survey procedures, data collection, editing,
and analyses would adhere to the highest standards.
As in all research activities involving human subjects, it is HRSA's
policy that adequate protection of participants be ensured in accordance
with the provisions of 45 CFR Part 46. Project plans and procedures must
be submitted to the appropriate IRB for approval, and the program must execute
all necessary Assurances of Compliance with OPRR. The NSSRN obtained IRB
review and approval for all survey methods and procedures.
For the National Sample Survey of Registered Nurses, widely accepted
standards as established by the statistical community are utilized to assure
data survey quality. For example, the median design effect for the survey
was 1.66, an accepted level by the statistical community for a large national
survey. The information disseminated in the report provides extensive detail
on the analytic issues (sampling and nonsampling errors, standard error calculation,
variance estimation, etc.). As with any sample survey, the results are subject
to sampling error. The report provides detail on measures of variability,
standard errors, information on the computation of the sampling variance
and the design effect.
Further, HRSA strives to demonstrate in its Paper Reduction Act
(PRA) clearance packages that each draft information collection will result
in information that will be collected, maintained, and used in a way that
is consistent with OMB, HHS and HRSA information quality guidelines.
V. � Agency Administrative Complaint Procedures
- Responsibility of the Complainant
To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.
- A complaint or request for review and correction of information shall be in written hard copy or electronic form;
- it shall be sent to the agency by mail or electronic-mail(e-mail); and
- it shall state that an information quality request for correction is being submitted.
The complaint shall contain
- a detailed description of the specific material that needs to be corrected
including where the material is located, i.e. the publication title, date,
and publication number, if any, or the website and web page address (url),
or the speech title, presenter, date and place of delivery; and
- the specific reasons for believing the information does not
comply with OMB, HHS or HRSA guidelines and is in error and supporting documentation,
if any;
- the specific recommendations for correcting the information;
- a description of how the person submitting the complaint is affected by the information error; and
- the name, mailing address, telephone number, e-mail address,
and organizational affiliation, if any, of the individual making the complaint.
Complainants should be aware that they bear the ‘burden of proof’ with
respect to the necessity for correction as well as with respect to the type
of correction they seek.
Complaints by mail should be directed to:
HRSA Reports Clearance Officer
HRSA/OPE, Room 14-45
5600 Fishers Lane
Rockville, MD. 20857
E-Mail complaints should be sent to www.infoquality@hrsa.gov
- Responsibility of the Agency
Based on a review of the information provided, the agency will determine
whether a correction is warranted and if, so what action to take. The agency
will respond to the requestor by letter or e-mail. The agency's response
will explain the findings of the review and the actions that the agency will
take, if any. The response will consider the nature and timeliness of the
information involved and such factors as the significance of the correction
on the use of the information and the magnitude of the correction. The response
will describe how the complainant may request reconsideration. The agency
will respond to all requests for correction within 60 calendar days of receipt.
If the request requires more than 60 calendar days to resolve, the agency
will inform the complainant that more time is required and indicate the reason
why and an estimated decision date.
- Appeals
If the individual submitting the complaint does not agree with the
agency's decision (including the corrective action, if any), the complainant
may send a written hard copy or electronic request for reconsideration within
30 days of receipt of the agency's decision. The appeal shall state the reasons
why the agency response is insufficient or inadequate. Complainants shall
attach a copy of their original request and the agency response to it, clearly
mark the appeal with the words, Information Quality Appeal, and send the
appeal to the specific agency appeals address:
Division of Information and Analysis
Office of Planning and Evaluation
HRSA, Room 14-45
5600 Fishers Lane
Rockville, MD. 20857
The agency official who resolved the original complaint will not have
responsibility for the appeal.� The agency will respond to all requests for
appeals within 60 calendar days of receipt. If the request requires more
than 60 calendar days to resolve, the agency will inform the complainant
that more time is required and indicate the reason why and an estimated decision
date.
- Comments and Requests for Information:
HRSA's Office of Communications (OC) has maintained a toll-free phone
number to answer questions and address comments on subject specific information.
This office has served as the point of entry for queries regarding HRSA information,
requests for publications, comments and corrections to information disseminated
to the public. To date this Office has not received any complaints about
information disseminated by HRSA. Calls have been made, however, by States
and grantees to correct and/or update information that is maintained on the
HRSA web site. These infrequent calls are received by the OC and directed
toward the appropriate Bureau for review, verification, and revision. The
States/grantees verify the correction/modification of their data to Bureau
staff and after confirmation with program staff the revisions are implemented.
In addition, the HRSA web site contains a detailed list of HRSA phone numbers
by Office and Bureau for comment or for additional information. Comments
and questions can be mailed to ask@hrsa.gov or comments@hrsa.gov, or by calling 1-888-275-4772.
VI. � Influential Scientific, Financial and Statistical Information
From time to time HRSA disseminates information that would generally be
regarded as influential. In those circumstances, we use the highest standards
of reproducibility and transparency for such information to be disseminated
to the public.
VII. � References
(Currently HRSA is in the process of updating these references)
HRSA Circular, No. 95-01, Identification of HRSA in Communications Materials
Logo Guidelines. HRSA, OC, 5600 Fishers Lane, Rockville, Md. 20857 HRSA Policy Circular, No. 96.05, Protection of Participants in HRSA
Research Programs. HRSA, 5600 Fishers Lane, Rockville, MD. 20857 Planning, Clearing, and Producing a HRSA Information Product. 1999,
HRSA, Office of Communications, 5600 Fishers Lane, Rockville, MD. 20857
Publications Catalog, 2001. HRSA, Information Center, 5600 Fishers Lane, Rockville, MD. 20857
Skinner, CJ. Aggregated analysis: standard errors and significance
tests. In: Skinner, CJ, Holt, D. Smith TMF, eds. Analysis of complex surveys.
New York: John Wiley and Sons, Inc. 1989. The Registered Nurse Population, Preliminary Findings, 2001. The
National Sample Survey of Registered Nurses, 2002. HRSA, Division of Nursing,
5600 Fishers Lane, Rockville, MD. 20857
VIII. � Other Agency Specific Policies and Procedures
HRSA,
through its Bureaus and Offices, administers a variety of service delivery
and demonstration programs. HRSA Bureaus and Offices also administer some
specific research programs and a variety of epidemiological and service utilization
studies, as well as evaluations which might be considered research. Participants
must be protected from potential risks that may be associated with any such
research projects. HRSA Policy Circular No. 96.05 establishes policies and
procedures to protect human subjects in research programs conducted or supported
by HRSA. This circular gives guidance for compliance with HHS regulations dealing
with protection of human research subjects. It states HSRA policy for various
levels of protection applicable to HRSA programs, and explains how to determine
the protection level appropriate to each program. It provides guidance on
exemption for certain research activities or public benefit or service programs,
requiring evaluation of such claims by program staff and approval by a HRSA
Human Subjects Protection Committee. If the public checks the HRSA resources for public information but
does not find adequate information, they may file a Freedom of Information
Act (FOIA) request. The HRSA FOIA Home Page, newsroom.hrsa.gov/efoia.htm
has the location and contact information for making such requests. The site
contains a guide for requesting information and records from HRSA as well
as a copy of the Freedom of Information Act, As Amended. In addition, this
site provides the HRSA Annual FOIA Report for Fiscal Years 1998 through 2001.
Last revised: May 14, 2004
|