Final Report for Conference Grant R13 HS016070-01
The purpose of this conference, held on November 15, 2005, was to define a knowledge-based and skills-based competencies common to all
health services research professionals trained at the doctoral level.
Principal Investigator: Christopher B. Forrest, MD,
PhD
Conference Manager: Anne Millman, MS
Conference Coordinator: Joyce Hines
Research Assistant: Erin Holve, MPP
Grantee Institution: Johns Hopkins Bloomberg School of Public Health
Project Period: 6/1/2005 to 10/31/2005
Sponsor: Agency for Health Research and Quality (AHRQ)
Grant Project Officer: Brenda Harding, AHRQ
Acknowledgment: Diane Martin, PhD from the University of Washington played a central role in the planning and conduct of the
conference supported by this grant. She also functioned as a co-moderator for
the actual conference. Her involvement in this project was instrumental to its
success.
Contents
I. Abstract
II. Purpose of Conference
III. Scope
IV. Methods
V. Results: HSR Doctoral Level Core Competencies
VI. Literature Cited
I. Abstract
Purpose: To conduct a conference
that defines the knowledge-based and skills-based competencies common to all
health services research professionals trained at the doctoral level.
Scope: Health services research
(HSR) is a scientific field that has been developing for the past 40
years. Given this maturity and established professional identity, it is now
appropriate to propose common learning objectives. Elucidating the knowledge
base and skills needed to be a successful health services researcher will aid
in defining the field, developing similar core competence among graduates from
various programs, and communicating to employers the unique expertise of health
services researchers.
Methods: Development of
the initial set of core competencies was based on a structured review of the
published and unpublished literature, AHRQ-sponsored training
grants, and academic institution accreditation self-studies submitted to
the Council on Education for Public Health (CEPH). Text analysis was used to
identify themes and educational domains.
Results: The draft set of
competencies and HSR educational domains were discussed and refined at a
conference held in Baltimore, MD on September 8-9, 2005. Conference attendees represented training directors and students from
AHRQ-sponsored National Research Service Award (NRSA) T32 training programs, directors of CEPH-accredited
doctoral health services research programs, AcademyHealth, AHRQ, CEPH, and
public and private sector stakeholder organizations that are employers of HSR
trainees. The conference produced a consensus-derived set of 14
doctoral-level core competencies. The results of this conference will be
presented at the 2006 annual research meeting of AcademyHealth, and they will
be disseminated in a peer-reviewed journal article.
Key Words: health services research;
doctoral education; core competencies
Return to Contents
Purpose of Conference
The overall goal of the conference was to provide a
forum to explore and document the challenges of preparing a well-trained cadre
of health services researchers.
The conference was designed to accomplish the following objectives:
- To provide a forum
for exploring the possibility of developing core competencies for the field.
- To define specific
knowledge-based and skills-based competencies common to all health services
research professionals.
- To produce a
consensus-derived set of knowledge-based and skills-based HSR doctoral training
competencies and their content areas.
Conference Assumptions
Doctoral Level Training
To focus discussion during the
conference, we elected to address doctoral training only.
General Doctoral
Training
Our goal was
to define the essential competencies that are expected of all HSR trainees,
irrespective of areas of specialization in content or method or disciplinary
focus.
Definition of
Health Services Research
We used the following definition of HSR to guide discussion at the conference: "Health services research is the multidisciplinary field of scientific
investigation that studies how social factors, financing systems,
organizational structures and processes, health technologies, and personal
behaviors affect access to health care, the quality and cost of health care,
and ultimately our health and well-being. Its research domains are
individuals, families, organizations, institutions, communities, and
populations." (Lohr and Steinwachs 2002)
Return to Contents
III. Scope
Background and Statement of the Problem
Health services research (HSR) is a scientific field of
inquiry that examines the structures, functions, and outcomes of health services
delivered to individuals and populations. It involves investigation into all service
sectors that affect health, not just the medical care system, and its purview
is global. HSR has been developing for
the past 40 years, and given this maturity and established professional
identity, it is now appropriate to propose common training goals and learning
objectives. Elucidating the knowledge base and skills needed to be a successful
health services researcher will aid in defining the field, developing similar
core competence among graduates from various programs, and communicating to
employers the unique expertise of health services researchers.
In the 1995 Institute of Medicine report, Health
Services Research: Training and Workforce Issues (Ed. Field, et al), four elements of HSR training were emphasized: it has a multi-disciplinary approach;
it involves basic and applied research; it examines conceptual and theoretical
relationships within and between health systems; and, the field involves
research on both populations and individuals (page 4). The report suggested
that "a single educational path is neither practical nor desirable (page 7)"
and HSR needs to offer a range of training opportunities. This perspective is
reflected in the variety of National Research Service Award (NRSA) programs
that support health services research training in varied disciplines and
topical areas.
HSR incorporates
a wide range of disciplines, most notably: psychology, economics, anthropology,
biomedicine, mathematics, political science, sociology, and management sciences.
Application of these disciplines to problems confronting health systems requires
a workforce with a diverse skill set. Although the
field has specific journals, professional societies, employers, sponsors, and
training programs, the competencies common to all doctoral-trained HSR
professionals have not been defined.
Graduates of HSR training programs have pursued a wide
variety of careers in academia, research, healthcare delivery, and policy
analysis. Despite the success of these graduates, they complete their
respective training programs unsure what it means to be a health services
researcher, how they should distinguish themselves from other health
professionals, and what their advanced training will enable them to do. Our
inability to provide these talented students with an adequate answer to their
quandary has been a principal motivation underpinning the development of a
common set of HSR competencies.
Defining HSR doctoral competencies would:
- Assist program directors with developing and identifying the most important learning
objectives for training students in HSR.
- Clarify for employers of graduates
of HSR training programs what the skills and abilities are for individuals in
HSR.
- Give HSR trainees a unique professional identity.
Return to Contents
IV. Methods
A team in the Department of Health Policy and Management
at the Johns Hopkins Bloomberg School of Public Health received an AHRQ
contract in July of 2004 to identify the doctoral level competencies that are
required of health services researchers.
The first draft
of the competencies and their content areas was reviewed by faculties from the University of Washington and Johns Hopkins Schools of Public Health. Comments from this review
were used to produce version 2 of the competencies and their content areas. Conference
attendees and their colleagues reviewed this version; their comments were
incorporated into the version presented at the conference convened in Baltimore, MD, September 8-9, 2005 for the meeting sponsored by this conference grant.
Meeting
Attendees
Conference attendees were selected to represent training
directors and students from AHRQ-sponsored NRSA T32 training programs,
directors of CEPH-accredited doctoral health services research programs,
AcademyHealth, AHRQ, CEPH, and public and private sector "stakeholder"
organizations that are employers of HSR trainees.
Meeting Agenda
The two day conference was divided into five sequential
parts:
-
Introductory
remarks.
-
General discussion
on HSR doctoral training and assumptions made for the consensus process.
- Development and
description of the core competencies and their content areas.
-
Small group
discussion on modifications to the core competencies, followed by a large group
consensus process.
-
Small group
discussion on modifications to the HSR educational domains followed by a large
group consensus process.
The specific agenda is presented below.
Health
Services Research Doctoral Training Competencies: A Consensus Conference
Agenda
Conference Location: Feinstone
Hall, Johns Hopkins Bloomberg School of Public Health, 615
N. Wolfe Street, Baltimore, MD.
Thursday, September 8, 2005
1:00: Welcome, Introductions, and Introductory Remarks
Welcome (Chris Forrest and Diane Martin)
Agenda (Chris Forrest)
Meeting Logistics (Anne Millman)
Introductory Remarks (Karen Rudzinski, AHRQ)
Attendee Introductions
1:45: Large
Group Discussion
(Facilitator: Diane Martin)
3:00: Break
3:15: Large Group Discussion Continued (Facilitator: Diane
Martin)
4:00: Development of Core Competencies.
-
Review of competency development
process (Erin Holve).
- Competency Domains, and
Competencies (Chris Forrest).
4:30: Group
Discussion on Core Competencies (Facilitator: Chris Forrest)
- Omissions and additions.
- Modifications.
- Consensus on final wording.
5:30: Break
6:00: Small
Group Discussions on Core Competencies
Working Dinner
Location: E9519 (9th Floor)
8:15: Wrap-up (Martin)
8:30: Adjourn
Friday, September 9,
2005
8:30: Breakfast (Feinstone Hall)
9:00 Core
Competencies Small Group Reports
(Facilitator: Chris Forrest)
Each group reports on its
deliberations. Goal is to reach consensus on
proposed changes
10:30: Break
10:45: Convene
small groups in conference rooms
Goal: review a set of 9
competency domains:
- Map domains to core competencies.
- Flesh out content of each domain.
12:30: Lunch
1:30: Small
group reports on competency domains and their content
Location: Feinstone Hall
(Facilitator:
Forrest)
Consensus process regarding
modifications.
3:00: Wrap-Up
(Facilitator: Martin and Forrest)
-
How should this document be
disseminated?
-
How should the document be used
by funders of training, training programs, employers, faculty, students?
- Other reactions to the process.
3:30: Adjourn
3:45: Shuttle leaves for BWI airport
Our
primary goal for the conference was to produce a consensus-generated set of
core competencies and HSR educational domains that are linked to the
competencies.
Meeting Evaluation Methods
We
conducted a Web-based meeting evaluation to assess attendees' perceptions of
how effective the meeting was in meeting its objectives, the usefulness of the
core competency development process to their organization, and their
willingness to continue participating in a national work group on HSR
training. The specific items were as follows:
-
How would you rate the HSR Competencies Conference overall? (Poor-Excellent, 1 to 5).
-
The HSR Competencies Conference
had two major objectives. Please rate the degree to which the conference
accomplished each objective:
-
To foster discussion about the
competencies that are (or should be) common to all health services research
professionals (i.e., the core competencies of HSR) (Poor-Excellent, 1 to 5).
- To produce a consensus-derived
list of core competencies (Poor-Excellent, 1 to 5).
-
How useful was the conference to
you? (Not all all-Extremely, 1 to 5).
-
Please provide any additional
comments or suggestions (open-ended text responses):
_____________________________________________________
-
What approaches would you
recommend for disseminating the core competencies? (journal article(s), a
national work group, special sessions at a professional society meeting such as
AcademyHealth, annual meeting similar to the work group meeting in Baltimore,
other).
- If you are involved with an HSR training program at your institution, will you use the core competencies developed at the
conference to: (n/a, generate discussion with faculty in your program, generate
discussion with students in your program, aid in curriculum development, use as
part of a training grant application, aid in strategic planning, other).
Return to Contents
Results
Core Competencies
The
conference consensus process produced 14 core competencies (Table 1). These
were redistributed to all meeting attendees after the conference for review and
comment, and the final version presented here incorporates these comments.
The following definition of a competency was used
at the meeting:
A
knowledge-based or skill-based asset that all doctoral trainees in HSR should acquire during their training. Core competencies are the common denominator across
all training programs. They are the skills and knowledge sets that all
graduates of HSR training programs should have when they finish their doctoral training.
There
was a vibrant discussion at the meeting on the implications of these core
competencies. Conference attendees expressed enthusiasm for beginning to
explicate the common knowledge bases and skill sets of health services
researchers. There was a fair bit of discussion on how to disseminate the
competencies to as broad an audience as possible-training programs, federal
sponsors of training programs, students (both current and prospective), and all
public and private employers of HSR doctoral trainees. At the same time, some
participants were concerned about embedding the core competencies into an
accreditation process that may have the unintended effect of stifling
innovation in HSR training.
The
14 core competencies provide an overview of the breadth of expertise that can
be expected of all graduates of HSR doctoral training programs. How much depth
to require for each of core competency was a lively topic of debate.
Table 2 presents our approach for characterizing the vertical dimension of
professional competence.
Conference
attendees, for example, did not agree that all HSR trainees should be able to
conduct independent primary and secondary data gathering studies. Instead,
these competencies were worded to reflect an expectation of an intermediate
level of competence—i.e., know how to apply or do something in a supervised
setting. It would be expected that individual trainees would choose one of the
two types of methodological approaches to gain independent expertise in during
their doctoral training.
Prospective
students could use the core competencies by examining the methods that
alternative training programs have selected to attain them as well as the
program's areas of emphasis. Current students can use the competencies as a
benchmark against which they can judge the effectiveness of their personal
educational experiences.
Educational Domains
Conference
attendees evaluated whether each of 36 educational domains of doctoral level
training were essential or optional. The work groups suggested that 9 original
domains either be merged with other domains or eliminated, leaving a total of
27 domains, 21 of which were considered "essential." (Table 3)
Some
domains subsume a descriptive (i.e., awareness of) level of competence as a
prerequisite for doctoral training. An example is that knowledge of
introductory statistics is assumed to be a Masters level set of competency.
The
bullets provide a list of exemplary content for each domain. The list of
content areas is not intended to be exhaustive. A more detailed explication of
each domain could be done at a future meeting of the national work group.
Linkage
Between Core Competencies and Educational Domains
The
educational domains are intended to provide a more detailed description of the
content of the core competencies. Although the conference attendees discussed
whether each domain should be considered an essential part of doctoral
training, the specific competencies within each domain were not explicated at
this conference. That level of specification, it was suggested, would be best
left for individual programs.
Table 4 shows the relationship between the 21 essential educational domains and
the 14 core competencies.
Additional
Discussion Points
Several
discussion points in addition to what was described above for the core
competencies were raised during the conference. These are summarized below:
-
Can we define the specific
methods, both in terms of data collection and data analysis, that form the core
of HSR?
- What are the common core
disciplinary and interdisciplinary theoretical models in HSR?
- The core competencies will need
to be revised on a periodic basis, probably every 5 years. The current set of
the core competencies should be considered a "Version 1."
-
How will the competencies be
used? We need to carefully consider whether they can or should be used for
accreditation purposes.
-
Future meetings could address
alternative educational methods and evaluation approaches for achieving the
core competencies.
Evaluation
Seventeen
meeting participants who were not conference meeting organizers or staff
completed the Web-based survey. The overall rating of the HSR Competencies
Conference was 3.8 on a 5 point scale. About 75% of attendees rated the
conference as very good or excellent.
Ratings
on the two major conference objectives were:
- To foster discussion about the competencies that
are (or should be) common to all health services research professionals (i.e.,
the core competencies of HSR) = 4.2 (on a scale from 1 to 5).
- To produce a consensus-derived list of core
competencies = 2.9 (69% gave this item a "good" rating).
These
results suggest that attendees valued the forum that the conference provided
for discussion on doctoral education. On the other hand, the lower rating for
producing a set of consensus-derived core competencies may be indicative of a
general concern on how these competencies would be used and lingering
uncertainty regarding how effectively the process captured the true core
competencies of HSR. Whether the core competencies are used over the long-term
by students, training programs, faculty, and employers may be the best
indicator of their usefulness.
The
mean rating for how useful was the conference to the meeting attendee was 3.5,
with the majority giving it a "very" or "extremely" useful rating.
The
two most common approaches attendees recommended for disseminating the core
competencies were a special session at a professional society meeting such as
AcademyHealth (94%) and a journal article (56%). We are pleased to report that
Drs. Forrest and Martin have been asked by AcademyHealth to organize a session
on HSR core competencies at the 2006 Annual Research Meeting of AcademyHealth,
and Dr. Stephen Shortell of Berkley University has agreed to moderate it. In
addition, we are preparing a manuscript for a peer-reviewed journal on the core
competencies.
Many
conference attendees were interested in being part of a national work group on
HSR education. We plan to seek additional sponsorship to maintain this group.
Additional funding will be needed for periodic conference calls and possibly an
annual meeting that expands the topics beyond what was covered in the conference
(e.g., Masters training, developing curricula, defining the core methods skill
sets, etc.).
The
most common responses for how attendees would use the competencies were to generate
discussion with faculty in their training program (63%) and to generate
discussion with students in their training program (50%). We have already sent
the final set of core competencies to all meeting participants, many of whom
have shared them within their respective organizations.
Return to Contents
Literature Cited
Adelman, Naomi, (2000). Health Services Research
Core Library Project, Module One, Submitted to the National Information Center
on Health Services Research and Health Care Technology, National Library of
Medicine, Number: 467-MZ-901214.
Calhoun, J. G., Davidson, P. L., Sinioris, M. E.,
Vincent, E. T., and Griffith, J. R. (2002). Toward an Understanding of
Competency Identification and Assessment in Health Care Management. Quality Management in Health Care 11(1):14-38.
Chauvin, SW, Anderson, AC, and Bowdish,
B (2001). Assessing the Professional Development Needs of Public Health
Professionals. J Public Health Management Practice 7(4);23-37.
Institute of Medicine (1995). Health
Services Research: Training and Workforce Issues (Eds. Field MJ, Tranquada RE,
Feasley JC). National Academy Press, Washington DC.
Kennedy, V, and Moore, F. A Systems Approach to
Public Health Workforce Development. J Public Health Management Practice 2001, 7(4):17-22.
Lohr KN, Steinwachs D. Health services research: an
evolving definition of the field. Health Serv Res 2002;37(1):7-9.
National Library of Medicine (2000). Health Services Research—A Historical
Perspective (video), National Information Center on Health Services Research
and Health Care Technology
Prybil, Lawrence (2003). Challenges and
Opportunities Facing Health Administration Practice and Education. Journal
of Health Management 48(8):223-31
Turnock, Bernard (2001). Competency-Based
Credentialing of Public Health Administrators in Illinois. J Public Health
Management Practice 7(4):74-82.
Wistera, W (2001). Competencies in Education: A
Confusion of Tongues. J of Curriculum Studies 33(1):75-88.
Return to Contents
Current as of August 2007
Internet Citation:
Health Services Research Core Competencies. Final Report. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/fund/training/hsrcomp.htm
540 Gaither Road Rockville, MD 20850