To improve the health care response to victims of domestic violence, hospitals
and health care systems are designing and implementing training, screening,
and intervention programs. Formal evaluations of the programs are essential.
The Agency for Healthcare Research and Quality (AHRQ) developed a consensus-driven
quality assessment tool for evaluation of hospital-based domestic violence programs.
Dr. Jeffery H. Coben, while AHRQ's Domestic Violence Senior Scholar-in-Residence,
based the instrument on the views of national experts who took part in an AHRQ-funded
Delphi process. The Family Violence Prevention Fund cosponsored Dr. Coben's
position.
The experts achieved consensus on 37 performance measures, which have been
expanded into a working instrument. The measures are in the form of questions,
each with a list of possible responses.
Select to Download Evaluation Tool.
Background
Tool Development and Testing
Suggested Uses
Guidelines for Completing Instrument
Preparing for Program Assessment
Tool Instructions and Interpretations
Scoring
References
More Information
Download Evaluation Tool
Background
Domestic violence has been recognized as a major public health problem in the
United States. Recent studies have demonstrated that 2-4 percent of all women
seen in hospital emergency departments have acute trauma associated with domestic
violence and another 10-12 percent of women have a recent history of domestic
violence.1-3 Similar studies report that 5-15 percent
of all women seen in other health care settings have a history of recent domestic
violence.4-6 While the majority of injuries sustained
by domestic violence victims are classified as superficial contusion, abrasions,
and lacerations,3,7 an estimated
73,000 hospitalizations and 1,500 deaths among women are attributed to domestic
violence each year.8,9
To address this important problem, health care professionals, victim advocacy
groups, and other professional organizations have been working together to improve
the health care response to victims of domestic violence. Across the Nation,
hospitals and health care systems are designing and implementing domestic violence
training, screening, and intervention programs. As these programs develop, formal
evaluations of their progress are essential to program planners, participants,
policymakers and funders. Unfortunately, instruments to assess these programs
have not been available.
Several prospective studies are currently underway to examine the long-term
impact of hospital-based domestic violence programs on outcomes of health and
safety. While awaiting the results of these studies, it is important that ongoing
efforts be directed toward implementing programs of the highest possible quality
and establishing measures to evaluate and monitor quality.
The quality of health care and the quality of health-care interventions for
domestic violence can be assessed using a health services research paradigm.
This approach attempts to measure quality by examining the structure of the
health care system, the process of health care delivery, and the outcomes associated
with the care provided.10 The paradigm examines
the following:
- Structure—attributes of the settings in which care occurs. This includes
the attributes of material resources (facilities, equipment, money), of human
resources (number and qualifications of personnel), and of organizational
structure.
- Process—what is actually done in giving and receiving care. It includes
the practitioner's activities in making a diagnosis, documenting their findings,
and implementing treatment.
- Outcome—effects of the program on the outcomes of interest, which usually
include morbidity, mortality, quality of life, health care utilization, and
health care costs.
A fundamental tenet of this approach is the belief that programs with good
structures in place will have an increased likelihood of having a good process
of care, and good process increases the likelihood of good outcome. While good
structures and process of care do not guarantee success, they increase the likelihood
of successful outcomes. Similarly, inadequate structures and poor process of
care will likely result in poor outcomes.
Most domestic violence initiatives in health care settings have multiple components.
Many components are structure- and process-oriented and include:
- Training health care providers.
- Establishing a hospital task force or team.
- Establishing specific policies and procedures.
- Modifying environments.
- Screening for victimization.
- Enhancing intervention services.
Formal measurement of these components will allow administrators, advocates,
program managers, and researchers to:
- Determine how well a program has been implemented at a site.
- Compare different programs across sites.
- Determine which program features are most important in producing positive
outcomes.
Return to Contents
Tool Development and Testing
The Agency for Healthcare Research and Quality (AHRQ) developed a consensus-driven
quality assessment tool for evaluation of hospital-based domestic violence programs.
Jeffery H. Coben, M.D., AHRQ Domestic Violence Senior Scholar-in-Residence,
based the instrument on the views of national experts who took part in an AHRQ-funded
Delphi process. His position was cosponsored by the Family Violence Prevention
Fund (FVPF).
Dr. Coben is Associate Professor of Emergency Medicine at Drexel University
College of Medicine and Director of the Center for Violence and Injury Control
Allegheny-Singer Research Institute, Allegheny General Hospital, Pittsburgh,
PA. An authority in prevention of injury and violence, he has published in peer-reviewed
literature on identification of domestic violence victims in medical settings.
An earlier version of the tool was first developed and used in a multi-site
study evaluating the effectiveness of a training model to improve the emergency
department response to battered women.11 The current
version was developed with input from a panel of 18 experts, using the Delphi
process of consensus development. A separate publication fully describes the
details and findings of the consensus development project.12
The instrument is designed to permit a formal assessment of a hospital's performance
in implementing a domestic violence program. Panelists were instructed to concentrate
on structure and process measures. Performance measures were identified and
agreed on by the panel of experts. These measures fell within nine different
domains (categories) of domestic violence program activities including:
- Hospital Policies and Procedures.
- Hospital Physical Environment.
- Hospital Cultural Environment.
- Training of Providers.
- Screening and Safety Assessment.
- Documentation.
- Intervention Services.
- Evaluation Activities.
- Collaboration.
The panelists also assigned weights to each of the categories and to each of
the individual measures, based on their perceived relative importance. These
weights were used to help derive the scoring procedures.
Following completion of the consensus development project, Dr. Coben further
operationalized the instrument by putting each performance measure into the
form of a question or several questions. In addition, a list of possible responses
to each question was developed, categorized, and codified.
Field testing of the instrument was then conducted to determine its feasibility
and inter-rater reliability. Reliability testing was conducted at four different
hospital sites, using two different pairs of coders. One pair of coders, Dr.
Coben and another trained evaluator, were experienced and very familiar with
the instrument, while the other pair of coders were inexperienced and received
minimal training on administering the instrument. In all cases, inter-rater
reliability was very high (Kronbach's alpha ranging from 0.97 to 0.99 in the
experienced coders and 0.96 to 0.99 in the inexperienced coders).
Return to Contents
Suggested Uses
The instrument can be used for a variety of purposes:
- The measures can serve as useful benchmarks or objectives for program achievement.
- An individual site's performance can be assessed repeatedly over time to
determine progress in program implementation. Ideally, sites should perform
a baseline assessment prior to implementing a new program and re-assess their
status annually.
- The instrument can be used by researchers and program administrators to
compare and contrast different programs across different sites.
- If linked to appropriate outcome measures, the instrument could be used
to help determine which program features are most important in creating positive
long-term outcomes for domestic violence victims.
The focus of this instrument is on hospital-based domestic violence programs.
While domestic violence programs have been implemented in many other settings,
the generalizability of the instrument to other settings, including other health
care settings (such as private physician offices or outpatient clinics), has
not been tested.
Return to Contents
Guidelines for Completing Instrument
The evaluation instrument can be self-administered or administered by an independent
evaluator in conjunction with a representative from the hospital domestic violence
program. In either situation, the individual who is most familiar with the domestic
violence program should participate in the process. The required information
can be best obtained via a "site visit" in the hospital. The hospital
should have sufficient time to assemble the documents to be reviewed. Three
weeks advance notice is suggested. The assessment procedures should include
a review of documents as well as a physical tour of the facility to examine
posters, brochures, documentation procedures, equipment (i.e., cameras), and
other supplies. Approximately 4 hours should be allocated for completion of
the instrument.
Return to Contents
Preparing for Program Assessment
To assist in assessment of the hospital's domestic violence program, the materials
below should be assembled and reviewed. The hospital site should be given sufficient
time to gather the information (usually 3 weeks). The site should be asked to
flag or mark the information if it is part of a larger document, such as a hospital
policy manual.
Suggested documentation:
- All written hospital policies, protocols and procedures regarding domestic
violence.
- Relevant department-specific policies and procedures regarding domestic
violence (such as Emergency Department, Security Department, obstetric/gynecologic).
- Information on domestic violence in the hospital's Employee Assistance Program.
- Documentation of the activities of the hospital's domestic violence task
force or committee including:
- Roster of participating individuals, departments, and agencies.
- Schedule of meeting dates.
- Prior meeting minutes or notes.
- Materials used and/or distributed in any domestic violence training for
hospital staff.
- Schedules of planned trainings or strategic plans for training employees.
- Forms or checklists used for domestic violence programs including:
- Domestic violence screening forms.
- Standardized documentation forms.
- Consent-to-photograph forms for domestic violence cases.
- Intervention checklists for staff to use when victims are identified.
- Standardized safety assessment forms.
- Referral forms.
- Information on prior evaluations used as part of the program including:
- Assessments of staff attitude and knowledge of domestic violence.
- Prior chart audits to assess for domestic violence screening.
- Other documented evaluation procedures.
- Documentation of hospital preventive outreach and public education on the
topic of domestic violence.
- Information on financial resources that the hospital provides for the domestic
violence program.
- Copies of brochures, pamphlets, or referral cards for victims of domestic
violence or available to the public in the hospital.
In addition, the site should conduct a chart review to determine compliance
with domestic violence screening. The site should review a random sample of
charts from an area of the hospital where domestic violence screening is supposed
to be conducted. The percentage of female patients who were actually screened
for domestic violence, based on chart documentation, should be recorded.
Return to Contents
Tool Instructions and Interpretations
Category 1. Policies and Procedures
1.1. This information should be obtained from a direct review of the
hospital-wide policies and procedures manual.
1.1(a). Simply requires a definition. Does not require that the definition
meet an external standard or commonly used definition.
1.1(b). If any new and/or existing employees are required
to obtain training, this will result in a "yes" response. If staff
is encouraged, but not required to obtain training, this will result in a "no"
response.
1.1(c). Universal screening refers to the screening of all women seen
in a clinical setting. If the screening policy includes both men and women,
this will result in a "yes" response. If the screening policy includes
one clinical setting (e.g. Emergency Department) or more than one clinical setting,
this will result in a "yes" response. Note, this question does not
require a "mandate," but does require that the hospital encourage,
support, and/or advocate universal screening.
1.1(d). Self-explanatory.
1.1(e). Assess if the hospital policy provides information and/or instructions
on the documentation required for victims of domestic violence.
1.1(f). Assess if the hospital policy includes information and/or instructions
dealing with the referral of victims of domestic violence. Note, a specific
domestic violence agency or hotline number does not need to be included in the
hospital policy.
1.1(g). Assess if the hospital policy provides information dealing with
the legal obligation of health care providers to report domestic violence to
other authorities.
1.2. A "domestic violence task force" could be called other
things, such as a "Working Group," "Abuse Committee," or
"Domestic Violence Coordinating Committee." To qualify as a "yes"
response, there needs to be evidence that the group focuses on the topic of
domestic violence, either exclusively or in conjunction with other closely related
topics (e.g., sexual assault, child abuse). This information should be obtained
through discussions with the site representative and by reviewing the
following types of material:
- Roster or mailing list of representatives on the task force.
- Schedule or announcement of meeting dates.
- Prior meeting notes or minutes.
1.3. "Direct financial support" refers to internal funding,
provided by the hospital, for materials, supplies, and personnel. This does
not include extramural grants or contracts for the program or "in-kind"
contributions such as space, electricity, and heat. This does include
salary and fringe benefits, if the hospital covers some or all of these costs
for the domestic violence program coordinator or other personnel directly involved
in the program.
1.4. Self-explanatory. Compare with 1.1(c). Requires
a written mandate for screening.
1.5(a). "Regular chart audits" means that at least a sample
of charts are reviewed on a regular basis (at least quarterly) to determine
if screening is being performed or documented.
1.5(b). "Positive reinforcers" would include awards or rewards
for screening, contests identifying the individuals or departments with the
highest screening rates, direct feedback to clinicians who are screening, and
prompts or signs to encourage screening.
1.5(c). "Punitive measures" would include the enforcement
of corrective and/or disciplinary actions to individuals or departments if screening
does not occur.
1.6. A "yes" response requires that specific written procedures
be identified. These procedures may be found within the security department,
the hospital's policies and procedures manual, the hospital's domestic violence
policies, or other similar materials.
1.7. The domestic violence coordinator could be a hospital employee
or employed by an outside agency. Typically, the domestic violence coordinator
would be responsible for managing the hospital's domestic violence task force,
coordinating training activities, scheduling meetings and providing leadership
for the programmatic activities.
Category 2. Physical Environment
2.1. This information is obtained from direct observations throughout
the hospital facility. Each separate location housing the public display of
domestic violence-related posters or brochures counts as one location. List
the total number of locations, up to 35. Separate rooms within the same location
(e.g., waiting room, triage area, and patient treatment room in the emergency
department) would each count as a separate location. Multiple posters or posters
and brochures in one room (e.g., 3 posters plus brochures all located in the
emergency department waiting room) would count as only one location.
2.2. Same as 2.1, except material must include referral information
(hotline numbers).
2.3. This information relates to how and where a victim is cared for
in the hospital while awaiting placement in a community-based shelter, and it
assumes that the victim does not require admission to the hospital for medical
reasons. The information may be obtained from a review of hospital domestic
violence-related policies and procedures or from direct conversations with the
domestic violence coordinator.
Category 3. Cultural Environment
3.1. Requires documentation of a previous written assessment (i.e.,
survey or questionnaire) of staff knowledge and attitude about domestic violence.
3.2. A good indicator of program start-up would be the date of the first
task force meeting.
3.3. Information should be obtained from a review of hospital policies
and procedures, written information from the Employee Assistance Program, and/or
discussions with the domestic violence coordinator.
3.4. Examine policies, training materials, and referral information
for evidence of cultural competency as described.
3.5. Self-explanatory. Documented participation should include more
than sponsorship.
Category 4. Training of Providers
The information for this section should be obtained from review of available
training materials and discussion with the domestic violence coordinator. If
the domestic violence coordinator is not directly involved with the training
program, he or she should provide this information in consultation with the
training staff.
4.1. A "formal training plan" indicates that training is systematic
and ongoing, rather than haphazard and irregular. Evidence of strategic planning
for systematic training should be provided. "Clinical staff" refers
to staff that have direct patient care responsibilities. "Non-clinical
staff" would include administrators, clerical staff, security staff, and
others without direct patient care responsibilities.
Category 5. Screening and Safety Assessment
5.1. Use of a "standardized instrument" should be assessed
by actually examining the instrument itself. The screening should include at
least three questions to qualify as a "yes" response, but these questions
can include "framing questions" about domestic violence as well as
specific questions on abuse. Determine if the instrument is a separate "stand
alone" form or if the questions are routinely incorporated/embedded in
the clinical record, for example, as part of the nursing assessment form or
part of the physician's review of systems history form.
5.2. This assessment requires that a sample of charts be retrieved and
reviewed for screening documentation. Charts reviewed should be of female patients
who have visited a clinical setting where screening is supposed to be conducted.
A random sample of a minimum of 25 charts is suggested.
5.3. A "standardized safety assessment" means that all patients
identified as in an active abusive relationship are asked a series of routine
questions regarding their personal safety. These questions may be in the form
of an assessment instrument (e.g., Danger Assessment) or may be verbally administered
by the individual(s) responsible for assisting victims. The safety assessment
should be standardized, meaning all victims are asked the same series of questions.
Category 6. Documentation
6.1. This information should be obtained by reviewing the actual documentation
form or instrument, if one exists.
6.2. Information on procedures for photography should be obtained from
the domestic violence coordinator and from clinical staff responsible for photographic
documentation. The camera should be readily available.
6.2(d). The consent-to-photograph form should be examined. To qualify
as a "yes" response, this form should specify that it provides consent
to photograph injuries from abuse.
Category 7. Intervention Services
7.1. A "standard intervention checklist" refers to a readily
available reference for clinical staff to use that provides information on the
required/suggested steps to be taken if a victim is identified. This checklist
may be in the form of a clinical algorithm, a "pull-packet," a computerized
reminder system, or other similar format. If the checklist is embedded in the
hospital policies and procedures manual, it should be readily accessible for
immediate staff use to qualify as a "yes" answer.
7.2. "Victim advocacy services" refers to service provision
above and beyond immediate safety assessment and referral. Advocacy services
usually include the provision of empathic support, empowerment and options counseling,
education regarding the cycle of violence, promoting self-efficacy, establishing
linkages with community resources, and maintaining followup if desired by the
victim. A "trained victim advocate" is someone who has received specific
training in providing the above services to victims of domestic violence.
7.3. Examples might include assessments for depression, post-traumatic
stress disorder (PTSD), and other mental health problems. Are these types of
assessments performed as a part of the evaluation/intervention for victims of
domestic violence?
7.4. Does the hospital provide transportation for victims if needed,
or vouchers for transportation (e.g., cab slips)?
7.5. A "yes" response would indicate that victims are offered
followup domestic violence support/counseling with personnel from the hospital
domestic violence program. Referral to another site (i.e., shelter program or
another support group) would not qualify as a "yes."
7.6. On-site legal counseling would include activities such as:
- Explaining victim's legal rights and options.
- Providing information on how to obtain a restraining order or protection
from abuse order (PFA).
- Providing assistance with obtaining a restraining order or PFA.
7.7. To qualify for a "yes" response, the
program would need to offer some service above-and-beyond routine child/adolescent
health services. Examples might include specific assessments for children who
witness domestic violence, assessments for co-occurring child abuse and domestic
violence, or providing child care during maternal followup domestic violence
counseling visits.
7.8. "Evidence for coordination" could include:
- A hospital domestic violence task force that includes mental health and/or
substance abuse professionals.
- Policies or procedures that include screening domestic violence victims
for mental health (depression, PTSD, etc.) concerns and/or substance abuse.
- Routinely assessing sexual assault victims for domestic violence and routinely
assessing domestic violence victims for sexual assault.
- Other similar activities.
Category 8. Evaluation Activities
8.1. To qualify for a "yes" response, there should be some
evidence that the program collects and analyzes data on a regular basis. The
data may be qualitative or quantitative and it may include:
- Evaluations of the domestic violence training activities.
- Assessment of staff knowledge and attitudes about domestic violence.
- Interviews or focus groups with providers on the topic of domestic violence
services.
- Assessments of program implementation.
- Chart audits to assess for domestic violence screening and documentation.
8.1(a). A program demonstrating evidence that charts
are audited to assess for domestic violence screening would receive a "yes"
on this item, in addition to a "yes" on item 8.1.
8.1(b). Peer-to-peer reviews involve professionals
discussing specific cases of domestic violence in an effort to improve the quality
of the program. A program demonstrating evidence of peer-to-peer reviews would
receive a "yes" on this item, as well as a "yes" on item
8.1.
8.2. Standardized feedback differs from peer-to-peer reviews, in that
standardized feedback can be provided by the domestic violence coordinator,
advocate, or other individuals involved in the program. Feedback on domestic
violence screening performance, the provider's chart documentation, or the outcome
associated with newly discovered cases of domestic violence are examples of
activities that would qualify for a "yes" response.
8.3. Self-explanatory.
Category 9. Collaboration
9.1-9.2. For these items, the hospital should provide evidence of collaboration
with the agencies indicated. Local domestic violence programs generally refer
to shelter programs and/or advocacy agencies. Local law enforcement could include
police, district attorney's office, sheriff, or other law enforcement agencies.
Participation on the domestic violence task force can be documented via a roster
or mailing list of task force participants. Other specific examples of collaboration
in the areas mentioned should be sought.
9.3(a). Examples of collaboration within the same health care system
might include:
- A hospital domestic violence program that also involves and collaborates
with an affiliated, off-site, out-patient clinic.
- Coordination between a hospital-based
domestic violence program and affiliated off-site private physician offices.
- Collaboration between two or more hospitals that are part of the same health
care system.
9.3(b). Examples would include collaboration between hospital domestic
violence programs that are in the same service region but not part of the same
parent health care system. If a hospital collaborated with multiple hospitals
in their region, including some within the same health care system and some
in different health care systems, they would receive a "yes" for both
9.3(a) and 9.3(b).
Return to Contents
Scoring
Scoring procedures are based upon a weighting scheme developed with input from
the Delphi panelists. For each of the nine categories, the site can receive
a score ranging from zero to 100 points, with 100 being the best possible score.
The total (raw) score for each category is simply a sum of the points received
for each question within that category. The point values assigned to each possible
response are noted in parentheses next to the response. It should be noted that
in some cases only a single (forced choice) response is permitted (e.g., question
1.3), while other questions allow for multiple positive responses (e.g., question
1.1).
To calculate a single total score for the entire instrument, the following
procedure should be followed:
Raw Score for Category 1 = ___ multiplied by 1.16 = A
Raw Score for Category 2 = ___ multiplied by 0.86 = B
Raw Score for Category 3 = ___ multiplied by 1.19 = C
Raw Score for Category 4 = ___ multiplied by 1.15 = D
Raw Score for Category 5 = ___ multiplied by 1.22 = E
Raw Score for Category 6 = ___ multiplied by 0.95 = F
Raw Score for Category 7 = ___ multiplied by 1.29 = G
Raw Score for Category 8 = ___ multiplied by 1.14 = H
Raw Score for Category 9 = ___ multiplied by 1.04 = I
Total Score = A+B+C+D+E+F+G+H+I divided by 10
The total score is also based on a scale of 0-100, with 100 being the best
possible score. Higher scores would indicate a more fully implemented program.
At the current time there are insufficient data to provide cut-off scores for
what may constitute an acceptable, exemplary, or ideal program. Rather, it is
suggested that sites use these scores to establish their own baseline and monitor
their progress over time. As more data becomes available, these data will be
posted so that individual sites can compare their results with the distribution
of scores across multiple sites.
Return to Contents
References
- Dearwater SR, Coben JH, Campbell JC, et al. Prevalence of
intimate partner abuse in community hospitals. JAMA 1998;280:433-8.
- Abbott J, Johnson R, Koziol-McLain J, Lowenstein SR. Domestic
violence against women: incidence and prevalence in an emergency department.
JAMA 1995;273:1763-7.
- Muelleman RA, Lenaghan PA, Pakieser RA. Battered women:
injury locations and types. Ann Emerg Med 1993;28:486-92.
- Jones AS, Campbell J, Gielen A, et al. Annual and lifetime
prevalence of partner abuse in a sample of female HMO enrollees. Women's
Health Issues 1999;9:295-305.
- McCauley J, Kern D, Kolodner K, et al. The "battering
syndrome": prevalence and clinical characteristics of domestic violence
in primary care internal medicine practices. Ann Intern Med 1995;123:737-46.
- Gazmararian JA, Lazorick S, Spitz AM, et al. Prevalence
of violence against pregnant women. JAMA 1996;275:1915-20.
- Coben JH, Forjuoh SN, Goldolf EW. Injuries and health care
use in women with partners in batterer intervention programs. J Fam Violence
1999;14:83-94.
- Crowell NA, Burgess AW, editors. Understanding violence
against women: panel on research on violence against women, National Research
Council. Washington, DC: National Academy Press; 1996.
- Tjaden P and Thoennes N. Full report of the prevalence,
incidence, and consequences of violence against women. Washington DC: National
Institute of Justice and the Centers for Disease Control and Prevention; November
2000. Also available at: http://www.ncjrs.org/txtfiles1/nij/183781.txt.
- Donabedian A. Evaluating the quality of medical care. Milbank
Mem Fund Q 1966;44(3 suppl):166-206.
- Campbell JC, Coben JH, McLoughlin E, et al. An evaluation
of a system-change training model to improve emergency department response
to battered women. Acad Emerg Med 2001;8:131-8.
- Coben, JH. Measuring the quality of hospital-based domestic
violence programs. Acad Emerg Med. In press, November 2002.
Return to Contents
More Information
For more information on the Delphi Instrument for Hospital-based Domestic Violence
Programs, contact:
Jeffrey H. Coben, M.D., Director
Center for Violence and Injury Control
Allegheny General Hospital
320 E. North Ave.
Snyder Pavilion, Suite 214
Pittsburgh, PA 15212
Phone: (412) 359-6260
Fax: (412) 359-6261
E-mail: jcoben@wpahs.org
Return to Contents
Download Evaluation Tool
Your browser may support downloading the evaluation tool from this Web page.
To download it, right click on the links and then select "Save Target As"
(Internet Explorer) or "Save Link As" (Netscape):
Return to Contents
Current as of September 2002
Internet Citation:
Evaluating Domestic Violence Programs. September 2002.
Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/domesticviol/