To improve the health care response to victims of domestic violence, hospitals and health care systems are designing and implementing training, screening, and intervention programs. Formal evaluations of the programs are essential.
The Agency for Healthcare Research and Quality (AHRQ) developed a consensus-driven quality assessment tool for evaluation of hospital-based domestic violence programs. Dr. Jeffery H. Coben, while AHRQ's Domestic Violence Senior Scholar-in-Residence, based the instrument on the views of national experts who took part in an AHRQ-funded Delphi process. The Family Violence Prevention Fund cosponsored Dr. Coben's position.
The experts achieved consensus on 37 performance measures, which have been expanded into a working instrument. The measures are in the form of questions, each with a list of possible responses.
Select to Download Evaluation Tool.
Background
Tool Development and Testing
Suggested Uses
Guidelines for Completing Instrument
Preparing for Program Assessment
Tool Instructions and Interpretations
Scoring
References
More Information
Download Evaluation Tool
Domestic violence has been recognized as a major public health problem in the United States. Recent studies have demonstrated that 2-4 percent of all women seen in hospital emergency departments have acute trauma associated with domestic violence and another 10-12 percent of women have a recent history of domestic violence.1-3 Similar studies report that 5-15 percent of all women seen in other health care settings have a history of recent domestic violence.4-6 While the majority of injuries sustained by domestic violence victims are classified as superficial contusion, abrasions, and lacerations,3,7 an estimated 73,000 hospitalizations and 1,500 deaths among women are attributed to domestic violence each year.8,9
To address this important problem, health care professionals, victim advocacy groups, and other professional organizations have been working together to improve the health care response to victims of domestic violence. Across the Nation, hospitals and health care systems are designing and implementing domestic violence training, screening, and intervention programs. As these programs develop, formal evaluations of their progress are essential to program planners, participants, policymakers and funders. Unfortunately, instruments to assess these programs have not been available.
Several prospective studies are currently underway to examine the long-term impact of hospital-based domestic violence programs on outcomes of health and safety. While awaiting the results of these studies, it is important that ongoing efforts be directed toward implementing programs of the highest possible quality and establishing measures to evaluate and monitor quality.
The quality of health care and the quality of health-care interventions for domestic violence can be assessed using a health services research paradigm. This approach attempts to measure quality by examining the structure of the health care system, the process of health care delivery, and the outcomes associated with the care provided.10 The paradigm examines the following:
A fundamental tenet of this approach is the belief that programs with good structures in place will have an increased likelihood of having a good process of care, and good process increases the likelihood of good outcome. While good structures and process of care do not guarantee success, they increase the likelihood of successful outcomes. Similarly, inadequate structures and poor process of care will likely result in poor outcomes.
Most domestic violence initiatives in health care settings have multiple components. Many components are structure- and process-oriented and include:
Formal measurement of these components will allow administrators, advocates, program managers, and researchers to:
The Agency for Healthcare Research and Quality (AHRQ) developed a consensus-driven quality assessment tool for evaluation of hospital-based domestic violence programs. Jeffery H. Coben, M.D., AHRQ Domestic Violence Senior Scholar-in-Residence, based the instrument on the views of national experts who took part in an AHRQ-funded Delphi process. His position was cosponsored by the Family Violence Prevention Fund (FVPF).
Dr. Coben is Associate Professor of Emergency Medicine at Drexel University College of Medicine and Director of the Center for Violence and Injury Control Allegheny-Singer Research Institute, Allegheny General Hospital, Pittsburgh, PA. An authority in prevention of injury and violence, he has published in peer-reviewed literature on identification of domestic violence victims in medical settings.
An earlier version of the tool was first developed and used in a multi-site study evaluating the effectiveness of a training model to improve the emergency department response to battered women.11 The current version was developed with input from a panel of 18 experts, using the Delphi process of consensus development. A separate publication fully describes the details and findings of the consensus development project.12
The instrument is designed to permit a formal assessment of a hospital's performance in implementing a domestic violence program. Panelists were instructed to concentrate on structure and process measures. Performance measures were identified and agreed on by the panel of experts. These measures fell within nine different domains (categories) of domestic violence program activities including:
The panelists also assigned weights to each of the categories and to each of the individual measures, based on their perceived relative importance. These weights were used to help derive the scoring procedures.
Following completion of the consensus development project, Dr. Coben further operationalized the instrument by putting each performance measure into the form of a question or several questions. In addition, a list of possible responses to each question was developed, categorized, and codified.
Field testing of the instrument was then conducted to determine its feasibility and inter-rater reliability. Reliability testing was conducted at four different hospital sites, using two different pairs of coders. One pair of coders, Dr. Coben and another trained evaluator, were experienced and very familiar with the instrument, while the other pair of coders were inexperienced and received minimal training on administering the instrument. In all cases, inter-rater reliability was very high (Kronbach's alpha ranging from 0.97 to 0.99 in the experienced coders and 0.96 to 0.99 in the inexperienced coders).
The instrument can be used for a variety of purposes:
The focus of this instrument is on hospital-based domestic violence programs. While domestic violence programs have been implemented in many other settings, the generalizability of the instrument to other settings, including other health care settings (such as private physician offices or outpatient clinics), has not been tested.
The evaluation instrument can be self-administered or administered by an independent evaluator in conjunction with a representative from the hospital domestic violence program. In either situation, the individual who is most familiar with the domestic violence program should participate in the process. The required information can be best obtained via a "site visit" in the hospital. The hospital should have sufficient time to assemble the documents to be reviewed. Three weeks advance notice is suggested. The assessment procedures should include a review of documents as well as a physical tour of the facility to examine posters, brochures, documentation procedures, equipment (i.e., cameras), and other supplies. Approximately 4 hours should be allocated for completion of the instrument.
To assist in assessment of the hospital's domestic violence program, the materials below should be assembled and reviewed. The hospital site should be given sufficient time to gather the information (usually 3 weeks). The site should be asked to flag or mark the information if it is part of a larger document, such as a hospital policy manual.
Suggested documentation:
In addition, the site should conduct a chart review to determine compliance with domestic violence screening. The site should review a random sample of charts from an area of the hospital where domestic violence screening is supposed to be conducted. The percentage of female patients who were actually screened for domestic violence, based on chart documentation, should be recorded.
1.1. This information should be obtained from a direct review of the hospital-wide policies and procedures manual.
1.1(a). Simply requires a definition. Does not require that the definition meet an external standard or commonly used definition.
1.1(b). If any new and/or existing employees are required to obtain training, this will result in a "yes" response. If staff is encouraged, but not required to obtain training, this will result in a "no" response.
1.1(c). Universal screening refers to the screening of all women seen in a clinical setting. If the screening policy includes both men and women, this will result in a "yes" response. If the screening policy includes one clinical setting (e.g. Emergency Department) or more than one clinical setting, this will result in a "yes" response. Note, this question does not require a "mandate," but does require that the hospital encourage, support, and/or advocate universal screening.
1.1(d). Self-explanatory.
1.1(e). Assess if the hospital policy provides information and/or instructions on the documentation required for victims of domestic violence.
1.1(f). Assess if the hospital policy includes information and/or instructions dealing with the referral of victims of domestic violence. Note, a specific domestic violence agency or hotline number does not need to be included in the hospital policy.
1.1(g). Assess if the hospital policy provides information dealing with the legal obligation of health care providers to report domestic violence to other authorities.
1.2. A "domestic violence task force" could be called other things, such as a "Working Group," "Abuse Committee," or "Domestic Violence Coordinating Committee." To qualify as a "yes" response, there needs to be evidence that the group focuses on the topic of domestic violence, either exclusively or in conjunction with other closely related topics (e.g., sexual assault, child abuse). This information should be obtained through discussions with the site representative and by reviewing the following types of material:
1.3. "Direct financial support" refers to internal funding, provided by the hospital, for materials, supplies, and personnel. This does not include extramural grants or contracts for the program or "in-kind" contributions such as space, electricity, and heat. This does include salary and fringe benefits, if the hospital covers some or all of these costs for the domestic violence program coordinator or other personnel directly involved in the program.
1.4. Self-explanatory. Compare with 1.1(c). Requires a written mandate for screening.
1.5(a). "Regular chart audits" means that at least a sample of charts are reviewed on a regular basis (at least quarterly) to determine if screening is being performed or documented.
1.5(b). "Positive reinforcers" would include awards or rewards for screening, contests identifying the individuals or departments with the highest screening rates, direct feedback to clinicians who are screening, and prompts or signs to encourage screening.
1.5(c). "Punitive measures" would include the enforcement of corrective and/or disciplinary actions to individuals or departments if screening does not occur.
1.6. A "yes" response requires that specific written procedures be identified. These procedures may be found within the security department, the hospital's policies and procedures manual, the hospital's domestic violence policies, or other similar materials.
1.7. The domestic violence coordinator could be a hospital employee or employed by an outside agency. Typically, the domestic violence coordinator would be responsible for managing the hospital's domestic violence task force, coordinating training activities, scheduling meetings and providing leadership for the programmatic activities.
2.1. This information is obtained from direct observations throughout the hospital facility. Each separate location housing the public display of domestic violence-related posters or brochures counts as one location. List the total number of locations, up to 35. Separate rooms within the same location (e.g., waiting room, triage area, and patient treatment room in the emergency department) would each count as a separate location. Multiple posters or posters and brochures in one room (e.g., 3 posters plus brochures all located in the emergency department waiting room) would count as only one location.
2.2. Same as 2.1, except material must include referral information (hotline numbers).
2.3. This information relates to how and where a victim is cared for in the hospital while awaiting placement in a community-based shelter, and it assumes that the victim does not require admission to the hospital for medical reasons. The information may be obtained from a review of hospital domestic violence-related policies and procedures or from direct conversations with the domestic violence coordinator.
3.1. Requires documentation of a previous written assessment (i.e., survey or questionnaire) of staff knowledge and attitude about domestic violence.
3.2. A good indicator of program start-up would be the date of the first task force meeting.
3.3. Information should be obtained from a review of hospital policies and procedures, written information from the Employee Assistance Program, and/or discussions with the domestic violence coordinator.
3.4. Examine policies, training materials, and referral information for evidence of cultural competency as described.
3.5. Self-explanatory. Documented participation should include more than sponsorship.
The information for this section should be obtained from review of available training materials and discussion with the domestic violence coordinator. If the domestic violence coordinator is not directly involved with the training program, he or she should provide this information in consultation with the training staff.
4.1. A "formal training plan" indicates that training is systematic and ongoing, rather than haphazard and irregular. Evidence of strategic planning for systematic training should be provided. "Clinical staff" refers to staff that have direct patient care responsibilities. "Non-clinical staff" would include administrators, clerical staff, security staff, and others without direct patient care responsibilities.
5.1. Use of a "standardized instrument" should be assessed by actually examining the instrument itself. The screening should include at least three questions to qualify as a "yes" response, but these questions can include "framing questions" about domestic violence as well as specific questions on abuse. Determine if the instrument is a separate "stand alone" form or if the questions are routinely incorporated/embedded in the clinical record, for example, as part of the nursing assessment form or part of the physician's review of systems history form.
5.2. This assessment requires that a sample of charts be retrieved and reviewed for screening documentation. Charts reviewed should be of female patients who have visited a clinical setting where screening is supposed to be conducted. A random sample of a minimum of 25 charts is suggested.
5.3. A "standardized safety assessment" means that all patients identified as in an active abusive relationship are asked a series of routine questions regarding their personal safety. These questions may be in the form of an assessment instrument (e.g., Danger Assessment) or may be verbally administered by the individual(s) responsible for assisting victims. The safety assessment should be standardized, meaning all victims are asked the same series of questions.
6.1. This information should be obtained by reviewing the actual documentation form or instrument, if one exists.
6.2. Information on procedures for photography should be obtained from the domestic violence coordinator and from clinical staff responsible for photographic documentation. The camera should be readily available.
6.2(d). The consent-to-photograph form should be examined. To qualify as a "yes" response, this form should specify that it provides consent to photograph injuries from abuse.
7.1. A "standard intervention checklist" refers to a readily available reference for clinical staff to use that provides information on the required/suggested steps to be taken if a victim is identified. This checklist may be in the form of a clinical algorithm, a "pull-packet," a computerized reminder system, or other similar format. If the checklist is embedded in the hospital policies and procedures manual, it should be readily accessible for immediate staff use to qualify as a "yes" answer.
7.2. "Victim advocacy services" refers to service provision above and beyond immediate safety assessment and referral. Advocacy services usually include the provision of empathic support, empowerment and options counseling, education regarding the cycle of violence, promoting self-efficacy, establishing linkages with community resources, and maintaining followup if desired by the victim. A "trained victim advocate" is someone who has received specific training in providing the above services to victims of domestic violence.
7.3. Examples might include assessments for depression, post-traumatic stress disorder (PTSD), and other mental health problems. Are these types of assessments performed as a part of the evaluation/intervention for victims of domestic violence?
7.4. Does the hospital provide transportation for victims if needed, or vouchers for transportation (e.g., cab slips)?
7.5. A "yes" response would indicate that victims are offered followup domestic violence support/counseling with personnel from the hospital domestic violence program. Referral to another site (i.e., shelter program or another support group) would not qualify as a "yes."
7.6. On-site legal counseling would include activities such as:
7.7. To qualify for a "yes" response, the program would need to offer some service above-and-beyond routine child/adolescent health services. Examples might include specific assessments for children who witness domestic violence, assessments for co-occurring child abuse and domestic violence, or providing child care during maternal followup domestic violence counseling visits.
7.8. "Evidence for coordination" could include:
8.1. To qualify for a "yes" response, there should be some evidence that the program collects and analyzes data on a regular basis. The data may be qualitative or quantitative and it may include:
8.1(a). A program demonstrating evidence that charts are audited to assess for domestic violence screening would receive a "yes" on this item, in addition to a "yes" on item 8.1.
8.1(b). Peer-to-peer reviews involve professionals discussing specific cases of domestic violence in an effort to improve the quality of the program. A program demonstrating evidence of peer-to-peer reviews would receive a "yes" on this item, as well as a "yes" on item 8.1.
8.2. Standardized feedback differs from peer-to-peer reviews, in that standardized feedback can be provided by the domestic violence coordinator, advocate, or other individuals involved in the program. Feedback on domestic violence screening performance, the provider's chart documentation, or the outcome associated with newly discovered cases of domestic violence are examples of activities that would qualify for a "yes" response.
8.3. Self-explanatory.
9.1-9.2. For these items, the hospital should provide evidence of collaboration with the agencies indicated. Local domestic violence programs generally refer to shelter programs and/or advocacy agencies. Local law enforcement could include police, district attorney's office, sheriff, or other law enforcement agencies. Participation on the domestic violence task force can be documented via a roster or mailing list of task force participants. Other specific examples of collaboration in the areas mentioned should be sought.
9.3(a). Examples of collaboration within the same health care system might include:
9.3(b). Examples would include collaboration between hospital domestic violence programs that are in the same service region but not part of the same parent health care system. If a hospital collaborated with multiple hospitals in their region, including some within the same health care system and some in different health care systems, they would receive a "yes" for both 9.3(a) and 9.3(b).
Scoring procedures are based upon a weighting scheme developed with input from the Delphi panelists. For each of the nine categories, the site can receive a score ranging from zero to 100 points, with 100 being the best possible score. The total (raw) score for each category is simply a sum of the points received for each question within that category. The point values assigned to each possible response are noted in parentheses next to the response. It should be noted that in some cases only a single (forced choice) response is permitted (e.g., question 1.3), while other questions allow for multiple positive responses (e.g., question 1.1).
To calculate a single total score for the entire instrument, the following procedure should be followed:
Raw Score for Category 1 = ___ multiplied by 1.16 = A
Raw Score for Category 2 = ___ multiplied by 0.86 = B
Raw Score for Category 3 = ___ multiplied by 1.19 = C
Raw Score for Category 4 = ___ multiplied by 1.15 = D
Raw Score for Category 5 = ___ multiplied by 1.22 = E
Raw Score for Category 6 = ___ multiplied by 0.95 = F
Raw Score for Category 7 = ___ multiplied by 1.29 = G
Raw Score for Category 8 = ___ multiplied by 1.14 = H
Raw Score for Category 9 = ___ multiplied by 1.04 = I
Total Score = A+B+C+D+E+F+G+H+I divided by 10
The total score is also based on a scale of 0-100, with 100 being the best possible score. Higher scores would indicate a more fully implemented program. At the current time there are insufficient data to provide cut-off scores for what may constitute an acceptable, exemplary, or ideal program. Rather, it is suggested that sites use these scores to establish their own baseline and monitor their progress over time. As more data becomes available, these data will be posted so that individual sites can compare their results with the distribution of scores across multiple sites.
For more information on the Delphi Instrument for Hospital-based Domestic Violence Programs, contact:
Jeffrey H. Coben, M.D., Director
Center for Violence and Injury Control
Allegheny General Hospital
320 E. North Ave.
Snyder Pavilion, Suite 214
Pittsburgh, PA 15212
Phone: (412) 359-6260
Fax: (412) 359-6261
E-mail: jcoben@wpahs.org
Your browser may support downloading the evaluation tool from this Web page. To download it, right click on the links and then select "Save Target As" (Internet Explorer) or "Save Link As" (Netscape):
Current as of September 2002
Internet Citation:
Evaluating Domestic Violence Programs. September 2002. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/research/domesticviol/
Return to Women's Health
Research Findings
AHRQ Home Page
Department of Health and Human Services