Appendix A. HIV/AIDS Prevention Research Synthesis (PRS) Project Purpose and Selection Criteria

In 1996, CDC began the HIV/AIDS Prevention Research Synthesis (PRS) project to create a database of all HIV/AIDS behavioral, social, and policy studies to meet the needs of HIV prevention researchers, service providers and users, planners, policy makers, and others. The PRS project has several aims:

1. to conduct systematic reviews that address the population, intervention, study design, setting, and outcome factors associated with intervention effectiveness;
2. to identify methodologically rigorous studies that have statistically significant positive results; and
3. to identify gaps in the existing research and directions for future study.

The scope of the PRS project includes all studies with a focus on HIV prevention. Studies conducted in and outside the United States are included to enhance our understanding of risk reduction. Both published and unpublished reports of the studies are included to minimize the effect of “publication bias.” Studies reported from 1988 onward are included in the database to coincide approximately with the start-up of HIV intervention research. A study typically was conducted two or three years earlier than the report date. The following types of studies are not considered part of our focus: biomedical-only, including vertical transmission; drug treatment only; occupational exposure; blood supply exposure; and referral-to-services-only.

Due to the time it takes to publish, index, and review studies, the PRS team, using the PRS database as of June 30, 1998, has reviewed about half of the studies reported or published between 1988-1996. The PRS team has not yet completed an extensive review of studies published or reported after 1996.

The PRS database and the Compendium will be updated periodically. At the time we completed the review for this Compendium, the PRS database contained approximately 4,000 citations on HIV prevention, which included 553 citations reporting on outcome evaluations of HIV risk reduction interventions. Among these, 276 studies were reviewed for PRS relevance and rigor criteria. Of the 276 intervention studies, 124 studies met all of the PRS criteria for relevance (measured at least one of the behavioral or biologic outcomes listed below) and methodologic rigor (used experimental or quasi-experimental designs with comparison groups). Of the remaining 152 intervention studies, 53 studies did not meet the PRS relevance criteria because they measured only psychosocial outcomes (Not Relevant) and 99 studies met the PRS relevance criteria but did not meet the criteria for methodologic rigor (Relevant Not Rigorous). The following describes in detail the PRS criteria for relevance and methodologic rigor, and Figure 1 illustrates the application of these criteria.

1. PRS Criteria
1. Criteria for Scope – These criteria are used to select studies that focus on HIV prevention interventions and that are not being studied extensively elsewhere. An in-scope study meets the following criteria:
1. HIV prevention focus
2. Reported from 1988 onward
3. Published or unpublished
4. Conducted inside or outside the U.S.
5. Not drug treatment only
6. Not biomedical only, (e.g., vaccine trials, AZT to prevent perinatal transmission)
7. Not occupational exposure
8. Not blood supply exposure
2. Criteria for Relevance – These criteria are used to select studies that aim to reduce sex- or drug-related risk behaviors or incidence rates of HIV or other STDs. The outcomes that PRS has determined to be relevant are those that directly impact the transmission of HIV or are indicators of HIV transmission. A relevant study must include one or more of the following outcomes:

3. Criteria for Methodologic Rigor – All relevant studies are evaluated for methodologic rigor, regardless of the findings (i.e., studies with negative or null findings are included). These criteria are based on study design and vary by intervention category¹. Behavioral and social intervention studies are classified as methodologically rigorous if they used random assignment to intervention and control groups (experimental designs) and reported at least post-intervention data. Behavioral and social studies are also considered rigorous if they used non-biased assignment (e.g., systematic assignment) to intervention and comparison groups (quasi-experimental designs) with equivalence of groups or used statistical adjustment for any nonequivalence, and reported pre- and post-intervention data. Policy interventions used these designs or less rigorous designs, such as designs with pre-post data but without a comparison group. These criteria are summarized below and in Figure 2.
   1. For behavioral and social intervention studies:
   • Random assignment to intervention and comparison groups WITH
   • post-intervention data

   • Non-random assignment to intervention and comparison groups using a non-biased method WITH
   • pre-post data AND
   • apparent equivalence of groups OR
   • adjustment for apparent non-equivalence of groups

   2. For policy studies:
   • Random assignment to intervention and comparison groups WITH
   • post-intervention data

   • Nonrandom assignment to intervention and comparison groups using a non-biased method, WITH
   • post-intervention data AND
   • apparent equivalence of groups OR
   • adjustment for apparent nonequivalence of groups

   • Pre-post data with no comparison group

Figure A.1 PRS Review Process: Reviewed OUTCOME STUDIES, as of 6/30/98
(Click on the image below to zoom)

^aSupplemental Background Reports provide information associated with primary studies, e.g. intervention description, methods baseline data
^bSupplemental Outcome Reports provide evaluation data associated with primary studies, e.g. subsequent data waves, subsets of participants.

2. Compendium Criteria

To identify interventions for this Compendium we reviewed the relevant and rigorous studies using additional selection criteria:
1. Studies conducted in the United States – this Compendium is intended to meet the needs of prevention service providers, planners and others in the US, therefore, interventions conducted inside the US were thought to be more applicable in context and content.
2. Behavioral and social interventions, excluding policy studies – Policy studies evaluate administrative or legislative decisions rather than an intervention per se. Most interventions that are the basis of a policy study have been evaluated elsewhere. For these reasons, only behavioral and social studies are included in this Compendium.

We then examined this subset, further selecting studies that met the following effectiveness criteria:

3. Positive results on PRS relevant outcome(s) – Outcomes that are relevant to PRS are those behaviors that would have a direct impact on HIV transmission and biological outcomes that are indicators of HIV transmission. Thus an important measure of a study’s effectiveness is a positive result on at least one of these outcomes.
4. Positive results that represent a statistically significant difference between the intervention and the control or comparison condition
5. No statistically significant negative findings on PRS relevant outcomes – It would not be expedient to recommend an intervention that produced negative results for a behavior that directly impacts HIV transmission or for a biological indicator of HIV transmission, even though it may have produced a positive effect on another relevant behavior or indicator.

Figure A.2 illustrates the application of these criteria. Applying all five of these criteria resulted in the 24 interventions contained in this Compendium. Within the constraints indicated by the criteria listed above, these represent the best state-of-the-science interventions available as of June 30, 1998.

Consistent with these pre-established criteria, many studies were not selected for the Compendium. We did not select, for instance, studies where there was no control or comparison condition in the study design. Many of these studies with other designs provide valuable information but are out of the scope of this Compendium. Also, another CDC project, Characteristics of Reputationally Strong Programs (described elsewhere) examines such programs.

Figure A.2 Identify Effective Interventions

Identify and review HIV intervention studies

Select those that meet outcome relevance and study design/rigor criteria (see A.2 & 3.)

Select behavioral and social interventions conducted in US (65 of 99 studies reviewed by end of review period) (see B.1&2.)

Identify studies that meet criteria for effectiveness (see B.3,4&5.)

Go to Appendix B