Syndemic Prevention Network: Overview -- Plan Evaluation

A syndemic orientation reinforces the idea that the focal mission of public health goes beyond epidemic control to include improvements in the public's health. To a profession that has become deeply fragmented, confronting syndemics could restore a sense of wholeness and common purpose. To be meaningful, this unification must also be incorporated in the values and procedures used to plan program strategy and to document achievements.

The methods for planning and evaluating syndemic prevention initiatives draw upon established techniques as well as innovative options that have been, and are continuing to be developed. Planners and evaluators of comprehensive health improvement initiatives will benefit by using a syndemic orientation because it provides a systematic framework for

Differentiating Between Attribution and Contribution
Valued outcomes in syndemic prevention include the control of specific diseases, but that is not all. To achieve meaningful gains in health, programs and policies must also be aligned across a number of problem areas so that they have a combined effect on reducing the burden of disease from interrelated causes. With this focus, discourse could shift from a fragmented emphasis on attribution (i.e., epidemic control) to a united search for contribution (i.e., systems change, health improvement, increasing health equity).

Through the analysis of contribution, new independent variables might be found, and linked groups of dependent variables could be used to differentiate effects for different degrees of collaborative action. Tools such as the outcomes network (Figure 5), which incorporate higher degrees of organizational depth than conventional logic models, can help clarify complex relationships and spot areas of convergence or divergence in planners' theories of change. This tool can also be used to negotiate lines of accountability, indicating the boundaries within which credit for accomplishments will be shared among all partners.

Expanding Outcome Measures to Include Summary Measures of Population Health
An expanded range of health status measures can also be used as the outcomes for syndemic prevention initiatives. Although there is near universal consensus that no single measure is appropriate for capturing the effects of health promotion initiatives, there remains widespread confusion about how to identify and measure outcomes that matter.

Usually, planners and evaluators must either conduct parallel analyses, substituting different outcomes for different facets of their program, or face the unpleasant task of looking for effects only in selected categorical areas. The latter option is often chosen, despite the fact that it alienates those partners who construe their missions differently and leaves undocumented important changes that might indeed have occurred. For example, even though an initiative might have triggered profound changes in community conditions and systems, the effects reported tend to be only those preferred by the categorical funding source(s).

The emergence in recent years of "summary measures of population health" (IOM, 1998) adds an important new class of outcomes to consider. Also known as "burden of disease" measures (Murray CJL, et.al., 1996), they combine information about morbidity and mortality or address morbidity alone, often across a wide range of health areas. As such, they possess an intrinsic syndemic orientation. Burden of disease measures summarize health status relative to clusters of conditions instead of singularly defined disease states. Compared with categorical measures, burden of disease measures provide decision makers with more complete and sensitive information about overall population health, al though they "incorporate critical but not necessarily obvious or well-accepted judgments about whose life or what kind of life has meaning and worth" (IOM, 1998).

Progress in developing and interpreting data on burden of disease is occurring rapidly throughout the world. Perhaps the most straightforward measure developed to date has been the CDC's "healthy days" index (CDC, 2000). Since 1993, the Behavioral Risk Factor Surveillance System has asked respondents to state how many days in the last 30 their (mental or physical) health was not good. Striking findings have been reported using these data (CDC, 2001). If confronted with summary measures of population health, program planners, policy makers, and the general public will likely call for interventions with a syndemic character, that is, interventions addressing directly the conditions that support overall health.

Defining Conditions for Health
The charge to assure the conditions for health is a charge to remake the world into a safer healthier place. It is a mandate to construct an ecology free of known health hazards, which protects people equally, and which is equipped to respond to emerging problems. Before intervention plans are approved or evaluation strategies selected, health planners must be clear about the world in which their constituents want to live and what legacies will be left for future generations.

Nainoa Thompson, lead navigator of the Polynesian Voyaging Society, understands well the responsibility of planning voyages. In 1995, he was instrumental in helping the children of Hawaii articulate their vision for the future, which eventually became formalized as the Ke Ala Hoku Critical Indicators (Hawaii Community Services Council, 1999). Here, Nainoa Thompson talks about an experience in which he and 18 school children came to appreciate the deep significance of assuring the conditions for health. He started by asking,

Charting Progress Using Navigational Statistics
The image of a navigational voyage is perhaps the most common metaphor used to describe public health ventures. Yet navigational properties are not incorporated formally into the procedures used for charting progress and understanding change in public health. Explicit mathematical models for navigation are, however, used in other branches of science, such as seamanship, geography, oceanography, zoology, and geology, where it is common to collect and analyze directional data (i.e., data describing movement from one place to another).

Scientists who study navigation use navigational statistics, also known as circular statistics because they are based on polar coordinates instead of the Cartesian grid (Figure 6; Jammalamadaka SR, Sengupta A, 2001; Fisher NI., 1993).

Geometry diagram of navigational statistics

** Adapted from: Baker RR. Human navigation and the sixth sense. New York, NY: Simon and Schuster. 1981.

These are the only valid approaches for analyzing directional data. Methodologists are unequivocal about the potential biases involved in using other procedures for data of this kind.

Directional data usually refer to movement through physical space, but with a suitable theory they may also be used to model transitions through social space, such as the movement from one set of community conditions to another. Public health professionals in the 19th century (e.g., Florence Nightingale) presented their work using data displays from circular statistics. A movement away from these procedures took place as Cartesian methods became the dominant techniques in medicine and social science. In the context of a syndemic prevention initiative, navigational statistics might well provide the elusive quantitative tools necessary to demonstrate the effect of community and systems change on health status.

Documenting Changes in Community Conditions and Systems
Community conditions and systems encompass the social, physical, organizational, and other ecological attributes that make each neighborhood or community unique. Because these factors have profound effects on health and well-being, public health advocates, especially those who operate from a syndemic orientation, must be concerned with identifying harmful conditions and creating positive changes. Those changes could include new or modified

Changes in community conditions and systems generally have an indirect effect on health status because they alter individual behavior (e.g., tobacco use) or biology (e.g., blood pressure), which in turn affects health. Some system changes can exert a direct effect on health, however, such as those that remove harmful exposures from the environment or eliminate obstacles to life-saving services (e.g., improve response time by police, fire fighters, or ambulances) (Figure 7).

Practitioners working to prevent syndemics ought to identify, advocate for, and celebrate positive changes while being vigilant about tracking unexpected or unwanted occurrences, particularly those that threaten health or undermine the effectiveness of public health programs. Indeed, documenting the persistence of harmful conditions can be a powerful tool for advocacy when positive changes are not occurring.

Measures of ecological changes are related to but different from indicators that aggregate individual behavior. For example, the proportion of children in a community who have up-to-date immunizations is a summary of individual behavior. This statistic might rise or fall depending upon factors like the number and location of immunization clinics in the neighborhood, the quality of those clinical services, or the level of community trust in health workers, all of which are attributes of the community conditions and systems.

Although achievements in health promotion must ultimately be measured as improvements in health status and quality of life, it often takes decades for those effects to become visible. Earlier indicators of progress are widespread changes in biology or behavior. Still earlier indicators are changing community conditions and systems, which provide a sign that health promotion initiatives are on track for success.

Thanks in part to pervasive information technology, systematically recording changes in community conditions and systems is becoming more and more feasible. In fact, health officials are now exploring ways of working closely with community members to build surveillance systems that monitor changes in community conditions, just as they now track trends in behaviors, diseases, and other health events. Unlike traditional objects of public health surveillance, many changes in conditions and systems can be recorded prospectively or identified retrospectively. This flexibility is due to the fact that these changes tend to be either present or absent (i.e., a walking trail exists or it does not; schools have a no-smoking policy for staff or not; etc.).

Recording changes in community conditions and systems is analogous to keeping a community journal and provides the foundation for telling a factual, evidence-based story about how the community has been changing. This is an important piece of the puzzle for understanding how successful initiatives to prevent syndemics work. Analyses that include measures of conditions and systems are stronger because they account for context (Figure 8).

One hypothesis for the disappointing ratio of health promotion programs conducted to those that achieve success is that there are unacknowledged and unmeasured contextual influences that mediate program effectiveness (Kreuter M, et al., 2001). From the perspective of a practitioner or an analyst, the relationship between intervention action and health objectives (either behavioral or biological) can be established more forcefully when community conditions and systems are taken into account.

Recognizing the Difference Between Communities and Other Objects of Inquiry
Part of the difficulty in planning and evaluating comprehensive community initiatives stems from the extent to which communities are unlike other objects of inquiry. Communities behave in ways that are more like complex adaptive systems than like stable bounded entities. In a community, for example, interconnecting parts function as a whole, with profound feedback and delay effects; the essential properties of community life can be changed or damaged if influences are added or removed; the arrangement of resources, including who has access to them, is crucial; and the behaviors of people or organizations are affected by the community's total structure; change the structure and the behaviors can change as well. Eoyang and Berkas (1999) have summarized the attributes of complex adaptive systems and gone on to identify tools and techniques for evaluation that seemed well-matched to those attributes (Table 3).

Distinguishing Between Principles of Research and Program Evaluation
Far more work remains to be done in identifying analytic methods that are appropriate for understanding how whole communities function and change over time, but methodologies can only be used appropriately when the principles guiding their application are explicit. Here, there is an opportunity to clarify much of the confusion that surrounds the evaluation of interventions that use a syndemic orientation.

Efforts to achieve directed social change can be thought of in multiple ways. Often they are seen as social experiments, at other times as an integral part of social learning. When thinking of community initiatives as experiments, it is logical to apply conventional research principles, but when the enterprise of social learning takes greater prominence, program evaluation principles are often a better fit. Few decision makers are trained to recognize the distinction between these perspectives; indeed, most people view evaluation as research. Not surprisingly, research procedures are often misapplied to the task of learning whether and under what conditions community interventions can be effective in improving health status.

The vast majority of public health work does not, and should not, take place in the context of experimental research. At the same time, everything attempted in the effort to protect the public's health ought to be the basis for learning and improvement. This distinction has profound implications for planning, decision making, framing questions, and nearly every other aspect of program design and development (Table 4).

Table 4: Conventional Principles of Research and Program Evaluation
	Research	Program Evaluation
Planning	Scientific Method • State hypothesis • Collect data • Analyze data • Draw conclusions	Framework for Program Evaluation • Engage stakeholders • Describe the program • Focus the evaluation design • Gather credible evidence • Justify conclusions • Ensure use and share lessons learned
Decision Making	Investigator-controlled • Authoritative	Stakeholder-controlled • Collaborative
Setting Standards	Validity • Internal (accuracy, precision) • External (generalizability) Repeatability	Program Evaluation Standards • Utility • Feasibility • Propriety • Accuracy
Framing Questions	Facts • Descriptions • Associations • Effects	Values • Merit (i.e., quality) • Worth (i.e., value) • Significance (i.e., importance)
Constructing Knowledge	Isolate Changes and Control Circumstances • Narrow experimental influences • Ensure stability over time • Minimize context dependence • Treat contextual factors as confounders that necessitate randomization, adjustment, or statistical control • Control or comparison groups are a necessity	Incorporate Changes and Account for Circumstances • Expand to see all domains of influence • Encourage flexibility and improvement • Maximize context sensitivity • Treat contextual factors as essential information using system diagrams, logic models, and hierarchical or ecological modeling • Control or comparison groups are optional (and sometimes harmful)
Collecting Evidence	Sources • Limited number (accuracy preferred) • Sampling strategies are critical • Concern for protecting human subjects Indicators/Measures • Quantitative • Qualitative	Sources • Multiple (triangulation preferred) • Sampling strategies are critical • Concern for protecting human subjects, organizations, and communities Indicators/Measures • Mixed methods (qualitative, quantitative, and integrated)
Analyzing & Synthesizing	Timing • Once (at the end) Scope • Focus on specific variables	Timing • Ongoing (formative and summative) Scope • Integrate all data
Making Judgments	Implicit • Attempt to remain value-free	Explicit • Examine agreement on values • State precisely whose values are used
Justifying Conclusions	Attribution • Establish time sequence • Demonstrate plausible mechanisms • Control for confounding • Replicate findings	Attribution and Contribution • Establish time sequence • Demonstrate plausible mechanisms • Account for alternative explanations • Show similar effects in similar contexts
Using New Knowledge	Disseminate to Interested Audiences • Content and format vary to maximize comprehension	Feedback to Stakeholders • Focus on intended users and uses • Build capacity Disseminate to Interested Audiences • Content and format vary to maximize comprehension • Emphasis on full disclosure • Requirement for balanced assessment

References

Baker RR. Human navigation and the sixth sense. New York, NY: Simon and Schuster. 1981. CDC. Ten great public health achievements — United States, 1900– 1999. MMWR 1999:48(12):241– 243.

CDC. Health-related quality of life: publications and reports. Atlanta, GA: Centers for Disease Control and Prevention, 2001. Accessed October 22, 2001.

Eoyang GH, Berkas T. Evaluation in a complex adaptive system. In: Lissack M, Gunz H, eds. Managing complexity in organizations. Westport, CT: Quorum Books, 1999.

Institute of Medicine. Summarizing population health: Directions for the Development and Application of Population Metrics. Washington, DC: National Academy Press, 1998.

Fisher NI. Statistical analysis of circular data. Cambridge, England: Cambridge University Press, 1993.

Hawaii Community Services Council. Ke Ala Hoku: critical indicators report 1999. Honolulu, HI: Hawaii Community Services Council, 1999.

Jammalamadaka SR, Sengupta A. Topics in circular statistics. River Edge, NJ: World Scientific Publication Company, 2001.

Kreuter MW, Lezin NA, Young LA. Evaluating community-based collaborative mechanisms: implications for practitioners. Health Promotion Practice 2000;1(1):49-63.

Murray CJL, Lopez AD, Eds. The global burden of disease. Cambridge, MA: Harvard University Press, 1996.

SenGupta A. A statistical package for the analysis of directional data. 7th International Conference of the Forum for Interdisciplinary Mathematics. Mumbai, Maharastra, India. December 19-21, 2000.

Thompson N. Reflections on voyaging and home. October 1, 2000. Accessed October 22, 2001. HTML

Syndemics Overview - What Procedures Are Available for Planning and Evaluating Initiatives to Prevent Syndemics?

References