Table of Contents

1. Day 1

1.1. Welcome and Overview of Meeting

1.2. Public Participation NVPO perspective

1.3. Study Circles

1.4. AmericaSpeaks

1.5. Deliberative Democracy Consortium

1.6. Enhancing Public Participation through Information Technology

1.7. READII Project (CDC)

1.8. AAAS Public Engagement Project

1.9. Environmental Protection Agency (Office of Water)

1.10. National Aeronautical and Space Administration

2. Day 2

2.1. Welcome

2.2. National Institute of Allergy and Infectious Diseases/NIH, Division of AIDS

2.3. National Institutes of Health

2.4. FACA Considerations

2.5. Environmental Protection Agency (Superfund)

2.6. Department of Defense

2.7. Vaccine Policy Analysis Collaborative (VPACE)

2.8. Public Comment

Members in Attendance

Members

• Ruth Katz, George Washington University School of Public Health and Health Services-Chair
• Dr. Charles M. Helms, University of Iowa Hospital and Clinics
• Mary Beth Koslap-Petraco, Suffolk County Department of Health Services

Dr. David Johnson, Aventis Pasteur

Members Absent

• Dr. Jerome O. Klein, Boston University School of Medicine
• Dr. Patricia N. Whitley-Williams, Robert Wood Johnson Medical School

Dr. William Schaffner, Vanderbilt School of Medicine

Ex Officio Members

• Dr. Bruce Gellin, National Vaccine Program Office, DHHS
• Matt Leighninger, Study Circles
• Carolyn Lukensmeyer, AmericaSpeaks
• Tonya Gonzalez, Deliberative Democracy Consortium
• Robert Carlitz, Deliberative Democracy Consortium
• Ginger Pinholster, AAAS Public Engagement Project
• Victoria Friedensen, Office of Space Science, NASA
• Leanne Nurse, Office of Water, EPA
• Tamara Kicera, READII Project, CDC
• Rona Siskind, National Institute of Allergy and Infectious Diseases, NIH
• Jaynce Hedetniemi, Office of Community Liaison, NIH
• Doug Thompson, Superfund, EPA
• Jim Murphy, Superfund, EPA
• Ellie Tonkin, Superfund, EPA
• William Pehlivanian, U.S. Army
• Emily Marcus Levine, Office of the General Counsel, DHHS
• Mary Davis Hamlin, The Keystone Center

Invited Speakers

Audience members

• Deborah Wexler, Immunization Action Coalition
• Dr. Thomas Vernon, Chiron
• Dr. Thomas Zink, GlaxoSmithKline
• Dr. Geoffrey Evans, Health Resources and Services Administration, DHHS
• Dr. Kathleen Stratton, Institute of Medicine
• Jennifer Zavolinsky, Every Child By Two
• Sarah Landry, National Vaccine Program Office
• Maguerite Willner, Health Resources and Services Administration, DHHS
• Rosemary Gunn, Information Renaissance

Day 1

Welcome and Overview of Meeting

Ruth Katz

Ms. Katz introduced herself as a member of the National Vaccine Advisory Committee (NVAC) and Chair of the Public Participation Working Group. She noted that most of the working group members were present and thanked them for attending. She asked the working group members to introduce themselves and noted that two members, Dr. Whitley-Williams and Dr. Schaffner, were unable to attend. She thanked the presenters for coming and asked them and the public attendees to introduce themselves. She then thanked Ms. Landry for her work in putting this meeting together.

About a year ago, there was increased interest in enhancing public engagement in national vaccine policy. In June 2003, the Vaccine Policy Analysis Collaborative (VPACE) was brought to the attention of NVAC as a model for involving the public in setting national vaccine policy. VPACE served as a catalyst for the formation of this working group. The purpose of this group is to learn more about enhancing public participation, which will be reported back to the full NVAC at its October meeting.

At the conclusion of this two-day meeting, the working group will prepare a summary of this meeting and make recommendations on addressing this issue. Those recommendations will include a specific recommendation regarding the potential effectiveness of the VPACE proposal. The working group will also make recommendations about other potential approaches or models. The only issue before this working group is how to better engage the public, not whether to engage the public. There is consensus that there should be better public participation. How to achieve this is the issue at hand.

There will be 45 minutes allotted for each of the presentations and at the end of the presentations there will be time for public comments. Tomorrow morning, Dr. Helms will be chairing the meeting while Ms. Katz hosts another engagement.

Public Participation-NVPO perspective

Dr. Bruce G. Gellin

Dr. Gellin reinforced the idea that this is a question of how, not if, public engagement can be enhanced-the VPACE group also went through a similar process. The working group is also interested in hearing how other sectors have tried to solve this issue. The need for more public participation grew out of concerns with vaccine safety. However, there are a broad range of policy issues beyond safety, including pandemic influenza preparedness, production problems, resistance to an antiviral medicine, vaccine supply, and vaccine financing.

When discussing the pandemic influenza preparedness plan in DHHS, it became clear that this would be a broad-reaching problem. Should there be a pandemic, the implications for society are so broad that public engagement became essential. The draft plan has been posted on the website on August 26 for a 60-day public comment period. This meeting is, in part, an attempt enhance that public participation process as they move from the draft plan to the next stage.

Study Circles

Matt Leighninger

Mr. Leighninger noted that he began working with citizen involvement 11 years ago at the Study Circle Resource Center. The center was founded by Paul Edgar, who made a fortune in the electronics industry and went into philanthropy. Mr. Edgar started the center in a farmhouse next to his home. He was a Unitarian and joked that when Unitarians die, they have a choice between heaven and a discussion of heaven. The mission of the center was to get people in communities to talk about issues in small groups. Mr. Leighninger worked there for seven years and continues work with them as a senior associate. He also works with the League of Women Voters, Neighborhood Reinvestment Corporation, and the National League of Cities.

Over the last ten years, there has been a big shift away from traditional forms of citizen involvement. There are several reasons for this shift. First, particularly at the local level in the 80s and 90s, there was a mandate for dialogue among citizens on race-this was a clear issue for communities such as Los Angeles. Second, there was increasing acrimony surrounding policy decisions from the neighborhood level to the federal level. This, in part, might be due to the increasing ease of becoming an instant activist on any issue. Third, public resources were insufficient for solving public problems, such as education and crime prevention. Therefore, there was a need and desire to tap into citizen potential and organizations to work with government. Fourth, there were growing frustrations with traditional community organizing. In the past, community-organizing efforts were conflict based. Now, community organizers are using different tactics to engage the public.

Over the last 15 years, there has been a proliferation of citizen involvement or "democratic organizing"' efforts. These new efforts have the following key characteristics. 1) They are recruiting a diverse critical mass of residents. 2) They are using both small and large-group meetings. Small meetings provide intimacy and learning capacity, while the large meetings help them to see the potential of their efforts and have their conclusions amplified to decision makers. 3) They are giving people a chance to compare personal experiences and consider a range of views. 4) They are effecting change in learning, relationship-building, policy input, and tangible action.

There have been more than 250 communities, three states, and two national efforts that have conducted democratic organizing. Successful small groups (sometimes called study circles) share the following characteristics. 1) They include eight to twelve people and meet for several sessions-the more time they spend together, the better. 2) They are facilitated by someone who can remain impartial. This person does not have to be a trained professional, but must remain impartial. 3) They are open to many different views. 4) They include a diverse mix of people, including race, ethnicity, age, political affiliation, etc. 5) They follow a sequence from personal experience to larger questions to action planning.

Successful large-group meetings (sometimes called action forums) share the following characteristics. 1) They summarize the conclusions reached in the small groups. 2) They are used to launch task forces or committees on action ideas, or connect people with existing action opportunities. 3) They allow public officials to hear and add to the conclusions reached in the circles. 4) They provide participants a chance to socialize, network, and compare notes.

It is a misconception that these efforts are only about talking. There is a lot of discussion, but they are also about action at various levels. These include individual volunteerism, small-group action efforts, changes made by organizations, and policy changes. As an example of volunteerism, a participant in a Kansas City project to reconnect schools with their communities was quite skeptical of the process, but ended up volunteering to set up an after-school program around dance and the arts. For small-group action efforts, there was a project on police/community relations in Buffalo. One of the neighborhoods had an issue between halfway house residents and business owners. As a result, a mental health response team was established. For changes made by organizations, there was a discussion about waste disposal and landfills in a county in Kansas. As a result of that project, people concluded that recycling should become a bigger priority for the county. For policy changes, the League of Women Voters and others met to discuss the penal justice and correction system in Oklahoma. As a result, a state bill was passed almost unanimously, though similar bills were previously rejected along partisan lines.

There are several new assumptions about public involvement in these new efforts. One is that solving public problems requires the whole community. It is something that government cannot do alone. This results in a potential for greater problem-solving capacity. However, there is a greater need for coordination and at times confusion over accountability. It also illuminates the need for regional collaboration.

A second assumption is that proactive, network-based recruitment is critical. It is not enough to call a meeting and hope that people show up. By reaching out and being proactive, you can reach people who have not been involved. However, it raises the question of who should provide staff time for organizing. It reveals different outreach capacities of different groups and forces recruiters to think about "what's in it for citizens."

A third assumption is that putting policy decisions in citizen context helps the discussions. This creates a potential for more broadly supported policies, with people helping the government to implement the policy. However, this can be more difficult to address technical or abstract issues and requires different way of writing about issues.

A fourth assumption is that citizens can be expected to learn, empathize, and change opinions. When people are engaged, especially in small groups, they often do change their minds. However, there is a challenge of bringing together people on different sides of an issue and a danger for officials in suggesting that citizen opinions should change.

A final assumption is that there is a larger sense of civic duty, including dialogue, learning, and action. There is a potential for more people to take leadership roles. It requires a need to provide more opportunities for people to exercise active citizenship. However, there is a danger of division between "active citizens" and others.

There are several important planning questions when implementing democratic organizing efforts. These include:

• What is the issue you want citizens to address and how should you describe it?
• Who are you trying to involve and why would they want to participate?
• How will you provide participants with the information they need?
• Who are your key partners?
• How will you staff this project?
• How should you get started?
• What do you hope to achieve?

Regarding the work on vaccines, the best projects in public involvement include stakeholder collaboration, as well as public involvement. The first step in this type of project is building relationships with organizations that can help with recruitment. This relationship will also help show that the project is inclusive of all different or conflicting perspectives. This is exemplified in the VPACE proposal, which includes stakeholder participation and broader citizen involvement. Additionally, the issue of the flu is well suited for a national citizen involvement effort because it is something by which everyone is affected.

Questions

Ms. Koslap-Petraco asked about the mechanics of getting people involved who do not feel that their opinion counts. There is a need for broad spectrum of people. Those that are adamant about a particular issue are usually the ones that get involved. However, there is a need to hear from those that usually do not speak up.

Mr. Leighninger responded that the two key pieces are message and recruitment. The message must be clear that people will have the opportunity to make a difference and that the effort is not based on the agenda of only one group. On the recruitment side, it requires the work of reaching out to organizations to recruit within their memberships. Advertising is sometimes appropriate, but relationship building is by far the most effective technique.

Dr. Helms asked if it was critical to have agreement among stakeholders before the public is involved.

Mr. Leighninger noted that there does not have to be agreement on policy, only an agreement that this issue requires broader public input.

Ms. Landry asked about examples of when these efforts have not worked.

Mr. Leighninger responded that there are many examples. Many of these efforts fail because they do not have the right staffing, lack funding, or when facilitators cannot remain impartial.

Dr. Johnson asked about how these efforts could be translated to national policy or recommendations.

Mr. Leighninger noted that there are many options. One example is the efforts of the Canadian government. It involved a thousand small groups on immigration policies and was treated as an input gathering exercise. It provided materials and instructions to anyone who wanted to conduct a small group meeting, and the process was influential on policy change. Another idea is the concept of working with paired organizations across the country. There would be national organizations as part of a national coalition and coordinators who would work with different chapters. A third approach is to use two or three communities to get some representative policy input.

Ms. Wexler noted that stakeholder organizations tend to be those that have an issue or concern. She asked about how to engage the larger public that does not have a concern.

Mr. Leighninger responded that it would not be much different from the way citizens are engaged. You have to explain why the issue is critical and how the effort will allow their constituents to have some impact. In addition, there are a number of broad national organizations that have an interest in this kind of work and would be willing to participate.

Ms. Katz asked how the questions to be discussed are determined.

Mr. Leighninger answered that a committee of people with diverse viewpoints would frame the issue. After they figure out what the questions are, they would test the questions to make sure that they are not missing anything.

Ms. Katz responded that there is a separate group that looks at framework, rather than the group itself.

Mr. Leighninger added that his projects have all had the questions and issues set up before recruiting discussion group members. Without this level of specificity, it would be difficult to form a group.

Ms. Katz asked about the difference in small groups where the goal is to change minds versus the goal of building a consensus among those who will never change their opinion. Is this a matter of conflict resolution?

Mr. Leighninger noted that conflict resolution is different from citizen participation. There is some overlap, but conflict resolution requires more from the facilitator, who is skilled in that type of work. In most public participation processes, there is the hope that people from different viewpoints will come to some common ground.

Ms. Katz added that one of the keys to public participation is having people who are more open-minded.

AmericaSpeaks

Citizens Role in Policy Development

Dr. Carolyn Lukensmeyer

Dr. Lukensmeyer noted that one of the most challenging social issues of our time is the disconnect between decision-makers and citizens. This is especially critical in government since the foundation for American democracy was built on the idea of "of the people, by the people, for the people." The core question becomes how to put people back into their rightful role in a democracy in terms of influencing policy. The question of "how" to engage the public, rather than "whether" to engage the public, is a unique perspective. There are many in the country that still consider whether it should happen. All other countries plan and budget for public participation.

From 1987 to 1991, Dr. Lukensmeyer served as Chief of Staff to the governor of Ohio. The governor was concerned that they had passed many laws that match his platform, but they were unable to implement and change the bureaucracy. In that experience, she gained real insight into the frustration from the politician and citizen sides. Most of the rhetoric focuses on the gap between the government and the public in terms of a crisis of apathy-that citizens would not take the time and are not informed. However, the fundamental issue is that we have a crisis of alienation and disempowerment-people have real reasons for not voting.

In 1995, she spent six months traveling around the country listening to citizens who organized themselves to push initiatives that made a change. She discovered what motivated them, what they cared about, and how they engaged decision-makers. She learned two critical lessons. For the last two decades, the mantra has been that the government should be more like business. Though there have been benefits from this approach, this has left citizens feeling disempowered. A book published last year, Downsizing Democracy, talks about how public messages have reduced people's sense of agency. The second lesson was that citizens are extremely sophisticated about the role of special interest groups. There is a difference between the general interest public and the special interest public. In terms of recruitment, the formula is to get 60 percent general interest public and 40 percent representation of special interest.

The first time the 21^st Century Town Meeting model was used was on a dialogue on social security. Social security was a good example of how entrenched special interest groups were when it comes to privatization, benefit reductions, and tax policies. As a result, they created a dual track strategy. They worked with all of the stakeholder interest groups and created an advisory board made up of top leaders from various stakeholder groups. First, the board worked on advising on a participant discussion guide. This allowed all stakeholders to put forth their positions. After months of debate, all members of the board approved the guide and called it fair and equitable.

Second, the public felt closed out of the process by experts. When they would attend public hearings, they often felt demeaned, disrespected, and made to look foolish by what they did not know. In Western culture, experts are not expected to translate their knowledge for the general public. As a result, public processes must be designed to include expert knowledge in a way that does not disempower the public. It must also allow the public to understand that good public policy requires the collective knowledge and wisdom of everyday people.

For example, Ohio's governor tried to radically reform education. To do this, he never met with experts first. Instead, he met with those who were most impacted. The first meetings were in eight cities in Ohio with populations more than 100,000. There he listened to kids in grade school, elementary school, and high school, and their parents. From these meetings, he learned that the most significant issue was safety. In 1985, safety would not have been on the agenda of any education expert.

For decision-makers, there is an issue of timing, which has been exaggerated since the 1980s. For example, politicians were expected to have a full policy position on terrorism immediately after the World Trade Centers fell. It is absurd where we have taken the role of media to demand politicians immediately to take a position on issues. Conversely, corporate executives are rewarded for changing their business strategies. Yet, politicians are rejected for changing their positions.

The mission of AmericaSpeaks is to engage citizens in the most important decisions that affect their lives. This process has been used at the national level with social security and a new project on youth obesity is about to begin. They have also used this process with U.S. mayors in setting budget priorities in their communities. They have done regional planning and helped in the decision-making process in the rebuilding of the World Trade Center site.

She presented a picture of the Jacob Jarvis Center in New York City, the site of the World Trade Center town meetings. These meetings had to be large enough to have the credibility that it actually represents all of the constituents of New York. In the picture, there are 500 round tables of ten to twelve people, where intimacy is gained at the table itself. Each table was diverse, in terms of gender, race, age, and class.

As a side note, she added that the economy of all major cities is dependent on undocumented workers. The low-end jobs at the World Trade Center were almost entirely held by undocumented workers-many of them died. AmericaSpeaks joined with an existing effort to give employers amnesty so that these workers could be identified. This allowed the meetings to have credibility with organizations that worked with these undocumented workers. As a result, 448 of the 5,000 meeting participants were undocumented workers.

For a successful meeting, it must be an informed dialogue. For the World Trade Center meetings, there were maps, conceptual designs of the architecture, and budget spreadsheets. Additionally, there is a tremendous amount of human support in the process. For example, when people came to discuss social security, they stated that what they really cared about were constituency issues. As a result, they had actual social security counselors on site to help the participants resolve their problems.

To go from intimate table discussions to a collective view of thousands of people simultaneously, they used standard information technology that has been used in private companies for large business meetings. Every table had a wireless laptop computer for citizens to collect input. The discussions were about 45 minutes, with 20 minutes being the shortest and one hour as the longest. One of the core topics dealt with was who the memorial was for-victims, families, New York, world, etc.

While the tables are collecting data on their laptops, those data go in real-time to a bank of computers managed by Theme Teams. The data are then placed into categories by themes, which also include minority reports. Once the data are collected into themes, they are projected onto large screens throughout the hall. Each theme included various amounts of options (data). These options were then voted on and prioritized using polling keypads, which were provided to each participant. The keypads were numbered and tied to the demographics of the voter.

It was important to the Port Authority to know how family members felt, so they agreed that family members would be over represented. Each table had one family member of a victim. So that that person did not have to carry the emotional burden of the table, there were at least two other persons who were hugely impacted by 9/11-emergency workers, people who lived near the site, etc.

After each day, preliminary reports were copied on site and given to each participant (decision-makers, media, and citizens). The public read and discussed these reports enthusiastically and with a high sense of empowerment. They took the reports to their churches, businesses, or rotary clubs to share the information. These reports also increased the sense of agency in the citizens.

There are four critical success factors. First, every voice must be at the table-all must be represented. Second, decision-makers must be involved at each step and must influence (not control) the process from the very beginning. If decision-makers only see the finished results, they have no choice but to treat it like a special interest group. Third, the content, structure, and process must be correct. Fourth, there must be outcomes that make a difference.

There were five fundamental lessons learned.

Scale transforms the potential for impact and enables people to experience the community as a whole. Small discussion groups are important, but they do not show the experience of the whole community.

Through active and creative recruitment, it is possible to reflect the full diversity of the community. The myth of "hard-to-reach" groups is untrue and results from using the wrong medium. Posting notices and radio ads are not always the correct communication vehicles. Outreach strategies must be specifically targeted to every relevant demographic.

Transparency is critical to people's sense of ownership of and confidence in outcomes.

Citizens are capable of dealing with complex information and making trade-off decisions.

The quality of citizen recommended outcomes are only as good as the quality of the design of the materials and the process.

Questions

Mr. Johnson noted that the examples seem to be one-time/one-issue processes for input. He asked whether those that require a process over a long period of time were doable under this mechanism.

Ms. Lukensmeyer replied affirmatively. None of their projects were done just once. Social security is an example of a long-term process. It was done over 18 months. Additionally, the mayor of Washington, D.C. has been using this process to set the city's budget since 1999.

Ms. Wexler asked how much time was involved for the participants and how they were able to get off work. She also asked who pays for travel expenses.

Ms. Lukensmeyer noted that, as a principle, no one is paid to participate. She added that events are primarily held on Saturdays or Sundays, while accommodations can be made based on demographics. They did pay for transportation, fed people well, and listened to what the community said they needed to participate.

Ms. Koslap-Petraco asked how the money was raised.

Ms. Lukensmeyer noted that each project was a mix of government, foundation, and private funds. There was one exception, social security. The Pew Charitable Trust wanted to be a source of nonbiased, nonpartisan information about social security, so they paid for the entire project. She added that the government in Canada and certain European countries have a budget for public consultation.

Dr. Helms asked who the decision-makers are in national immunization issues-Congress, the public health service, etc.

Ms. Lukensmeyer noted that if decisions could be made by the agencies, it could be treated administratively. If legislation is involved, as in social security, both houses of Congress and the White House needed to be involved.

Ms. Katz noted that in their social security project, Congress ended up doing nothing. In light of the public's cynicism, would they be even more cynical since nothing happened.

Ms. Lukensmeyer added that they did not have the funds to track those people. However, she did not feel that citizens necessarily become more cynical. During the social security meetings, they allowed materials and people in the lobby from all advocacy groups. This allowed participants to know and communicate with the various advocacy groups. For example, a participant, who changed his views to favor some government control over privatization, linked up with the Kato Institute to try to influence their actions. Additionally, the project website continues to get 20,000 visitors that keep the discussion current. If you link people to something at the local level that allows them to continue forward, it is possible that the experience did not raise people's cynicism.

Ms. Katz asked if the complexity of the issues (e.g., transportation versus vaccines) matters to the success of these projects.

Ms. Lukensmeyer offered an example. In Denmark, the Danish Technology Board is responsible for doing a number of public engagement processes on a range of issues. One of these engagements dealt with biogenetic engineering. At these meetings, the outcomes of what the citizens say move directly to the debate in Parliament. In summary, the public can be educated on anything.

Ms. Katz noted that Oregon took the issue of Medicaid to the voters directly. She asked whether this same process should be used for vaccines.

Ms. Lukensmeyer noted that she is a proponent of representative democracy. She added that referendums have been distorted by big money. For example, she noted that each presidential election cycle has its top issue. In 1992, healthcare was the top issue and there was a mandate by the Americans for a national solution. This issue could have been taken out of the partisan legislative process and put into public dialogue. The results could have been moved directly to the House floor, bypassing the committee structure.

Dr. Zink asked what the costs are.

Ms. Lukensmeyer replied that it ranges from $70 to $400 per person (New York).

Mr. Johnson noted that there are a number of fundamental questions in immunizations. There are also questions on individual vaccines and vaccine interventions. This process can be used for larger questions, but could the process be applied to short-term issues where public input is needed very rapidly?

Ms. Lukensmeyer noted that the Department of Agriculture has a service called the USDA Extension Service, which has two-way satellite communication capabilities in every state. This is already funded by the U.S. government and could allow for a rapid turnaround. There are two issues that would most determine the length of the process. One is the development of materials. The second issue is outreach-getting people in the room. Though it drives up costs, an alternative is using a scientific random sample.

Ms. Wexler noted that undocumented workers live in crisis and that it is difficult for them to prioritize a discussion on vaccines. She asked how you could get people who do not have time in the first place to get involved in an issue that may not be very important to them. She noted that many people do not take the time to even vote.

Ms. Lukensmeyer invited Ms. Wexler to see the process and noted that this was the best way to understand how it is done. In response to the idea that people do not vote, she added that she learned that people do not vote because they do not believe that it is a fair system. She added that people would be more inclined to discuss vaccines than to vote. Once they go through several of these discussions where they see how they have made a difference, then they might begin to feel that the system is more fair and vote.

Deliberative Democracy Consortium

Tonya Gonzalez

Ms. Gonzalez introduced herself as the Director of the Deliberative Democracy Consortium (DDC), which was started two years ago by AmericaSpeaks. The idea of DDC from the beginning was that practitioners and researchers in the field of deliberative democracy thought they could do more together than they could alone. Ms. Lukensmeyer is the chair of their steering committee, which is comprised of representatives from most of the major public participation organizations, both in the United States and abroad.

The nascent movement for deliberative democracy in the United States and around the world served as the impetus for starting DDC. Deliberative democracy, which is about engaging citizens and giving them a voice in decision-making, is the cornerstone of our democracy. Outside of the United States, deliberative democracy has a long history-from the Danish Board of Technology to, more recently, the World Social Forum in Brazil.

DDC is led by a steering committee, which addresses critical areas of work through task groups-National Infrastructure, Knowledge Building, Link to Government, and Online Deliberation. Dr. Carlitz is a member of the Online Deliberation Task Group. These task groups work on questions, such as:

• What methods work best when?
• How do you carry a one-time event over a long process?
• How do you create a more deliberative public?
• In what are we engaging people?
• How do you educate people about complex issues and is there a dialogue that needs to happen first?
• Social movements in this country have been determined by advocates' views, so where does this work fall into deliberative democracy?
• If you have a group of parents or a community that thinks something else should be done about vaccine policy, how do you get them to come to the larger consensus?
• What are the costs involved?
• What are the carry-on effects?
• What mix of methods should be looked at?

Questions

Ms. Katz inquired whether a group should go to DDC to ask about the best method for a particular issue, rather than just go directly to an organization such as AmericaSpeaks.

Ms. Gonzalez responded that a group with such questions could go directly to AmericaSpeaks, but DDC would be a resource that offered a different level of understanding.

Dr. Zink asked who could join DDC.

Ms. Gonzalez responded that anyone could join. They have members from CDC and EPA. Other members include Senators, researchers and practitioners of the field, people who come from related practices (e.g., conflict resolution), and those who have participated in some type of deliberative forum.

Enhancing Public Participation through Information Technology

Dr. Robert Carlitz

Dr. Carlitz stated that he is part of a non-profit organization called Information Renaissance (IR). IR was founded in 1996 and grew out of several infrastructure projects to connect schools, libraries, and community centers to the Internet. It evolved to using that online infrastructure as a mechanism to facilitate citizen participation in government. IR views itself as the intersection of public need, governmental resources, and the technology tools that can merge these resources together. Rule making is a primary interest of IR, for it has enormous potential for citizen involvement. The notice and comment process invites input from many interested parties, which is strongly weighted by the agencies as they make their rules.

Traditionally, it was difficult for citizens to participate in rule making because a paper-based system required the participants to be physically located where the information was. However, the process is moving online, with 24-hour access. This shift has the potential of affecting a major change in how things work. Currently, online rule making is largely implemented out of the CIO office of most agencies

The potential reach of online events is enormous. Online events are simultaneously nationwide and do not require a series of hearings around the country. Additionally, online mechanisms allow for comprehensive resources to be built overtime, and materials from hearings and research can all be stored in one place.

A key issue in public participation is how managing information and authoritative sources. Authoritative records, which are meant to be accessible by the public, should be updated in an online repository. This achieves a high degree of transparency because the materials are in their entirety, rather than edited down versions, such as press releases.

Online resources are available on demand from anywhere in the country-participants do not have to travel to any particular site. Additionally, the online environment is also a good place for background and educational materials. As a result, it is possible to construct an online resource around a particular issue, which not only shows the proceedings of a particular committee, but also provides background materials. The net effect of online processes is a deepening of the level of public participation. In addition to people being more aware, they participate in a more intelligent and more informed manner.

Online techniques can be applied anytime the process involves public input, such as proposed legislation, planning for local construction projects, issues scoping within an agency, proceedings of Federal Advisory Committees, rule making, and the implementation specific legislation.

The preparations for an online event are very similar to an in-person event. People need to be notified and recruited to achieve balance in terms of demographics. In the participatory planning phase, the agenda is developed and the panelists and participants are determined. As people sign up for the event, the online mechanism can capture biographical information. There can be surveys to map out the territory and get a summary of attitudes, which can inform modifications to the agenda and set or adjust priorities.

Online mechanisms are a combination of the informal and anonymous. People who are normally afraid of speaking at a public meeting would be willing to speak in this more anonymous online forum. Additionally, agency members who appear remote and inaccessible in hearings can be made more personable.

Online events allow for participant introductions. They can provide biographies and collect demographics. They also allow for participant feedback. Basic resources can also be available, such as surveys and a comprehensive library so that participants have access to a complete set of information. Through this, it should be possible for citizens to have the same information that the experts do to analyze the issue. Additionally, online events allow for participant feedback, which can help set priorities and agendas.

Online events are self-documenting. Materials put online stay online, unless an effort is made to remove them. As materials accumulate, they can be indexed and searched. This creates an important archival component that allows non-participants to learn about the process.

Deeper online participation can help deepen public participation. "Websites" or "online resources" become collaborative workspaces. People actively participate in the process and are not just reading what is there. They provide information and input. They can determine or influence the agenda and help set priorities, which is accomplished through a voting mechanism. These processes get people more interested in the activity and allow them to feel that they have a voice. There can be on-going monitoring of results either by the group as a whole or various subgroups.

Another technique is notification. A problem with websites is that they can rapidly accumulated a large amount of material, making it difficult for participants to keep up with the information. A solution is a technology called RSS feeds, used by newspaper sites to collect headlines of articles. RSS feeds can be sorted by keywords, resulting in a customized newspaper with new items in various areas of interest show up.

Online dialogue involves pulling together a group to talk online about a particular subject. These events typically involve 1,000 people and have lasted two or three weeks in length, which has proven to be a convenient time period. They tend to be intense and involved, and have generated an enormous amount of interest. When staff support has been available to digest the data from these events, it has been extremely useful for the agencies involved. Online dialogues involve the following elements:

• Moderator-to help organize the discussion
• Threads-organizing the discussion around particular topics
• Discussion summaries-daily summaries posted back to the group
• Tracking-notification if someone responds to a posting
• Archive-information resource
• Panelists-to launch particular topic areas

There are problems with online events. 1) People without network access cannot participate. Possible solutions to access include public access sites, community sites, libraries, and government offices. There is also the evolution of technology in telephone access, interactive television, and other mechanisms becoming more broadly available. Though these solutions are creating greater access, online access is presently a real problem. 2) Reaching certain populations that are not showing up at these events require particular effort. 3) Literacy is a problem universally for public participation. If people are not articulate, afraid to stand up in front of a group, or lack the vocabulary to address the issues at hand, it is going to be hard to get them to participate. 4) There is also the issue of comfort levels with technology.

There are also benefits. 1) Young people tend to be more comfortable with computers than standing up and giving a speech at a public hearing. 2) Rural populations, young parents, and people unable to travel will be able to participate online. 3) People who cannot participate during the workday will also be able to join the online discussion because it is available 24 hours a day. Most of what is done online is asynchronous, meaning immediate participation is not necessary-a message is posted and a reply can be posted later. 4) People with disabilities who are comfortable with computers have easy access to this type of forum. 5) The background materials and library of resources available online allow interested participants, who do not know a lot about the subject, to learn more and become informed participants.

There are several concerns with online events. 1) The sponsoring group needs resources and expertise to host an online event. Merely announcing an online event is not enough, often resulting in off-topic comments or no comments. Online events require more preparation than a public hearing because they are nationwide. 2) Legal constraints to online events are beginning to be considered and are showing up in law journals and literature. These constraints are problematic because they are new and often interpreted differently from one agency to another. These interpretations are mostly based on comfort level with the technology. On one side, online events are just a different vehicle for public participation. On the other, these events are revolutionary and should be approach with caution. 3) In collecting data, there are restrictions in terms of privacy issues and paperwork reduction issues. The EPA's general counsel, for example, believes that it is an invasion of people's privacy if their names shows up in an author index if they have submitted a comment online. A search capability allowed contributions to be found, but author indices were suppressed. Another question involves whether the actions of a moderator trying to keep the discussion on topic are violating free speech rights of the participant. Similarly, restrictions on prolific commenters is problematic.

There are advantages to face-to-face events. They can be more personal and allow for facial and body language to be seen. However, online events allow for more detailed information and structure. Face-to-face and online events can also work together. Online mechanisms can be used to announce face-to-face meetings, which can be tracked online. For example, face-to-face meetings can be broadcast over the web at low costs, called webcasts. This can include video clips and archives of previous meetings.

Generally, online discussions can be more open-ended than face-to-face meetings because they can stretch over a longer period of time. As a result, online resources can serve as a comprehensive, long-term coordinating structure for all public participation events and all of the information associated with these events.

Questions

Ms. Katz asked if Dr. Carlitz had been involved in any work where the same question was looked at online and in face-to-face meetings. She asked if the results were compared for differences in the response.

Dr. Carlitz responded that one of the early events was done with Ms. Lukensmeyer on social security reform. There had been a number of face-to-face meetings, accompanied by an online discussion. The online discussion tended to be more in-depth and covered a longer period of time. Congressional staffers who participated were interviewed after the event for their opinions of the process. They noted that in their day-to-day jobs, they tended to produce sound bites for their bosses. However, they found members of the public to be interested in more detail and enjoyed sharing their research and expertise by answering questions in detail. Some staffers put an enormous effort into the event.

Ms. Katz inquired whether the issues brought up online and at the in-person meeting were similar or different.

Dr. Carlitz replied that they were similar, partly due to the fact that each group was exposed to similar background information.

Dr. Johnson posed the question of how people are recruited to participate online and if they self-select.

Dr. Carlitz responded that they are self-selected. They tend to advertise through groups that already have a presence online. They post announcements and ask people who are interested to tell their friends about it. If the groups are well organized, the announcement will be success. In the social security example, the online audience was skewed toward Libertarians because they were very active in talking about this topic online, and they were excited about talking to people who were writing legislation on this topic. A flaw in online mechanisms is that the demographics of the participants lean towards the demographics of the online community. Surveys on recruiting mechanisms show that online mechanisms worked best.

Ms. Landry asked if there is a criteria on selecting online library materials?

Dr. Carlitz explained that they try to work with bi-partisan forums in selecting the materials. In the social security discussion, there was a White House conference on social security that had bi-partisan sponsorship. The groups who were invited to participate in that meeting were asked to submit a one or two-page statement of their outlook. Their statements, showing all sides of the various issues, became key materials in the online library. Groups that were not involved in the White House conference were also allowed to post their statements. It is important not to be perceived as being biased towards a particular position.

Ms. Landry inquired if there were criticism from people who thought they were not giving enough opportunities.

Dr. Carlitz responded that during the social security event, someone posted a message stating that the event was a way for Prudential to push its agenda on privatizing social security since part of the funding came from Prudential. There was a response by a Libertarian who said that the participants were being fooled into thinking that the forum was actually open and inclusive. There was criticism, but in general, the participants believed in the process.

Ms. Wexler posed the next question on whether there is usually a specific issue asked to be addressed online?

Dr. Carlitz responded that with the FCC rulemaking event on connecting schools and libraries to the Internet, there was a preamble that posed a number of questions. They structured the events so that each topic had equal time, and the discussions were given specific topics to cover-it is ideal to have a focus and an outcome. The social security discussion was weak because it did not end with a policy change. Rulemakings are nice, because after the 60-day period is up, a rule is issued. People will participate when they know there will be an action.

Ms. Koslap-Petraco asked how participants are recruited. She works with an underserved population. She cannot imagine her Hispanic patients getting involved in something online, which would result in an enormous segment of the population being left unheard.

Dr. Carlitz agreed.

Ms. Koslap-Petraco asked if this would be an adjunct to face-to-face meetings.

Dr. Carlitz noted that online events would be an adjunct to face-to-face meetings. One thing that motivates people to participate online is the opportunity for their voices to be heard worldwide. Audiences that are too dispersed or cannot access the online medium can participate through intermediaries, using small in-person meetings.

Ms. Koslap-Petraco wanted more information on how to get the word out to isolated and dispersed populations.

Dr. Carlitz noted that online events use the same techniques as other events. However, the most successful are online advertisements and mechanisms. It is also important to reach out to groups that represent those that are isolated and dispersed. He added that they have done a lot of work in low-income areas, which has a lot of interest. Opportunities to participate can be provided at library and community sites.

Ms. Gonzalez commented that complex issues require longer citizen engagement, rather than single events. The social security event had mixed components in the discussion, and the best approach may be a combination of methods to get the level of recruitment for which you are interested. There is a learning curve and as online events go on and people begin to realize that they do want to have input, recruitment may look very different two years from now.

Ms. Koslap-Petraco suggested doing some of the recruitment in languages other than English.

Ms. Gonzalez added that AmericaSpeaks and Study Circles do ongoing work with various populations, and materials for Study Circles for various events have been translated into Spanish.

Ms. Koslap-Petraco commented that she did not want to see Hispanics left behind.

Dr. Carlitz responded that even with the expense of translation, it has to be done.

READII Project (CDC)

Tamara J. Kicera

Ms. Kicera noted that READII stands for Racial and Ethnic Adult Disparities in Immunization Initiative. She noted that the READII demonstration project is somewhat different from the earlier presentations with respect to soliciting public input-there is a difference between getting assistance with implementation and policy or decision-making. For the most part, READII projects engage partners to implement intervention strategies that have already been shown to be effective.

Ms. Kicera presented an overview of the READII project. She indicated that she would not be presenting much data or background information during this presentation since she is scheduled to do so at the October NVAC meeting. Instead, she would be focusing on experiences with the Rochester or Monroe County, New York project. She noted that Dr. Nana Bennett and Christine Long, the leads on this project, were unavailable to attend this meeting.

One of the key messages from the Rochester experience was that starting or funding something new is not always necessary. Existing community groups, stakeholders, or infrastructure could perhaps be enhanced or expanded rather than starting from scratch.

The READII project genesis began in Spring 2002 when Dr. Walter Orenstein (NIP/CDC director at the time) made a presentation in the DHHS Secretary's office about the need to take a closer look at the racial and ethnic disparities among older adults, particularly with respect to the influenza and pneumococcal vaccinations. DHHS was supportive of this initiative since eliminating racial and ethnic disparities has, and continues to be, a top priority. On July 31, 2002, DHHS announced that the project would move forward. They immediately moved to engage other federal agency partners, including CMS, HRSA, AoA, and AHRQ. They then awarded the funds to the five demonstration sites late that year. After obligating the funds, they quickly moved to identify stakeholders and other community partners, and develop community plans to begin the demonstration projects.

The immunization programs being funded had little or no adult infrastructure in place, unlike the longstanding, well-supported childhood immunization programs. Unfortunately, many immunization and health department personnel were asked to respond to the smallpox vaccination programs being implemented at the same time, and with the influenza season that had already begun, a tremendous burden was placed on these personnel.

Ms. Kicera then discussed what is currently happening with the project. She noted that sites and agencies formed new, important partnerships and have actively engaged their communities. They are better understanding infrastructure gaps and the difference between childhood and adult programs.

The five READII sites are Chicago, Milwaukee, San Antonio, Rochester, and the Delta Region (19 counties in Mississippi). The sites have been encouraged to employ evidence-based interventions, most of which are taken from the Community Guide for Preventive Services. Some of the more effective ways for increasing adult immunization levels include standing orders and provider recommendations. Formative research is generating ideas for targeted health education messages to patients and providers, including small-scale media campaigns.

Ms. Kicera showed a slide of Dr. Julie Gerberding (CDC Director) holding one of the provider tool kits that was developed as part of the READII project. The tool kit was a set of materials intended to help providers understand the importance of their vaccine recommendations, particularly with respect to elderly African-Americans and Hispanics. She added that the READII project activities have been widely publicized, particularly in READII project site communities.

Ms. Kicera discussed what has been learned. First, new challenges are faced every year. Challenges in the first year included smallpox efforts, SARS, and monkey pox. Another challenge was the early flu season last year, in which there were highly publicized pediatric deaths resulting from a vaccine shortage. She noted that duplicating childhood successes for adults may not work and that new methods, interventions, and partnerships might be necessary. She added that they are trying to form partnerships with agencies such as AoA since many of the existing partnerships target preventive services for children rather than adults.

Ms. Kicera noted that negative attitudes among health care providers, especially nursing staff, can negatively affect the success of even a standing orders policy. In Rochester and Chicago, many private practices do not have the traditional structure of providers with many registered nurses (RNs). Because of costs, nursing shortages, and other reasons, many practices operate without any RNs. Instead, they operate with nurse's aides and LPNs, who do not necessarily have the education level or attitudes about the importance of promoting vaccinations. Even in places with standing orders, staff with doubts or misconceptions about vaccines move away from recommending vaccination to high-risk patients.

Ms. Kicera emphasized that changes of this magnitude take time and they have not done very well with the general public with respect to influenza and pneumococcal vaccinations levels and are doing worse with minority populations. To effect such change cannot be done in a two-year project, but would require eight to ten years of targeted intervention to see a dramatic change.

A mid-course review meeting took place in March 2004 with the five grantees, the federal partner agencies, and several senior officials from the Deputy Secretary's office and other DHHS agencies. The purpose of the meeting was to review and discuss successes and challenges thus far in the project; to allow discussion of issues and challenges with CDC, partners, and invited panelists; and to share site-specific project information with senior officials from DHHS and other partners.

The demonstration projects are scheduled to end December 31, 2004, followed by aggressive evaluation activities. The experiences with the 2003-04 influenza season present opportunities. In response to the media frenzy about being immunized, facilities in Rochester were flooded by white, middle-class, or suburban populations, while minority groups did not have the same reaction. Determining interventions that will be most effective with black and Hispanic elderly populations is a challenge. CDC has been charged with creating an influenza vaccine stockpile and hopefully the demand will be there. Experiences in the five sites will be valuable to future programmatic and research efforts. Some NVPO funds have already been used to launch studies about attitudes, beliefs, and misconceptions of nurses and other healthcare professionals with respect to vaccinations. The findings will be disseminated with the hope that successful efforts will be replicated.

Ms. Kicera then discussed Rochester's Community Action Plan using slides presented by Christine Long in Atlanta. This was a collaboration of the University of Rochester and the Monroe County Department of Public Health. The Community Action Plan was developed at the front end of the project and included the following components: a community advisory board; community-based organizations; a communications plan; several targeted projects; and eight or nine private vaccine provider groups, which have shown very promising results. The primary care practice intervention had other components, including a communications strategy, providing public vaccine, targeted projects, and community-based organizations.

The Community Advisory Board engaged groups such as the Urban League of Rochester, economic development groups, senior citizen organizations, and a coalition of African-American churches. The people in Rochester felt it was critical to address these groups at the front end of the project. She noted the importance of involving decision-making groups at the front end, rather than developing a report to present to them at the back end of the project.

Ms. Kicera noted that Rochester, Milwaukee, and Mississippi targeted black, elderly populations. San Antonio targeted the Hispanic elderly, and Chicago targeted both blacks and Hispanics.

In Rochester, the goal of the Community Action Plan was to promote the messages that vaccines are important and necessary, and that disparities do exist in the community. Another goal was to provide direction on where and how to get vaccines. She also noted that none of the five sites were overtly trying to solicit input from communities about policy or the recommendations or interventions being promoted. The recommendations and interventions came from the Community Guide for Preventive Medicine or ACIP. The demonstration sites were asked to engage their community stakeholders to help promote these messages.

The groups did not want new projects funded or new groups brought in because they felt there was a wealth of resources within the community that could be worked with or expanded upon. They found that one-to-one or small group formats worked best and that there was a need for patience because these types of changes take time. Most groups that were members of the Community Advisory Board volunteered their organizations through in-kind contributions. They volunteered to be speakers with the media or church groups.

The Community-Based Organizations (CBOs) were, for the most part, engaged in outreach and education for their own constituents (e.g., foster grandparents, case management clients, Head Start programs). The group from the health department working specifically on the READII project supplemented these activities by providing low literacy brochures and training for the CBO staff. Feedback from the CBO staff indicated that the training was extremely helpful in terms of raising their awareness about the importance of the vaccines.

In the first year, there were 23 staff members from 12 CBOs that participated in outreach training on adult vaccines and cultural competency, and ten CBOs routinely reported on the activities (e.g., how many people they were reaching, number of events they sponsored, any forum where immunization activities were being promoted).

While Rochester targeted their African-American populations, they found that they needed to do a better job of engaging their Hispanic community. There was not only a need, but also an interest among the Hispanic groups to participate. They also wanted to engage CBOs from other parts of the city. Rochester found that one problem was identifying providers that serve minority, elderly populations. There was initially the assumption that providers serving minority pediatric populations would either serve the adults or direct them to adult providers. This was not the case and identifying providers in the community who saw a large number of the African-American elderly was difficult.

Ms. Kicera then discussed some of the communications. The NIP/CDC conducted focus groups at each of the five sites. One of the messages was not to limit the discussion of immunizations to the influenza or the influenza season. For example, pneumococcal vaccine can be given anytime during the year, but is often coupled with influenza vaccination campaigns. In Rochester, they found that priming people to consider talking to their providers in the summer about being immunized in the fall was very helpful in terms of moving people towards getting vaccinated. Another message that resonated in the fall was that being vaccinated will also protect loved ones.

Ms. Kicera discussed the lessons learned from the Community Action Plan. First, the interpersonal approach worked best-working with CBOs, churches, senior housing, etc. They found that African Americans in Rochester did not respond to the national media frenzy that created last season's influenza vaccine emergency the same way that white populations did. It is difficult to evaluate whether it would be more beneficial to invest funds in media campaigns versus more targeted activities with community groups or providers.

They emphasized that the involvement of the Community Advisory Board contributed to the success of the interventions. Incorporating the message into their ongoing activities is better than beginning a new campaign. Trusted members of the community promoted these messages.

Questions

Dr. Johnson asked about taking national policy recommendations to the community and whether there was any sense of pushback from the community on these policies.

Ms. Kicera responded that there was a lot of pushback and commented that they were surprised that Rochester was one of the sites with so much resistance. Rochester has a long-standing relationship with CDC with respect to immunization policies. She noted that the demonstration projects that took place 10 to 12 years ago, which resulted in CMS placing influenza vaccine as a covered benefit, occurred in Rochester. The provider community in Rochester was already primed about the importance of providing immunizations, so the resistance they encountered from providers was surprising. Some providers refused to implement standing order policies. In facilities where standing orders were implemented, many nurses and other staff often were from the same minority groups as the target patient populations and shared many of the same myths and misconceptions about the vaccine. As a result, if patients asked these nurses whether they should receive the flu shot, the nurse would advise against it. Rather than nurses advocating for immunization, there was a lot of subtle, negative information passed around. She emphasized the need for an effort to understand these negative attitudes, to educate these workers, and to increase immunization rates among healthcare professionals.

Ms. Katz asked whether there was any reason to believe that resistance was because the people were not involved in the formulation of the policies, or whether there would have been resistance regardless of their level of participation.

Ms. Kicera responded that they were hoping to do additional research in this area to get a better understanding. She noted that feedback showed that some providers, through liability issues with implementing standing orders, did not like relinquishing control over their patients. Although standing orders are routine in many settings, they encountered the same resistance in hospitals, nursing homes, etc. She added that it appeared that providers were putting up barriers that are not true barriers. In facilities where standing orders have been implemented, there have been dramatic increases in immunization rates without any liability issues with respect to screening and immunizing patients meeting specific criteria. She noted that in larger practices or managed care facilities, if one can approach the person who makes policy decisions (e.g., medical director, administrative manager), the policies work well and the providers are happy that this does not take their time away from their patients. However, in smaller practices, providers are stretched in terms of the time they have with their patients and the number of messages they are expected to relate. Adults do not visit their providers for preventative medicine-they go in when they are ill or have some type of chronic medical condition requiring follow-up visits. Even though Medicare reimburses for the cost and administration of the vaccine, providers do not make money and in some cases lose money, particularly in small settings where overhead is high, time is short, and billing Medicare is complicated. Some providers, while they recognize that immunizations are important, particularly for high-risk patients, hope that patients get immunized elsewhere.

Dr. Zink asked about alternate provider sites and whether any of the demonstration sites tried to collaborate with different delivery systems.

Ms. Kicera noted that this has been done, but less so in Rochester than the other sites. She noted that CDC is recognizing that they will have to move away from the childhood model to succeed with adults, and that they will need to effectively partner with mass immunizers and other ways to immunize large numbers of adults. As seen with pandemic influenza planning and the smallpox experience, infrastructure for immunizing large masses of adults is not established in this country. If we had to gear up quickly to do this, it would be very difficult. They have looked at working with, for example, pharmacies. However, not every state allows pharmacists to immunize. State law and legislative differences influence these efforts. In Rochester, other than physicians, only RNs can administer vaccines, and as discussed earlier, many facilities do not have RNs on staff.

Dr. Zink asked whether nurses who had negative opinions about vaccines had views that varied from one year to another depending on the specific situations-whether some years are good matches as opposed to others.

Ms. Kicera responded that this was not the case. They found that myths and misconceptions of healthcare professionals were echoed by the patient population (e.g., the flu vaccine gives you the flu). She noted that they received "schizophrenic" messages during work with some of the focus groups, where, for example, there was distrust of government healthcare programs, particularly among African Americans. However, their most trusted messengers were their healthcare providers. They noted that if their healthcare providers recommended it, they would get it. In most cases where adults have not been immunized, it was because their providers did not tell them they needed it. She commented that the attitudes seen among healthcare workers have not been sophisticated in terms of good matches one year versus another year.

Ms. Koslap-Petraco commented that she was trying to fit this into public participation in terms of the healthcare provider group, particularly nurses since she is one of them. She noted that healthcare providers need to be educated first. This was the topic of a paper she wrote for CDC. In her health department, immunization rates went up after an educational program was provided for healthcare workers. Workers were asked their concerns about why they did not want to be vaccinated and the educational program was built around these concerns. She clarified that in the state of New York, RNs are the only nurses permitted to vaccinate, but LPNs can vaccinate under the supervision of an RN. She noted that in Suffolk County, pharmacies hire RNs to administer vaccines. She acknowledged that this is more complicated on a national level with different laws in different states.

Ms. Kicera added that one thing they found surprising going into providers' offices was the level of disorganization and state of disarray. In many cases, the providers could not identify their high-risk patients or even patients 65 years of age or older. The providers indicated that they did not have the time or resources to do this type of organization, but told READII staff that they were welcome to come in and do it for them. Ms. Kicera noted that they had been under the impression that the providers could just pull up a list of, for example, all patients over 65. This was not the case.

Ms. Nurse asked how much of the attitudes found among minority groups were related to issues of aging (i.e., these people are old and going to die anyway and do not need special treatment).

Ms. Kicera responded that this has been a topic of conversation between Roger Bernier (NIP), herself, and others for a number of years. She noted that they are baffled by the lack of outrage at the staggering number of hospitalizations and deaths from influenza and pneumonia in this country-90 to 95 percent of which are occurring among older Americans. If these types of numbers were to occur among any other population group, there would be a tremendous public outcry about what is being done to prevent it. The vaccine recommendation is for people 50 and over, many of who are still very active. An estimated 114K to 115K excess hospitalizations and 50K to 60K deaths occur annually from vaccine-preventable illnesses. They are having difficulty understanding the societal attitudes around the aging issues. As the population continues to age, they hope this issue becomes a national priority.

Ms. Nurse asked about the use of media research in terms of the efficacy of print campaigns versus radio and third party campaigns. There is increasing evidence that people are not trusting print media campaigns, particularly those from government sources. Because of the oral tradition in black and Hispanic communities, people will listen to their neighbors, ministers, etc. She asked whether this recent research is being utilized in terms of radio and third party campaigns.

Ms. Kicera responded that Mississippi and Milwaukee have launched aggressive radio campaigns. They have also looked at alternate sites such as church-based groups. Mississippi recently had a huge faith-based summit in which Garth Graham from the DHHS Office of Minority Health participated. She noted that disparities between the African-American and Hispanic elderly and the general population persist despite controlling for things such as socio-economic status, access to healthcare, insurance coverage, etc. They have therefore, in addition to implementing interventions that have been shown to work, encouraged creative, new ways to reach out to these communities.

Leanne Nurse invited Ms. Kicera to present the Rochester findings at the 2005 Community Involvement Conference in Buffalo.

AAAS Public Engagement Project

Ginger Pinholster

The American Association for the Advancement of Science (AAAS) mission is to "advance science and innovation throughout the world, for the benefit of all people." The practical goals are to preserve the long-term AAAS reputation (circa 1848), increase visibility, and engage the public. She noted that in her office, the Office of Public Programs (OPP), the specific mandate is to rapidly, equitably, and efficiently communicate with the public and reporters worldwide. This is relatively new-they used to be the news and information office at AAAS.

OPP uses conventional press techniques to reach out to the public. They have also used animal wrangling, such as Rico the Wonder Dog, a dog that can remember names of things through a process of "fast-mapping" or cognitive elimination. Like most science authors, he initially refused to talk to the press, but unlike most science authors, he crawled under the table to get away from Reuters. Traditional approaches are press briefings, followed by public briefings.

In 2004, AAAS had two initial public engagement activities. Family Science Days was held at the AAAS annual meeting in Seattle and drew 10,000 people. Oceans for Everyone town hall meeting was another activity.

A key public engagement vehicle for all AAAS work in reaching out to the press and the public is EurekAlert!, which is their editorially independent website that reaches 540,000 public visitors per month. Research news is presented in a variety of languages and there are 4,800 reporters from 50 countries registered with this site. One of the most important destination-specific sites on EurekAlert! is the Science for Kids feature. This was started as a way to keep the science press package team interested in their jobs. The site now draws some 10,000 public and reporter visitors per month. A spin-off of this feature is the "What Would Kids Ask?" public engagement effort. Each week, two children are invited to engage directly with Science authors by asking them questions. This idea came from a public briefing with the three cloned mules where a child asked Dr. Gordon Woods whether the mules had the same memories. Dr. Woods responded that they have different birthdays.

Ms. Pinholster then gave a brief background on the public appetite for science based on information AAAS disseminated to the public and the press. She showed a list of the best selling Science features and noted that they have not included much for children in terms of medical or health news.

She noted that at each AAAS annual meeting they serve about 1,200 press registrants. She also highlighted the stories that generated the most coverage in 2004, with stem cell research as the top story. Based on the latest Science press package survey, medicine was at the top of the list of what individual reporters wanted to know about from Science and AAAS. She noted that the more whimsical topics (e.g., dinosaurs, animals) were towards the bottom of this list, while 2003 coverage trends showed that animals were popular. This is a result of the difference between what reporters want and what their editors may be able to get in newspapers or what producers may be able to put on television. She commented that it is encouraging that the topic of health threats is among the top three categories and noted that there is tremendous coverage whenever there is a health-related paper.

Ms. Pinholster then discussed the AAAS Center for Public Engagement with Science and Technology. The two primary goals of this new center are to provide an open forum for the public's concerns and hopes regarding scientific advances, and to provide an opportunity for the public to help frame future research agendas. The mandate was set forth by AAAS CEO, Dr. Alan Leshner, in Science magazine. This is a departure from their well-respected, 30-year old public understanding initiative, which is more education-driven. Dr. Leshner had noted that it may not be realistic to assume that the public can always understand science and technology issues in the same way that a physician or scientist might, but this does not mean that the issues do not affect their lives or that they have no right to have any say.

Ms. Pinholster then discussed the Oceans for Everyone town hall meeting. The meeting was initially a half hour briefing for 60 reporters with five speakers. It grew and was advertised as a three-hour event. It lasted four hours and there were over 230 attendees, including 34 local policymakers, and five speakers. Ira Flatow was recruited as the moderator.

The town hall meeting was structured as two simultaneous events. There were a series of breakout or dialogue groups, followed by the central town hall meeting that was kicked off by Ira Flatow and Alan Leshner. At designated intervals, the breakout groups would have their discussions and then feed summary reports to Ira Flatow. Their input was summarized to rank the choices they felt would be the most realistic approach to deal with ocean issues in the Seattle area. Results were posted on the web and passed out to participants as they left.

They worked with one of their media partners, the Science Channel, for a real-time webcast played at the Annual Meeting Internet Café, a delayed broadcast radio spot, and an advance feature piece for television. They also worked with Public Agenda, a public engagement group, and Porter-Novelli's social marketing unit to put this together.

Porter-Novelli conducted a public opinion survey of 2,400 adults in advance of the event to get data to use as a news hook for a premise for discussion. Adults know that the health of the ocean is being endangered and yet, very few feel empowered enough to do anything. This led to the second, indirect message that AAAS is there to help.

Ms. Pinholster commented that they were impressed with the work of Public Agenda, but that perhaps Porter-Novelli's help was not necessary. In terms of participation, they expected about 300 participants and ended up with 230. One lesson learned was that they should over book. The twelve collaborating organizations were not pushed very hard to work through their communications channels to recruit people. This affected the socio-economic and ethnic diversity of the group and some groups were under-represented. She commented that the breakout groups did not work as well as planned. The choice work model given by Public Agenda was overly complicated for what AAAS was trying to do. She did not feel they needed that level of programmatic complexity. Their biggest problem was that some of the panelist discussions were too long-winded. There was, however, good dialogue. Dr. Leshner gave this first effort a B+.

The proposed format for future town hall meetings is to use a reactive-mode format that may stimulate dialogue and insights. As stated earlier, all AAAS public engagement activities are intrinsically linked to their press outreach activities. Much of their success depends on how well they convey their message to the media. She gave the example of The Washington Post article on the 2003 event at MIT where the Dalai Lama and Eric Lander were both asked to look at the color blue and explain what went through their minds. With this model, the event becomes the news. It is also possible to harness the power of celebrity to reach many more people.

She added that they are interested in public health as it affects families and children. She also displayed a list of other possible topics that they have under evaluation.

Questions

Ms. Katz asked if, from the AAAS point of view, the purpose of the town hall meetings was to get media attention to the issue.

Ms. Pinholster responded that this was not the only purpose, but was critical to their success.

Ms. Katz asked how the topic of oceans was chosen.

Ms. Pinholster responded that the topic was chosen primarily because of the venue. They knew there was a strong marine science community in Seattle. They looked at the peer-reviewed programmatic content for the meeting and saw that a significant percentage had to do with marine science. They spoke extensively with a number of partnering organizations that were willing to work with them. They felt that it would work and that there would be public demand for this type of information.

Ms. Katz pointed out that this was different from the previous presentations in that this was more of an educational event-AAAS did not have to make a decision about what had to be done with the oceans.

Ms. Pinholster noted that AAAS was not as interested in engaging the public in policymaking as they were with raising awareness about some of the issues and giving the public the opportunity to participate.

Ms. Wexler asked if this was their first foray into getting the public engaged.

Ms. Pinholster responded that this was the first town hall that they had done.

Dr. Zink noted that there was interest in the United Kingdom about engaging the public and scientists, particularly with respect to genetic foods. He asked for comment on any AAAS involvement or study in this area.

Ms. Pinholster responded that they have had no involvement. She commented, however, that this was an interesting question because they saw the 9/9 UK announcement about the availability of public engagement money, and with the major nanotechnology seminar coming up at the 2005 AAAS annual meeting, one can imagine this being video-conferenced with the United Kingdom.

Ms. Landry asked if there was any feedback from the participants on whether they felt this was worthwhile, whether there has been any measurable change, or whether more people are now interested in the issues.

Ms. Pinholster responded that they conducted a post-participation survey and people pretty much graded it the same as Dr. Leshner (B+). People commented that some of the speakers were too long-winded and there was some grandstanding that the moderator was not able to contain. The questions that were never fully answered were what to do next and how to sustain the interest. They established a marine science portal on EurekAlert!, which has become a very viable virtual community for many of the participants.

Dr. Vernon raised the issue of stem cell research, an area where people are likely to have more hardened positions. He noted that this is more the type of environment they are likely to encounter with immunization policy discussions. He asked Ms. Pinholster if she could speculate about what might be done differently under these conditions.

Ms. Pinholster responded that, as Dr. Helms had commented earlier, everyone invited must agree on the premise that they will not necessarily agree on the specific points, but agree that they are worth discussing. With Public Agenda, the level of advance staging was phenomenal and they anticipated virtually every comment and prepared a moderator for responding to them.

Ms. Wexler asked about the 2,400 people surveyed before the meeting and how they were selected. She also asked whether the 230 participants were drawn from this pool.

Ms. Pinholster responded that the number was from Porter-Novelli's random sampling. She added that the 230 participants did not necessarily come from the 2,400 surveyed-they worked with the 12 collaborating organizations to recruit participants.

Ms. Katz asked whether the audience had any difficulty understanding the science.

Ms. Pinholster responded that she believes anything can be explained, and added that they did not include a high level of technical detail.

Environmental Protection Agency (Office of Water)

Common Sense to Common Practice-Improving Public Involvement at EPA

Leanne Nurse

Ms. Nurse's presentation focused on work in the recent Executive Order on Cooperative Conservation that the White House issued last week, as well as new EPA Administrator Mike Leavitt's initiative on collaborative problem solving. She noted that there is some controversy over cooperative conservation. Some people feel it is a timely, election-oriented effort to show how friendly the administration is and how much it is listening through the agencies.

However, this kind of work has been happening at EPA for some time, in terms of social science and environmental research. The largest body of information on civic engagement is in the environmental field, much of which is a result of EPA's legislative mandate. They have a lot of good work on which they are building. Their office, the National Center for Environmental Innovation, was tasked with developing the policy for public involvement. Bob Carlitz from Information Renaissance will help her discuss with more detail on the online dialogue that helped them formulate and launch the policy.

She expressed her appreciation for Patricia Bonner and noted that she is very ill. Ms. Bonner has been a pioneer at EPA and many know that she developed its customer service policy, which provided leadership for the federal government. Similarly, the public involvement policy is the first agency-wide policy. They are getting a lot of feedback from their federal partners through ongoing discussions, which NVAC will hear about tomorrow with the Deliberative Democracy Consortium and Link to Government Group, and through high-level conversations at the Council on Environmental Quality (CEQ). The goal of this work is to enable common sense things that they agree on (e.g., being respectful in listening and giving timely feedback) to become common practice across an agency with 18,000 people.

EPA has a huge job in explaining and building the body of knowledge showing that effective civic engagement and public involvement creates better environmental outcomes. This is a hard sell to business and science-oriented managers who have been convinced through their training and social models that they are the experts, they have the answers, and if we trust them with the work, then the outcomes will be good. That is meeting the rising public expectation that ordinary people have informal knowledge and have other kinds of expectations and information that need to be taken into account as public decisions are made.

They have, through a long series of conversations with citizens and experts in the field, developed the "Seven Basic Steps for Effective Public Involvement." She noted that copies of the presentation, a brochure that answers basic questions, and online tools are available.

The most important of these steps is the one least likely to take place, which is to plan and budget for public involvement activities. Most people have been involved in situations where someone calls a meeting quickly to check it off their list and then calls it public involvement. Fortunately, there is a lot of good work throughout public and private organizations where people recognize that it is helpful to include stakeholders in the strategic business model. For many, this idea continues to be a surprise and an affront to their boundary issues. It is important not to take for granted that many people do not understand the value of building relationships as part of their business model, their pathway to effectiveness, and their final decision-making. Care must be taking in how managers and technical people are addressed.

Ms. Nurse then reviewed a chart outlining the spectrum of decision-making and public participation processes. This chart has been particularly helpful for managers as they walk them through the EPA collaboration process. She then reviewed the International Association for Public Participation (IAP2) public participation spectrum and the range of activities. They can talk about collaboration and can have negotiated settlements, but all of this more complicated work is built on very basic steps, such as providing basic, effective, and timely information. The meeting attendees may be familiar with the IAP2 spectrum, which can be found on the IAP2 website.

She then reviewed the consultative processes EPA uses-how they actually conduct business across the spectrum of engagement. They do basic things like tours, workshops, and roundtables, which are essentially one-way means to get out information. Unfortunately, this is the bulk of what happens, but they are also aware that through advisory committees, the FACA process, and joint task forces with various partners, they are able to have a more productive and thorough ongoing dialogue. The collaboration work that the Administrator and the White House have been discussing is to look at some these larger scale conversations and situations in which an opportunity exists to equitable, negotiated settlements. Power sharing is among the issues people are facing. How do those who have power share that power? How do those who have access to proprietary information agree to share that information with those they do not trust upfront?

One of the tools that they are using and that may be useful for NVAC to examine is a contract through their Center for the Prevention and Resolution of Conflict. It is an alternative dispute resolution (ADR) contract (www.epa.gov/adr), which allows third party neutrals to work for the agency and to be involved in situations where the agency many not be the convener of record, but may be a participant in a particular situation.

The tools they have include a series of booklets, which the Working Group will receive. They have brochures that include the Seven Steps and discuss working with the environmental justice community and improving public meetings and hearings. A lot of this material came from the dialogues that Information Renaissance helped them develop, the public comment process, and the areas of ongoing learning, such as appreciative inquiry. They are now looking at appreciative inquiry specifically for internal marketing and to help develop a community of practice in the agency.

They have a small network called the Public Involvement Improvement Council, and representatives from each of the program offices-such as Air, Water, Solid Waste Emergency Response, each of the ten regional offices, the laboratories, and the Office of Research and Development-are involved. They are planning a two-day retreat in November in which they will use a process of strength-based or appreciative inquiry. Looking at good practices, they will look at where they are now, how to get where they want to be, and how to build on what they already know.

They also have laminated cards and bookmarks, and a portable exhibit that they are revising for different settings. They are also waiting for the next fiscal year to release money for training, which will provide some of the baseline work that is being done in the agency to improve the practice of high-level practitioners. This will begin to provide some leveling of the knowledge for others in the agency about basic practices, as well as a set of five modules for managers that are less about classroom training and more about engaging them in teachable moments. A key issue is how to reach managers when they are already "flooded out" in their respective meetings or situations and how to tailor materials to address specific program issues and managers' responsibilities. At the Senior Executive Service (SES) meeting, with which Tonya Gonzalez helped, the managers were adamant that the agency respect and acknowledge the existing work they are doing. They have a very strong sense of pride and ownership of their work, but they are also interested in new tools.

They also have a series of feedback tools. The surveys and the information requests for the surveys are at the Office of Management and Budget (OMB), which will hopefully be released soon so that they can start using them. The survey materials will be available online, so that people in the regional and local offices can download them. Eventually, this will become a part of a searchable database accessible to others in the agency.

Ms. Nurse then reviewed some of the online tools that they are using. Their public website (www.epa.gov/publicinvolvement) includes a large amount of information and materials, including policy and other kinds of resources and tools that may be of interest to the Working Group. Their favorite tool is the Public Involvement Resources and Training (PIRT) database, and they hope to migrate it to the public website. They have provided the URL, user name, and password to give the Working Group an opportunity to learn more about it. The Feedback/Evaluation Tool Box is available to the public at www.epa.gov/evaluate. The dialogues with Information Renaissance are also available (www.network-democracy.org/epa-pip). EPA also has the lead for the e-government rule making process, and they will try to keep the Working Group up-to-date.

PIRT now contains over 700 entries related to public involvement. These entries are very specific and searchable by type of activity. They include academic research, trainings, and conferences around the world. At www.epa.gov/evaluate is the evaluation resources toolbox for stakeholder and public involvement. Again, these summaries can be searched by topic or by alphabetical order. The feedback tool components include the user guide, the survey tools (pending OMB approval), and a data input and analysis spreadsheet program.

Ms. Nurse then introduced Mr. Carlitz to talk about the online dialogue that helped support the development of the policy and the tools, and to get into more detail about how EPA worked with them and other groups to make the dialogue a success.

Robert Carlitz, Information Renaissance

Mr. Carlitz provided an overview of the online dialogue they did with EPA in 2001. This event was successful because EPA staff, Pat Bonner, in particular, recruited people. There were 41 EPA hosts that included all of the regional offices. People participated very enthusiastically, and Ms. Bonner was able to get the staff past the nervous stage where they feared people would yell at them. The audience did not ignore problems they had with EPA, but they put issues on the table. EPA staff was clearly viewed as facilitators in addressing these problems, rather than the causes, and a very positive environment was created. There were 1,166 participants and 36 expert panelists, and each day, hosts from different EPA offices and regions were invited.

Previously, they had done work with EPA for a dialogue with librarians. The topic was libraries as a resource for environmental information, and there were about 550 participants. There was some overlap between these two events, and the library dialogue was very useful in setting a positive tone and overcoming the resistance to participate.

Initially, Information Renaissance sent a grant proposal to do the online dialogue, which was not the appropriate vehicle, and the work was put out for a contract. Information Renaissance secured supplemental support from the Hewlett Foundation, which allowed them to put together things such as the extensive online library, giving them an advantage in competing for the contract. Consequently, the joint sponsors for the online dialogue were EPA and the Hewlett Foundation. There was also an evaluation report completed by Tom Beierle at Resources for the Future

Mr. Carlitz then reviewed the list of topics, each of which was given a couple of days for discussion. They had moderators who kept the discussion focused. The topics included:

• State, tribal, and local government issues
• Effective collaboration decision processes
• Required public participation (for permitting, rule-making, etc.)
• Technical and financial assistance
• Local environmental partnerships (Superfund, Brownfields, etc.)
• Evaluation and accountability
• Identifying and reaching the interested public and those hardest to reach

There were also some lessons they learned coming out of the dialogue. There are several stages to this process. Before the dialogue, there were questions about who and how to recruit, and what kind of agenda and briefing book to put together. They were able to work collaboratively with EPA on these issues. EPA identified possible people to recruit and the recruiting was aggressive. They sent out repeated emails, phoned groups they wanted to target, and ended up with a large audience.

Similarly, they tried to make the briefing book very complete to put a good resource online. It is unlikely that all of the participants read everything that was in the briefing book. They have looked at traffic before and after the dialogue started, and found that people reviewed the material to prepare for the dialogue and used it as a resource after the discussion began.

As mentioned earlier, there were different expert panelists and EPA hosts each day. They tried to stay on top of all of the responses each day and emailed or called people daily as questions were asked. Inspired by the ethics in Ms. Bonner's office, there was an effort to accommodate everyone thoroughly. An interesting dynamic was created. EPA's strong attention to detail reinforced Information Renaissance's interest in wanting to do a good job, and the participants appreciated the strong support.

The dialogue itself ran for two weeks and there was ongoing coordination with the various panelists to keep the discussion flowing among the participants. Mr. Carlitz did not believe the moderator was extensively used, since there were no shouting matches, rude comments, or major deviations from the topic. He believes the dialogue was harmonious because there were people who participated in a similar event the previous year and they jumped in and set a very positive and constructive tone. Issues, however, were not avoided. There were clearly people who were upset and angry with local issues. One of the interesting features of the dialogue was what to do with the information that was provided.

After the dialogue, EPA suggested that Information Renaissance go through the more than 1,000 messages that were posted during the dialogue and to pull out messages that point to EPA's best practices and problem areas. These issues were tabulated, put into a database, and made available to participants and those interested in the event. It allowed a compact way to see what EPA learned from the event. There was a commitment to disseminate best practices across EPA and to address each of the problems that were identified. Someone was contacted within one of the EPA offices, who then contacted the person who originally raised the problem to tell him or her what was being done to try to address the situation. This is an extremely good way to deal with public input because it is almost a guarantee that the agency will look at the issue and do something about it. In individual cases, the answer may not be satisfactory, but it shows that the concern was reviewed.

The dialogue was arranged to be as accommodating as possible. EPA received a fax from someone who wanted to participate in the dialogue but did not have Internet access. The fax was scanned and posted as a message on the web dialogue. This person posted a number of messages, each of which were scanned, converted, and posted. A lot of work went into accommodating this person, perhaps too much, but it made a good impression on participants.

The evaluation report found that 85 percent of the participants believe EPA should do this more often. Mr. Beierle also tried to imagine how the dialogue would have taken place if it were not online, which would have required a much larger effort.

Questions

Ms. Landry noted that there are pros and cons of doing the dialogue online versus face-to-face. She asked why EPA chose to use a web-based approach.

Mr. Carlitz noted that he had Ms. Bonner's notes on this question. He explained that there were no funds for regional public meetings, but they wanted more public input. They had a certain comfort level with online dialogues because of their previous online experience with the library dialogue. There was a feeling that if they were ever going to do something like this, this was the time. The background to the dialogue is that EPA had attempted a public participation policy in 1981, which was never implemented. After 20 years, the agency wanted to try to come up with another public participation policy. One of the things that happened in the interim was the Internet, which provided a completely new mechanism for conducting public involvement. The feeling was that if they were willing to be innovative in their programs, why not be innovative in how they develop the programs. Internally, there was the feeling that they could put their money into something that did not work so well in the past, or they could try something new that might or might not work.

Ms. Gunn noted that there has been a lot of interest in recruitment. One of the things that make EPA special is that they have developed an enormous number of relationships with their stakeholders. Also, the development of environmental justice movements has meant that there are identifiable groups that can be talked to about issues concerning people with lower incomes and education levels.

Ms. Nurse explained that, in general, EPA has a huge network of existing and resilient relationships that, through their particular staff, are primarily informal. They work with their colleagues who are the designated federal officers for FACA, the program people in community-based environmental protection, and their Superfund community involvement people. They are in touch with people that do this as part of their daily work. She also explained that she and Ms. Bonner have an enormous range of experience. Ms. Nurse is going into her 15^th year of service, but also previously worked in commercial and public television, and with nonprofits and NGOs. They have a complementary mix of networks that goes into a wide range of people who do not read the Federal Register. These are people who will not necessarily look for opportunities to engage the agency on a particular rule-making process, but if given the opportunity to discuss a particular issue, will participate or get their colleagues to represent them.

She noted that some meeting attendees may know their Environmental Justice Associate Director, Charles Lee, but the National Council on Churches did some of the earliest environmental justice work on toxic waste and race about 25 years ago. People talk about environmental justice as a new idea, but it has been around for a long time. They are able to take advantage of these informal networks and are fortunate that people are responsive when they are invited to participate. They are trying to show their colleagues the value of complementary knowledge and relationships to the primary mission of the agency-how it is that they protect the environment and public health by doing things that may not have been in the tool or rule book when people went to school before coming to serve the public. For example, they are using graduate interns to evaluate their work over the next year. They are hoping that funding will continue for their particular part of the work, although she is more inclined to believe that the work that is happening under the collaboration umbrella will influence some changes in that direction. They have a two-year work plan, but as she mentioned earlier, they have 2.3 full-time employees for 18,000 people in EPA, and are doing their best.

Ms. Katz noted that EPA seems to be ahead of other agencies in this work. She asked Ms. Nurse if she had any examples of when they asked the public for their views and built public policy from there, as opposed to being far along in the policy and asking for input on implementation. EPA has been terrific in collecting responses on what they have decided to do and then making adjustments.

Ms. Nurse explained that given the huge volume of existing rule making and projects, a lot of their near-term work is about improving work that is underway. They are trying to seed a culture change so that publics are engaged as soon as people initiate new work or policy. For example, they now have a module in their three-day training in the regulatory development group for people being trained to write rules. During the training, they talk about public involvement and collaborative problem solving. Before this year, this module did not exist and they did not see the need to solicit public comment or engage people in some formal way. Now, they talk to them about the range of tools and the ability to do better work by engaging a wider range of stakeholders. There are not a lot of examples, however. There is Cleveland Air Toxics and several showcase projects that are on the website.

Ms. Nurse hopes that they will hear more from EPA by the end of the year. The challenge to their deputy-level civil service people, the Innovation Action Council, next month will be to commit funds for a series of projects like that. They have a pool of projects that are primarily community-based, multimedia projects of regional significance and will have to make specific decisions about where the funding will go. The biggest complaint they had, other than the resistance to starting a new formal way of doing or accounting for work, was that they did not want to have to find new money. Her office's response was that this money is an upfront investment in engaging a broader range of people in more conversations, and it will pay off in dividends and measurable environmental progress at the back end.

Ms. Nurse explained that the nature of federal-level public work is that so much of it is based on legislative mandates and, in EPA's case, is driven by enforcement and public health. These two histories and identities have never been resolved in terms of a corporate identity. To talk about doing primary mission work is not something everyone is comfortable hearing about yet.

National Aeronautical and Space Administration

Victoria Friedensen

Ms. Friedensen noted that her presentation will focus on the communication model NASA uses, its history, and how it is approaching dialogue with the public. She noted that it is a new process for NASA, despite the fact that they spend a substantial amount of time talking to the public. Outreach, however, is a one-way transmission of information that people can take away for their own purposes. Engagement is a conversation where information is exchanged between people-it is the concept of a dialogue. She is the director of a brand new NASA program, Communication, Engagement, and Outreach, which has never before been an integrated function.

Project Prometheus is about exploring the solar system and the power to fuel exploration. Looking at comparative distances of planets, some of which are very far away, the range of technologies that they can use is varied. The closer they are to the sun, the better the solar power. As you get farther out, the only technologies that NASA has available are nuclear technologies. The Cassini mission to Saturn is powered by 72 pounds of plutonium 238, which is a highly radioactive, non-weapons grade material. If they are successful with their development project for Prometheus, they will be developing a nuclear reactor for space exploration. Russia developed them, and the United States flew one in the 1960s. It will be the size of a 5-gallon paint can and will provide 100 kilowatts of power for expanded exploration. They have no other technology outside of the sun's orbit where they can do some of the scientific activities they would like to do.

Project Prometheus is a technology development program that engages the public and the only program that has generated negative public opinion when it comes to NASA. They have sustained broad support for all of their programs. Even the loss of Columbia in 2003 did not damage the base of support in the United States. The only thing that has received sustained opposition is the use of nuclear power in space.

The communication model they have selected is based on risk communication techniques. They are using an expanded proactive dialogue, and specifically, she is using Cantor and Rayner's How Fair is Safe Enough? They are looking at public engagement and trust, liability, and consent (TLC), EPA experiences, and the Army's Assembled Chemical Weapons Disposal Program. They are looking at building understanding and trusted communication pathways and social amplification of risk theory models. In other theory models, the idea of trusted communication paths can be more important than the message. In addition, there is the understanding of perception-you cannot build a communication program if you do not know what people are worried about.

Most of the risks that NASA talks about are probabilistic uncertainties. This situation, however, talks about probabilistic uncertainties combined with a measure of dread and harm (e.g., an accident involving nuclear material). Looking at the model, they have high consequence and extremely low probability of accident. They design the systems so that they contain the fuel under an extreme range of accidents. Because of the statistical uncertainties, they go out of the basic knowledge and expertise range, where they talk about "the facts," and into "trust us, we're the expert," which involves more risk. Finally, they go into world views and values where they say, "You do want to explore space, don't you," which generates a higher intensity of conflict and a higher degree of complexity. It could, for example, involve a conflict of different values, or a question of whether someone is trustworthy or viewed as competent to answer the necessary questions. Looking at the work of Ortwin Renn, they can take the idea of values versus competence versus expertise/facts and look at where controversy develops and how agencies and organizations respond.

Ms. Friedensen then reviewed the risk management escalator model, which has four phases-routine operation, scientific risk assessment, risk balancing and risk assessment, and risk tradeoff and deliberation. Routine operation is NASA's day-to-day work, such as launching rockets and sending satellites into space. NASA's return to flight for the space shuttle is currently under scientific risk assessment; it is a cognitive discussion about what they know and do not know. As they move into risk balancing and risk assessment, they move into an evaluative component in which they have involved stakeholders and reflective discourse. There are also uncertain risks involved. When controversy really gets heated, they have tradeoffs and deliberation. Project Prometheus is in the second phase of the model, where they have a potential conflict but they have extensive risk assessment that they will be putting into place.

They are very early in the program. The next launch of a traditional radioisotope powered system is in 2007, which have been used for 30 years. The launch of a nuclear reactor is scheduled for 2015. This is a long-range development program; these kinds of engineering projects take time to develop.

They care at this time because public attitudes have changed towards NASA post-Challenger. They launched radioisotope-powered systems (RPS), for example, with Viking, Voyager, and Pioneer, which were all plutonium powered. Post-Challenger, there were issues of safety. The Ulysses mission was originally to be launched off the shuttle, but it was redesigned and launched off another spacecraft. Galileo was launched off the space shuttle, even in the face of public opposition.

What did this mean to NASA as an agency? Lawsuits were filed, which was new to NASA. Letters were written to Congress, which people rarely did previously because they expected NASA to do the job correctly. These letters protest the use of nuclear power in space. There was a concerted writing campaign in 1997 and 16 members of Congress asked NASA to stop the Cassini project before it launched-this was unheard of in NASA's history. What this meant for agency resources was millions of dollars expended in a highly reactive mode where they were scrambling to stay above the controversy and keep going. They were successful and launched. That mission has been a tremendous success, as is evident from the Saturn photos and some incredible data. As NASA moves forward, however, they want to make sure that they do not keep facing this problem.

Since early in the Project Prometheus initiation in early 2002, they have been looking at what people are saying. These concerns include:

• NASA is looking at the militarization of space.
• The technology is viewed as untested and untrustworthy-that NASA does not know what it is doing.
• Costs are undetermined and incalculable-NASA does not know how to pay for it.
• Reactors are considered more dangerous than radioisotope power systems.
• There are other, higher priorities, such as the war, that should concern the nation.

NASA has a series of risk communication plans that it is developing. The first was in the Office of Space Science. Then there was Mars, Project Prometheus, and JIMO (Jupiter icy moons orbiter), which will be the first application of the nuclear reactor. Their basic goal is to ensure open, honest, inclusive dialogue and communication. To do this, they have to provide proactive (e.g., before something happens), cooperative engagement with a broad range of potential stakeholders. They want to increase trusted communication pathways and the public's and likely interested public's basic working knowledge of the program with the hope of increasing broader support. The likely interested public includes the rank and file NASA supporters that love space but do not know much about it. In a democracy, people are allowed to be interested in what they are interested in, and they need to place information in front of them in places where they can go and get it when they need it.

Communication, engagement, and outreach will ensure clear, accountable communication processes and enable extended inclusive engagement with NASA's stakeholders. This is new for NASA. They educate and inform, but they rarely engage in a dialogue. This will be the first time that all of the messages will be fully integrated across the agency.

They wanted to identify their stakeholders' issues. They knew that past statements about radioisotope systems probably would not apply, and they wanted to understand what the perceptions were about using a reactor. For these reasons, they went to the Keystone Center, which conducted a series of interviews on NASA's behalf and submitted a report with recommendations. One of these recommendations was to create a dialogue group, an independent group of diverse groups and individuals that would discuss NASA issues.

In the Keystone study, they went to a broad range of organizations that represented a spectrum of interests in NASA and a spectrum of opinions in the use of nuclear power. For example, the Federation of American Scientists opposed the Cassini mission at one point and then retracted their opposition later. However, the Physicians for Social Responsibility and the Union of Concerned Scientists believe that the use of nuclear energy at all-whether for space or not-should be opposed. They also interviewed a range of thought leaders, and some declined because they were either too busy or represent complete opposition and would not speak to them under any condition.

Essentially, all of the messages were the same as what they have heard before. The only new theme was the question of international implications for the nuclear non-proliferation positions held by the United States. This question indicates that NASA has a substantive opposition that knows what it is doing. They responded to these messages in a way that allowed them to engage in a broader communication with those individuals who may not have formed a concrete opinion yet. They wanted to reach the people who are aware of NASA but do not have a pro or con position.

The dialogue group will be asked to provide NASA advice on the public's concerns related to worker, public, and environmental risk and safety. This is an essential issue. What, in the event of an accident, will happen around the launch area and to the rest of the planet, depending on the nature of the accident? Can they protect NASA's and the Department of Energy's (their partner) workers? What are the benefits, costs, and risks? Because of the perception that they are really fronting for the Department of Defense and that this is a weaponization program, how can they talk about the benefits of using nuclear energy without implying that they are seeking to advance tendencies that are more aggressive?

Also, what does credible, independent review look like? This is an essential question because they already have two independent reviews. One of which responds directly to the President through the President's Science Advisor is not perceived as independent enough. Who do they need to bring to the table so that they are trusted when NASA comes back to them?

They hope to have the first meeting of the dialogue group in November. They want to have it FACA-chartered and they are in the process of getting the package completed to take to GSA. It will be a fixed-term committee early in the program; they will hold six to eight meetings and then will probably be released. They will have an external facilitator and the NASA Deputy Administrator, who is also the former Associate Administrator for Safety and Mission Assurance, will be the lead agency representative on the dialogue.

When the dialogue is completed, NASA will:

• Respond to the findings as best as they can
• Integrate findings into other programs such as communications and outreach
• Identify opportunities for specific conversation with additional stakeholders
• Create opportunities for real time input from Congress on perceived issues and responses
• Integrate agency-wide programs responding to the Columbia Accident Investigation Board Report
• Communicate the process, findings, and results of the dialogue to NASA and DOE personnel, contractors, and other aerospace stakeholders
• Analyze the dialogue process to identify further opportunities for engagement

Program initiation is a hard thing when you are talking about outreach and engagement. This is the first time that NASA has dedicated two full-time staff to this process. They have a budget that is called out, which is up to one percent of the total Project Prometheus budget. They are going to have an integrated plan that brings together three disparate parts of NASA, and they are looking at a 10 to 15-year program, which is also unheard of for engagement and public outreach programs at NASA.

They want to know what their metrics will look like-generally, how well it goes. One of the things they will look at with the dialogue group, for example, is the satisfaction of the participants and how NASA completes the milestones with them. These will be highly placed people with the ability to walk away from the table if things go badly. They are also looking at their training frequency and the use of their communication products and how they can get real-time input from interested individuals from the public.

Questions

Ms. Landry asked about NASA being in a reactive mode and trying to keep their head above water. She noted that they sometimes feel that way with vaccines. Acknowledging that the program is in its very early stages, she asked if they feel they have made progress in this area and if it has made a difference.

Ms. Friedensen replied that they believe they have made progress. Anecdotally, they prepared extensively for their budget announcement of the Nuclear Systems Initiative because they were very worried about backlash, and it went very smoothly. Every question that they anticipated would be asked was asked. Her boss, Associate Administrator Wiler, thought that people did not care and that it was easy, but it was not-they had to be and was prepared. From a practical perspective, they got a budget and very little Congressional input, and the letter writing has been very small. The technologies themselves are controversial, but since they are early in the program and NASA has been consistent in their responses, reporters have not noticed any internal controversy.

Ms. Landry then asked about what had happened the previous week. Were they able to get on top of the situation right away?

Ms. Friedensen noted that this involved a different division in NASA. The problem with their analyses is that they always know what went wrong, even though they do all of the work and preparation. That is what happened with Genesis, so they were able to get there and respond with what had been instituted for Columbia, which is to keep it completely open (e.g., we don't know what happened; we're looking into it; and we'll get back to you). It will be the same with Project Prometheus.

Mr. Carlitz asked if the dialogue is being conducted to have the public play a role in making policy or to gain public acceptance for a policy that NASA for which has decided.

Ms. Friedensen replied that this a good question, and they have struggled with the desire to have public acceptance and to do the right thing versus how do you really create an opportunity for the public to make policy. What is not on the table is whether or not they are going to explore the solar system; this is NASA's job. Right now, the only technology they can see to do this effectively is nuclear power. With these two provisos, what policy do you recommend to make NASA do it more safely and effectively and create trust? These are policy questions. For example, they know that they have five safety assessments that happen internally. They have one that will happen through the Department of Energy's Naval Reactors Program. What policies and areas of risk assessment would they have dialogue about where the public would have a positive impact on the program?

Mr. Carlitz explained that he understands NASA's dilemma about the exploration of outer planets, but on larger scale, the agency is in a paradoxical situation. If NASA were to have a public dialogue on the overall agency policy in much broader terms (e.g., manned exploration versus unmanned probes where NASA's money has been going for decades and an area that has been criticized significantly from the scientific community), it seems like this is the way to build the sort of trust so that this kind of issue would be viewed in perspective. NASA is in a difficult situation. From a scientific standpoint, they are probably right in what Ms. Friedensen is saying, but with the structure they have set up, it seems like they will never have the opponents change their view when NASA says what they are doing and that it is their mission and there is no other way to do it. It is hard to get opponents' buy-in because there is nothing for them to suggest that is different.

Ms. Friedensen replied that this is true. The Aldridge Commission, which conducted its meetings in the spring and early summer this year, was convened to get public opinion on what NASA should be doing. They discussed questions such as, what kind of investments should they be making and in what? If they are going to be investing in humans going to Mars, how do they keep them productive and safe? What kinds of technology should they use? She can see using open questions for further dialogue in the way discussed in the AAAS presentation with a town hall or other public format.

Mr. Carlitz noted that the relevance for the present committee is the importance of doing something early enough in the policymaking process to allow for some choices. The dialogue mechanism is a good idea. It is important to do it early enough in the policy formulation process so that you do not get into a situation where any of the opponents feel squeezed out. Otherwise, they will feel that there is nothing they can do but make a lot of noise and say it was the wrong decision. It is important to involve them when there are still some choices on the table and they are in a position to influence some things. They become partners in the process, and even if they lose in some decisions, they feel like they are being listened to and have a role. There is a difference in public involvement in the formulation of policy and explaining to the public that you have done everything right and they ought to trust you.

Ms. Wexler asked if NASA regularly asks the public what their views of NASA are. She doe not think that they do that well with immunization. They do not have routine screenings of the public about how they are doing and what their concerns are. She heard about the recent study but asked if they regularly check with the public about what they think of NASA's work.

Ms. Friedensen explained that as a federal agency, they are prevented from conducting opinion polls. It has to take another guise, and it depends on if you have money for a social study. They did get permission to do an opinion poll on the exploration systems, which is the new vision for returning man to the moon and going on to Mars. They got the same ratings approval that they have gotten since 1978. Thirty percent believe they are doing a fantastic job; 30 percent think they are doing an okay job; and 24 percent do not know. Those polls were done every two years from 1978 to 1996, and then there was a bit of an hiatus. As she mentioned earlier, they have broad, substantive, and uninformed support. They have database website information, unsolicited comments from the public, and responses from Congress, which is their first public this early in the program. They are also looking at ways to routinely assess what they are doing.

Ms. Katz asked Mr. Leighninger if there is a way to explain to the public what the degree of potential harm is. Is there a way to say that the risk is 1(10^-17) in a way that the public will understand? Ms. Katz agreed that it is difficult as Ms. Friedensen explained and noted that she would not know how to do it. Are scientific questions harder to get people to understand and engage, as opposed to simply listing what the government priorities should be (e.g., space versus education)?

Mr. Leighninger explained that there are two different challenges they are facing. The easier challenge is whether people can understand once you get them in the room. The harder question is how to get them in the room in the first place. Referring back to Mr. Carlitz's comments, he said it would be interesting to see people become involved in a whole range of issues facing NASA, including the safety issues. In general, the more you can broaden things, the broader the pool of participants you get because people are interested in different things. Also, people's ways of making an impact are broader. There may be a number of decisions or pieces of a project that people feel they have impacted, or, even better, they feel like they can do something about an issue. The ways to take action personally in the space program are not the same in public health. However, the more public involvement opportunities you give people, where they can look back on an experience and feel like they made a difference, the better for a number of reasons. At the same time, he acknowledged how difficult it is politically and for other reasons to do something that is broader than the real mandate. In their discussions so far, people in the public engagement field have asked why the vaccine issue cannot be about children's health. There are several reasons why vaccines become more difficult as a public engagement topic.

Ms. Davis asked Ms. Friedensen if she would be willing to discuss some of NASA's fears and resistance in pulling this together, since NASA has been trying to pull together the dialogue for several years.

Ms. Friedensen explained that this is a new project, a new way of communicating, and a new way of forming relationships for NASA. They have done an intensive internal education campaign, and they hope to do an external education campaign. They are engaging their different managers and decision-makers internal to NASA, so that they really understand what the public's concerns are and that they have to change their program to address those concerns as much as they can. The first concern is that they cannot tell us that we cannot go to space, but what can they do that it is genuine and responsive. They work very hard to listen to their internal decision-makers in order to pique that genuine moment of public decision making available to us. They have intensive programs. In fact, last week, she had several meetings with legal council in which they were bringing in all of the FACA rules, the government advisory rules, and everything else into the room to discuss how to talk to the public about how nuclear electric engines are going to work.

Ms. Katz noted that Ms. Friedensen emphasized that the public cannot tell NASA not to go to space or explore certain areas. She suspects that one of the issues people will want to explore is whether vaccines should be mandatory, which they are by state law. There are some people who would argue that they should not be. If they truly want to engage the public, one of the issues is that they do not begin the process with saying that people cannot tell us not to give vaccines.

Ms. Friedensen responded that it depends on how they set the agenda and the dialogue. However, she agreed with Ms. Katz. She noted that DHHS has a regulatory responsibility in ensuring immunization. NASA is non-regulatory and voluntary and really relies on the public's input. They have the mandate to explore space, but compared to vaccines, wars, and the economy, how important is this mandate?

Ms. Landry asked if the participants and stakeholders have been chosen for the dialogue and whether this list was public.

Ms. Friedensen responded that they have been selected, but are not public yet. There are essentially five types of organizations: environmental, peace and justice, space, representatives of the scientific community, education, and thought leaders. They have 19 primary and 17 backup candidates, representing a diversity of organizations.

Ms. Landry asked if they were happy with turnout and if they got the representation they were hoping.

Ms. Friedensen noted that there were a couple of organizations that she would personally like to draw in later. One is too busy and does not have enough money and the other is leery. Some organizations do not have enough resources, even when NASA is paying for the costs for travel and other expenses. NASA wanted this organization involved because they had brought a lawsuit. The organization was interested but does not have the resources. The other organization declined for political reasons and did not want to be co-opted and fed propaganda by NASA.

Ms. Landry asked if there was anyone from the general public selected.

Ms. Friedensen explained that they have struggled with this issue as well. Each meeting will include time for public comment since they are FACA-chartered. They also view all of the non-federal organizations as representing the public. She believes that the ground rules also allow asking for distinctions between a person's "representative hat" versus their "own hat." They also got intensive involvement from the launch-area community near Kennedy Space Center. Several organizations are from that area, and there are two elected officials from the county government.

Dr. Helms noted that with Keystone's help, they determined that over the last 10 years, there has not been much change in public acceptance of NASA's performance and mission.

Ms. Friedensen clarified that Keystone identified message and the key concerns, and NASA knows from past experience that these concerns have not changed. Keystone did not derive this determination.

Dr. Helms asked what is driving NASA's efforts. It cannot be to change the way people perceive the agency if the perceptions have not changed. Ms. Friedensen explained that the key concerns have not changed, which means they have not addressed them.

Dr. Helms asked if the goal is to change these perceptions. Ms. Friedensen explained they want to do something different-such as their risk assessments, modeling, and testing procedures-with the hope of generating change in these perceptions.

Dr. Helms noted that for NVAC, to get involved in this would be to affect the long-standing concern over individual rights and immunization. As far as they know, that is a major issue.

Ms. Friedensen noted that one of her coworkers was writing some bullets for her and wanted substantive milestones every six months for her program, which is like asking for social change every six months. Mapping the milestones of social change is an issue. There is the acceptance of vaccines and the challenge of integrating the message into the media each flu season. It is very complex.

Referring to Dr. Helms' comments, Ms. Wexler noted that one of the things they have not done is survey the American population on a regular basis, as NASA has, to ask what people think of the U.S. vaccination program. A random, across-the-board survey could ask what they like and do not like about the program, whether they are neutral, what they have done, and what their concerns are. They would have a list every two years that will tell them the public's concerns and what could be improved. It would be useful to get some baseline information so that they would know the concerns across the nation, rather than just hearing from certain interest groups that they hear from all of the time.

Dr. Gellin added that the British have been doing this for about 10 years with waves of surveys twice a year. It is pretty elaborate and is something about which NVAC should learn more. They have not published much; it is mainly for internal guidance on a number of things, including services or programs and some of the broader issues, such as the philosophical concerns people have about vaccines. It is mostly oriented towards making changes on the delivery end. They ask a broad range of questions and they have a methodology with individual surveys with 1,000 mothers per waves. In the United States, they have scattered surveys that they pull together but nothing with a consistent methodology over time.

Ms. Friedensen offered to provide the citations. Her own studies in risk communication involve not only identifying the concerns, but also what generated the concern. With vaccines, there are the media scares and personal influence. For something that has never happened, such as an American accident of a space nuclear power supply, it is how you identify the concern and to what it relates. In the United States, it relates to environmental issues, such as Three-Mile Island, Chernobyl, and the scare movies. With post-9/11, it relates to potential terrorist attacks. People are also still worried about the unknown nature of radioactive materials. They do not understand it and NASA's analogies are not very effective. They do not have the idea of dread, harm, and no choice. In their messages, they are really looking at what generates trust and understanding of liability and how to get consent (e.g., What would it take for you to say yes to me?). They are basing their approach on the work of Paul Slovic, Nick Pidgeon in England, and Howard Kunreuther.

Day 2

Welcome

Dr. Charles M. Helms

Dr. Helms noted that he is a professor of medicine at the University of Iowa and the Chair of NVAC. He is taking the place of Ruth Katz, who is overseeing a public health policy debate at George Washington University. Ms. Katz is expected to arrive by the afternoon. The goal is to be done by 2:30 pm. After 2:30 pm, the NVAC working group will be meeting to discuss the findings of this meeting and to give guidance to Ms. Landry and her staff about the report that will be presented at the next NVAC meeting. There will also be an adjunct federal agency discussion.

National Institute of Allergy and Infectious Diseases/NIH, Division of AIDS

Rona Siskind

Ms. Siskind noted that she is with the Division of AIDS (DAIDS) at the National Institute of Allergy and Infectious Diseases at NIH, and that she will be discussing how DAIDS involves the community in its research process. She began by providing a history of when the community first became involved in their research effort. AIDS first hit in the early 1980s and NIH responded immediately by looking at the causes of AIDS and the development of new treatments. It was not until 1986 that the Division of AIDS was officially established. The Division's mission is to:

• Support research to increase understanding about HIV
• Support the development of prevention strategies, including HIV vaccines (highest priority), as well as non-vaccine prevention strategies, such as topical microbicides
• Support the development and discovery of new treatments for HIV and its complications

The AIDS Clinical Trials Group (ACTG) was established in 1987 and was the first national multi-center research network devoted to the development and evaluation of HIV/AIDS treatments. However, the pace was slow and there was only one treatment available for HIV and not enough for the infections associated with it. There was an under-representation of women and minorities in the clinical studies. As a result, the activists in May 1990 stormed NIH. Dr. Fauci, the Director of the National Institute of Allergy and Infectious Diseases, began meeting with these activists to determine a way they could be involved without hindering the research process.

By the end of 1990, community representatives were invited to attend the eighth ACTG meeting-before this, activists were showing up at ACTG meetings uninvited. This was the formation of the Community Constituency Group (CCG), consisting of 22 national representatives. Within a year, CCG began participating as members on protocol teams (to develop studies) and scientific committees (to develop research agenda). The key lesson learned was that you could either include the community and hear from them directly, or exclude them from the process and hear from them indirectly.

There are three types of communities: the investigator/scientific community, the industry, and the affected community. The affected community is very broad. It includes those infected with HIV, at-risk populations, advocates, and current and former trial participants. Additionally, there are other ways to segment this community-either they are hard to reach or they have special needs. These include children or youth, pregnant women, older adults, injection drug users, racial/ethnic minority communities, etc. There are also domestic and international communities.

The overall goal of including the community is to build trust and acceptance of the research agenda. In order to build trust, they ensure that there is an exchange of information between DAIDS, researchers, and the community, and that the study participants and the community have a voice. They also share research results with the community and allow for a community role in the development of the research agenda and specific studies. Ultimately, this will encourage trial participation.

There are several ways DAIDS involves the community. There are advisory committees on scientific and outreach and education issues. There are consultations, town meetings, and community advisory boards. As advisors, the community does not have a formal role or legal authority to require changes to a program or trial. They are providing advice and the ultimate decisions rest with DAIDS and NIAID.

Advisory committees

The advisory committees (scientific and outreach/education) fall under FACA guidelines. At the institute level, the National Advisory Allergy and Infectious Disease Council advises the director of NIAID. The AIDS Research Advisory Committee (ARAC) focuses on the research conducted by DAIDS. ARAC was established by law in 1988 and it was designed to advise DAIDS on all aspect of its research, including progress and productivity, gaps in research portfolio, and approving new programs and funding levels. There are 13 voting members, of whom two are public representatives.

Another committee is the HIV Vaccine Communications Steering Group, which does not operate under FACA guidelines. It was established six years ago to guide DAIDS on all of its communications strategies to educate the public about HIV vaccine research, targeting specifically racial and ethnic minority groups. The goal of the group is to create a supportive environment for future vaccine studies. Its membership includes vaccine advocates, community-based organizations, industry, government, and communications specialists. The steering group has been successful in identifying knowledge and attitudes about HIV vaccine research. They did a survey of the general public involving 2,000 individuals, as well as 500 individuals from each of the targeted communities. They have also broadened public outreach through the HIV Vaccine Awareness Day and by funding community-based and national organizations.

The lessons learned from these advisory committees include:

• They need a clearly defined mission, purpose, and role
• There needs to be a timely exchange of information
• There is a limited ability to represent all sectors of the affected community and all aspects of HIV/AIDS research and science
• There needs to be representation from targeted communities
• The engagement needs to be meaningful

Consultations and town meetings

The DAIDS is going through a process of reorganizing seven of its major clinical research networks that includes HIV treatment research for adults and children, vaccine research, and prevention research. When starting the discussions, they knew that this would create a great deal of controversy within the scientific and other communities. As a result, they started talking to the leadership of their networks, scientists, and the community. They held over 30 community consultations. Some were at preplanned meetings and some were specifically planned as public participation meetings. They created a webpage that listed all of the consultations, included summaries and presentations, and provided a link for comments, suggestions, questions, and concerns.

The efforts were successful in soliciting a great deal of input from a broad range of individuals. What they learned was that it was important to include the community very early in the process and to provide regular feedback. They also learned that the community is larger than they think and that there is always a segment that believes it was left out.

Community advisory boards

The community advisory boards (CAB) are the foundation for community involvement. CABs are a group of volunteers from the general public and from the diverse communities affected by AIDS. They are organized to assist and advise the Director and DAIDS, as well as the researchers within a given network or site. The mission of CAB is to provide guidance on all aspects of HIV/AIDS research process, including setting the agenda, research related care, and program management.

CABs operating at the national or network level are called Community Constituency Groups (CCG) or National and Global Community Advisory Boards (NCAB or GCAB). Those at the local or site level are called Community Advisory Boards. The prototype network structure places CABs just under the Executive Committee level. They sit on each of the Scientific Committees, as well as on the Executive Committee.

For researchers, the value of CABs includes building trust, recruiting and retaining diverse populations, and increasing acceptance and awareness of their studies. For the community, CABs gives them the opportunity to be involved in science that affects their lives, to address ethical issues, and to ensure appropriate and culturally sensitive information and materials.

CABs are made up of representatives from the area in which the research is being conducted and representatives of the population involved in the research. Each national CAB has its own bylaws and its own structure. There are one or two representatives from each site within the network. They have a chair and co-chair, and a committee structure to address specific issues. They typically do business via monthly conference calls, annual retreats, and national meetings. They receive support through the operations office at each of the networks. At the site-level, CABs hold monthly or bi-monthly meetings.

The CABs are the link between researchers and the broader community. At the national level, they can help establish the research agenda by sitting on the Scientific Committees. They participate in study development, review informed consents, review educational materials, and serve as spokespeople.

CAB members were asked if they if they had any impact on decisions. One of the responses noted that the input of the community tends to reduce the number of number of clinic visits and blood draws, which increases participation and compliance. Another commented that the trial inclusion criteria were changed to allow participation of people with diabetes. Another was able to change a questionnaire to include a behavior of injection drug users of which researchers were unaware (i.e., renting needles versus sharing them).

At the local level, CABs can advice the study staff and the principal investigator on all on all aspects that affect the conduct of a clinical trial. They can let the investigator know if there is interest in a study. They also provide input on operational issues, including transportation, childcare, and hours of operation. They can foster a supportive environment for trial participation and help provide legitimacy to the research site. They serve as a liaison between staff and volunteers.

Components of a successful CAB include having a diverse and representative membership of the population being studied. CAB members must have a commitment to learn and stay current about research processes and the science. There needs to be ongoing and sufficient support and clear mechanisms for information exchange. Meetings must be easily accessible and volunteers must feel appreciated.

There are many challenges as well. Since the community is not one voice, there are differing bases of knowledge, experiences, and expertise within the broader community. Inclusion of hard to reach populations can be difficult. As the research effort moves internationally, effectively including international participants is a challenge, due to time zone issues, language barriers, and cultural barriers. Additionally, sustaining a CAB over a period of time is a challenge.

The lessons learned include that the commitment of DAIDS and researchers is essential; there needs to be clear channels of communication between DAIDS, researchers, and CABs; and ongoing support and training is critical. This training and support includes not only the science involved, but also general board development, mission statement development, negotiating with investigators and study staff, and new member orientation. They also learned that all network and site CABs share common concerns, interests, needs, and goals. There is a real value in sharing ideas and pooling resources. There is also a role for non-network affiliated community representatives.

To address these challenges and lessons learned, they have held many cross-CAB activities and trainings. They have tried to facilitate information exchange across CABs. They have done a number of trainings to look at the science, CAB development, and other areas of interest. They have also conducted outreach at conferences to share information on CABs and how people can get involved.

As they reorganize all DAIDS funded clinical research networks, community involvement will remain a critical component. They are also looking to create a new committee, Community Partners, which will be an overarching committee that would transcend all of the networks. Community Partners may include representatives from each CAB, as well as non-network affiliated community representatives.

In summary, you can either include the community and hear from them directly, or exclude them from the process and hear from them indirectly. Before community involvement, the activists were protesting NIH. Now, they are cooperating with DAIDS.

Questions

Dr. Helms asked if DAIDS has a specific budget for CABs and how it is set.

Ms. Siskind commented that DAIDS does not have a specific budget for CABs. DAIDS funds each individual research network and requires the network, in their applications, to propose how they will involve the community. At the network level, they have staff assigned to support the CAB. The degree of support varies and may include education, outreach, and training or may include someone to set up conference calls. She does not have a figure for the national level, but noted that it is probably done at a broad range. It is also a range at a site level, because it depends on how committed the principal investigator is. On average, it may cost $60 per meeting for a given site, mostly for transportation and food. Internationally, site budgets are greater to include travel to the Untied States.

Dr. Helms asked if there is research that is outside of networks and if they are required to have a CAB process as well.

Ms. Siskind noted that there is research outside of the networks. These are individual grants and they are not required to have CABs. However, a medical institution may have some type of community input.

Ms. Koslap-Petraco asked how they recruit a broad spectrum of the community.

Ms. Siskind noted that recruitment is an ongoing challenge. At the site level, it begins with trial participants, family members, partners, and friends. As they come in for a study, a staff member will talk to them about CABs. Then there is outreach into the broader community at every conference, including a brochure that describes CABs.

Mr. Leighninger asked how the consultations are structured. He commented that NAIDS has both temporary opportunities (consultations) and permanent structures (CABs) for community involvement. He asked if there is a connection between these two groups.

Ms. Siskind answered that in the reorganization process, there was a great deal of overlap. Many of the consultations were held in conjunction with a national meeting. For example, if the AIDS Clinical Trials group was meeting and soon there after the Pediatric AIDS Clinical Trials group was meeting, a consultation may be held between the two meetings to make sure individuals from both groups participated, as well as CAB members and other interested parties. In terms of the success of the consultations, it is important to make sure that there is enough time for input and discussion.

Dr. Vernon commented that the purpose of these CABs is to offset angry demonstrators by giving them a sense that they are included. Beyond that, the primary purpose is the facilitation of a research agenda-to make sure the research agenda is accepted in the community- and to recruit participants. However, how are the CAB members involved in public policy decisions, which involve the larger public values that we hold?

Ms. Siskind noted that Dr. Vernon is asking how they are involved in more than just setting the research agenda. She added that there are different levels of input. Consultations and input from specific solicitations are entrées into DAIDS and NIAID. As policy decisions are being made, the community knows that they can provide input.

Ms. Landry commented that she and Ms. Siskind have worked together before. She added that the parallel track program was developed by community interest and CABs. This led to changes in how drugs were approved and licensed, allowing drugs to be used much earlier.

Ms. Siskind added the community has an active role with the Office of AIDS Research (OAR), which is at NIH and not within NIAID. For example, there was controversy as to whether DAIDS was soliciting enough input in their consultations, which prompted OAR to hold its own working group to solicit more input about the reorganization.

Dr. Vernon asked how well the online feedback on their website worked, relative to what was learned in the community meetings.

Ms. Siskind replied that it worked extremely well, especially to capture input when there was not enough time at the consultations. It also provides a good venue for long narratives. As a result, they received a lot more feedback and constructive suggestions. Many also pooled their ideas and submitted comments as one voice. It also gave DAIDS an opportunity to compile and study the comments.

Ms. Gunn noted that an important aspect of working with a community is its ability to see agency officials as people, rather than faceless bureaucrats.

Ms. Siskind added that before, activists were storming NIH. Now, they know that they have access to Dr. Fauci and other routes to express their opinions. Though there is not that animosity anymore, there is not always agreement. However, there is at least a way to channel it.

National Institutes of Health

Jan Hedetniemi

Ms. Hedetniemi noted that she would be talking about an agency model of public involvement and that she would not be using slides. When she became the Director of the Office of Community Liaison at NIH, there was a great deal of public outcry against the agency. Unlike the former director of facilities, she purposely never attended meetings with an entourage because she did not want to overwhelm the community members. She noted that she would be talking about advocating personalization (e.g., becoming familiar with people and faces, picking up the phone, etc.) in terms of the model.

In 1993, Congress announced its intent to double the NIH budget over the course of the next decade. The Bethesda campus of NIH was drastically in need of expansion and renewal. However, NIH is located in the heart of one of the most sophisticated communities in the state of Maryland, bounded on three sides by homes, by a young women's parochial school, and the Bethesda Naval Center. It was dedicated in 1939 by President Roosevelt on the steps of Building 1. NIH and the community ignored each other and had no interest in what the other did. NIH maintained that the homes in Bethesda community were elevated because NIH gave stature to the community, and the community saw NIH as just something to walk across and drive around.

This changed in 1994 when NIH found itself in a hot bed of controversy. NIH was operating under the same master plan since 1972. Master planning in the Federal triangle requires approval from the National Capital Planning Commission (NCPC). NCPC had to give its approval before NIH could build a new building or do renovation. NCPC became impatient with NIH and told NIH they needed a new master plan because they would no longer do things piecemeal. NIH therefore developed a master plan. The community almost succeeded in blocking NIH from doing any construction (e.g., building laboratories, parking garages, utility improvements) on the campus with this master plan. In other words, a group of organized residents almost stopped NIH's growth. They managed to do this with a voice, a reason, and fear, and by provoking anxiety and distrust among the community. The community engaged the efforts of a very zealous local reporter who published stories in the local newspaper about NIH poisoning its neighbors out the back door while trying to advocate public health measures through the front.

Ms. Hedetniemi noted that the NIH director at the time was very aggressive and ambitious, and the master plan was one of the things she wanted to leave as her imprint. When the master plan was developed, it was massive. It proposed bringing everyone from the outlying buildings in the community onto the campus, which would essentially double the population on the NIH campus. There would in turn be increased traffic, growth, and congestion. She wanted to level the existing Clinical Research Center (the largest building on the east coast) and rebuild it. She proposed to have essentially fence-to-fence dense buildings, which would have been an eyesore and a nightmare in terms of noise, pollution, and environmental management. In addition, she only gave the community one month to approve the plan, and this incensed them immeasurably.

She commented that this may have gone unnoticed, except for the fact that the master plan would have allowed NIH to retrofit three incinerators on the campus to burn medical and pathological waste, along with other things. The Dickerson incinerator in Montgomery County was a hot spot of great protest because of its emissions of dioxins and other harmful gases. Environmentalists were looking for reasons to fight this incinerator and polled major federal and hospital units in Montgomery County. They found that every one of them (e.g., Navy, Suburban, Walter Reed) had shut down their incinerators in favor of more modern technologies, except for NIH. Environmentalists appeared and NIH was forced to take notice that there was a community around it. Congress got involved and NIH was questioned by the appropriations committee about its public attitude and corporate behavior. As a result, NIH went through a transition and did some things that further enervated this discussion. NIH built the Natcher Conference Center, the construction of which was hastened because they wanted to build it while Natcher was still alive. In addition, part of the appropriation was for a parking garage that would be within 100 feet of one of the most tranquil neighborhoods in the area. People complained that the garage would create unnecessary lighting in their patios and noise, and there were pictures of people lying down in the path of bulldozers.

It was into this type of unruly, non-defined controversy that Dr. Harold Varmus assumed leadership of NIH. Dr. Varmus was a Nobel laureate from New York by way of California who rode a bicycle to work and donned a suit only when absolutely necessary. He was a firm believer in talking to people when there is a problem and made the unprecedented step of talking to their neighbors. Before this, no NIH director had ever met with any member of the community. Dr. Varmus not only met with the community-he agreed with them. He said he would stop incineration, test the water and soil on the campus for contamination, and make public all reports on safety.

Dr. Varmus scrapped the master plan and started over again with the involvement of the community. He also started a program of community relations and public outreach. Ms. Hedetniemi noted that she was appointed to head this office. Her mandate was to engage the community in such a manner to gain support of the master plan, maximize public participation in the process, and find a means by which to inform and educate the public about NIH, its mission, and its value. She noted that she had to overcome a lot of anger, distrust, and mistrust, internally and externally. NIH, as a corporate agency, had little interest in dealing with its community, its neighbors, the county, or the state-they were not funded by them and it made no difference to them if NIH was awarded big appropriations.

NIH took a leap of faith into a corporate behavior of public outreach and involvement. They began with the obvious thing-meeting with the community. They invited all who were interested and willing to participate in an effort to redo the NIH master plan. She sent out 75 letters with no idea of what she would get back. They engaged a community group called the Working Group on the Community Master Plan, which included about 50 people. They muted the yelling behavior by making it a formalized process-there were notebooks and agendas, and the community sat at the table and NIH sat around the room. The community set the agenda and decided the topics of discussion at the next meeting. NIH provided them with briefings and written documents prepared by architects, engineers, and scientists. They were also given background information so that when they were asked to concur in a recommendation, they were informed. NIH listened more than they talked. There were minutes, action items, and subgroups that took on specific topics-environmental management, transportation management, and adjacency issues. Adjacency issues included walkways, fences, grounds keeping, etc.

As a deliberate measure, NIH operated based on consensus. Ms. Hedetniemi commented that consensus is often mistakenly misinterpreted as unanimity. Consensus is not unanimity nor is it reached through voting-it is achieved through the discussion and acceptance of general will.

The community liaison model established at NIH continues today, and the core of this model is the Community Liaison Council, which is the outgrowth of the Working Group on the Master Plan. The Council provides NIH with comment and concurrence on all master plan activities, and also serves as a mediator between NIH and the community on issues of concern. The Council consists of 35 community association representatives from communities throughout Bethesda and Montgomery County, as well as liaison representation from the NCPC, the Maryland National Capital Parks and Planning Commission (MNCPPC), the local government services center, the Office of the County Executive, and other groups that have interest in NIH (e.g., YMCA, Suburban Hospital). Representatives of the Maryland State Legislature and Congressional staff were often in attendance as well. The Council meets the third Thursday of every month except for August. It has an executive committee that meets in between regular meetings to identify agenda items and to discuss information and speakers needed for the next meeting.

There are standing subgroups-transportation, soil testing, etc. The Council and the subgroups are co-chaired by an NIH representative and a community member. The current Director of Community Liaison at NIH co-chairs the Community Liaison Council with a community member.

Ms. Hedetniemi shared other lessons learned when dealing with the public on sensitive or highly charged issues. First, when communities and the public are angry, they should not be ignored or isolated, and a power play will not make them go away. Facing and engaging the public in discussion and conversation is the only way to begin negotiations. She noted that she always wants to include people who are against NIH in every community dialogue on the principle that she would rather have her enemies where she could see them than be blind-sided by not knowing what they are doing.

Second, it is a mistake to assume that the loudest voices represent the public. Broadening the base of community involvement usually presents a spectrum of attitudes, from the most angry to the modestly interested, providing balance between the ranges of positions. The Community Liaison Council was a community-driven group in that the community found its own way to the table. NIH issued a broad invitation and did not deny access to anyone interested in participating. Ms. Hedetniemi noted that she found that by not denying people the opportunity to participate, a major hurdle to dealing with angry people was eliminated. She noted that the loudest protestor often loses interest when there is no barrier and others can begin to engage in the process. The protestor is sometimes more interested in the sound bite than the hard work.

Third, the more voices representing the spectrum of attitude, the less likely it is that the loudest voices will dominate, allowing for space to insert good information and negotiation points.

Ms. Hedetniemi added that, as Ms. Siskind noted earlier, public outreach and public participation cannot be confined to an office. You have to go out to the community and participate where they are active-take the program to them and show interest in what they see. She noted that she went up in a local news helicopter during rush hour to see the traffic patterns around NIH so that they could better manage their own traffic. She was on the rooftop of the National Library of Medicine to make sure that the placement of a satellite dish would not be obtrusive to the view of people in local high-rises. She was under the eaves of the NIH power plant to test emissions because of a lawyer who e-mailed and wrote letters every time something was emitted. She noted that it was well worth the heat because there was nothing the lawyer could contest. She added that she was able to have the story printed in the local newspaper and people began to understand there was nothing to his protests.

She noted that when you have a community council, you cannot be accused of hiding things. They had a website, sent out newsletters to thousands of people, made safety records public, and an environmental reading room accessible to the community.

She noted that she found that using media as education rather than public relations worked well. They also institutionalized the process of the Community Liaison Council. It is now ongoing and has been around for almost 10 years. It still meets and is a means by which things can be resolved.

Ms. Hedetniemi discussed the NIAID laboratory that will house a BSL 3 facility, over which there was much controversy. It was the Council that served to inform, educate, and assist the community in dealing with the perceived fear. She noted that in the mid-1990s when NIH opened a BSL 4 laboratory, it was entirely run through the Council. The same over-zealous lawyer mentioned earlier talked about how the laboratory was capable of studying anthrax, Ebola, etc. even though it was only to be used for multi-drug resistant tuberculosis. Before the laboratory was opened, there were tours and educational forums for people.

Today, the community liaison model established by NIH is being used in different ways. She noted that she wanted to amend something that Ms. Siskind discussed. NIAID was given funding to build BSL 3 and BSL 4 laboratories as part of the National Bio-Defense Research agenda, recognizing that building a BSL 4 in a community could be fraught with concern. As part of the applications process, applicants had to submit a community relations plan identifying: 1) its intent to involve and engage the community as it planned, implemented, and built the facility; 2) how it was going to manage transport of agents to and from the facility; 3) how safety and security were going to be addressed so the community could feel that there were no environmental dangers presented by the facility; 4) how the community would be involved in the ongoing process about the reason for BSL 4 and BSL 3 laboratories in bio-defense research.

The model has also been adapted to the Fort Detrick facility, where a national bio-defense campus is being built, and at the Rocky Mount laboratories when they began to design and construct a BSL 4 facility. It is now being used as an offset to community angst and concern with regard to bio-defense. The model has proven its worth. The master plan was approved and NIH has continued credibility with its efforts to engage the community in health forums and other issues. She noted that in her retirement, she has found that people continue to cite the kinds of things NIH did as uniquely posturing a Federal agency into a public-private partnership that works and has some adaptability.

Questions

Ms. Wexler commented that Ms. Hedetniemi appears to have done a remarkable job with the process of engaging the community. She noted that NIH had a local problem that was not being addressed with the neighbors and NIH continued to use them as an advisory board. Ms. Wexler asked if the group has, in any way, been used to address issues that have nothing to do with the neighborhood (e.g., research plan).

Ms. Hedetniemi responded that the group has gone beyond the communities of Bethesda and Montgomery County. At times, NIH is asked to talk to the Maryland State Legislature about what NIH is doing and how it is potentially able to help other agencies within the state with outreach and education-the kinds of things becoming more and more prevalent where community groups are formed to assist in clinical trials and convey research results at the ground level. The Council is the point of credibility for NCPC. NCPC will not act on a building in NIH's master plan unless they can be assured that the community concurs with it. They accept testimony from the community. NIH took the operating principle that, if they were going to build a laboratory, the Council had to understand its purpose. Annually, the Deputy Director of NIH presents the agency's budget and research emphases to the Council. The new Clinical Research Center was designed from the ground up with the Council-they helped select the architect, do the design, concurred in the management of traffic, etc. Indirectly, they had an impact on the types of research that can be conducted on the community. Without them, NIH would not have the facilities to conduct the research. It does not have the full array of community involvement at the grass roots level that Ms. Siskind had described-it is a different model.

Ms. Landry asked how the group sustains their energy between the master plan and the TB lab and what other issues.

Ms. Hedetniemi responded that NIH is a dynamic campus and there is always something going on. She then told a story about the type of thing that energized the group. When they were designing the Clinical Research Center, some trees had to be cut down. There was a tree with a yellow ribbon around it. A neighbor who walked her dog on campus (before 9/11) decided that the tree had a history of its own-that it was there prior to the nation's founding, that it played a role in the Civil War, how Indians rested under it, etc. She engaged a local reporter in a campaign to save the tree on the premise that NIH was a bad neighbor that was cutting down all green things. NIH has a policy of replacing anything it takes down with one to two new things. The clinical research plan had been approved by the NCPC and the community council. It was after this approval that these individuals went to the NCPC with the road design that the tree was being taken down for, and the NCPC voted against the design of road until NIH could justify taking down the tree. NIH spent $10,000 to show that saving the tree would have meant constructing a road so dangerous that emergency vehicles could not travel it. Furthermore, the Clinical Research Center would have to be redesigned to accommodate the tree. The Council was offended that someone had come from nowhere and second-guessed them on their ideas and on their votes. The Council testified on the behalf of NIH to the NCPC and it was agreed that the tree would come down. When the tree was cut down, it was found only to be 180 years old. The tree was the subject of so much publicity, that the Navy asked for the wood to use it retrofit and renovate the U.S. Constitution in Boston. People from a mill in Rock Creek contacted her to get wood. Because the Council had vested itself in the process of this project, they became NIH's advocate against the outlier. The woman has become one of the most active volunteers in the Clinical Research Center.

Dr. Helms noted that he is from the state of Iowa and teaches at the University of Iowa, and that, like other colleges, they periodically run into problems with the local residents. He added that over time, everybody realizes that it is critical to maintain reasonable, good relations with the community in semi-formal structures. He commented that the story Ms. Hedetniemi told was successful, but perhaps they should have realized that it was going to be a controversial issue and addressed it ahead of time. He commented though, that NIH did a great job of damage control in turning the situation around.

Dr. Vernon noted that he was trying to draw a parallel as best as possible to the types of issues needed to be addressed in the vaccination and immunization community. He asked how NIH addresses situations where a group of citizens has concerns about particular high-level NIH policies or practices (e.g., the use of laboratory mice and rats, stem cell research).

Ms. Hedetniemi responded that there are several ways that NIH does this. Typically, it is done through the individual institutes, which advocate biomedical science in general through their professional meetings, grantee information, outreach, and through programs such as the one described by Ms. Siskind. She noted, however, that these types of issues are usually addressed through the press. They try to get information to the public through the press and communications in media to offset misinformation. For example, CDC and NIH got confused in a medical program called Medical Investigation. Tom Shales, a movie critic, thought it was terrible. Dr. Varmus wrote a three-page piece in the New York Times talking about the fact that NIH has no helicopter and does not break into a person's house to take a swab. He noted that this was a missed opportunity by the networks to help advance what biomedical science is all about. Rather than erroneously attributing CDC's role of epidemiology to NIH, they could have taken the opportunity to help people understand the importance of, for example, immunology, vaccinations, preventive health care, etc. There is no real working model to take community outreach and formalize it in processes that address advocacy for these larger issues (e.g., stem cell research) except in the institute models where there are community groups. Bigger issues are addressed through press relations and press briefings.

FACA Considerations

Emily Marcus Levine

Ms. Levine noted that FACA had been mentioned several times during the meeting, and her role is to briefly explain what FACA is and plant the seed that this will be an issue. She will be working with Dr. Gellin, Ms. Landry, and others at NVPO whenever there is a recommended proposed solution or a new way of dealing with the public. She will be looking at the implications for FACA and the agency, since this is an agency decision and these are issues the agency will have to deal with in evaluating the different proposals.

FACA is the Federal Advisory Committee Act, which was originally passed in 1972. It was part of the openness in government initiative to make the government's deliberations open to public. FACA allows various stakeholders to know what is happening when the government deliberates on a specific issue. FACA created special rules for the advisory committees subject to it. With limited exceptions, the meetings have to be open to the public and must be announced a certain number of days in advance in the Federal Register. There are also requirements for balance in the committee's membership in terms of race, age, and other similar characteristics, but also different viewpoints depending on the issue. Other requirements are imposed on the members in terms of conflict of interest and ethics. Committee members must be forthcoming about their other obligations so that the public is aware of any potential conflicts. NVAC is a FACA committee and subject to these requirements.

Any advisory group is subject to FACA, whether it is called an advisory committee, task force, a panel, or any other group that has at least one member that is not a federal employee. A group of federal employees who come together to meet on an issue is not subject to FACA, but as soon as an outside member of the public is invited, it may be. The wording in FACA refers to groups "established or utilized by a federal agency or the president." There is a lot of case law on this, and the Supreme Court has basically said that "utilize" means to be managed or controlled by the agency. This is a very fact-specific inquiry based on how the committee is composed and run, but it is a pretty broad reading. Some things that they may think do not show management or control have been identified by the Supreme Court as satisfying the requirement. The idea is that it is in the interest of obtaining advice or recommendations from the public. Yesterday, NASA discussed some of its dialogue with the public, which they have subjected to a FACA committee; this means they will be putting the group under all of these requirements.

There are many exceptions to FACA. Depending on where they want to go to have dialogue with the public, the group they compose may be subject to one of these exceptions. These exceptions include groups that are composed of certain state, local, and tribal officials; formed to exchange facts and information but not produce advice or recommendations; and created by non-federal entities and not managed or controlled by the government. The most important exception in the vaccine context is groups assembled to provide individual advice, where no consensus advice is sought or provided.

This does not mean that consensus would be required but that it is sought. If you bring a group of people together, as NVAC does, and then ask them to deliberate on an issue, mull it over, and then come back with a group recommendation to an agency official, then it is seeking consensus. However, if you get a diverse group of people from the population together, ask them for their input, and then tabulate the information somehow for the agency (e.g., 20% of the people thought X) without trying to reach consensus, then the group would not be subject to FACA. Based on Ms. Levine's knowledge of how contentious the vaccine issues can be, it would be difficult to get consensus on these issues. There could be a way to ask all the members of the public and all of the groups for their opinions on consensus advice.

FACA can be important, especially with controversial issues. There are groups that file lawsuits against the government when they feel something should have been subject to FACA but was not. Some judges have ruled in the favor of these groups, invalidating the decisions made by the committee. Everyone wants to avoid this situation. Even if they develop an approach that is not subject to FACA, they still may want to think through the requirements and voluntarily adopt them as good public policy. These requirements include having meetings open to the public, providing people with the minutes, announcing meetings in advance, and balancing the membership. There are several other legal issues that may come up for the committee, but she will work with NVPO to address them.

Questions

Dr. Johnson noted that he has heard about periodic or one-time consultations from various groups. If CDC or NVPO were to set up a one-time consultation process with, for example, a large representative sample of people across the country, would this be subject to FACA?

Ms. Levine replied that keeping it to one time does not exclude it and the other requirements would need to be reviewed. There could be an argument that a one-time meeting means there is less management and control, but it could still be subject to FACA, depending on how it is organized.

Ms. Gunn noted that FACA seems to be open to interpretation. She asked Ms. Levine if there are resources that she would recommend to stay informed about the most recent developments. Also, are there any cross-agency discussions on the interpretations, or is it mainly a discussion between the agencies and GAO?

Based on what she has heard, Ms. Levine explained that there are varying interpretations of and approaches to FACA. It is very different at each agency. Some are more conservative, while others are more liberal. Ultimately, it is up to the court to decide. DHHS has been more "middle-of-the-road" with FACA. She believes they have more FACA committees than any other agency, so they are very aware of the rules and requirements, particularly in the more controversial areas of public health. They want to be careful. In terms of any special resources, she will look into this, and if she finds anything, she will forward it to Ms. Landry for distribution.

Dr. Vernon asked, under the rules and requirements, if there is specific language on how public comment must be considered for existing FACA committees. Are there certain routines that committees, like ACIP, would be required to follow, and would there be monitoring to ensure they are being followed?

Ms. Levine explained that there are a lot of rules and most rules are procedural, such as keeping documents and making them accessible to the public or how votes are taken. Committee members can choose to weigh the information provided by the community.

Dr. Vernon asked if they would be required to have a certain percentage of a meeting devoted to commentary from the general public.

Ms. Marcus Levine replied no. They have to give the public an opportunity to comment, but different federal agencies do this in different ways. She has been to advisory committee meetings that are closed to the public throughout the meeting until the last 15 minutes for public comment. In others, the public is much more active. It is up to the chair of the committee and the federal official to decide how they want to run it. There must be an opportunity to be heard and to provide written comments.

Environmental Protection Agency (Superfund)

Doug Thompson (via teleconference)

Mr. Thompson apologized for not being able to attend in person and commented that the meeting sounded very interesting. He noted that he would try to compensate for this by including Jim Murphy, the community liaison in the Superfund program, who is knowledgeable about public involvement, and Ellie Tonkin, who runs the Alternative Dispute Resolution (ADR) program in the region. He noted that Ms. Tonkin was not physically with him, but was calling in from her vacation.

Mr. Thompson noted that they would be making some key observations about their experiences, which he hoped would be relevant to the purpose of the meeting, and then have a discussion.

Mr. Thompson began by noting some of their observations. He noted that they could perhaps categorize EPA's involvement in this issue in three basic contexts. First, EPA can have a catalyzing or convening role, where it is trying to involve or engage the public in some fashion. He pointed out that Mr. Murphy does this on the ground in the Superfund context. Second, EPA can be a participant or a stakeholder-they may or may not have a decision-making role, but they are not the architects for the process and are not running the process. Third, EPA may have the role of providing neutral assistance. This may be done directly, through the ADR program where they provide mediation or facilitation services. Other times, if the issue involved is large-scale or involves long-term work, they may help arrange for services from outside providers.

Mr. Thompson then discussed seven themes, lessons, or operating principles-ideas they have been able to draw from their experiences. First, they found it helpful to take a sort of diagnostic approach rather than a "cookbook" approach. This would involve looking at each situation individually and looking at their own interests as well as the interests of the other parties and stakeholders. They analyze whether the agency work or decision would need information from the public or whether they need to inform or educate the public. Often times, it is both, where information is flowing in both directions. Typically, public involvement issues are a mix of procedures, relationships, and substance. The underlying dynamics of the values of the parties, how much credibility the agency has or does not have, and what the power of the relationships is will vary. It is important to analyze these things.

The second theme is clarity about roles. EPA often convenes or participates in the public process without surrendering regulatory or decision-making authority. There have been misunderstandings about this. Sometimes people come in thinking that both the discussions and the ultimate decisions will be consensus-based, and this is not always the case. It is important to be as clear as possible up front with everyone involved about how things are going to work.

The third theme is to meet people where they are. This means understanding where people are coming from. He noted that it may be better to be restrained in terms of how much the agency will be able to help or a sales approach about what a great process they are about to get involved with-sometimes lower expectations are much easier to manage. Government agencies are often greeted with skepticism or mistrust, and coming in with a proselytizing attitude is not likely to be helpful.

The fourth theme is to try to have as common an information base as possible. At EPA, sometimes disputes are genuinely technical, while others are masquerading as technical. At some point in the process, there should be an orderly approach to having a common information base and then identifying points of consensus and looking for the points of departure so that there is at least understanding on those matters.

The fifth theme is that it is difficult to engage large segments of the public unless they are already aroused about the issues. Life is short, and absent the sense of immediacy, people are understandably reluctant to get involved. This is a vexing problem when the agency is trying to get people involved and is unable to get them to the table-not because they are resisting, but because they are apathetic or have higher priorities. Mr. Thompson noted that he knew of no quick remedy for this.

Sixth, in small and large-scale processes, neutral assistance is very helpful in everything from getting logistical support to help in getting an accurate read from the parties. He emphasized that they are big supporters of getting neutral assistance in these endeavors.

The seventh and final theme is to make it in the interest of other parties to participate or at least make it easy to participate, to the extent possible. As an example, he discussed literally meeting people where they are. If you can go to the stakeholders, or pay for their travel, this may induce them to participate.

Mr. Murphy emphasized that for public participation, it is imperative to show how the process will affect them, their family, their interests, etc. Second, making it easier for people to participate is very important. This may be done simply by scheduling things when it is convenient for them. Multiple outreach meetings could be held in different places. Smaller focus group meetings could be held rather than large advisory board meetings. Food and beverages, child services, elderly care services, etc. could be provided. Finally, communication materials should have the appropriate level of information and could perhaps be presented in different levels and formats.

Ms. Tonkin noted that to the extent that much of this is common sense, she wanted to reinforce some things that she found surprising in terms of the magnitude of their importance. She discussed, with respect to expectations, the idea of under-promising and over-delivering. It is very difficult to go in the other direction. The more that can be offered by way of participation as the process goes along, the better. It is hard to offer less when you have given them the idea that more might be possible. Initial sessions involving just listening or exchanging information are a good way to start. It then becomes apparent that more might be possible by way of accommodating people's needs in government decision-making or information sharing.

Second, the persistent perception that the government is withholding information makes it very important to behave in ways that create an appearance (and are in fact a reality) of sharing information and making it accessible and understandable. They should also inquire as to whether the information is being offered in a useful form. Once trust has been established that information is not being withheld, it is remarkable how trusting people are about its sources and form. This has to be repeatedly proven.

Going to people and putting their own face on it is also important. She noted that she and Mr. Thompson were recently involved in a very technically-focused consensus process in Vermont, where a respected local professor ended up being a very visible person in establishing a tone of neutral inquiry on the technical sharing of information. They could go to meetings that people are already having (e.g., PTA, organizational meetings) and have information conveyed and meetings facilitated by local leaders.

Questions

Ms. Landry asked what the issue was that initially got EPA interested in community participation.

Mr. Murphy responded that typically, from his experience, EPA gets involved at the request of the community because there is a hazardous waste site in the area. EPA may also come into the area to inform the community that there may be a problem for which they are unaware.

Ms. Tonkin offered the perspective of the ADR program. When they first got involved in providing neutral assistance, almost all of EPA's involvement was crisis management. There was a public controversy where nobody knew what else to do, so a process was established with some type of neutral assistance. In the last 10 years, there has been a marked trend towards collaborative decision-making and conflict prevention. There has been a shift from about 90 percent of the work being after-the-fact crisis management to at least half now being public participation or collaborative and facilitated discussion. For example, 10 years ago, many community groups were outraged about emissions from asphalt plants, whereas more recently, they have been involved in things like consortiums of state and federal agencies in the New England area to address the problem of conflicting fish advisories. The range of possible approaches is very different. There is much less need to spend time at the outset on trust building and press issues. On the other had, in the crisis management situation, they initially attempt to get people to comfortably sit down at the table together. This is where neutral assistance has been very effective. Where trust is initially an issue, getting people to the table together has always been a means to overcome trust issues.

Dr. Helms asked, in terms of outcomes, whether there were any satisfaction measures or ways to measure the efficacy of what they are doing.

Mr. Thompson responded that there is not a lot of rigorous, quantitative data on that. There is their collective experience and anecdotal evidence in favor of that. One of the difficulties in evaluating these things is that you do not know what would have happened with an alternative course of action. As Ms. Tonkin alluded to, over the past 10 years, people have become more sophisticated about public involvement, and they have seen an increase in credibility and a decrease in mistrust. To the extent that the avoidable conflicts can be minimized, they are getting better. One downside is that these processes are very time-consuming and an issue to be considered would be to have EPA less as a convener of a process and perhaps just as a participant. Time is an issue. With respect to some of the larger EPA projects, there is a desire to participate in the collaborative processes, but there are only one or two FTEs for the entire region.

Ms. Tonkin added that when they first started doing this in the Superfund public controversy over remedy selection, they embarked on a number of very long-term, resource-intensive processes. They were, without exception, successful in managing the initial crisis, but they lasted a long time and it was not clear whether they were good at ending them at the right time. There was a real need to develop more short-term, surgical strike kinds of models. This is something that has happened. There is less of a presumption that any situation calls for an intensive process, but rather that a public involvement plan with the involvement of the public might be designed. This would involve minimal meetings. There should not be assumptions about all processes being so intensive and long-term. She noted that now that they have other models and that people are not immediately "freaked out" by the suggestion that public participation might be needed.

Mr. Thompson added that the U.S. Institute for Environmental Conflict Resolution has quite an interest in the question of looking at the efficacy of various public participation processes and processes involving neutral assistance, and would be a good resource for information on evaluation.

Ms. Koslap-Petraco asked for clarification on how to make the diagnosis of what the problem is that needs to be discussed.

Mr. Thompson responded that this involves a mix of art and science issues. This would involve listening very carefully in every possible way. What are we dealing with? What are our interests? What are the interests of the other party? How much is technical, legal, or political? What is the level of engagement or credibility or trust among the parties? Mistrust should to be addressed before technical discussions take place. There are structural issues (e.g., few or many parties) that should be considered. It is like doing an analysis, for example, in the ADR field referred to as conflict assessment or situation assessment. He noted that while these can be helpful, he cautioned that it is not always called for and that it depends on the situation.

Ms. Tonkin noted that this is one area where the experience of their program has pulled them in a different direction from some of the common thinking in the field, which involves full-blown, exhaustive series of interviews and written reports. There are times when this is appropriate and useful, but more often than not, what is really helpful is listening to what is going on and then working with whoever is engaged and whoever it appears should be engaged. In a more organic way, they should start fashioning ways for people to work together and do what needs to be done. This is quicker, more efficient, and more responsive to the needs of the parties. Large written reports and exhaustive series of interviews can be distracting. At the outset, people are not sure what the issues are and people are bogged down anticipating obstacles that do not emerge once people start talking to each other.

Ms. Hedetniemi added that another way to echo Ms. Tonkin's point is to say they pick their fights. Rather than respond to every jab in the newspaper or meetings, they wait for a proper forum to provide information to everyone. A tactic that an activist group might use would be to interview you, get your take on something, and then turn around and use your words to talk to someone else. Then suddenly there are conflicting responses about the same issue, which makes you look like you do not know about what you are talking. So, they pick their fights and wait for the right opportunity to get the information out in a responsive or informed atmosphere.

Ms. Tonkin noted that activist groups have tactics that work for them and objectives that have to do with certain types of publicity. Often times, the agencies involved assume that everyone should be invested in or interested in a particular kind of forum. However, it is unclear that that would be responsive to the needs of particular activist organizations. This is a reality that has often worked itself out when there is an incremental approach to bringing people together to talk to each other. Ground rules should be limited to accomplishing the short-term purpose if possible so everyone can become engaged. By preserving everybody's ability to use whatever approaches they need to use, it is possible to have the initial conversation from which others can be built in ways that protect the concerns of everyone involved. She noted that there have been times when they have been surprised by the way some organizations they thought would find it not in their interest to participate ended up participating because they found a safe way to figure out whether it was going to work for them or not.

Department of Defense

William Pehlivanian

Mr. Pehlivanian is the Deputy Program Manager for the Assembled Chemical Weapons Alternatives Program (ACWA), formerly the Assembled Chemical Weapons Assessment Program. In the early to mid-1980s, Congress mandated that chemical weapons be destroyed. There are currently eight chemical weapons stockpiles in the continental United States. After much research, the Army chose to use incineration as the method to destroy these weapons. Several incinerators were constructed, one of which was in operation out of the Johnson Atoll Chemical Agent Destruction System (JACADS). Anti-incineration activists convinced Congress to pass a law that required the Army and DoD to look at alternative technologies. Mr. Pehlivanian and his boss, Mike Parker (ACWA Program Manager), set out to do this.

They wanted to implement a transparent program, unlike the "decide, announce, defend" scenario that the Army has been accused of in the past, so they enlisted the support of a neutral third party, the Keystone Center. The Keystone Center convened a diverse group of people that had a lot of trust in the communities across the country and were well-known in the activism community to form what was called "the Dialogue" on ACWA. The Dialogue was formally established in June 1997 and consisted of approximately 32 representatives from across the United States, who were stakeholders representing the different stockpile communities. The diversity included people who were anti-incineration, as well as people who were strongly pro-incineration. The Keystone Center took the lead in developing what became the Dialogue. If the Army had taken the lead, it would have given the appearance that the Army was trying to bias the process. They did not want this-they wanted an open, transparent program. Even though the Army had to hire the Keystone Center and pay for meetings, administration, etc., everyone agreed that this was the appropriate process and everyone was happy with it.

Mr. Pehlivanian noted that one of the most important things about establishing this was that everyone was coming to the table with the same thought and no false expectations were created. Everyone knew that the ACWA Program Manager and the Deputy Undersecretary of Defense for Acquisition and Technology had the ultimate decision. This group was to give input at various decision-making points in the process of finding alternative technologies.

Once the Dialogue was formulated and everyone began building criteria by which to assess alternative technologies, it became evident that the technology was high above what they were used to dealing with. While there were PhDs and chemistry professors representing different locations, there were others who wanted expertise to help them understand the highly technical nature of what was going to be done. As a result, the Citizens' Advisory Technical Team (CATT) was formed. CATT included four Dialogue participants and a technical consulting firm. They signed confidentiality agreements allowing them to serve as the Dialogue's "watchdog" at many ACWA internal meetings, including procurement-sensitive information. CATT assured the Dialogue that ACWA was keeping to their word and following the criteria that was agreed upon and developed. They met 13 times over two and a half years and each meeting lasted two to four days. These meetings were scheduled when ACWA needed input on a decision that had to be made to move forward.

The Dialogue put together a goal before they moved forward. Mr. Pehlivanian commented that he was amazed at how long this took because there were varying views on what needed to be accomplished. The goal was to draw on a wide range of experiences, perspectives, and expertise in support of efforts to identify, demonstrate, and deploy safe, effective, and broadly acceptable methods of disposing chemical munitions and any resulting materials and waste streams. He noted that they spent six hours on the word "deploy"-it was not included in the law, but the people wanted to be assured that if an alternative was found, it would be implemented.

The general Dialogue objective was to solicit stakeholder input into the Congressionally mandated program at key decision-making points during the process. They were a part of every step in setting up the process and reviewing the technologies. When it came to a decision process before the Defense Acquisition Board (where the Undersecretary of Defense makes a decision), Dialogue representatives from the communities where the alternative was going to be deployed came to the meeting at the Pentagon to give their input to the Undersecretary.

Mr. Pehlivanian noted that CATT was integral in quickly moving the process along. Though they did not always agree, they were included in the process and a trust was built. As a result, when ACWA made a decision, they accepted it. He commented that this was interesting because usually when they say "no," it meant there would be a standoff. There was a lot of discussion about why certain things could not be done, but through education and explanations, they were able to move on. Although the Dialogue had to be repeatedly reminded that, by law, the program Manager for ACWA had the ultimate decision-making authority, they accepted it.

Mr. Pehlivanian then went over ACWA's accomplishments. The Program Evaluation Criteria development took time. They put out requests for proposals for industry to come in and bring their alternative technologies. They received 13 original proposals, which were more than they had expected. The criteria were used to cull the 13 proposals down to six, for which demonstrations of alternative technologies were completed at various test facilities across the country. CATT visited these sites and viewed the testing. They concluded with four viable technologies that could be deployed at any given facility or stockpile.

At the time, there were still nine facilities, most of which were constructed. There were incinerators at four facilities and some bulk sites (i.e., just agent and not weapons). Other sites had the agents in the weapons and this was the most dangerous scenario. The two remaining sites (Pueblo and Kentucky) had no facility constructed and when the law first came out, everyone thought it was for these particular sites. Because there were facilities built at the other sites, it was not economically feasible to deploy the alternative technologies there. Neutralization, followed by biotreatment, was selected for full-scale pilot testing at the Pueblo Chemical Depot and neutralization, followed by supercritical water oxidation, was selected for the Blue Grass Army Depot.

After these decisions were made, the original program was completed. They had completed the charter-alternative technologies were found and DoD made their decision about their deployment. ACWA was then going to turn the program over to the existing Program Manager who had been implementing the incineration and bulk destruction. However, the members of the Dialogue and stakeholders were so pleased with the process, they went to their representatives and another law was passed to have the existing ACWA program manage the new site-specific programs.

ACWA therefore went from assessing chemical weapons disposal technologies to implementing full-scale pilot testing of alternative technologies at these sites with the same staff. The focus shifted from a national level to site-specific public involvement in Pueblo and Kentucky, since these were the remaining stakeholders who had a voice in what was happening in their communities.

Each site (including the incineration sites), by law, has a Citizens' Advisory Commission (CAC), for which the governor appoints eight people, within a 50-mile radius of the stockpile, to watch the process. Mike Parker wanted to maintain the transparency they had at the national level and went to the communities and asked the for vehicle by which they wanted to work with ACWA in a transparent approach.

In Pueblo, they wanted to maintain their CAC, but noted that they might need to form subgroups as they developed an understanding of what was going to be done. Three subgroups were formed-the Public Involvement Working Group, the Permitting Working Group, and the Acceleration Options Working Group. The Public Involvement Working Group was formed to assess public involvement opportunities and ensure the community was informed. The Permitting Working Group was formed to expedite the planning, development, and implementation of the environmental permitting process, which typically takes three to seven years. They used an RD&D approach in which they phased the different types of state permits and got the first permit in seven months. In the shadow of 9/11, they received a mandate from the Undersecretary to look at accelerating the project, while keeping things as safe as possible. The Acceleration Options Working Group was formed to study the potential impacts of accelerating the disposal of chemical weapons. Three of the four acceleration methods were accepted.

In Blue Grass, the community did not feel that their CAC could do the job the way they wanted it done and wanted to set up an entirely new process. ACWA agreed because they did not want to have a "cookie-cutter" approach to things and wanted the community to be comfortable with the process. They established the Chemical Destruction Community Advisory Board (CDCAB). CDCAB was a larger group (about 20 members) than Pueblo's CAC, in part because while the stockpile is located in Richmond, Kentucky, it is very close to Berea, Kentucky. As a result, both communities were involved in the process. The objectives were very similar to those for Pueblo.

Mr. Pehlivanian then discussed lessons learned. He noted that public image is a byproduct of the trust that must be built from the ground up. He noted that Mike Parker received a prestigious award in San Francisco from the Sierra Club, which had been a very vocal detractor to the way the Army had been working with incineration. The award was for the process, not the technology.

With respect to education, the Army has used the term "outreach." Communities do not like the way the Army comes in, provides a large document, and tells people to contact them if they have any questions. Involvement is the key. While outreach is required, if people want to get involved, this should be facilitated. ACWA relied on the Dialogue to let them know how the communities were going to react to different decisions because this varies from community to community-education goes both ways.

With respect to ownership, if you have bought into a process and agreed to try to reach consensus, you own part of the product and want it to be the best it can. He commented that this was certainly the outcome to their program and the foundation by which they conduct their program.

Mr. Pehlivanian noted that including public participation can slow a program down early in the process. However, in the end, the ultimate decision comes more quickly and with much more ease. This was the case with the permits. He concluded with the idea that if you are willing to set up this process, you need to get everyone from the top down to commit to it. Keystone was big on this. If someone does not buy into it, there will be problems down the road.

Questions

Ms. Landry asked Mr. Pehlivanian to elaborate on his last point regarding the buy-in and whether he meant buying into the concept of the importance of public engagement or the actual process.

Mr. Pehlivanian responded that he was referring to buying into the product that comes out of the process, whatever it may turn out to be. He noted that the senior leadership and the Army wants to do things right. The public would not see this if they were not allowed to be a part of the process. The commitment is that whatever comes out of the process is what will be done, and this is what happened with ACWA. They found viable alternatives that were accepted by the community and the decisions were made to go with the alternatives as founded by the ACWA program.

Dr. Vernon commented that he was intrigued by the process in Kentucky, where they had a CAC they did not trust and had to form the CDCAB. He explained his interest in this because as the head of the Colorado State Health Department in the 1980s, there was a phenomenon where you had a body that was responsible for the oversight committee formed often of citizens, such as the Board of Health. The Board of Health was suspected of being bought in, so another level was formed to overlook the Board of Health. He asked why there was mistrust of the CAC.

Mr. Pehlivanian responded that perception becomes reality and that this was true throughout the entire process. He added that this is why it was so important to be very careful of the perception that was being created. The reverse of what happened in Pueblo happened in Kentucky, where the CAC membership had been around for many years. The Chemical Weapons Working Group (CWWG), a very enthusiastic, activist community against incineration, is headquartered in Berea, and the perception became that the CAC was too activism slanted. Mayors, city councils, and much of the public wanted them to be a part of it because they are nationally known and go to the stockpiles to fight against incineration. But because of the geography, the perception in the community was that they should have something more than the CAC-they wanted a more even, diverse group, and the CAC was too slanted to the environmentalist side. Mike Parker agreed because everyone was happy with the decision. Even the CWWG agreed that they should form a CDCAB, as long as they were included.

Vaccine Policy Analysis Collaborative (VPACE)

Mary Davis Hamlin

Ms. Hamlin noted that Dr. Roger Bernier began the VPACE project as a part of his sabbatical on enhancing public participation. With a large amount of research, Dr. Bernier began developing a process to enhance public engagement. He wrote the "Starting Point" proposal, which served as the initial framing of deliberations and was widely vetted before convening the Wingspread Group.

There are different ways to engage the public. At one end of the spectrum is public education. This is an important component, but is not conducive to public input. Many agencies are currently the stage of gathering input from the public, which tends to be a one-way process (e.g., public hearings, meeting comments, and surveys). The Wingspread Group looked at creating the capacity to collaborate.

There are two main groups of collaborative public participation. The first is ongoing, multi-interest stakeholder forums. Examples of these include NASA, ACWA, and the Citizen Advisory Boards. The second is general public deliberative processes. Examples of these include Study Circles, AmericaSpeaks, and consortiums. Some agencies have decided to move from input to having the option of collaboration.

Dr. Bernier's initial framing focused on the deliberative and collaborative process. He explored both stakeholder forums and deliberative public processes. He held ad hoc stakeholder forum to discuss the process of creating a collaborative process. The forum included key interests and collaboratively problem-solved to jointly develop the wisest and most supportable recommendations. At this point, The Keystone Center became involved as a facilitator for the Wingspread Group.

Keystone began by pulling together the key interests involved in the vaccine community. The Wingspread Group had more than 30 participants from supportive NGOs, vaccine concerned NGOs, neutral and general public NGOs, government agencies, health professional organizations, industry, advisory committees, academia, minority groups, and public participation professionals. The group was formed to satisfy issues of key interest, balance, and diversity.

There are five basic approaches to a collaborative process to minimize debate and conflict. First, you welcome the diversity of interests. Second, you educate and understand interests, rather than starting with solutions. Third, you jointly work to address the range of interests. Fourth, you jointly conduct fact-finding to inform deliberations. Finally, you build the most supportable and wisest agreements.

At this collaborative process, participants decide to come to the table, stay at the table, and agree to outcomes based on their perception of how the process meets their interests. There were procedural and substantive interests that affected the perceptions of the process. The procedural interests were:

• Is this the right group?
• Are there logistical issues?
• Am I willingness to sit across the table and collaborate with other participants?

The substantive interests included:

• Am I willing to talk about the issues on the table?
• Does the emerging agreement meet my needs?

Based on these concerns, two key interest groups withdrew. At this point, the remaining group must work to address the concerns or find other representatives. They must also decide whether the group has lost some legitimacy or credibility since two important groups have withdrawn. The remaining members of the Wingspread Group weighed their procedural and substantive interests, and decided to proceed with the process.

They decided to stay because they believed that there was a need to build a collaborative capacity to address:

• Complex and important issues with long-term consequences and unclear or competing authorities;
• Problems that are driven by values and require much consideration of opinion, beliefs, and convictions;
• Issues with a high level of controversy with active and mobilized groups of competing interests;
• Situations where parties are not communicating or communicating at a distance or through extreme positions.
• Need to build collaborative capacity to address. Need to do this to make good decisions. Be strategic.
• Issues where resource and technical expertise alone will not solve the problem;

Other core interests included:

• Increasing understanding of the full range of the public's interests and moving away from framing the issue in terms of extreme interest groups;
• Providing the ability to wrestle with the implications of the limitations of science and questions involving competing values and trade-offs;
• Creating a safe forum for honest communication among diverse interests;
• Building trust and constructive relationships among stakeholders;
• Supporting wise decisions that achieve buy-in from stakeholders;
• Supporting early effective input into decisions;
• Supporting informed deliberations regarding vaccine safety;
• Supporting commonly held values around health and safety;
• Supporting continuous improvement of the national public health vaccine program.


• Providing a forum for constructive collaborative influence, instead of using costly adversarial forums;

The principles of good public engagements include:

• Commitment of and link to decision-makers;
• Adequate resources to support the participation of public stakeholders;
• Neutral process and technical support;
• Representation of all key interests;
• Safe and independent forum for all stakeholders to participate;
• Respectful and equal treatment of participants with no undue influence of any one group on the deliberations;
• Jointly developed and enforced clear protocols for the group's operations that support collaborative deliberations and these principles;
• Recognized legitimacy of numerous types of knowledge (e.g., scientific, experiential, local, cultural, personal, etc.);
• Respectful questioning of all information about validity, accuracy, authenticity and reliability;
• Transparency of the process;
• Evaluation during and after the process.
• Services that fill a gap and does not duplicate other efforts;

Based on these principles, the Wingspread Group came up with the following recommendations:

• Type-ongoing stakeholder forum are the foundation and core of public engagement; implement larger deliberative public processes to inform deliberations if deemed useful, appropriate, and cost effective;
• Links/accountability with decision-makers-establish concurrent NVAC Working Group/IOM Roundtable to consider outcomes and brief forums regarding issues and actions taken;
• Framing of issue-issues framed by agencies and advisory groups;
• Products of deliberations-offer a rich analysis and problem-solving deliberation to flesh out a deep understanding of the range of interests and options;
• Representation-involve key interests (a microcosm including a range of public perspectives);
• Type of deliberations-ensure dialogue is interest-based and collaborative; no voting;
• Protocols-are jointly developed to support collaborative deliberations;
• Evaluation-are on going, with a sunset provision.
• Support-include neutrals and independents (e.g., conveners or facilitators) and technical experts (e.g., IOM);

Based on these deliberations, the VPACE proposal was delivered to NVAC for consideration. In the meantime, Keystone would like to use funds from a grant to conduct a one-issue pilot. The pilot would be a joint partnership with the Wingspread Steering Committee, NVPO, and NVAC, and the issue to be examined would be pandemic flu. NVAC may also be asked to partially fund the pilot.

Questions

Ms. Koslap-Petraco noted that she has not seen what parents think of this. She asked how the views of an average parent who may not have any vaccine complaints are to be collected. She asked how these groups would be involved, in addition to those who already have a known stake. She added that she trusts IOM implicitly and looks to them to evaluate vaccine issues. As a result, to have IOM as a facilitator may be a conflict of interest. She concluded that without an assessment of what the average parent thinks of immunizations, a solution cannot be determined.

Ms. Hamlin noted that it is possible, though a challenge, to get all interested parties to the table. The process does not have to include only special interest groups. In the Wingspread group, there were representatives from the League of Women Voters and the PTA.

Dr. Stratton noted that she worked on vaccine safety issues at IOM for the last 12 years. She was also a part of the Wingspread group. Once it became clear that there was a potential for an IOM role, she did not serve as an IOM advocate.

Ms. Katz asked how the proposed stakeholder forum is different from what NVAC does now.

Dr. Evans, National Vaccine Injury Compensation Program, noted that he was on the Wingspread initiative. He added that public participation in vaccine decision-making is very limited. The typical setting that requires the public to speak into a microphone while others listen silently is not the most comfortable communications framework. In the last 15 years, there has been a shift away from when immunization was taken for granted. People are now questioning vaccines and this questioning has created more work and more difficultly for many people. A lot of time is spent dealing with a small segment of the public that for various reasons is not content with the decision-making process. In the last couple of years, workshops were conducted with various stakeholders. These workshops dealt with DTP, mandates, and OPD, which resulted in important intangibles. Now, the issues are more wide-ranging, including genetic predispositions and long latency adverse events. The way public comments are conducted today is not adequate to give the public a sense that they have a meaningful role as a stakeholder in the decision-making process.

Ms. Katz noted that the question for this working group is not whether there should be greater public participation, but how to best achieve that. Standing in front of a microphone is not an easy thing, but how is the proposed structure different.

Dr. Evans responded that parent input is just as important as the "ring of people." With that ring, there is a closed door and people have to deal with each other for the first time. When consumers showed up at these meetings, they were the outsiders and the ones taking a chance. However, when everyone was brought together, especially those who are not use to dealing with people, it was a new situation. This was a learning experience and what was gained was the opportunity to better understand each other. In this setting, participants are forced to communicate with each other and discuss substantive issues. Over time, minds may not change, but understanding is achieved.

Ms. Hamlin added that there is a perception of mistrust in the government among some stakeholders. To get those people to the table to collaborate, it is important to have a sense of neutrality.

Ms. Katz asked if NVAC should go all the way and use a forum such as AmericaSpeaks, where the agenda is set by the public.

Ms. Hamlin responded that the Wingspread group made recommendations for both general types of collaborative processes. One is the stakeholder forum and the other is the AmericaSpeaks or Study Circles model. However, the stakeholder forum should be the foundation.

Ms. Katz added that the government agency would set the agenda.

Ms. Hamlin noted that the group does not set its own agenda, but provides the collaborative capacity to create a safe forum to get a broader perspective on various issues.

Ms. Katz asked what would happen at the first stakeholder meeting and what would be the agenda.

Ms. Hamlin noted that the group would first meet at NVAC's request when it needed a broader perspective on a particular issue.

Ms. Katz added that the government would set the agenda. In the previous models, the agenda is set through the process.

Dr. Evans noted that the agenda setting is back and forth to some degree. People were asked what the issues should be for such a group, while government advisory committees were also asked for input.

Ms. Hamlin added that there is a framing of the issues and topics for discussions.

Mr. Pehlivanian commented that there does not seem to be a good connection with what happened with chemical weapons. For example, at the first meeting, there were many people who were in a rage about what the government was doing in the process of destroying chemical weapons. There were also those who were supportive. When they were first brought to the table, they were educated on everything dealing with the chemical weapons. Then they were able to give their input regarding their concerns. It became clear that they were initially talking past each other. For example, the environmental activist community wanted a "closed-loop system" when destroying chemical weapons. This community did not want any hazardous waste to get outside of the destruction process. The Army noted that that was impossible and explained that there would be hazardous byproducts. After much deliberation on this one issue, it was learned what the environmentalist really meant by a closed-loop system. They wanted the weapons to be destroyed, tested, made sure to be safe, and then released. This was a major breakthrough and got the group to a point where they could begin to understand each other.

Ms. Wexler noted that she participated in the first Wingspread meeting. She was initially concerned that other groups were not invited and that the invitees were handpicked. She was displeased with the functioning of the group. At the meeting, there was no dialogue. Rather, people were taking turns speaking at each other. There were threats and the antivaccine community was overrepresented. Afterwards, she withdrew from the process. She has spoken with a number of people and learned that there is concern among the medical community about Wingspread. She added that though individuals were supposed to only represent themselves, her name and organization is listed on the VPACE proposal as a participant. The other organization that withdrew was PKIDS. PKIDS and Ms. Wexler's organization were the two vaccine supportive groups. She asked who supports and endorses the VPACE proposal.

Dr. Evans added that Lou Cooper, who chaired the CDC blue ribbon panel, commented that the Wingspread project is a good example of how to bring people together to discuss tough issues.

Dr. Stratton commented that the VPACE proposal got the most votes of the four or five options. It was not unanimous, but it did receive the majority vote.

Ms. Wexler asked who was there. She asked if there were organizations that support this project and who the steering committee is. She added that she was not asked about this proposal, yet her name is on it.

Dr. Stratton noted that is was probably a mistake to keep Ms. Wexler and PKIDS on the proposal. She added that industry had an alternative position. It is not listed who voted for the proposal, but the group as a whole agreed that this would be the action taken.

Ms. Hamlin added that the intention behind listing Ms. Wexler was to show her involvement in the initial discussions.

Ms. Wexler noted that it misrepresents others on the list-people who attended only one meeting. She asked who vetted the proposal.

Ms. Hamlin responded that at the final discussion, the group deliberated on the options and decided to go forward with the recommendations. She, as the neutral, then drafted the proposal. Afterwards, it was sent to all of the remaining participants.

Ms. Katz noted that this working group was asked to look at the VPACE proposal, regardless of who endorsed it. She added that the record would reflect the question about what groups formally endorsed the proposal. However, she asked the participants to focus on the substance of the proposal.

Mr. Carlitz commented that there needs to be a process for ongoing issues that is open and scalable. It must be open and well documented in terms of what went on. It must be scalable to allow for new participants to join. If it is not scalable, there might be people who would feel excluded from the process. Online mechanisms are intrinsically transparent and scalable.

Dr. Zink noted that it took him three years to get comfortable in front of a microphone. Industry is part of the public and his group has taken the stance that they are guests at these types of meetings. He thanked the committee for allowing him to speak, even before the public comment period. Additionally, he added that the NVAC subcommittee meetings allowed the public to literally sit at the table. As a result, NVAC might be one of the better groups that are looking at how to hardwire further public participation as an inherent skill set of the committee. That is why he believes something like VPACE should be incorporated as a routine systemic skill set of this committee.

Dr. Zink added that he is on the VPACE steering committee. However, he was surprised when he heard about the ACIP opportunity. He believes this opportunity helped to destroy some trust. He had thought that this opportunity would have been brought to the NVAC Public Participation Working Group. As a member of VPACE, he has serious concerns about the timing. He is concerned about a number of systems that have yet to be created, including outcome monitoring and financial sustainability. He is also concerned about the willingness of VPACE to integrate into NVAC. There appears to be a strong inclination within the group to remain independent. The plan to find VPACE inner core and outer core members is still too fluid at this point. Consensus on the reporting of the results and the deliberations has not been vetted. There was an agreement that there would be no voting, but there was a vote. There was an agreement that there would not be any reporting outside of the client structure. If NVAC is the client, then NVAC would report on VPACE activity. However, Dr. Bernier's letter indicates that a preliminary report will be released. The biggest worry is that VPACE will satiate those for which VPACE is being designed. Some members have been members on NVAC and have turned their backs to NVAC. VPACE can be sharpened and fine-tuned, but that may take another meeting before it is brought to NVAC for consideration.

Ms. Hamlin added that the report provides the general concept of the group's proposal. The final plenary session deliberated in detail on many of these issues. For example, the report would only be released to NVAC and not to the press. However, the hope is that many of these issues would be fleshed out in a pilot.

Ms. Katz added that this working group was not asked to support or not support a pilot project. They were asked to look at VPACE as a model for further public engagement. The development of a potential model came after the charge to this committee.

Ms. Hamlin noted that they hope to go forward with the pilot and asked for NVAC's participation and partnership.

Dr. Johnson noted that in chemical weapons meetings, there was a general agreement that there needed to be something that was doable and scientifically sound. One concern about the VPACE model is the notion of recognizing the legitimacy of numerous forms of knowledge (local, cultural, etc.) all on par with scientific knowledge when it comes to decision-making and recommendations that have to be fundamentally based in science.

Mr. Leighninger added that one reason VPACE is difficult to access is that it is trying to combine the best of all of the different models. Many of the other models represent "republican" systems. They are a set of citizen advisory committees of various forms that create a representative sampling of the population. An advantage of these standing bodies is that they are there always available. The difficulty is that there is no guarantee that they are always representative of the broad public. On the other side, there are "democratic" processes, such as AmericaSpeaks and Study Circles. One drawback of these groups is that they are temporary. As a result, both types of processes are needed. When there is a representative group that is making decisions on the basis of the larger public input, that group must have credibility in the larger community. The VPACE proposal might not have clearly articulated that there are two equal and complimentary pieces. However, the intent of those working on VPACE is to have both the republican and democratic system.

Dr. Stratton addressed Dr. Johnson's concern about different types of knowledge. She noted that the questions to be posed to the group, by design, are not purely scientific in nature. For example, this group would not take on the question of what strain should be in next year's flu vaccine. However, discussion on trade offs, ethics, and legal issues are not purely science-based.

Dr. Johnson responded that he was surprised to see a proposal for a pilot to look at creating recommendations for a specific vaccine, which are very technically based issues.

Dr. Stratton added that it was not to create recommendations. It was to explore the considerations for the various options that are already in existence. For example, rationing scarce vaccines is not only a technical issue.

Dr. Vernon noted that he believes in government. Though it has not always performed as well as it could have, his first responsibility is to help improve the process. The VPACE solution is based on a mistrust of government. It is attempting to create another organization that is beyond government to make sure it is doing things right. It is defeating if we step away from government to take on tasks that the government should be doing.

Ms. Hamlin commented that this was why the Wingspread group was careful not to duplicate efforts. The group does not work outside of the government. It works through NVAC and provides the capacity to have a certain level of deliberations to work through issues of trust. It provides some of the input to help inform NVAC's decision, not to usurp it.

Ms. Katz asked who would select the members of the stakeholder forum. For NVAC, the administration must approve the nominations. If VPACE is an extension of NVAC, would the administration have to approve the nominations?

Ms. Hamlin noted that working groups do not require the same official procedural channel. She added that she would work with NVPO, the Wingspread steering committee, and NVAC to pull together the best recommendations on what would be a representative and balanced group.

Ms. Levine added that the approval process would depend on whether this group was a subcommittee/working group or a public participation group. There can be a subcommittee of NVAC that has participants that are not NVAC members. This might require government approval, though it might not be as rigorous as being a member of an advisory committee.

Dr. Evans noted that working groups have non-members and Ms. Landry added that they are technically considered consultants.

Dr. Helms expressed that if NVAC suggests a study such as VPACE and creates policy recommendations based on this study, NVAC could be viewed as advising itself. Secondly, it creates the possibility of a faulted study, especially if the inner core of VPACE does not contain major individuals who stay outside of the process, in which NVAC might be in a position to reject it.

Dr. Stratton responded that NVAC would not be advising itself because there would be no advise. There is only an exploration of various options. Secondly, there is a somewhat complicated proposed role from the VPACE group for an IOM roundtable. At these forums, they do not make recommendations, but discuss issues. As a result, IOM would be the conduit for the technical or substantive discussions. IOM would appoint a roundtable of interested parties. IOM would agree to include members of the NVAC working group. The NVAC members would then report back to NVAC themselves. IOM would not create this summary report for NVAC.

Dr. Helms asked if NVAC needs to be involved in the setup, rather than just receiving the report.

Dr. Stratton agreed that NVAC would not officially need to be involved. There were earlier discussions about not including NVAC and only creating IOM workshop reports. However, a link with NVAC was deemed necessary and a working group was determined to be a good way to accomplish that.

Ms. Hamlin added that they did not want to create any unnecessary layers. They wanted to enhance and be a part of government.

Dr. Evans responded to Dr. Vernon. Though mistrust has driven many of these kinds of efforts, there is also a lot of paternalism. He added that the activists were not overrepresented and were actually in the minority.

Ms. Zavolinsky asked how the stakeholder forums and the general public groups would interact with each other.

Ms. Hamlin commented that the process would start with the stakeholder forums. This would be the group that would generate the issues to be tested in the larger forum. Then, the larger deliberative process would feed into the stakeholder deliberations. However, the larger process could have its own report.

Public Comment

Ms. Wexler noted that a common theme of this meeting is that there is a problem that needs to be resolved. However, she does not believe that they have taken a step back to determine what the problem actually is. For example, the United Kingdom conducts an assessment of what parents think every six months. The United States should do a similar assessment to learn what the general public thinks, rather than hearing only from the activists. As a citizen, her concerns deal with paying for vaccines for uninsured adults. Another concern is whether influenza vaccine should be mandated for all healthcare workers.

She then read from a proposal that she is submitting to NVAC. The need is to provide timely information on immunization and vaccine knowledge, attitudes, and practices among the public. First, primarily qualitative information should be gathered using validated surveys and focus groups, conducted in regularly occurring cycles among population groups affected by immunization practices. This would provide information and trends over time on such variables as age, race, ethnicity, socio-economic level, and risk groups. Second, the implementation of the survey and focus groups would be the responsibility of professionals in NVAC or NIP. Third, generation of survey and focus group topics would be the responsibility of the above professional group and guided by an advisory committee comprised of representatives of a number of professional and nonprofessional organizations. Fourth, the results of the surveys would be reported regularly to the advisory committee and NVAC or ACIP.

Ms. Gunn commented that she is surprised that there has not been a discussion about provider attitudes. It may be a good idea to survey providers about their attitudes towards vaccinations to influence their perceptions. This may be a lower-cost solution than to change the population as a whole. Additionally, the first task has to be determining the objectives.

Ms. Katz reiterated that the task of this working group is to look at different models on how to engage the public more than it currently is. It is not to set the agenda for any of the proposals.

Ms. Gunn added that when it is difficult to engage the public, you have to work through the intermediaries, which are the healthcare providers.

Ms. Katz thanked Dr. Helms and Ms. Landry for their help, and thanked the participants for their involvement. She added that the working group would meet and prepare a report to the full NVAC committee, along with some recommendations. She then adjourned the meeting.