Health Emergency Assistance Line and Triage Hub (HEALTH) Model

Chapter 8. Testing

Valuable information was garnered from the implementation of the Smallpox Vaccination Program Support Service and from implementation of the Colorado Health Emergency Line for the Public (CO-HELP) for West Nile virus (WNV). Not all aspects of the Health Emergency Assistance Line and Triage Hub (HEALTH) can be tested in these two applications; for example, use of the Incident Command System (ICS) cannot be tested, as these were planned events not necessitating an Incident Command System, and exportability cannot be tested as the applications were limited to Colorado. Funding of the proposed systems upgrades (e.g., IVR, "screenpop" function, Web-chat) must occur before those elements in the multi-channel system concept can be tested. It is clear that different types of events and testing opportunities will be required to test all components of the HEALTH model.

In attempting to evaluate our service delivery, we went back to the original objectives of this project (presented in Chapter 3) to assess our progress.

Objective 1: Determine Best Practices, Challenges, and Shortfalls of Communications

Best Practices

Quality Control in Real Time

With only a 0.5 full-time equivalent (FTE) project manager, we were still able to review 100 percent of call records (n = 6,687) for incomplete documentation. The project manager monitored approximately 30 percent of calls for quality by listening to the call in real time.

A CentreVu® Call Management System allowed the manager to track call metrics in real time. This means that the manager was aware when calls went into queue and how many were in the queue (that is, callers had heard the complete message and were on hold for the information specialist). The manager was also able to know whether information specialists were logged out of the system, and keep track of abandon rates all in real time (rather than 1 month after the fact).

Complaints and praise for the hotline were immediately input and E-mailed by the information specialist to the project manager, allowing the project manager to immediately take whatever actions were possible to remedy the situation.

Providing a Communication Conduit Between the Public and Public Health

All information shared with callers had to be approved prior by the State Health Department. We provided frequently asked questions (FAQs) on 55 topics during the smallpox program and 244 topics during WNV. New information was required throughout the delivery of both programs, and processes were in place to accommodate these occurrences.

During WNV, our system advanced to the point at which an information specialist handling a call for which there was no approved FAQ completed an E-mail request for information while the caller was on the line. The E-mail was sent immediately to the State epidemiologist and the project manager. The information specialist would take the caller's contact information and the caller would be called back (usually within 1day) with the answer. There was a total of 21 WNV FAQs added to the 223 original FAQs the program started with (a 9-percent increase). A summary of the types of FAQs added are presented in Table 8.

Over time, we were able to identify areas of concern for citizens (e.g., concern about who to call about standing water, information on mosquito spraying) and share that with local and State public and environmental health agencies. This feedback allowed those entities to adjust their public information messages accordingly. We also identified a demand for information from health care providers and veterinarians. An information sheet was developed and available to be faxed or E-mailed to those providers.

Assuring Delivery of a Consistent Message

Our use of software embedded with decision triage assured consistency in the information that was shared. Our quality reviews confirmed this. Something of particular concern in the WNV hotline was the frequent demand on the part of the public for evaluation of symptoms. Because only a Level One service level agreement (SLA) was established with the State Health Department, this was not a competency that was available to the public. For information specialists to offer medical advice would have produced issues of liability both for Denver Health Medical Information Centers (DHMIC) and possibly for the State Health Department. The decision triage software, along with approved FAQs and quality review of staff, offered the highest level of protection from this possibility.

Challenges

Unknown Call Volumes

Because there is a lack of information on this kind of service, we had no idea what to expect in terms of demand for the hotlines. The smallpox vaccination program initially had plans of vaccinating more than 2,000 people but the majority refused vaccination. Thus, the volume of calls for that project (193 calls) was lower than anticipated. The lack of information made it difficult to anticipate scheduling needs and budget needs. This also made it a challenge to convince the client (State Health) that the service was necessary for WNV. However, calls to the Colorado Department of Public Health and Environment (CDPHE), during the 2002 WNV outbreak, caused a lot of disruption to CDPHE's normal operations. The success of the smallpox vaccination program hotlines suggested that CO-HELP should be activated during the 2003 WNV season. Though we staffed CO-HELP for 1,000 calls per week, the actual demand far surpassed these predictions as the outbreak got progressively worse in August 2003 (Figure 3).

We tasked 3 full-time employees as information specialists for WNV and 0.5 full-time employees to manage the project. We discovered that surges in call volumes could be expected immediately following media coverage that included publication of the hotline number. This meant that surges could be predicted to start at 7:00 a.m. following a newspaper article, and at 11:00 a.m., 4:00 p.m., and 5:00 p.m. following local news coverage. Call volumes were primarily related to media coverage and less related to occurrence of deaths or reports of illness due to WNV. A sample of how media reports impacted call volume is shown in Figure 4.

Intermittent Employees

Because the smallpox program was only to be offered in the short-term, staff was initially hired as intermittent. Knowing that their employment would end shortly, intermittent staff generally had greater allegiance to their primary job. This caused some scheduling challenges. For WNV, the staff was hired as permanent employees, which improved job performance and attendance.

Shortcomings

No Fax Server

Physicians preferred to get information sent to them by fax; none wanted to receive our information sheets by E-mail. Unfortunately, our system was not set up with a fax server. This meant that information specialists had to wait until they had coverage, or slow time, to go to the fax machine (in another room) and send the fax manually.

Demand for Medical Triage and Symptom Support

About 1,300 callers reported WNV symptoms and were referred to their physicians. Many callers reported getting confusing or dismissive treatment from their health care providers; other callers reported that they had no insurance and did not know where to get treatment. If the client (State Health) had opted to include Level Two in the service level agreement (SLA) for WNV, registered nurses could have offered symptom management and triage support, similar to support services offered for smallpox vaccinations. As it was, many of the callers were disappointed with referrals to a local clinical or primary health care provider for symptom management advice, and that no public services for testing, evaluation, or treatment were available. We suspect that many of these callers could have benefited from education as to what symptoms they should watch for in the progression of their illness, assistance in evaluating the danger of their symptoms, and education/information that would have supported them in making informed decisions about their health care needs.

Demand for Information on County/City Services

For WNV, there was a great demand for environmental information regarding clean-up of standing water, applications of larvicide, and spraying for adult mosquitoes. This information varied by municipality. We tried to develop FAQs regarding these requests; usually they took the form of a city or county contact. Many of these callers felt that they had received the runaround when calling city and county agencies and were frustrated with the referrals we provided.

A small number of callers were reporting symptoms from adverse reactions to mosquito repellant. Because of our location within a poison center, those calls were immediately linked to a poison information specialist.

Objective 2: Determine Types of Information and the Communication Pathways Requested by the Public. How Was the Information Perceived?

Communication Pathways

The State Health Department provided a WNV Web site, "FightTheBiteColorado.com" that complemented our telephone service and showed the potential value of a multi-channel contact center. We referred many parties (especially schools and researchers) to the Web site, and also received many calls from people who accessed the hotline number from the Web site. The public seemed open to accessing Web-based information and receiving information from us by E-mail. The physician community preferred fax to E-mail.

Public's Perception of the Information Provided

A satisfaction survey for the smallpox program was sent to a sample of 39 callers. Overall, response was positive; of 12 surveys returned, 3 negative responses were received. (One caller was a nurse/paramedic who was preparing for a paramedic lecture and complained that, "I did not get all of my questions answered for my paramedic lecture; [I] was referred to other resources." The remaining two calls were responding to the question, "Did you receive satisfactory information and/or support when you called the military resources that we gave you?" Two of three respondents answered, "Strongly disagree.")

Ten out of 12 respondents reported that they were satisfied with the information and support they had received, and that it had aided them in more effectively dealing with their situation. Health care providers reported that the service reduced their call loads: "It's a great service, and I think it should be available for Phase 2. We gave the number out. It reduced our on call work- [it] made our lives easier."

No such survey was done for the WNV hotline, because we did not collect contact information unless we specifically needed to return information to the caller. The State will be surveying county health departments for their feedback on this service.

For both implementations, callers generally reflected their pleasure that the State was offering the service. Dissatisfaction was uniform with matters beyond the scope of DHMIC. Callers either wanted services (e.g., dead birds to be picked up, free WNV serological testing, WNV symptom support, standing water to be cleaned up for them) that were not offered in this service, or were frustrated that they had not been able to get through to their local city or county officials.

During the WNV outbreak there was a surge of community spirit; callers reporting dead birds felt that they were doing a community service. Dead bird reports accounted for the greatest proportion of calls taken. This may be a powerful force to consider in future disasters. As was discussed in the section above on risk communication, the public wants to be engaged in the solution during an emergency and a hotline or contact center can provide a conduit for both communicating information on how they can get involved and collecting information from them.

Media Interest

We received a lot of positive media interest both because we were considered a resource for WNV information (among other things, we were able to give out the number of confirmed cases and deaths), and because of interest in the type of service being offered. Five on-site media visits were done, and we received coverage from local and national radio and television news programs including National Public Radio and Fox News. We also received inquiries from a Canadian radio station that did a story on our program.

Objective 3: Determine What Special Populations Were Encountered

Military Referrals

Concurrent with the Colorado Smallpox Vaccination Program, approximately 10,000 regional military personnel were also being vaccinated for smallpox. As a result, several calls to the Colorado Provider and Hospital Information Line (CO-PHIL) and CO-HELP were received from military vaccine recipients, their civilian health care providers, or family members and contacts. The number one category of referral for the CO-PHIL line was military referral, or 25 percent of calls fielded on that line. This use of the telephone lines was unforeseen, and the military referral link had not been developed prior to initiation of the program. Within the first week of the call center's initiation, nine military related calls had been received, with two of those calls reporting an adverse event. Three additional reported adverse events occurred in civilian contacts that had a possible inoculation from a military vaccine recipient. The military was contacted and a referral to the military's emergent medical toll-free line, toll-free medical information number, and vaccine-specific Web sites were provided. Military caller information and adverse event reporting data collection continued, although all calls that were identified as military-related were also given the military referral.

Health Care Providers

During WNV, health care providers emerged as an unanticipated population with very particular information needs. We developed a fact sheet with information on current recommendations and locations of testing facilities that was faxed or E-mailed to them, per request.

Spanish Speaking Callers

During WNV, a total of 87 callers required Spanish translations. The CO-HELP recorded message was provided in Spanish, and then information specialists accessed a translator to assist with the call. All brochures, and the Web site for the "Fight the Bite Colorado" campaign included the advertisement that the hotline had Spanish translation capabilities. However, the State had not advertised the hotline on Spanish radio or television. Around the second week of August 2003, Spanish calls began to pick up. After that, we asked Spanish-speaking staff to let us know if any Spanish media had published the hotline number. We observed that peaks in calls accessing the translation line followed such announcements on Spanish radio. The radio stations, following the intensive English media coverage, had also initiated these reports.

We also received one TTY (teletype for the hearing impaired) call that reportedly went well. The information provider reported that she repeated the caller's question back to the caller and asked if the caller had understood the answer provided, which the caller did.

Veterinarians and Concerned Animal Owners

Coloradoans own a lot of horses and other animals. So it should have come as no surprise that questions about pets arose during the WNV outbreak. We had to add several FAQs on WNV and animals. The Centers for Disease Control and Prevention (CDC) was receiving reports of infected horses, but received no reports for cats or dogs. Several frustrated veterinarians called about the lack of reported information regarding positive cases of WNV in cats and dogs.

Senior Citizens

Senior citizens were the largest group at risk in our population during the WNV outbreak; because we were only offering an information line, we did not actively collect information on the age of callers. However, many callers voluntarily identified themselves as seniors. We also received calls from managers of senior centers with questions on how to protect their grounds from mosquitoes.

Recreation and Travel

We did not have a particular FAQ category for tourism-related questions, but we did receive a lot of queries regarding recreational activities, such as whether it was safe to go camping, how to protect yourself from receiving mosquito bites on camping trips, should outdoor events be cancelled, and the like. We also received calls from 33 States because people who had traveled, or were planning travel, to Colorado had concerns about WNV.

Uninsured Callers

Many callers during the WNV outbreak intimated that they had no regular doctor and had no health insurance. This was a difficulty when they were calling about symptoms that were compatible with WNV. These callers were referred to a local hospital to find out about what clinics might be available with free or sliding-scale fee services.

Children, Infants, Pregnant Women, and Breast-Feeding Women

These were groups with a high level of concern. Often, the caller's health care provider did not know the answer to questions regarding WNV and while breast-feeding or during pregnancy. We shared FAQs developed by the State having the most current information on the effects of WNV. Common concerns surrounded the guidelines for use of DEET insect repellant on children, and we provided information consistent with the American Academy of Pediatrics.

Objective 4: Determine Requirements for Data Storage and Retrieval

For both the smallpox and WNV implementations, the client (State Health) required reports in simple Microsoft® Excel spreadsheets sent by E-mail with encryption. The CDC required Vaccine Adverse Event Reporting System (VAERS) forms during the smallpox program, and we included the same data fields in our reports to CDPHE. In the future, the CDC will have new requirements for disease reporting formats, but they have not been determined to date. Call records required a security clearance permitting only the project manager to make corrections to protect data integrity.

Objective 5: Determine Needs for Infrastructure Integration Between CO-HELP/CO-PHIL and State and Local Health Departments

Information Input/Output Mapping Aids Coordination Between Agencies

We developed a virtual integration with the State Health Department. While the communication process during WNV was fairly simple, the process for smallpox was quite complex and involved not only the State Health Department, but on-call physicians and access to epidemiologists at the State and the CDC. The process used in the smallpox service is available in Appendix E. The processes and call flows that were developed were successful in handling all types of calls to the smallpox support services lines. The process and call flow diagram allowed those delivering the service to quickly identify bottlenecks, agree on solutions, and address any unique problems that emerged.

Objective 6: Develop Criteria for Electronic Data Collection and Conveyance

For the WNV application, we modified current software, LVM E-Centaurus, which is used by our NurseLine. The software is normally used to support telephone nurse triage. Modifying the software for our purposes required building a database of FAQs and a decision tree structure to aid the information specialist in appropriately responding to caller requests. Additional features were added later as demand developed. A dead bird reporting survey was developed at the request of counties that were being overwhelmed with reports of dead birds. We added additional data fields (zip code and city) at the request of counties.

For the smallpox program, all information shared with callers was based on CDPHE vaccination protocols and CDC developed information, and finally approved by the epidemiologist in charge of bioterrorism preparedness for Colorado. The smallpox support project installed an internal requirement of a 24-48 hour turnaround in responding to information requests, when the requested information was beyond the scope of the current content as approved by CDPHE. Recording of the caller's county and zip code, and specifics of information requested, allowed for a feedback loop from the vaccine recipients and health care providers to public health agencies on informational concerns.

Information on adverse events was recorded in the same fields used on the VAERS form required by the CDC, and those data were forwarded to CDPHE within 24 hours. Vaccine recipients experiencing vaccine-related adverse events, who were not already under the care of a physician for the adverse event, were immediately connected with trained call center nurses for assistance with symptom management. CDC-developed clinical decision trees were used in supporting vaccine recipients and their health care providers.

Objective 7: Determine Call Metrics Required for Public Health Agency

We found that the client (State Health) was not aware of call metrics (abandon rates, queue times, etc.) or their importance to call center operations. Throughout the administration of WNV, because of the high volume of calls, the State health representative developed an awareness of some of the terms, particularly call queuing and abandon rates. A major concern for the client was minimizing costs. They began to see that knowing when call volumes were peaking (thus queuing was occurring and calls were being abandoned) would allow more appropriate and cost-effective scheduling.

Objective 8: Determine the Facility Specifications

During the smallpox project, workstations were borrowed from the NurseLine. Because this would take away from the NurseLine in the long term, we acquired five new workstations for CO-HELP and CO-PHIL. (WNV coincided with a move for DHMIC to a new building; we provided adequate space for the expansion). We also acquired system upgrades for some administrative personnel to allow them to enter the system in response to surges and to assist with answering calls.

Objective 9: Determine the Technical and Equipment Requirements

Each workstation required a headset, a computer equipped with LVM E-Centaurus and linked to our call center network, and a digital telephone connected to a telephone switch with automatic call distribution capability. This allowed the user to log in to the telephone system and receive calls. Internet access was required to allow us to send E-mail responses to callers. As was mentioned above, we determined that a fax server is an essential future addition to the system, and we have invested in equipment to provide this capability.

Objective 10: Determine the Most Effective, Feasible Technical Solutions

The smallpox and WNV programs use the "voice channel" or telephone as the method for providing information to clients. In both programs, half of the callers were satisfied with the recorded message alone and did not opt to route to the information provider. During WNV, many clients connected to the Fight the Bite Colorado Web site for their informational needs, and therefore did not need to contact the hotline. E-mail and fax were well received by callers needing printed materials that we could provide. With these capacities alone, and 1 to 2 information specialists on staff at a time, with backup from administrative staff in other departments, we were able to manage a peak of 537 calls in 1 day (on August 11, 2003) without any detrimental impact to our existing services. Had call volumes increased to higher levels, the need for other technological solutions to assist staff in handling the demand would have become more apparent.

Objective 11: Develop Methods for Rapid Ramp-up for Surge Staffing

Six additional administrative personnel in the DHMIC were cross-trained on the system to be called up in case of surges. We also acquired system upgrades for some administrative personnel to allow them to enter the system in response to surges. We had to use these additional staff resources several times during WNV operations.

Installation of the CentreVu® Call Management System allowed the project manager to monitor the volume of calls, and whether or not calls were going into the queue. This was vitally important because the call volumes were unknown and initially unpredictable. When the manager recognized that callers were on hold, she could access backup information providers to help clear the backlog.

Because call surges were so clearly linked to media reporting of the hotline number, the ideal disaster hotline facility would be equipped with televisions to allow the staff to monitor news coverage.

Return to Contents
Proceed to Next Section