Appendix 4. Developing an Interactive Response Tool: The HELP Model (continued)

5.0 Interactive Response Tool Implementation Document

The implementation phase includes building or purchasing the technology/system, testing and piloting it, training staff on it, migration of it into an active environment, and then going live. During this phase, the technology solution undergoes comprehensive testing first by the project team and internal users and then by external users. This allows for both coarse and fine tuning to validate that application designs meet the requirements and objectives. The implementation plan should incorporate appropriate users in the process, establish realistic time frames for tasks, provide adequate opportunity for feedback, and establish criteria for measuring success. Depending on the environment of your call center, you may consider using change management strategies to create a positive environment for learning and change among the staff impacted by the implementation. Our project team was responsible for ensuring that each component of the implementation phase was addressed following their clearly defined roles and responsibilities.

Building the Technology Solution

Since we previously purchased an IR system, our first task was to build a technology solution utilizing this equipment. This included development of the IR applications and making programming changes to the telephone switch to support IR application functionality. We contracted with an IR consultant and developer to oversee the development of the applications in accordance with our business requirements and specifications. The IR consultant developed the applications, provided administration and maintenance training for our Information Technology (IT) group and assisted with modifications to the applications after testing. Our IT group made the appropriate programming changes to our telephone switch to support the IR applications.

Implementing the Solution

The following is a project time line for the implementation milestones, the responsible person(s), and the timeframe for completion. The implementation phase occurs after completion of the planning, analysis, and design phases (approximately 6 to 8 weeks). The timeframes listed do not necessarily occur in series; many can be done concurrently while other milestones are in progress. There also may be delays between milestones due to other demands on the responsible person(s) and the resources they require to support them.

The following table outlines the major implementation milestones for the four applications (approximately 34 weeks to complete):

Milestone	Responsible Person(s)	Timeframe
Develop IR applications	IR Consultant	8 weeks
Test IR applications on IR consultant system (test environment)	IR Consultant	4 weeks
Internal testing of QI Monitoring application prototype	Telecommunications Engineer/Project Manager	2 weeks
1st External testing of QI Monitoring application prototype	Telecommunications Engineer/Project Manager	4 weeks
Install applications on IR system	Telecommunications Engineer/IS Manager	2 weeks
Record messages and announcements for all IR applications	Telecommunications Engineer	2 weeks
Internal testing and validation of all four applications	Telecommunications Engineer/IS Manager	2 weeks
2nd External testing of all applications	Project Manager	4 weeks
Train IT group to administer and maintain applications	Telecommunications Engineer	2 weeks
Adjust and refine applications and their recorded messages	Telecommunications Engineer	4 weeks
Document application designs and administration procedures	Telecommunications Engineer	4 weeks

Return to Appendix 4 Contents

5.1 Testing

We used an iterative testing approach that included four cycles of testing:

1. Technology testing (external testing by the IR consultant and internal IT group testing).
2. Internal testing by our call center staff.
3. External testing by a rural user group.
4. External testing by an urban user group.

This testing approach gave ample opportunities to validate that the applications function as required and that any difficulties in using the applications were identified and addressed (user acceptance testing). These testing cycles are already included in the overall implementation timeline in the previous section. Total time for testing required approximately 16 weeks to complete.

Return to Appendix 4 Contents

5.2 Training

The training strategy for this project was to ensure that the general public with a touch tone telephone should be able to navigate through these IR applications and have a positive overall experience.

In a testing environment, we could not convey instructions through the media or other channels that we would use in an actual emergency situation. Therefore, we had to incorporate clear directions for users within the applications themselves so that there would be a high probability of user acceptance. Some of the directions that were incorporated into the applications include:

• An option to connect to an information provider (live person) when there is an appropriate need or if the user gets lost within the application.
• An option to repeat a message so that the user can hear it again to fully capture the information being relayed.
• An option to utilize the application again (such as with POD and DI applications) so they do not have to call again to enter a different response.
• An option to return to the main menu (such as with FAQ Library application) so they can get answers to more than one question they may have.

There is always a trade off with devoting time at each step in an IR application with providing instructions in addition to the critical messages, so a balance in providing good guidance while keeping callers moving through the application has to be achieved. Testing of applications with user groups is one of the best ways to determine that balance.

It would be best to incorporate clear instructions into the QI application; however, we would still recommend public health agencies develop an instruction sheet for quarantine that includes what is expected of those in quarantine, what they should expect during the quarantine period, and what resources to contact for anticipated difficulties in maintaining quarantine (food, medications, finances, etc.).

Return to Appendix 4 Contents

5.3 Migration

The migration plan has both a technical and a business component. The technical component (conversion plan) describes how the new system will be installed and where applicable, how data in the previous system will move to the new system (this does not apply for this technology solution). Installation of the IR technology solution and the steps involved were discussed in previous sections. The business component includes a change management plan that helps users understand the change and motivate them to adapt to the changes.

The change management plan for this effort began early in the process and continued throughout the project. The project team, steering group and external advisory panel represented the key stakeholders and users that the technology system would impact. We focused on communication and feedback throughout the process to ensure members understood the costs and benefits for the organizations involved and end user groups, our plan for implementing the technology solution, how we would measure success, and the feedback process for continuous improvement. The iterative testing strategy provided a feedback loop for participants and users to identify what worked and what did not work so that appropriate changes could be made to the applications. These ultimately should result in applications that are well designed, efficiently deliver the required capabilities and do so in a manner that is effective and acceptable to the user.

Return to Appendix 4 Contents

5.4 Success Measures

It is critical to develop criteria for measuring success of any technology under development. Measures may include ease of use, successful installation, accurate programming performance, system security, accurate reporting, database accessibility, hardware and software reliability and maintainability. Success measures should reflect the essential criteria that the organization expects the solution to have. Success measures for the entire project included those for the management and development of call center applications to support outpatient health care and monitoring in a major public health event. For the tool itself and the applications developed, the measures of success were the following:

• Project completed within allotted timeframe and budget.
• Project goal met – developed IR applications that community health call centers could implement.
• IR applications developed within available IR technology/system.
• IR applications addressed the anticipated response needs for the users.
• IR applications were designed for a high probability of user acceptance (good user ratings for the following eight criteria):
  • Directions given by the IR were easy to follow.
  • Recorded voice on the IR was easy to understand.
  • Recorded voice on the IR went at a proper speed.
  • Recorded voice on the IR was at a proper volume.
  • User satisfied with experience using the IR.
  • User had a positive opinion of the IR.
  • User would trust receiving supportive contact or information via an automated system like the IR during a public health event,
  • User would accept receiving supportive contact or information via an automated system like the IR during a public health event.

Each of the above criteria and the measurements and feedback received from testing will be reviewed in the Evaluation Document below.

Return to Report Contents
Proceed to Next Section