Appendix 4. Developing an Interactive Response Tool: The HELP Model (continued)
5.0 Interactive Response Tool
Implementation Document
The implementation phase includes building or purchasing the technology/system,
testing and piloting it, training staff on it, migration of it into an active
environment, and then going live. During this phase, the technology solution
undergoes comprehensive testing first by the project team and internal users
and then by external users. This allows for both coarse and fine tuning to
validate that application designs meet the requirements and objectives. The
implementation plan should incorporate appropriate users in the process, establish
realistic time frames for tasks, provide adequate opportunity for feedback,
and establish criteria for measuring success. Depending on the environment
of your call center, you may consider using change management strategies to
create a positive environment for learning and change among the staff impacted
by the implementation. Our project team was responsible for ensuring that each
component of the implementation phase was addressed following their clearly
defined roles and responsibilities.
Building the Technology Solution
Since we previously
purchased an IR system, our first task was to build a technology solution utilizing
this equipment. This included development of the IR applications and making
programming changes to the telephone switch to support IR application functionality.
We contracted with an IR consultant and developer to oversee the development
of the applications in accordance with our business requirements and specifications.
The IR consultant developed the applications, provided administration and maintenance
training for our Information Technology (IT) group and assisted with modifications
to the applications after testing. Our IT group made the appropriate programming
changes to our telephone switch to support the IR applications.
Implementing the Solution
The following is a project time line for the implementation milestones, the
responsible person(s), and the timeframe for completion. The implementation phase
occurs after completion of the planning, analysis, and design phases (approximately
6 to 8 weeks). The timeframes listed do not necessarily occur in series; many can be
done concurrently while other milestones are in progress. There also may be delays
between milestones due to other demands on the responsible person(s) and the resources
they require to support them.
The following table outlines the major implementation milestones for the
four applications (approximately 34 weeks to complete):
Milestone |
Responsible Person(s) |
Timeframe |
Develop IR applications |
IR Consultant |
8 weeks |
Test IR applications on IR consultant system
(test environment) |
IR Consultant |
4 weeks |
Internal testing of QI Monitoring application
prototype |
Telecommunications Engineer/Project Manager |
2 weeks |
1st External testing of QI Monitoring application
prototype |
Telecommunications Engineer/Project Manager |
4 weeks |
Install applications on IR system |
Telecommunications Engineer/IS Manager |
2 weeks |
Record messages and announcements for all
IR applications |
Telecommunications Engineer |
2 weeks |
Internal testing and validation of all four
applications |
Telecommunications Engineer/IS Manager |
2 weeks |
2nd External testing of all applications |
Project Manager |
4 weeks |
Train IT group to administer and maintain
applications |
Telecommunications Engineer |
2 weeks |
Adjust and refine applications and their
recorded messages |
Telecommunications Engineer |
4 weeks |
Document application designs and administration
procedures |
Telecommunications Engineer |
4 weeks |
Return to Appendix 4 Contents
5.1 Testing
We used an iterative testing approach that included four cycles of testing:
- Technology testing (external testing by the IR consultant and internal IT group testing).
- Internal testing by our call center staff.
- External testing by a rural user group.
- External testing by an urban user group.
This testing
approach gave ample opportunities to validate that the applications function
as required and that any difficulties in using the applications were identified
and addressed (user acceptance testing). These testing cycles are already included
in the overall implementation timeline in the previous section. Total time
for testing required approximately 16 weeks to complete.
Return to Appendix 4 Contents
5.2 Training
The training strategy for this project was to ensure that the general public with a touch
tone telephone should be able to navigate through these IR applications and
have a positive overall experience.
In a testing environment, we could not convey instructions through the media or other
channels that we would use in an actual emergency situation. Therefore, we had to
incorporate clear directions for users within the applications themselves so that there
would be a high probability of user acceptance. Some of the directions that
were incorporated into the applications include:
- An option to connect to an information provider (live person) when there
is an appropriate need or if the user gets lost within the application.
- An option to repeat a message so that the user can hear it again to fully
capture the information being relayed.
- An option to utilize the application again (such as with POD and DI applications)
so they do not have to call again to enter a different response.
- An option to return to the main menu (such as with FAQ Library application)
so they can get answers to more than one question they may have.
There is always a trade off with devoting time at each step in an IR application with
providing instructions in addition to the critical messages, so a balance in
providing good guidance while keeping callers moving through the application
has to be achieved. Testing of applications with user groups is one of the
best ways to determine that balance.
It would be best to incorporate clear instructions into the QI application; however,
we would still recommend public health agencies develop an instruction sheet
for quarantine that includes what is expected of those in quarantine, what
they should expect during the quarantine period, and what resources to contact
for anticipated difficulties in maintaining quarantine (food, medications,
finances, etc.).
Return to Appendix 4 Contents
5.3 Migration
The migration plan has both a technical and a business component. The technical
component (conversion plan) describes how the new system will be installed and
where applicable, how data in the previous system will move to the new system (this
does not apply for this technology solution). Installation of the IR technology solution
and the steps involved were discussed in previous sections. The business component
includes a change management plan that helps users understand the change and motivate
them to adapt to the changes.
The change management plan for this effort began early in the process and continued throughout
the project. The project team, steering group and external advisory panel represented
the key stakeholders and users that the technology system would impact. We
focused on communication and feedback throughout the process to ensure members
understood the costs and benefits for the organizations involved and end user
groups, our plan for implementing the technology solution, how we would measure
success, and the feedback process for continuous improvement. The iterative
testing strategy provided a feedback loop for participants and users to identify
what worked and what did not work so that appropriate changes could be made
to the applications. These ultimately should result in applications that are
well designed, efficiently deliver the required capabilities and do so in a
manner that is effective and acceptable to the user.
Return to Appendix 4 Contents
5.4 Success Measures
It is critical
to develop criteria for measuring success of any technology under development.
Measures may include ease of use, successful installation, accurate programming
performance, system security, accurate reporting, database accessibility, hardware
and software reliability and maintainability. Success measures should reflect
the essential criteria that the organization expects the solution to have. Success
measures for the entire project included those for the management and development
of call center applications to support outpatient health care and monitoring
in a major public health event. For the tool itself and the applications developed,
the measures of success were the following:
- Project completed within allotted timeframe and budget.
- Project goal met – developed IR applications that community health
call centers could implement.
- IR applications developed within available IR technology/system.
- IR applications addressed the anticipated response needs for the users.
- IR applications were designed for a high probability of user acceptance
(good user ratings for the following eight criteria):
- Directions given by the IR were easy to follow.
- Recorded voice on the IR was easy to understand.
- Recorded voice on the IR went at a proper speed.
- Recorded voice on the IR was at a proper volume.
- User satisfied with experience using the IR.
- User had a positive opinion of the IR.
- User would trust receiving supportive contact or information via
an automated system like the IR during a public health event,
- User would accept receiving supportive contact or information via an
automated system like the IR during a public health event.
Each of the above criteria and the measurements and feedback received from
testing will be reviewed in the Evaluation Document below.
Return to Report Contents
Proceed to Next Section