Highlights of User Education Working Group Helpdesk Tours

NLM Tour - May 29, 1998

Carolyn Tilley, Annette Nahin, Sara Tybaert

The NLM library reference desk is staffed by about 20 people. The helpdesk for the databases on the Medlars system is staffed by 30-35 people total. They work two-hour shifts on phone or e-mail. The groups are working toward a more virtual environment.

Teloquent, software for the phone menu system, was installed in May 1997. The price was approximately $60,000. Previously there were two 800 numbers, one for the Medlars desk and the other for Grateful Med. With the move of Grateful Med from a disk-based product to the web, there are fewer hardware questions to be answered at the Grateful Med Desk.

The Teloquent system uses one server for the recordings. A central server controls what is heard. They are looking to balance out the calls. Rerouting can be done from the supervisor’s workstation. Monthly reports have been found to be the most effective, providing information about agent availability, number of calls in, the abadonment rate, calls by time of day, etc. The information can be downloaded to an Excel spreadsheet for further manipulation.

Teloquent works with special phone sets, which are actually computers. The NLM Library reference desk has chosen to have the keyboards on the screen instead of the separate keyboard.

Classes were conducted by the vendor on how to set up the routing programs. The library staff is supporting the software itself. It does not require a LAN, but can be networked.

The main 800 number for the library was a very deep phone menu. Everyone wanted to be at the top of the phone message menu. In this implementation, they have tried to shorten and flatten the menu.

In 1996/97, NLM conducted an interactive voice recognition telephone survey. They asked 6 "touchy-feely" questions about customer satisfaction. This was conducted as part of the National Performance Review. It provided a benchmark for comparison after changes were made. 700 people were queried, with a response rate of 20%. The majority of the responses were "down the middle", indicating general satisfaction.

A new call tracking system, called CUSTOMERQ (Quintus Corp.), was installed in November 1997. The Medlars, History of Medicine and Reference sections are using it as of this meeting. E-mail was incorporated first, followed by phone requests. In mid-March, 1998, PubMed e-mails were added. The system categorizes and keeps track of e-mail requests. The staff log on as agents. It supports workgroups and default routing. The question and response are indexed for future retrieval. The agent assigns a category and subcategory. This supports both the development of stock responses and provides feedback to the NLM product development teams. The categories also support the development of the NLM customer service knowledge base. It allows reports to be run on what people are asking about and how fast the agents are responding.

The first introduction of CUSTOMERQ was with NLM’s evening and Saturday contractor staff. This was done because they have more time to do the indexing and description that are necessary for this level of retrieval. Eventually it is hoped that customer service can be conducted via telecommuting from home.

Approximately 3,000-3,500 e-mails are received per month. 40% of the e-mail traffic is foreign. (Alta Vista’s translation service is used for e-mails received in foreign languages.) There is a service desk schedule for e-mail which is staffed seven days a week. The NPR benchmark is 100% of the e-mails answered in five days or less. The NLM rate is 100% in 3 days or less.

Over 80 stock replies to frequently asked questions have been developed by a team of helpdesk and reference librarians. The library technicians look at the e-mails first, responding with stock replies developed by the professionals, if possible. The Windows environment of CUSTOMERQ allows the agent to cut and paste into e-mail messages as necessary. They can also cut and paste from other help information, for example the guidance provided regarding copyright on their homepage, into the CUSTOMERQ e-mail messages.

NLM has a project called NUDUS which is evaluating e-mail composing responses to e-mail automatically.

The implementation of CUSTOMERQ is relatively simple. NLM has avoided heavy customization. Not all capabilities of the system are currently used. There is more customer data that could be collected, but this is not being done because it is too time consuming.

Stephen Hunt, Keith Stubbs

NLE described its digital references service, AskERIC. This is an e-mail based question answering service. The goal is to provide a "personalized Internet-based service providing education information to teachers, librarians, counselors, administrators, parents and others throughout the US and the world." The number of novices online has increased the need for this type of help. The idea is to promote the user’s ability to use the ERIC. In the case of students, sometimes the goal is to guide them to the answer rather than give it to them directly. (One issue is how to encourage people to go to their local libraries? ) The questions are also used as feedback for NLE system and product developers.

In November 1992 AskERIC was developed as a gopher site. It was moved to the Web in 1993. Approximately 200,000 questions have been answered since its introduction. The questions are up to about 1,000 per week, up from 300 per week in 1994.

AskERIC (www.askeric.org) is managed by the ERIC center at Syracuse University. Syracuse examines the central mailbox and routes questions as appropriate to the other 15 clearinghouses and other volunteer groups. Online forms in which the users profile themselves have been developed to help with the routing of the information.

AskERIC system consists of three components. A question and answer service, a virtual library, and development and research activities.

The Q&A system is based on triage. An attempt is made to answer the question up-front and if not, to route the question to the most appropriate location. An interim response may be sent while a more detailed response is being formulated, perhaps by another center. This type of routine requires a tracking system. It is also supported by a knowledge base of existing answers.

There are a number of tools in place or planned that support efficient response to these messages. Templates are used for responses. The responder can import sections, including URLs, to the e-mail response. A System-Wide Digital Reference Guide is being developed as of this meeting.

Digital reference is extremely problematic in some countries. This is particularly true when the respondent is restricted as to connection time within the country.

The Virtual Library consists of Lesson Plans, the ERIC Database, Infoguides in specific topical areas maintained by graduate students, listserv discussion groups, companion materials for television series, and special projects. Lesson plans on the Internet are extremely popular. This accounts for 30% of all the Web hits. A virtual library also provides links to other organizations, educational resources, etc. A User Survey has been conducted, with approximately 200 responses. The respondents indicated that they found the ERIC citations and the URLs for Internet sites in Infoguides most valuable, and the listservs and lists of educational organizations of lesser value. Free accounts from AOL and other ISPs have an impact on such surveys and to some extent responses, because they encourage use by more novice users.

NLE contracted with Chuck McClure and Carol Hurd for a critique of the web site. Several valuable suggestions were made including a disclaimer statement (what you do and do not do for the user), warnings about load time, power supplies, etc. The areas of internal management structure, usability testing, policy, log analysis, etc. were also reviewed.

The Virtual Reference Desk (www.vrd.org) is a project to "establish a national cooperative digital reference service to coordinate participants in the growing field of online question answering services for the K-12 education community." AskERIC and other similar services have suffered from a linear cost model. NLE is paying a fixed price for each answer. An objective of the Virtual Reference Desk (VRD) is to find a way to reduce the load on the government, by creating a national cooperative service. This project brings together answering services for K-12, using an architecture similar to that of the AskERIC clearinghouses. It uses meta-triage by incorporating more distributed voluntary groups in the routing and answering load balancing. The Web site also provides a locator service (AskA+ Locator) for the organizations who are involved. Other services may include matching experts with these organizations who can use their services, and creating new digital reference services through training, software and consultation. The VRD project provides guidelines and a quality baseline for establishing an online reference service.

An automatic parser is under development that evaluates the e-mail question up-front and sets up possible answers and possible referrals. This is then followed by human evaluation. The human sends the results to the tracking database. The answer always cycles back to the knowledge base.

Quality criteria is being defined. Such a system needs to match user expectations. There is a vast difference between answering questions on your organization (which is what many organizations and government agencies do) and answering questions within a specific subject (digital reference). Helpdesk software vendors were contacted, but many of them focus on much smaller areas of expertise and on much less distributed environments. The answers are often much shorter in length. Some helpdesk vendors have indicated an interest in expanding into digital reference.

Another area of research is the metadata that travels with the question. This helps in tracking, answering, and reuse of the answer.

While the NLE and NLM systems are addressing different markets, they are both feeling the stress of increased usage by more users, and more novice users. Both are seeking new methods of organizing themselves and new automated supports to address the increases in reference traffic. A key problem area for both was increased foreign e-mail traffic and the communication problems entailed in responding to this increase.