Tremendous effort has been devoted to mining information in newswire, news broadcasts, and conversational speech and in developing interfaces to metadata extracted in these domains. However, little has been done to address such applications in the more challenging and equally important meeting domain. The meeting domain includes a large number of subdomains including judicial proceedings, legislative proceedings, lectures, seminars, board meetings, and a variety of less formal group meeting types. All of these meeting types could benefit immensely from the development of automatic recognition, understanding, and information extraction technologies which could be linked with a variety of online information systems. Currently, no such technologies exist.

The development of smart meeting room technologies which can recognize information about meeting participants, speech, interactions, relationships, decision-making activities, and information dissemination will provide an invaluable resource for a variety of business, academic, and governmental applications. Such core metadata can provide a valuable basis for the development of second-tier meeting applications incuding automatic minute/transcription generation, summarization/abstraction, translation, retrospective search and retrieval, and semantic analysis. Third-tier applications will provide a context-aware collaborative interface between live meeting participants, remote participants, meeting archives and vast online resources. Given that the core meeting recognition technologies are in a fledgling or nonexistent state, it is essential that these first tier recognition technologies be developed before the higher tier applications can be made useful.

NIST has begun a project to address development of the core recognition applications and is collaborating with several other Government agencies and research institutions working on developing such technologies in the meeting domain.

Introduction

NIST has created a project to support the development of audio and video recognition technologies in the context of human-human meetings. The project includes periodic technology evaluations and workshops as well as an extensive data collection effort. In addition to collecting its own multi-media meeting corpora, NIST is collaborating with several other data collection sites to provide a broad base of corpora for research, development and evaluation. The recognition technologies currently addressed by the effort include speech-to-text transcription (STT), speaker segmentation, and video information extraction.

NIST conducted a first exploratory evaluation of speech recognition in meetings in Spring 2002 (see Rich Transcription). The evaluation addressed STT and speaker tasks. The evaluation showed that speech recognition performance for this domain is significantly worse than for other more heavily researched domains (conversational telephone speech and news broadcasts). This is not merely an issue of research funding. Rather, it is because the meeting domain poses several acoustic and language challenges not found in these domains. NIST is also collaborating with other agencies in the area of video analysis and content extraction, to support evaluation of several video extraction technologies in the context of meetings including person/object identification/tracking, text extraction, gesture recognition, and other video event-based tasks.

Unique Technical Challenges

The meeting domain has several important properties not found in other domains and which are not currently being addressed in other research programs: