![]() |
![]() |
The Rich Transcription evaluation series promotes and gauges
advances in the state-of-the-art in several automatic
speech recognition technologies. The goal of the evaluation series is
to create recognition technologies that will produce transcriptions
which are more readable by humans and more useful for machines. As
such, a set of research tasks has been defined which are broadly
categorized as either Speech-to-Text Transcription (STT) tasks and
Metadata Extraction (MDE) tasks. The evaluation series was started in 2002 and continues to
this day. The next evaluation will be conducted in the Spring of 2009. The Rich Transcription 2002
Evaluation (RT-02), the first in
the RT evaluation series, was implemented in April 2002. It included
STT tasks for broadcast news speech, conversational telephone speech,
and meeting room speech in English. The evaluation also included one
MDE task speaker diarization for broadcast news and conversational
telephone speech also in English. The Rich Transcription
Spring 2003 Evaluation (RT-03S),
implemented in March-April 2003, focused on STT tasks. The STT tasks
not only included the different data domains but also extended to
different languages in addition to English. Unlike the RT-02 STT tasks,
the RT-03S STT tasks did not include the meeting data domain because
that domain was moved to another RT evaluation. The RT-03S evaluation
also included one MDE task, "Who Spoke When" speaker diarization for
broadcast news speech and conversational telephone speech in English. The Rich Transcription Fall
2003 Evaluation (RT-03F), implemented in October 2003, focused on
MDE tasks. The MDE tasks
included "Who Said What" speaker diarization, sentence boundary
detection, and disfluencies detection for broadcast news speech and
conversational telephone speech in English. The Rich Transcription
Spring 2004 Evaluation (RT-04S) was
implemented in 2004. It included separate STT and MDE tasks as well as
integrated STT and MDE tasks for the various domains and languages. The Rich Transcription Fall
2004 Evaluation (RT-04F),
implemented in October 2004, included STT tasks for broadcast news
speech and conversational telephone speech in English, Mandarin, and
Arabic and MDE tasks also for broadcast news speech and conversational
telephone speech but only for English. The Rich Transcription
Spring 2005 Evaluation (RT-05F) focused on the English Meeting
Domain speech. There were four
evaluation tasks: STT, MDE Speaker Diarization, MDE Speech Activity
Detection, and MDE Source Localization. The cross site evaluation
corpora included "conference" room meetings and "lecture" room
meetings. The Rich
Transcription Spring 2006 Meeting
Recognition focused on the English Meeting speech. There were three evaluation task supported, STT, MDE Speaker Diarization, and Speech Activity Detection. The Rich
Transcription Spring 2007 Meeting
Recognition focused on the English Meeting speech. There were three evaluation task supported, STT,MDE Speaker Diarization, and a new task Speaker Attributed STT. The Rich Transcription evaluation series is sponsored by NIST
and
is open to all interested participants. Contact Information If you would like to participate in the evaluation or obtain
additional information, send an email to our staff.
Page Created:
September 18, 2007 |
|
Speech
Group is part of IAD
and ITL NIST is an agency of the U.S. Department of Commerce |
Privacy
Policy | Security
Notices Accessibility Statement | Disclaimer | FOIA |