The NIST Speech Group's Spoken Term Detection (STD) Project is an open evaluation series of technologies that search vast, heterogeneous audio archives for occurrences of spoken terms in three languages: Arabic, English, and Mandarin.  The goal of the project is to facilitate research and development of this key technology component which is important precursor to more sophisticated retrieval technologies.   The project began 2006 with the 2006 Pilot Evaluation and Workshop.  A second evaluation will be likely occure during the Fall of 2009. 

What Sets STD Apart

The focus of the evaluation is technology-centric as opposed to user-need-centric retrieval evaluations: the difference being the terms selected for the evaluation. For example, terms for the evaluation, /bill gates/, /grasshoppers/, /basketball camp/, and /school/, are selected to test the technology's ability handle the acoustic properties of terms rather than with the need of a user in mind. STD technologies would enable information retrieval research such as TREC, SDR, and CLEF. STD is an open vocabulary task unlike word spotting. The search terms are not known a priori to the speech processing modules. Therefore, vocabulary independence is a key component of STD research.

2009 Evaluation

A second evaluation will be conducted during March/April 2009 and a Workshop will be held in May 2009. The evaluation will focus on two challenges of the technology: quickly processing larger test collections and improving robustness to varied audio sources. A new style of speech corpus will be constructed during 2008 that will challenge the even the best of systems!

Contacts

If you are working on or interested in this type of technology, please send email std-eval@nist.gov to join the community.