This evaluation is intended to help promote technology development for event detection in video surveillance. The goal is to leverage machine learning techniques for detecting a pre-defined set of events. This evaluation task focuses on detecting events in airport surveillance video. We have a mailing list for announcements/discussion -- contact travis.rose@nist.gov and jonathan.fiscus@nist.gov to subscribe.

Evaluation Planning

As we write the evaluation plan, we will be asking for input from the research community. Planning telecons will be held during the following dates/times:

• Minutes of the Jan. 16, 2008 telecon
• Minutes of the Jan. 23, 2008 telecon
• Minutes of the Jan. 30, 2008 telecon
• Minutes of the Feb. 6, 2008 telecon
• Minutes of the Feb. 13, 2008 telecon
• Minutes of the Feb. 27, 2008 telecon
• Minutes of the March 19, 2008 telecon
• Minutes of the March 27, 2008 telecon

The evaluation plan is evolving and the following resources are available:

• Initial Discussion Documents:
  • The following slides outline several factors of the evaluation which we plan to discuss.
  • These slides outline some additional considerations of the evaluation protocol.
• Evaluation Plans:
  As the evaluation plan changes, two versions will be published: a normal text version and a comparison document. The comparison document will contrast the latest version with the previous version to focus the reader on recent changes.

  Normal Text Version	Comparisons Document
  Straw Man Proposal - V1
  Eval Plan - V2	V1 compared to V2
  Eval Plan - V3
  Eval Plan - V4
  Eval Plan - V5
  Eval Plan - V6
  Eval Plan - V7

• Event Definitions:
  • The following table lists 20 candidate events and responses from the forum.
  • The following paper EventDet-ViPER-format-v4.pdf documents XML Schema Definition (XSD) proposed for a ViPER XML Representation of Events.
  • The following TrecVid08_XSD_v2.tar includes the XSD and a sample XML file for events.
  • An annotation guidelines document TRECVid08_Guidelines_v1.6.pdf is now available.
• Experimental Control Files:
  • The following TRECVid08_ecf_v2.tar includes the XSD and a sample XML file for Experimental Control Files (ECFs).

Software

Download: F4DE-1.2-20080912-1800.tar.bz2

Data Resources

A TRECVid Event Detection microcorpus is now available. Please contact travis.rose@nist.gov and jonathan.fiscus@nist.gov for details.

Dry Run

The dry run for the TRECVid Event Detection Evaluation has now commenced. The procedure for participating is as follows:

• Download the Experimental Control File (ECF): expt_08_RetroED_Dev08_eng_s-camera_NIST_061808.xml
• Run experimental system on the video intervals specified in the ECF
• Package system outputs in ViPER format and make sure it validates against the TrecVid08.xsd: TrecVid08_XSD_v2.tar -- see the software download (above) for a validation script
• Prepare a tar/gzip of all outputs as described in the evaluation plan and submit the archive via FTP
• Submissions are due 11:59PM Eastern Time (UTC -4), July 21, 2008
• Results of the dry run will be returned no later than August 1, 2008

Questions may be sent to travis.rose@nist.gov and/or the discussion list. Thanks for your participation!

Related

TRECVid Home Page