Multimodal processing technologies
Projects that focused on audio, visual, biometric processing evaluation.
- Multimodal interface
- TRECVID Automatic Segmentation, Indexing, and Content-based Retrieval of Digital Video, originated by TREC
- CHORUS FP6 Coordination Action on Multimedia Content Search Engines
- AMI Augmented Multi-party Interaction targets computer enhanced multi-modal interaction in the context of meetings
- CHIL Computers in the Human Interaction Loop
- IM2 Interactive Multimodal Information Management
- FAME Facilitating Agent in Multicultural Exchange
- NIST Meeting Room project
- TC-STAR Technology and Corpora for Speech to Speech Translation
- MUSSLAP Multimodal Human Speech and Sign Language Processing for Human-Machine Communication
- AMIDA EU FP7 project, follow-up of FP6 AMI project
- ImageCLEF (since 2003 until now)
- CLEAR 2006 Evaluation Campaign and Workshop (Classification of Events, Activities and Relationships in human-human interaction scenarios)
Spoken language technologies
- Speech Synthesis
- ECESS European Centre of Excellence in Speech Synthesis
- Festvox CMU’s TTS project, organizing the Blizzard Challenge
- MBROLA
- DISC Project on Spoken Language Dialogue Systems and Components: Best practice in development and evaluation
- EVASY
- Speech Processing
- TC-STAR Technology and Corpora for Speech to Speech Translation
Written language technologies
- Machine Translation
- NIST MT 09
- ISLE International Standards for Language Engineering
- MT NIST annual MT evaluation campaign
- CESTA Machine Translation Evaluation Campaign
- IWSLT Evaluation campaign (International Workshop on Spoken Language Translation)
- Information Retrieval
- CLIA Cross Language Information Access: Consortium project focusing on IR, Summarization, and machine translation for Indian languages
- INEX Initiative for the Evaluation of XML Retrieval
- Amaryllis-Evaluation campaign for French Information retrieval systems
- CLEF Cross-Language Evaluation Forum: cross-language IR and QA, European languages.
- NTCIR Test Collection Project NII Test Collection for IR Systems: cross-language IR and QA, Asian languages
- SCHEMA Network of Excellence in Content-Based Semantic Scene Analysis and Information Retrieval
- TREC Text REtrieval Conference
- Information Extraction
- ACE Automatic Content Extraction
- General / other domains
- PROMISE: Provide a virtual laboratory for conducting participative research and experimentation to carry out, advance and bring automation into the evaluation and benchmarking of such complex information systems, by facilitating management and offering access, curation, preservation, re-use, analysis, visualization, and mining of the collected experimental data.
- EVALITA NLP Tools for Italian
- FIRE Forum for Information Retrieval Evaluation: IR initiative for Indian languages
- QUAERO: Germano-French collaborative research and development program, centered at developing multimedia and multilingual indexing and management tools.
- SemEval 2010 All-words Word Sense Disambiguation on a specific domain (WSD-domain)
- DEFT2010
- Technolangue program and EVALDA evaluation campaigns
- NIST 2006 HLT evaluation campaigns: TREC (Text REtrieval Evaluation Conference), RT (Rich Transcription workshop), MT (Machine Translation), ACE (Automatic Content Extraction), etc.
- SENSEVAL Evaluation exercises for Word Sense Disambiguation
- 863 Evaluations HTRDP Chinese Information Processing and Intelligent Human-Machine Interface Technology Evaluation