Academic institutions, research centres and funding organisations active in the field of Human Language Technologies evaluation.
In this section, we presents links to centres or institutions of which the main activity is monitoring the implementation of both European and international HLT evaluation initiatives, as well as organizations and institutions which are involved in different HLT evaluation activities.
- BAS
BAS is the Bavarian Archive for Speech Signals.
- CELCT Center for the Evaluation of Language and Communication Technologies
The CELCT was set up by The Center for Scientific and Technological Research (ITC-irst) and DFKI in order to promote the technology innovation in the field of the information and communication technologies. Its activities concern studying evaluation procedures, editing evaluation guidelines and launching evaluation campaigns.
- Chinese LDC]
Chinese LDC is the Chinese Linguistic Data Consortium.
- CMU
CMU Carnegie Mellon University is dedicated to speech technology research, development, and deployment.
- COCOSDA
COCOSDA, International Committee for the Co-ordination and Standardisation of Speech Databases and Assesment Techniques. It supports the development of spoken language resources and speech technology evaluation.
- CSLU
Speech Synthesis & Speech Enhancement Evaluation Lab at CSLU (Center for Spoken Language Understanding of Oregon Graduate Institute of Science and Technology OGI) supports research projects on speech technology evaluation and performs custom evaluations.
- CST Centre for Language Technology
The CST is involved in several EU evaluation projects and activities, it also carries out the development of evaluation methodolgies and metrics.
- DFKI Evaluation Center for Language Technology
The DFKI LT evaluation center is part of COLLATE project which has been funded by the German Federal Ministry of Education and Research (BMB+F) since April 2001. The objective is to conduct evaluations of the overall usability of language technology systems and advances knowledge of relevant usability issues and evaluation methods.
- ELRA European Language Resources Association
Established as a not-for-profit organisation, ELRA’s missions are to promote language resources for the Human Language Technology (HLT) sector, and to evaluate language engineering technologies. These activities are achieved through ELRA’s operational body ELDA (Evaluation & Language resources Distribution Agency).
- ICT
ICT, Institute of Computing Technology of Chinese Academy of Sciences supports the China 863 evaluation programs by organizing annual HLT evaluations.
- IDIAP
IDIAP research Institute, Institut Dalle Molle d’Intelligence Artificielle Perceptive, leading house of IM2 (the Swiss National Center of Competence in Research (NCCR on Interactive Multimodal Information Management), is involved in numerous European and international activities in the area of multimodal interaction: AMI, EARS, PASCAL Network, etc.
- ILC
ILC, Institute for Computational Linguistics at Pisa is involved in a large number of international and national activities in the HLT evaluation field such as SENSEVAL, WRITE, EAGLES, ISLE, etc.
- ISCA
ISCA, International Speech Communication Association aims to promote Speech Communication Science and Technology, both in the industrial and Academic areas.
- ISL
ISL, Interactive Systems Laboratories, joint center of CMU (Carnegie Mellon University) and TH (University of Karlsruhe) carries out research projects in speech and multimodal technology evaluation such as CHIL, TC-STAR, etc.
- ISSCO
ISSCO Dalle Molle Institute for Semantic and Cognitive Studies, part of the School of Translation and Interpretation (ETI) / Multilingual Information Processing Unit (TIM) at the University of Geneva, conducts numerous research projects in the field of HLT evaluation.
- ITC-IRST
ITC-irst, Centre for Scientific and Technological Research. The Cognitive and Communication Technologies (TCC) division conducts its research activities on HLT and supports projects of HLT evaluation as TC-STAR, PF-STAR, CHIL, etc.
- LDC
LDC is the Linguistic Data Consortium
- LIMSI-TLP
LIMSI-TLP, Spoken Language Processing Group of Laboratory of Computer Science for Mechanical and Engineering Sciences conducts researches on automatic speech processing and is involved in speech technologies evaluation projects such as TC-STAR, [http://isl.ira.uka.de/fileadmin/tem…], MEDIA, etc.
- MITRE
MITRE, not-for-profit corporation in the U.S. Its Human Language Technology research group provides expertise and supports HLT research projects funded by DARPA’s program as TIDES, Babylon, CAST.
- NTCIR
National Institute of Informatics (Japan)
- NIST Human Language Technology Evaluations
The NIST IAD supports the development of a number of cutting edge language technologies via open community evaluations. NIST HLT Evaluations, partially sponsored by DARPA’s programs, revolve around the following tasks: information retrieval, machine translation, automatic content extraction, etc. NIST Speech Group supports activities related to the speech technology evaluations: speech transcription, speaker recognition, etc.
- NordTalk
- OTA
OTA is the The Oxford Text Archive.
- SENSEVAL
SENSEVAL, International organization which organizes and runs evaluation of word sense disambiguation (WSR) system.
- SSE System and Software Evaluation Center
The SSE center is a research unit of the Institute of Information Science and Technologies (ISTI) of the Italian National Research Council (CNR). SSE center provides evaluation and certification services, e.g. study of evaluation criteria, models and methods for software measurement and evaluation, etc.
- UPC
UPC Speech Processing Group of Department of Signal Theory and Communications at Technical University of Catalonia (UPC) is involved in several evaluation projects such as BioSec, [http://isl.ira.uka.de/fileadmin/tem…], TC-STAR, etc.
- WTEC
World Technology Evaluation Center (WTEC) conducts evaluation studies in the field of information technologies.