Diagnostic and Evaluation tools for Natural Language Applications

Logo: DiET

As industrial use of Language Technology is flourishing and the market for competing software systems is expanding, there is an increasing need for the assessment of NLP components on the level of adequacy evaluation and quality assurance. However, effective and efficient assessment is often restricted by the lack of suitable test material and technology. While ISO norms and standards already exist for mainstream software, the basis and techniques for this are still largely lacking in the area of Language Technology. DiET will develop data, methods and tools for the glass-box evaluation of NLP components, building on the results of previous projects covering different aspects of assessment and evaluation. It will extend and develop test-suites with annotated test items for grammar, morphology and discourse, for English, French and German. DiET will provide user-support in terms of database technology, test-suite construction tools and graphic interfaces. The project will result in a tool-package for in-house and external quality assurance and evaluation, which will enable the commercial user to assess and compare Language Technology products.

Funded by:European Union (Language Engineering Sector of the Telematics Application Programme)
Project Manager:Hans Uszkoreit (Hans.Uszkoreit@dfki.de)
Contact:Hans Uszkoreit (Hans.Uszkoreit@dfki.de)
Duration: 01.04.1997 - 31.03.1999
Partners:GermanyDFKI (coordinator),
GermanyIBM, Heidelberg,
IrelandLocalisation Resources Centre, Dublin,
GermanySRI International Cambridge,
SwitzerlandISSCO, Genf