Annual International Contest S3 on

Semantic Service Selection

Retrieval Performance Evaluation of Matchmakers for Semantic Web Services

While the increasing number of already available Semantic Web applications and conventional Web services appears impressive, useful and potentially attractive also to major business stakeholders, there still is a major lack of implemented and deployed tools for Semantic Web services (SWS) retrieval and engineering in practice. SWS matchmakers are commonly used for selecting relevant SWS in any application setting.

This contest provides means and forum for the comparative evaluation of the retrieval performance of publicly available SWS matchmakers and the simultaneous development of required test collections.

The Semantic Service Selection (S3) contest initiative formed at the 5th International Semantic Web Conference ISWC 2006 in Athens, USA. It aims at encouraging the rapid and innovative development of tools for semantic service selection (matching and ranking); traditionally these selection tools are called SWS matchmakers.

In this respect, the S3 contest complements the SWS challenge (www.sws-challenge.org). The call for participation in any edition of the S3 contest is continuous (no deadline in principle but for each year’s edition only). Evaluation of semantic Web service matchmakers will base on classic performance metrics recall/precision, F1, average query response time.

Each entry will be peer evaluated by members of the S3 organisation committee.