Word Sense Disambiguation (WSD) is a long-standing task in Natural Language Processing and Artificial Intelligence. While progress has been made in recent years, the evaluation of WSD models has been limited to a set of (mostly SemEval-based) standard datasets.
The SemDeep-6 workshop includes a challenge (shared task) based on a new multi-domain evaluation benchmark for WSD “Target Sense Verification for Words in Context” (WiC-TSV).
The main difference between WiC-TSV and common WSD task statement is that in WiC-TSV there is no standard sense inventory that systems need to model in full. Each instance in the dataset is associated with a target word and single sense, and therefore systems are not required to model all senses of the target word, but rather only a single sense. The task is to decide if the target word is used in the target sense or not, a binary classification task. Therefore, the task statement of WiC-TSV resembles the usage of automatic tagging in enterprise settings.
For the WiC-TSV challenge training, development and test sets will be provided (training and development sets already available). For more information and instructions on how to participate, please visit WiC-TSV: Word-in-Context Target Sense Verification