Coding book:
http://www.hcrc.ed.ac.uk/~jeanc/
Authors: Carletta, J. C., Isard, A., Isard, S., Kowtko, J., Doherty-Sneddon, G. and
Anderson, A.
Title: HCRC Dialogue Structure Coding Manual
Human Communication Research Centre HCRC TR-82, University of Edinburgh, Edinburgh, Scotland; 1996

A slightly shortened version of the coding instructions can be found in
Authors: Carletta, J. C., Isard, A., Isard, S., Kowtko, J., Doherty-Sneddon, G. and Anderson, A.
Title: The Reliability of a Dialogue Structure Coding Scheme. Computational Linguistics, 23, 13-31. 1997

Number of annotators:
Main Map Task corpus was annotated using four different coders.  In all, at least 50 people have tried the scheme, with around a dozen research projects employing it. Most of the annotators were PhD students in linguistics or psychology; one was just someone with a degree in modern languages.

Number of dialogues:
128 in the original Map Task corpus (English) , plus at least as many again coded using the same scheme or  minor  variants, comprised of Map Task in other languages, in other conditions (audio-only, video-mediated, children), and dialogues for other tasks (e.g., travel planning, financial services simulations, simpler children's tasks).

Evaluations of scheme:
Full results published in Carletta, J. C., Isard, A., Isard, S., Kowtko, J., Doherty-Sneddon, G. and Anderson, A. (1997) The Reliability of a Dialogue Structure Coding Scheme. Computational Linguistics, 23, 13-31.

For act segmentation, k =.92 (N=4079,k=4).
For act classification on main corpus, k =.83 (N=563,k=4).
For act classification using naive coders and written instructions, k =.67
(N=139,k=3; agreement k =.69 when coding developer added to pool).
For main distinction between initiation, response, or ready, using naive coders, k =.84.
Disagreements were between CHECK and QUERY-YN, INSTRUCT and CLARIFY, and ACKNOWLEDGE, READY, and REPLY-Y.

Underlying task:
Linguistically motivated, but developed on map task (and therefore likely to be missing categories for goal negotiation).

List of phenomena annotated:
Primarily dialogue acts, but the papers also describe coding and reliability for higher level discourse structure built from the acts, in terms of goal-oriented dialogue games and transactions relating to dialogue planning divisions.   These levels are not part of the MATE specification.

• Initiating Moves

• Instruct

• Explain

• Check

• Align

• Query-YN

• Query-W

• Response Moves

• Acknowledge

• Reply-Y

• Reply-N

• Reply-W

• Clarify

• Ready Move

Markup language:
Current mark-up specified in an HCRC internal document; SGML-conformant and based on the TEI.

Existence of annotation tools:
No tools publicly available; in-house tools for move coding operate in python using LT-XML and Tk, and in Microsoft Word.   Nb can be configured to perform the annotation (implemented in TCL/Tk).

No semi-automatic annotation available.

Usability:
Has been used to structure the dialogue planning element of an SDS, to learn how to mark dialogue moves based on topic spotting, and to train the relationship between prosody and move type.

Contact address:
Maptask@cogsci.ed.ac.uk

Last ModificationL 27.8.1998 by Marion Klein