Peter Bohlin, Robin Cooper, Elisabet Engdahl and Staffan Larsson Information States and Dialogue Move Engines

[Full Text] [send contribution] [debate procedure] [copyright]

C1. Johan Boye (16.12.99):

One thing that bothers me about your paper is that you do not clearly state what kinds of dialogue applications your architecture is intented to manage (is it just info-seeking applications as in the example, or do you expect to handle a broader class?). The reader might easily get the impression that your approach is more general (or less general) than it really is.

Another matter I find obscure is the relation between your general architecture and the concrete system GoDiS. By the example in Section 6, one gets the impression that GoDiS has something to do with travel information, but that is not clearly stated anywhere. Or are you saying that GoDiS is an instance of the general framework where the update and selection rules are specified, but the domain is not fixed? Please clarify this issue.

You should also motivate the choice of data structures in your architecture. As it is presented now, you have fixed the "QUD" and "AGENDA" fields to be "stacks", the "PLAN" field to be a "list", and the "BEL" field to be a "set". Though this is not spelled out in your paper, it seems likely to me that this will have operational consequences for every dialogue application which is built on top of this architecture; e.g. the actions on the agenda will always be executed in the opposite order they were put on the agenda; the questions under discussion will be addressed in an order opposite to that they were put on the QUD stack. It is also not clear to me what you mean is the difference between a "list" and a "set"; is it that a list can contain repetitions while a set cannot, or is it that the order of the elements is of importance in a list but not in a set? Or is it both (or neither)?

Considering that you claim your architecture to be "general", it is somewhat surprising that you have already taken a lot of low-level design decisions, especially as they are not very well motivated in the text.

Another somewhat surprising property of your architecture is that there doesn't seem to be any explicit representation of the objects under discussion (a "flight" would be a typical object in your example). I first thought this was the role of the "QUD" field, but after looking at the example in Section 6, I understand that this is not so. In that example, it seems to be implicit that all questions, actions and beliefs concern the same object/flight; your formal language doesn't seem to be able to represent a situation where several flights are under discussion. But this would be a very common situation in your flight-booking scenario; consider the case where the system has suggested two flight alternatives and the user wants to discuss their relative merits.

Some detailed comments:
In two places in the article you refer to "our dialogue" (beginning of Section 5.1 and beginning of Section 5.2). I assume that this is the dialogue mentioned in one of the footnotes. As this example dialogue seems to have been quite influential when you designed your architecture, it would be useful for the reader to have a proper reference to where that dialogue can be found.

In Section 4.2, you say "The semantics (if it deserves the name)...". What do you mean by this? Either the semantics is appropriate for your needs or it isn't. If it isn't, then you should tell the reader what is wrong with the semantics, and how you intend to modify it for a future version.

In the selection algorithm in Section 4.1, one gets the impression that also selection rules can perform updates to internal data structures, although this is not mentioned when selection rules are introduced (in Section 2).

Best regards,
Johan Boye,
CTT and Telia Research

A1. Staffan Larsson et al. (27.1.00):

Johan Boye comments:

"Information States and Dialogue Move Engines".

Johan Boye comments:

The authors reply:

There are various levels of specificity involved here. The toolkit TRINDIKIT specifies a general architecture and a format for update rules, selection rules, information states and modules that can be interfaced with the dialogue move engine (control, interpreter, generator, input and output modules). This allows the implementer to specify a number of modules to make a generic dialogue system that can be combined with different modules to make particular specific instances of dialogue systems. For example, the implementer may decide to use TRINDIKIT to specify, information state type, update rules, selection rules and other modules external to the dialogue move engine (control module, interpreter, generator, input and output modules). The collection of these modules that we have specified for GoDiS form such a generic dialogue system. In order to make a fully instantiated system you in addition need a lexicon, a database and domain knowledge. The idea of that you should be able to substitute different instances of these modules with the same update rules etc and thus obtain different instances of generic GoDiS. We should emphasize, however, that what we have implemented at the moment are very small and in some cases quite trivial intances of such modules to illustrate the design we intend.

Johan Boye comments:

Johan Boye comments:

Considering that you claim your architecture to be "general", it is somewhat surprising that you have already taken a lot of low-level design decisions, especially as they are not very well motivated in the text.

The authors reply:

Johan Boye comments:

Another somewhat surprising property of your architecture is that there doesn't seem to be any explicit representation of the objects under discussion (a "flight" would be a typical object in your example). I first thought this was the role of the "QUD" field, but after looking at the example in Section 6, I understand that this is not so. In that example, it seems to be implicit that all questions, actions and beliefs concern the same object/flight; your formal language doesn't seem to be able to represent a situation where several flights are under discussion. But this would be a very common situation in your flight-booking scenario; consider the case where the system has suggested two flight alternatives and the user wants to discuss their relative merits.

The authors reply:

Johan Boye comments:

Some detailed comments:

In two places in the article you refer to "our dialogue" (beginning of Section 5.1 and beginning of Section 5.2). I assume that this is the dialogue mentioned in one of the footnotes. As this example dialogue seems to have been quite influential when you designed your architecture, it would be useful for the reader to have a proper reference to where that dialogue can be found.

The authors reply:

Johan Boye comments:

In Section 4.2, you say "The semantics (if it deserves the name)...". What do you mean by this? Either the semantics is appropriate for your needs or it isn't. If it isn't, then you should tell the reader what is wrong with the semantics, and how you intend to modify it for a future version.

The authors reply:

Johan Boye comments:

In the selection algorithm in Section 4.1, one gets the impression that also selection rules can perform updates to internal data structures, although this is not mentioned when selection rules are introduced (in Section 2).

The authors reply:

C2. Joris Hulstijn (1.2.00):

I have a comment for the discussion of Boylin et al, and especially on their reply to comments by Boye. It concerns the notion of QUD from Ginzburg's work and the way it has been applied in the Trindi architecture.

1. With regard to the definition of QUD as a stack:
There are two ways to think of QUD: (1) as a datastructure that can be used in a dialogue manager, (2) as a structure in a semantics of dialogue. Both are valid and useful points of view. Ideally (1) should mimic (2) as best as practically possible.

From the reply I take it that Bolin et al follow the datastructure view. Then it makes sense to make QUD a stack, because it can handle the initiative-response structure of typical exchanges like question-answer, or proposal-acceptance, etc. Such exchanges of the take the shape of a 'dialogue grammar'. However, in that case QUD partly overlaps in function with the Agenda data-structure.

If you treat it as a semantics that can handle some (technical) kind of relevance, Ginzburg's partial order makes more sense. In that case QUD should be closed under all kinds of aspects that influence relevance (questions related to current goal, questions related to topic, ...).

2. With regard to `objects under discussion':
An object under discussion is generally called a topic (or focus of attention or backward looking centre). Various people, most notably Jan van Kuppevelt (1995) have tried to combine question-based notions of topic with more general notions (e.g. Reichman 1985). Often one assumes for each topic a 'topic space' of possible attributes and possible sub-topics. These can be used in anaphora resolution for example.

If an object is introduced as a new topic, usually questions as to its attributes and questions about its sub-topics will become relevant too. So a semantic QUD ought to be 'closed under' such topic-related questions. To get rid of some redundancy, in simple task domains you can do away with the topic structure and only model the questions structure, or vice versa. For an information-seeking system, it makes sense to have questions at the heart of the dialogue manager.

Van Kuppevelt (1995) "Discourse Structure, Topicality and Questioning", L&P(31), 109-149

Reichman (1985), "Getting Computers to Talk like You and Me" MIT Press, Cambridge Mass.

Best Regards,
Joris Hulstijn,
University of Twente

C3. Staffan Larsson et al. (3.2.00):

We would like to thank Joris for some useful and interesting discussion.

Joris Hulstijn comments:

1. with regard to the definition of QUD as a stack There are two ways to think of QUD: (1) as a datastructure that can be used in a dialogue manager, (2) as a structure in a semantics of dialogue. Both are valid and useful points of view. Ideally (1) should mimic (2) as best as practically possible.

The authors reply:

You are right that there are two rather different notions of QUD at issue. However, we would not have drawn the distinction the way you are. We are indeed taking a simplified view of Ginzburg's idea so that it can be applied to simple implemented systems, so in that sense we are interested in defining QUD as a datastructure which will be used in dialogue management. However, we think it would be a mistake to separate semantics on the one hand from dialogue management on the other. A lot of what Ginzburg is talking about in his semantic approach is a theoretical approach to aspects of dialogue management and that is a large part of the interest in it. What we see as the main distinction between the QUD that we have used and Ginzburg's original notion is that ours is a local QUD. It represents the precise questions that are currently under discussion, i.e. that are so to speak up front and have been explicitly introduced into the dialogue. Ginzburg's original notion, as you say, was much more of a "global" QUD representing questions that arise from what has been said. We suspect that both notions of QUD will ultimately be necessary, though perhaps our plan does some of the work of a global QUD.

Joris Hulstijn comments:

From the reply I take it that Boylin et al follow the datastructure view. Then it makes sense to make QUD a stack, because it can handle the initiative-response structure of typical exchanges like question-answer, or proposal-acceptance, etc. Such exchanges of the take the shape of a 'dialogue grammar'. However, in that case QUD partly overlaps in function with the Agenda data-structure.

The authors reply:

Having opted for a local QUD the issue then arises as to what kind of data structure to use. Our original choice of a stack as opposed to a partial order was, well, simply because you have to work harder to implement a partial order and it's not something you would want to do if you can get away with a stack. But given what we have seen so far (and remember we don't have a large coverage dialogue system here) it seems that you are absolutely right -- a local QUD can be regarded as a stack. It is also the case that there is a tight coupling between the Agenda and the QUD. Popping agenda items often (particularly in the kind of information seeking dialogues we have been looking at) gives rise to new questions on the QUD. (e.g. if you raise a question, you may assume that that question is under discussion). New questions on the QUD give rise to new agenda items. (e.g. if your dialogue partner raises a question then that creates an obligation (agenda item) for you to respond.) We think that you need both Agenda and QUD in order to be able to handle grounding phenomena. e.g. the fact that you have raised a question (and popped the Agenda), does not necessarily mean that you are licensed to put the question on QUD (which represents your view of shared information). On a cautious view of grounding the question only gets pushed onto QUD once confirmation has been received from the other dialogue participant.

Joris Hulstijn comments:

If you treat it as a semantics that can handle some (technical) kind of relevance, Ginzburg's partial order makes more sense. In that case QUD should be closed under all kinds of aspects that influence relevance (questions related to current goal, questions related to topic, ...).

The authors reply:

Yes, it seems clear that a stack structure would be inappropriate for a global QUD.

Joris Hulstijn comments:

2. With regard to `objects under discussion'
An object under discussion is generally called a topic (or focus of attention or backward looking centre). Various people, most notably Jan van Kuppevelt (1995) have tried to combine question-based notions of topic with more general notions (e.g. Reichman 1985). Often one assumes for each topic a 'topic space' of possible attributes and possible sub-topics. These can be used in anaphora resolution for example.

The authors reply:

Right, nothing we have said in the paper has any bearing on this. We hope, of course, that what we have done is not inconsistent with adding a treatment along these lines. We will see.

Joris Hulstijn comments:

If an object is introduced as a new topic, usually questions as to its attributes and questions about its sub-topics will become relevant too. So a semantic QUD ought to be 'closed under' such topic-related questions. To get rid of some redundancy, in simple task domains you can do away with the topic structure and only model the questions structure, or vice versa. For an information-seeking system, it makes sense to have questions at the heart of the dialogue manager.

The authors reply:

Yes, and here would probably be the motivation for having a global QUD in addition to a local QUD. However, it seems that some of the work of this is done by our notion of plan and plan accommodation. That is, the agent has dialogue plans, largely a list of relevant questions that need to be raised, in what we have at the moment, and such plans get pulled into the agent's dialogue information state when certain issues are raised. See the discussion of the example "Flights to Paris" in the paper. We need to think carefully about the relationship between this and the more general notions you are raising.

C1. Ingrid Zukerman (22.2.00):

Having read the paper and the discussion, I have some further comments. As I indicated in my comments to Aretoulaki and Ludwig, I come from a slightly different background, so I hope my comments make sense.

In general, I have trouble with some of what I have read, in the sense that new approaches for dialogue are being presented, but then the examples being handled can be handled by existing/simpler techniques (see my comments to Aretoulaki and Ludwig). The need for a new approach should be motivated by showing that existing approaches cannot handle a particular phenomenon, but this is not done here. Clearly, existing approaches haven't solved the dialogue problem. However, in this case, it should be easy to come up with examples that are not being handled by existing approaches and should be handled by the new approach. For instance, a case in point is the idea of AGENDAS and PLANS to handle short-term and long-term dialogue goals respectively. This is a nice idea, but I believe that some of the problems handled by this approach are addressed by the PRS architecture. A summary of this architecture appears in PRS Summary. I am not an expert on this architecture, but I think the issue deserves to be looked at.

In your reply to the first comment (by Johan Boye) you say that the general architecture is in TRINDIKIT and that the GoDiS system was designed to illustrate possible information state transitions. This should be clarified in the paper. Specifically, what are the dialogue features GoDiS is testing. Further, it would be good to separate the general TRINDIKIT features from the specific design decisions made for GoDiS to study specific phenomena. In this context, I didn't find your answer ``[GoDiS] is meant to illustrate ideas that we have about possible information state transitions'' illuminating. Could you please be more specific?

Another comment that ties together the last two comments is as follows. The idea of accommodation is neat. However, in light of the examples covered, it is not clear that it is required. The example covered in the paper, ``flight to Paris'', could be easily handled with a semantic parser, and with slots that should be filled with subsequent information provided by the user (and queried about). [Raskutti and Zukerman 1997] describes one inference-based approach for handling this problem. So, like before, it is not clear what problems are being addressed by the general TRINDIKIT architecture, the specific GoDiS implementation, and why are new approaches necessary.

Additional (more detailed) comments

• In page 2 (.ps version), you distinguish between the roles performed by update and selection rules, but in the following paragraph you state that selection rules also update the DIS, like update rules (+ selecting a dialogue move). In the discussion, you then say that selection rules update the data structure specifying the next move. What happened to the DIS updating they used to do? Who is doing it now? And if they still do DIS updating, why can't this be done by update rules?
• How are the rules activated? Are their effects order-independent? i.e., is it possible that RuleA can be applied before RuleB but not after it? If so, how is this problem addressed?
• I am not entirely clear about the entries in the rules in (2) and (3), in particular about the differences between them. I think a walk-through of these rules would be appropriate.
• You advocate an optimistic approach to grounding whereby anything said by the system is assumed accepted by the user. Is this a GoDiS shortcut, or is it part of the general architecture? If the latter, then it precludes the modeling of misunderstandings.
• In page 4 (.ps version), second column, you talk about ``the accommodation rules described above''. Which rules? I thought (2) and (3) are update rules. Further, you talk about ``improving the conversational behaviour of the system by adding some new rules''. Could you please give some examples?
• One of your answers to Boye's questions confused me a bit. You wrote ``[QUD] represents questions that are currently being addressed, rather than a collection of questions that could be relevant and taken up.'' Actually, I understood that they were the second thing, since during accommodation you are trying to match the input to answers to possible questions.
• In the example, how is the next action determined? (asking the user where s/he wants to travel from) i.e., how does the system decide which parameter to query next? (one approach to solve this problem is described in [Raskutti and Zukerman 1997]).

References
Raskutti, B. and Zukerman, I. (1997), Generating Queries and Replies during Information-seeking Interactions. International Journal of Human Computer Studies 47(6), 689-734.
<-- Answer -->

C1. Bohlin et al (29.5.00):

First we would like to thank Ingrid for her comments, and we hope our (belated) answers are useful.

Ingrid Zukermans comments:

Having read the paper and the discussion, I have some further comments. As I indicated in my comments to Aretoulaki and Ludwig, I come from a slightly different background, so I hope my comments make sense.

In general, I have trouble with some of what I have read, in the sense that new approaches for dialogue are being presented, but then the examples being handled can be handled by existing/simpler techniques (see my comments to Aretoulaki and Ludwig). The need for a new approach should be motivated by showing that existing approaches cannot handle a particular phenomenon, but this is not done here. Clearly, existing approaches haven't solved the dialogue problem. However, in this case, it should be easy to come up with examples that are not being handled by existing approaches and should be handled by the new approach. For instance, a case in point is the idea of AGENDAS and PLANS to handle short-term and long-term dialogue goals respectively. This is a nice idea, but I believe that some of the problems handled by this approach are addressed by the PRS architecture. A summary of this architecture appears in PRS Summary. I am not an expert on this architecture, but I think the issue deserves to be looked at.

The authors reply:

Judging from a quick look, PRS (Procedural Reasoning System) seems to be a fairly classical planning system. General planning and inference systems represent one end of the complexity spectrum for dialogue system design; at the other end we have simple slot-filling or finite-state systems used in most practical dialogue system applications today. One of the ideas with the TRINDI architecture is to make it possible to explore the rest of the spectrum. The idea is to avoid the complexity problems that come with general reasoning and planning, but still be able to diaplay complex and natural behaviour.

When the paper was written, our research had not focussed on agendas and plans, but rather on grounding and accommodation. Lately, however, we have been experimenting with translating menu-driven interfaces to dialogue systems by converting menu structures into dialogue plans. This requires a more complex plan representation, with conditionals and embedded subplans. While the dialogue plan in the paper was very simple and very much like a slot-and-filler frame, the menu-derived plans go beyond what can be done with frames as standardly conceived. This shows, in our view, that the dialogue plan approach is more general than the frame-filling approach, while still not being too computationally complex.

Ingrid Zukermans comments:

Ingrid Zukermans comments:

Additional (more detailed) questions In page 2 (.ps version), you distinguish between the roles performed by update and selection rules, but in the following paragraph you state that selection rules also update the DIS, like update rules (+ selecting a dialogue move). In the discussion, you then say that selection rules update the data structure specifying the next move. What happened to the DIS updating they used to do? Who is doing it now? And if they still do DIS updating, why can't this be done by update rules?

The authors reply:

At the time when we wrote thew paper, we made a distinction between update rules (which update the DIS) and selection rules (which specify next moves and (optionally) updates the DIS). In the discussion we refer to the later version, where selection rules are a subset of update rules. Selection rules modify the DIS (or rather, the TIS as it is now called) by adding moves to the next_moves set. The next_moves variable is a part of the TIS, so to update next_moves is to update the TIS.

Comments:

The order in which rules are activated is determined by the update algorithm. Originally, we used the simple algorithm descirbed in the paper (basically, pick the first rule whose preconditions are true and apply it; repeat this until no more rules apply). In the current version, the update algorithm is more complex; rules are divided into classes and the algorithm specifies which class(es) can be activated at any given stage of execution. The algorithms are specified using traditional imperative constructs (repeat, if-then, etc).

Comments:

U-RULE: integrateLatestMove(answer(usr))
PRE:	valRec( SHARED.LM, answer(usr, A) )	P1
	fstRec( SHARED.QUD, Q )			P2
	relevant_answer( Q, A )			P3
EFF:	popRec( SHARED.QUD )			E1
	reduce( Q, A, P )			E2
	addRec( SHARED.BEL, P )			E3

The second rule is the same, except for two things: since the system knows which question it was answering, it is not necessary to check for question-answer relevance; the system only answeres questions which are topmost on QUD. A second complicating factor is that the rule in (3) also assumes that the system's move has not yet been grounded - it is stored in PRIVATE.TMP.LM rather than SHARED.LM. In later versions, optimistic grounding of system moves is handled by a separate rule, thus simplifying the integration of system moves.

Comments:

The optimistic approach to grounding is a part of GoDiS; the TRINDI architecture itself says nothing about grounding. We believe that dialogue participants are able to switch between optimistic and cautios grounding strategies; however, we have chosen to use only the optimistic approach in the current GoDiS version. It is true that we cannot model misinderstandings with this simple strategy; however, we are able to handle basic grounding behaviour. We hope to explore this issue further in future research.

Comments:

The idea is that we have start out from a basic system for information-seeking dialogue. By adding a couple of additional rules (the two accommodation rules, which are a type/class of update rules) we improve the conversational behaviour so that the user can give information in any desired order, and does not have to explicitly explain her errand.

Comments:

The set of "questions that are currently being addressed" is subsumed by the set of questions that could be relevant. During accommodation we first try to match against the QUD, but if this fails we match against against all possible questions in all possible plans - these are "the set of questions that could be relevant".

Comments: