Sharon Oviatt
Center for Human-Computer Communication
Oregon Graduate Institute of Science & Technology
oviatt@cse.ogi.edu, http://www.cse.ogi.edu/CHCC
Computational work on multimodal system design has focussed on a number of phenomena that may best be described as "myths"- or misperceptions of the linguistic nature of typical multimodal constructions. Unfortunately, computational presumptions about multimodal constructions could lead to misdirected system building by the community at large. In this presentation, we will focus our discussion on three common myths about multimodal input, findings from recent empirical work that uncovered these myths and, finally- our own different view of what constitutes a prototypical multimodal construction for the case of combined speech and pen-based input.
To focus discussion, we will limit ourselves to a discussion of the following three myths:
Speech + pointing is the dominant multimodal integration pattern.
Multimodal constructions involve redundant propositional content supplied by the input modes involved.
We also will discuss other input modalities that we view to be promising, and research strategies that can be used to build a scientific foundation of information on modes other than speech and pen. Finally, we will discuss general questions that we think need to be asked and answered by multimodal researchers in the field at large, as well as major outstanding challenges for multimodal/media system design.
The results to be presented and discussed (i.e., including simple graphic summaries of data relevant to the three myths articulated above) are included in the following paper:
Oviatt, S. L., DeAngeli, A. & Kuhn, K. Integration and synchronization of input modes during multimodal human-computer interaction, in Proceedings of Conference on Human Factors in Computing Systems: CHI '97, New York, ACM Press, 415-422.
This and other related publications also can be viewed and printed from the "publications" section of the following URL: http://www.cse.ogi.edu/CHCC.