Next:2.2 Terms related to
Up:2 Basic Notions
Previous:2 Basic Notions
It cannot be denied that there is a lot of confusion about the
fundamental notion of media. One reason for this misunderstanding
is certainly due to the fact that the term media is used with
different meanings in different contexts, such as in semiotics,
psychology, telecommunications, or computer science. The closely related
term modality is a further source of confusion. Some authors
seem to use both terms as synonyms, while others tend to reserve
modality only for input (to be processed by a machine interpreter),
and vice versa, media only for output (to be produced by a machine
generator). Each view has its pros and cons, depending on the
aspect under consideration. It is not our intention to provide the ultimate
solution to the medium/modality-debate across disciplines.
Rather, our use of terms resulted from a pragmatic ``merge'' of
different approaches. Our view of the medium/modality distinction
is illustrated in Fig. 2.
Figure 2: The Media/Modality Distinction
The leftmost column represents a human user sensing the environment
which, among other things, may include output delivered by dedicated
display devices. Taking this view, we associate different meanings
with the notion medium depending on the particular focus or
Less surprisingly, the term multimedia refers to the use of
However, one has to take into account which meaning of media is
meant. In the first case, one may think of a common physical space in
which different perceptible entities can be realized. Examples include
the simultaneous production of visibles and audibles. In the second
case, multimedia stands for a composition of some basic media
types. For example, one may compose a medium video-with-sound
of the two media video and audio. A characterization of
composed media may include additional properties such as temporal
relationships between the involved media.
- medium as physical space to realize perceptible entities
With the human sensory apparatus as ``target system'', we can consider
a medium as being a certain physical space in which perceptible
entities are realized.
As there are different types of perceptibles, say
visibles, audibles, hapticables, olfactables and
so forth, one may use these terms for differentiating between media as well.
However, further refinements can be made, e.g., to differentiate
between mono- and stereo visibles or audibles. In any case one may
characterize a medium among the physical dimensions which are relevant
to realize a particular perceptible entity.
- medium as a type of information and/or representation format
In this view, medium refers to a certain type of information and/or
the representation format in which information is stored. Examples are
pixmap graphics, 3D-scene descriptions, sequences of video frames,
audio data, etc.. It is assumed, that for each medium there is
a dedicated physical device which is able to produce perceptible
entities. In other words, devices serve to display media objects
(before the user's eyes, ears, etc.).
Apart from screens, projectors, printers, loud-speakers, etc., there are also
more futuristic examples of devices such as spatial light modulators
for electronic holography, or devices which can produce smells.
A framework for a feature-based characterization of media types is
currently under development in the context of the PREMO
A further important distinction is that between types
and instances. Again, what we understand as medium is decisive.
When considering medium as a physical space, instantiations are particular
perceptible entities. For example, a beep is an instantiation of an audible.
In the other reading of media, an instance corresponds to a particular
set of data in a certain format, for example the ascii source file of
this document is a media object.
During the last years, research in multimedia systems has concentrated
on the representation, creation, storing, conversion, distribution,
and display of media objects. When dealing with IMMPSs, however, the
information content of a media object and the way it is encoded
have to be addressed, too. The right most column of Fig. 2
illustrates what is meant by information content. Assuming a human
author, information content may be an intellectual entity existing only
in the author's mind. Assuming authoring will be done by a computer
system, there must be an explicit representation of the information
content. As these representations are usually not in a format
presentable to human users, we may speak of raw data, or of
knowledge structures in the case of highly structured representation
formats. In order to bridge between information content on the one
hand, and media objects on the other, we introduce the following
In the context of this paper the ultimate goal of using media and
modalities is to communicate information to the user. However, the
use of media/modalities may be further distinguished with regard
to different aspects, e.g., sequential vs. parallel use with
regard to time, or redundant vs. complementary use with
regard to the encoding of information, or informative
vs. decorative use with respect to the communicative function.
- refers to a particular way or mechanism of encoding
information which is to be presented to humans or machines in a
physically realised intersubjective form. Examples include 2D- and 3D
graphics, written and spoken natural
language, music and earcons.
For a more fine-grained classification of presentation modalities we
refer to a paper by Bernsen (see this issue).
- refers to the use of multiple modalities to
encode information. For example, this proposal is a bimodal presentation
since it utilizes written text and 2D diagrams to describe the
reference model. Note that a single medium may be used
as a common physical realization of several modalities, e.g., the
medium audio for spoken natural language accompanied with some
Concerning the relationship between media and modalities, it has to be
noted that the choice of a certain modality also places a restriction
on the medium to be used.
Moreover, in many concrete system implementations, there is even a
one-to-one mapping from a certain modality to a certain type of media objects.
In such a case, it does not bother so much if modality and medium are mixed up.
In the following, we introduce a few more terms related to media and
- will be used as a general term for any
composition of media objects which have been created for the purpose
of communicating information to a user. Depending on context,
we will use the term ``presentation'' either to emphasize the material
aspect of media objects (i.e, presentation = document),
or to emphasize the activity of communicating information to a user.
- multimedia/modal referring expression
- a composition of at least two
different media/modes objects included in a presentation with the intention
to refer to an object or a subject matter in the
- cross-media/modal reference
- an expression included in a
presentation with the intention to refer to another part of a
presentation (e.g., text in the caption of a graphic
that refers to a subpart in the graphic).
It is assumed that the expression and the referenced
part are realized in different media/modalities.
Next:2.2 Terms related to
Up:2 Basic Notions
Previous:2 Basic Notions
Last update: Sun Jan 19 00:29:35 MET 1997
Send comments to firstname.lastname@example.org