1. Architectural Design
The high-level architecture shown in Figure 1 presents
the basic functional blocks of the SPECTER system and their fundamental
interrelations. This conceptual base architecture and its modularization
into main components directly reflects the underlying organization
into major research topics.
Figure 1: Conceptual Base Architecture
The necessary interconnection of the various components
requires an ontological infrastructure for knowledge exchange inside
SPECTER. The two central aspects of our basic approach are the use
of shared ontologies and to utilize emerging standards for the notation
of data and metadata, like XML, XML Schema, RDF/S, OWL, XQuery,
etc. Shared ontologies provide common vocabulary to enable declarative
representations for data exchange between system components. The
knowledge engineering task can draw on general results in the following
The architecture sketch in Figure 2 provides a
more detailed view on the different software components of the overall
SPECTER system. This diagram as well highlights the important role
of user model and personal journal as central and persistent knowledge
sources. The modular structure shown here indicates that instrumentation
within a mobile context does not only relate to the local environment,
but includes also mobile equipment carried around by the user, like
a handheld computer and specific sensors to monitor his or her affective
Figure 2: Refined Component Architecture
SPECTER needs to be able to interoperate with different
kinds of instrumented environments. A dedicated interface component
encapsulates the technical details of gaining access to the specific
functionality provided by the current environment.
2. Instrumented Environment
A major challenge for linking into an instrumented
environment is the heterogenity of available infrastructure frameworks.
Current approaches use different technologies like Jini, UPnP, Bluetooth,
Web Services, or custum research prototypes and none of them directly
supports semantic interoperability. The approach followed within
SPECTER can be characterized as follows:
- The specific instrumentation is conceptualized as a set of services
which can be regarded as virtual sensors and effectors.
- A separate interface component provides a generic bridge to
the local infrastructure and maintains a semantic model of the
- The representation of services relies on OWL-S and related specifications.
- Individual Jini services as well as standardized UPnP and Bluetooth
profiles can be recast as Semantic Web Services to provide the
The initial test environment for SPECTER mimics
an instrumented shop with RFID equipped products and readers within
the shelf and basket (see Figure 3). In a mobile context, such as
a shopping scenario, all direct interactions between user and SPECTER
itself are performed via a PDA.The laptop on the right-hand side
provides the software infrastructure of the instrumented shop, enables
communication between shop and mobile assistant, and provides SPECTER
with input. Using radio-frequency identification tags, the system
can detect what product the user has taken from the shelf or put
into the basket.
Figure 3: Experimental Setup for an Instrumented Shop
The initial version of our experimental environment
for the shopping scenario relies on the SmartShop software provided
by our cooperation partners at Universität
des Saarlandes. It provides a Jini-based software infrastructure
for instrumented environments and includes basic service implementations
forRFID-sensors, object identification and product information.
Information access is realized through virtual shared memory using
the tuple space paradigm. Future versions of the infrastructure
software are planned to incorporate a growing list of more advanced
shopping services (e.g. for cross-selling). Figure 3 presents an
overview of the different runtime components on the infrastructure
level, including standard Jini services as well as environment-specific
Figure 4: Runtime Components of the Infrastructure Software
3. Sample Interaction
This section presents a typical Interaction with
the initial prototype of the fully integrated system. This early
demonstrator has also been documented in a short video, which is
available for download
in AVI format (approx. 150 MBytes).
Figure 5: User picks item from shelf
Using RFID tags the system is able to detect what
product the user has taken from the shelf. SPECTER stores all observed
interactions in an extended episodic memory, the personal journal.
Figure 6: SPECTER retrieves related offers and recommends alternative
Depending on what the user chose, SPECTER can provide
assistance. An acoustic signal is used to draw the user's attention
to new information on the handheld display. In the current context,
SPECTER automatically queried the shopping service for product information.
In this case, SPECTER recommends an alternative product of the same
category that provides more features and can also be found in this
store. All interactions between user and SPECTER itself will also
be added to the journal. For example, an immediate confirmation
provides valuable information as SPECTER is also able to learn from
user feedback to further improve its decision making.
Figure 7: User puts item back and takes other product
The user follows the recommendation and puts the item back. This
action can also be recognized.
Figure 8: User puts item into basket
The record of the user's actions and affective
states in the personal journal can be mined to learn a model of
the user's preferences, habits, and typical affective reactions.
SPECTER exploits the user model and the personal journal to provide
appropriate recommendations and assistance in the user's current
situation. Even while walking around, the user can have immediate
access to his updated personal journal for further reflection and
Figure 9: Interactions can be Viewed within the Personal Journal
Also on the PC at home, the user can explore and
modify the personal journal as well as the user model in order to
assist Specter in its learning process. Journal entries can be annotated
to provide explicit ratings for the corresponding low-level events
or high-level interpretations. Such subjective evaluations offer
an added value for affect-aware personal assistance. All modifications
provided by the user during the reflection and introspection phase
will be exploited for the automatic update of the persistent user