Alexis Heloir, Research

Feasability study: Sign Language Avatars for the Web

We are currently conducting a feasibility study for the Bundesministerium für Arbeit und Soziales (German ministry for labour and social welfare) to explore the potential of avatars for (semi)automatic sign language translation. Apart from surveying existing work we will conduct structured expert interviews and empirical tests to find out the practical limits of avatar-based sign language communication.

Gaze and speech alignment

Gaze is known to be a an important social cue in face-to-face communication indicating focus of attention. Speaker gaze can influence object perception and situated utterance comprehension by driving both interlocutor's visual attention towards the same object such that grounding and disambiguation of referring expressions is facilitated. The precise temporal and causal processes involved in on-line gaze following during concurrent utterance comprehension are, however, still largely unknown. Specifically, the alignment of referential gaze and speech cues may be essential to such benefit. we rely on eye-tracking studies exploiting a virtual character to systematically investigate how speaker gaze influences listeners' on-line comprehension.

EMBR: A Realtime Animation Engine for Interactive Embodied Agents

Embodied agents are a powerful paradigm for current and future multimodal interfaces, yet require high effort and expertise for their creation, assembly and animation control. Therefore, open animation engines and high-level control languages are required to make embodied agents accessible to researchers and developers. In this paper, we present EMBR, a new realtime character animation engine that offers a high degree of animation control via the EMBRScript language. We argue that a new layer of control, the animation layer, is necessary to keep the higher-level control layers (behavioral/functional) consistent and slim, while allowing a unified and abstract access to the animation engine, e.g. for the procedural animation of nonverbal behavior.

Evaluating data driven style translation of motion

This work presents an empirical evaluation of a ``Style Transformation'' method which consists of applying an automatically style on a neutral input motion to generate an appropriate style variant. Transformation parameters are extracted from an existing captured sequence. This data-driven method can be used either to enhance artist-generated gesture animations or to modify captured motion sequences according to a desired style. Although we used French Sign Language gesture data to perform the experiments described in this paper, the method may be applied to any kind of skeleton-based character animation.

Qualitative and quantitative evaluation of style in sign language gestures

We propose a qualitative and quantitative analysis of styled motion gesture data. The temporal, spatial and structural differences between styled gestures are confronted with the phenomenons described in the literature dedicated to sign language phonology. It is already well-known that gesture style modifies the temporal and the spatial aspects of gestures. In this paper, we address how style may also influence the structure organization of some lexical units. We then briefly present new insights for taking into account both the spatio-temporal and structural variations induced by style in existing gesture specification frameworks.

Temporal alignment of communicative gesture sequences

We address the problem of temporal alignment applied to captured communicative gestures conveying different styles. We propose a representation space is more robust to the spatial variability induced by style. By extending a multilevel dynamic time warping algorithm, we show how this extension can fulfill the goals of time correspondence between gesture sequences while preventing jerkiness introduced by standard time warping methods.

Modeling sign language discourse

Topological indexation of 3D meshes

This work has been dedicated to topological characterization and indexation of 3D surfaces based on Reed graphs.

This exploratory work has been conducted between January 2004 and July 2004, during my master thesis. For a complete investigation on the subject, I recommend the thesis manuscript of Julien tierny .

EMBR: A Realtime Animation Engine for Interactive Embodied Agents

EMBR is a realtime character animation engine that offers a high degree of animation control via the EMBRScript language. In EMBRScript, animations are described using key poses over parts of the whole agent. A pose may specify a sub-skeleton configuration, a shader parameter value, a set of kinematic constraints or a combination of morph targets. The public release of EMBR is scheduled for december 2009.

SMR: a motion analysis/synthesis library

SaMsaRa Animation and motion analysis library is a portable C++ library which has been designed to offer a consistent framework dedicated to the analysis and synthesis of human motion. SMR provides common methods for motion analysis ( PCA analysis, Time warping, Interpolation ) and straightforward data structures commonly used in animation frameworks (skeletal structures, kinematic chains), SMR provides easy access to forward and inverse kinematics algorithms as well as a collection of functions dedicated to load common animation assets (bvh and asf/amc motion files). SMR is generic (as generic as possible), although a minimal openGL visualization framework is provided, SMR may be integrated into existing rendering frameworks like Horde3D or Panda 3D.

Samasara is Protected with number: IDDN.FR.001.370002.000.S.C.2008.000.21000. Send me an Email for more information.

Using Signing Agents to Produce French Sign Language Utterances

Gesturing virtual agents have the potential to help deaf people with reading difficulties accessing information at any time and for low cost. One challenge in automated sign language production is emotional expression because it often contributes in a meaningful way to the message.

One topic of this dissertation is modeling the differences between several motion captured realizations (neutral, weary and angry) of the same French sign language sequence. By using principal component analysis and multi-scale dynamic time warping, we are able to extract style translation filters that can be used to add emotional expression to other neutral input sequences.

Another aspect of the work is the proposal of a discourse model based on the Iconicity model [Cux00].

[Cux00]: La langue des signes Francaise: les voies de L'iconicité Ophrys(Gap.) 2000.

Get the manuscript.

Character Animation (Course)

Character animation is the art of creating moving characters with the use of computers. It is a subfield of computer graphics. Until recently, crafting animation was a manual time expensive process. The arrival of new interactive medias like video games and on-line application established new requirements and constraints to the animation production process in terms of realism and flexibility.

This course will first introduce the essentials of computer animation: how traditional animation principles have been applied to computer animation an how interpolation techniques has increased animation productivity. It then presents the major breakthrough observed in the field during the last decades: Automatic motion generation using forward ,inverse kinematics and physical simulation, realistic motion playback through motion capture and the subsequent data driven animation trend. It finally gives an overview of current trends and perspectives: hybrid motion generation methods and high level description languages for character animation.

Character Animation (Seminar)

Animating hierarchical, articulated structures like human bodies is a challenging task. A human skeleton has many degrees of freedom that, ideally, should all be controlled automatically according to high-level commands (walk, grasp, gesture, change posture) and parameters (fast, expansive, like an old person, jerky). The ultimate aim is synthetic movement that looks natural (i.e. human-like), is synchronized (e.g. to speech or music) and expressive (style, personality, emotion). In this seminar, we will look at the most important contributions to this field, ranging from representational issues and inverse kinematics to motion capture and questions of high-level style parameters.