Publication
Move with the theremin: Body Posture and Gesture Recognition using the Theremin in Loose-Garment with Embedded Textile Cables as Antennas
Hymalai Bello; Bo Zhou; Sungho Suh; Sanchez Marin Luis Alfredo; Paul Lukowicz
In: Xianta Jiang (Hrsg.). Frontiers in Computer Science, Vol. 4 - Mobile and Ubiquitous Computing, Pages 1-15, Frontiers, 6/2022.
Abstract
We present a novel intelligent garment design approach for body posture/gesture
detection in the form of a loose-fitting blazer prototype, “the MoCaBlazer.” The design is
realized by leveraging conductive textile antennas with the capacitive sensing modality,
supported by an open-source electronic theremin system (OpenTheremin). The use
of soft textile antennas as the sensing element allows flexible garment design and
seamless tech-garment integration for the specific structure of different clothes. Our
novel approach is evaluated through two experiments involving defined movements (20
arm/torso gestures and eight dance movements). In cross-validation, the classification
model yields up to 97.18% average accuracy and 92% f1-score, respectively. We have
also explored real-time inference enabled by a radio frequency identification (RFID)
synchronization method, yielding an f1-score of 82%. Our approach opens a new
paradigm for designing motion-aware smart garments with soft conductive textiles
beyond traditional approaches that rely on tight-fitting flexible sensors or rigid motion
sensor accessories.