In: ICDAR. International Conference on Document Analysis and Recognition (ICDAR), IEEE, 2017.
Zusammenfassung
This paper presents a novel approach, called
AirScript, for creating, recognizing and visualizing documents in
air. We present a novel algorithm, called 2-DifViz, that converts
the hand movements in air (captured by a Myo-armband worn
by a user) into a sequence of x; y coordinates on a 2D Cartesian
plane, and visualizes them on a canvas. Existing sensor-based
approaches either do not provide visual feedback or represent
the recognized characters using prefixed templates. In contrast,
AirScript stands out by giving freedom of movement to the user,
as well as by providing a real-time visual feedback of the written
characters, making the interaction natural. AirScript provides
a recognition module to predict the content of the document
created in air. To do so, we present a novel approach based on
deep learning, which uses the sensor data and the visualizations
created by 2-DifViz. The recognition module consists of a Convolutional
Neural Network (CNN) and two Gated Recurrent Unit
(GRU) Networks. The output from these three networks is fused
to get the final prediction about the characters written in air.
AirScript can be used in highly sophisticated environments like
a smart classroom, a smart factory or a smart laboratory, where
it would enable people to annotate pieces of texts wherever they
want without any reference surface. We have evaluated AirScript
against various well-known learning models (HMM, KNN, SVM,
etc.) on the data of 12 participants. Evaluation results show
that the recognition module of AirScript largely outperforms
all of these models by achieving an accuracy of 91.7% in a
person independent evaluation and a 96.7% accuracy in a person
dependent evaluation.
@inproceedings{pub9673,
author = {
Dash, Ayushman
and
Sahu, Amit
and
Shringi, Rajveer
and
Gamboa, John
and
Afzal, Muhammad Zeshan
and
Malik, Muhammad Imran
and
Dengel, Andreas
and
Ahmed, Sheraz
},
title = {AirScript - Creating Documents in Air},
booktitle = {ICDAR. International Conference on Document Analysis and Recognition (ICDAR)},
year = {2017},
publisher = {IEEE}
}
Deutsches Forschungszentrum für Künstliche Intelligenz German Research Center for Artificial Intelligence