Skip to main content Skip to main navigation

Publication

Few-Shot Human Activity Recognition Using Lightweight Language Models

Federico Cruciani; Stefan Gerd Fritsch; Ian Cleland; Vitor Fortes Rey; Chris Nugent; Paul Lukowicz
In: 2025 International Conference on Activity and Behavior Computing (ABC). International Conference on Activity and Behavior Computing (ABC-2025), 7th International Conference on Activity and Behavior Computing, April 21-25, Abu Dhabi, United Arab Emirates, IEEE, 2025.

Abstract

The lack of labeled data and model generalization abilities have historically represented some of the major obstacles for Human Activity Recognition (HAR). Few-Shot Learning (FSL) addresses both of these issues. However, its application to HAR, is particularly difficult, since transferring models into new environments requires the model to adapt not only to a new set of activity labels but also, in many cases, to a different sensor configuration and possibly even a different sensor modality. The ability of Large Language Models (LLMs) to understand and interpret natural language, together with their versatility as few-shot learners, can greatly simplify model transfer, at least in all cases where sensor activations can be transformed into text descriptions, since such descriptions abstract from the specific sensor configuration. Unfortunately, the application of LLM at the edge is typically hindered by their hardware requirements, making it unfeasible or highly inefficient to deploy these models to be used locally within smart environments. In this context, we propose a lightweight LLM-based FSL approach to facilitate model transfer with only a few labeled data samples, while using a model small enough to be deployable at the edge. Our results show that a relatively small BERT-based LLM architecture can outperform larger models (including Chat-GPT). The approach was evaluated on two challenging datasets, namely the ``VK'' and the "VK houses'' datasets. On ``VK's'', we obtain a macro average F-score of 52.75% vs. a baseline F-score of 26.63% using Chat-GPT. On the ``VK houses'' dataset, our macro average F-score is 36.02%, in contrast to 15.46% for Chat-GPT.