Publication
Conversational Business Process Modeling using LLMs: Initial Results and Challenges
Chantale Lauer; Peter Pfeiffer; Alexander Michael Rombach; Nijat Mehdiyev
In: Luise Pufahl; Jana-Rebecca Rehse (Hrsg.). EMISA 2025. International Workshop on Enterprise Modelling and Information Systems Architectures (EMISA-2025), 15th International Workshop on Enterprise Modelling and Information Systems Architectures, located at EMISA 2025, Heilbronn, Germany, Gesellschaft für Informatik e.V. Bonn, 5/2025.
Abstract
Generating process models in textual conversation with a large language model (LLM) is an emerging topic in business process management (BPM). The underlying problem of text-to-model has been considered a challenging task in BPM for a long time.The emergence of LLMs drastically improved the feasibility of this task and the quality of the resulting models.However, as we analyze in this paper, several challenges remain for conversational process modeling with LLMs while new ones arise.To illustrate those challenges, we conducted experiments with various open-source LLMs of different sizes on the text-to-BPMN task. The resulting process models are evaluated regarding their syntactic, pragmatic, and semantic quality to compare the performance of LLMs in this task.