A4 Vertaisarvioitu artikkeli konferenssijulkaisussa
Using Large Language Models for Black-Box Testing of FMU-Based Simulations
Tekijät: Mughees, Abdullah; Sudheerbabu, Gaadha; Ahmad, Tanwir; Truscan, Dragos; Manngård, Mikael; Klemets, Kristian
Toimittaja: N/A
Konferenssin vakiintunut nimi: European Control Conference
Julkaisuvuosi: 2026
Lehti: European Control Conference
Kokoomateoksen nimi: 2026 European Control Conference (ECC)
Aloitussivu: 3116
Lopetussivu: 3123
ISBN: 979-8-3315-5755-3
eISBN: 978-3-907144-13-8
ISSN: 2996-8917
eISSN: 2996-8895
Julkaisun avoimuus kirjaamishetkellä: Ei avoimesti saatavilla
Julkaisukanavan avoimuus : Ei avoin julkaisukanava
Verkko-osoite: https://ieeexplore.ieee.org/document/11625516
We present an approach for black-box testing of Functional Mock-up Units (FMUs) that leverages Large Language Models (LLMs) in a human-in-the-loop workflow. The tool aims to reduce the manual effort involved in designing test scenarios for dynamic simulation models while improving the interpretability of results. The approach takes the functional and interface specifications of an FMU as input, and prompts an LLM to generate structured scenario goals in Given-When-Then format that define the initial input conditions of the simulation, a possible change in those conditions, and the expected output behaviour of the system against those changes. The corresponding scenario plans specify input patterns and add assertion oracles that describe expected output patterns defined in scenario goals. The approach generates a complete input time series for the scenario plans, runs the FMU simulation, and evaluates assertions on the recorded outputs. We evaluate the approach on a Lube Oil Cooling system and discuss design choices that make the approach practical for everyday use. Results suggest that LLM-assisted scenario generation can facilitate automatic test design and verification of dynamic simulation models.
Julkaisussa olevat rahoitustiedot:
This work was funded by the Finnish Ministry of Education and Culture’s Doctoral Education Pilot under Decision No. VN/3137/2024-OKM-6 (The Finnish Doctoral Program Network in Artificial Intelligence, AI-DOC) and Business Finland via the Virtual Sea Trial project (VST), under grant 7187/31/2023.