A4 Vertaisarvioitu artikkeli konferenssijulkaisussa
Risk Detection in E-commerce with LLMs: Annotation Challenges and Lessons from Real-World Business News
Tekijät: Davoodi, Laleh; Salimi, Sima; Ginter, Filip; Lorentz, Harri
Toimittaja: Achilleos, Achilleas; Forti, Stefano; Papadopoulos, George Angelos; Pappas, Ilias
Konferenssin vakiintunut nimi: IFIP Conference on e-Business, eServices, and e-Society
Kustantaja: Springer Science and Business Media Deutschland GmbH
Julkaisuvuosi: 2025
Lehti: Lecture Notes in Computer Science
Kokoomateoksen nimi: Pervasive Digital Services for People’s Well-Being, Inclusion and Sustainable Development : 24th IFIP WG 6.11 Conference on e-Business, e-Services and e-Society, I3E 2025, Limassol, Cyprus, September 9–11, 2025, Proceedings
Vuosikerta: 16079
Aloitussivu: 146
Lopetussivu: 160
ISBN: 978-3-032-06163-8
eISBN: 978-3-032-06164-5
ISSN: 0302-9743
eISSN: 1611-3349
DOI: https://doi.org/10.1007/978-3-032-06164-5_11
Julkaisun avoimuus kirjaamishetkellä: Ei avoimesti saatavilla
Julkaisukanavan avoimuus : Ei avoin julkaisukanava
Verkko-osoite: https://link.springer.com/chapter/10.1007/978-3-032-06164-5_11
The growing complexity of e-commerce supply chains has amplified the need for effective risk monitoring systems. While Large Language Models (LLMs) have demonstrated potential in various domains, their application to real-world risk detection in e-commerce remains underexplored. This study introduces a novel, manually annotated dataset of 121 business news articles covering five major e-commerce-related steel companies, ArcelorMittal, Tata Steel, POSCO, NLMK, and ThyssenKrupp, annotated using the Cambridge Risk Taxonomy. We evaluate the performance of two advanced LLMs in detecting and classifying risks across multiple categories using few-shot prompting and semantic similarity-based example selection. Our results show that LLMs can approximate human annotation with moderate micro F1-scores and high coverage, though challenges remain in recognizing complex Geopolitical risks and avoiding overgeneralization. The findings provide actionable insights into the potential and limitations of LLMs for automated, domain-aware risk monitoring, laying the groundwork for future applications in supply chain risk management.
Julkaisussa olevat rahoitustiedot:
We gratefully acknowledge the financial support provided by Business Finland, as this research was carried out within the framework of the AI-SIM project.
