A4 Refereed article in a conference publication

Multi-source Food Names Mapping Using OpenAI vision, Manual Dictionary and Fuzzy Matching Techniques;




AuthorsBhetuwal, Shyam; Koivunen, Lauri; Khalil, Rehan; Koskimäki, Sanna; Lähde, Hanna; Houttu, Veera; Laitinen, Kirsi; Mäkilä, Tuomas

EditorsAhram, Tareq Z.; Kalra, Jay; Karwowski, Waldemar

Conference nameInternational Conference on Applied Human Factors and Ergonomics

Publication year2026

Journal: AHFE International

Book title Artificial Intelligence and Social Computing : Proceedings of the 17th International Conference on Applied Human Factors and Ergonomics and the Affiliated Conferences, Istanbul, Turkey, 20-24 July 2026

Volume203

First page 41

Last page50

ISBN978-1-964867-79-3

ISSN2771-0718

DOIhttps://doi.org/10.54941/ahfe1007316

Publication's open availability at the time of reportingOpen Access

Publication channel's open availability Open Access publication channel

Web address http://dx.doi.org/10.54941/ahfe1007316

Self-archived copy’s web addresshttps://research.utu.fi/converis/portal/detail/Publication/533885175

Self-archived copy's licenceCC BY NC ND

Self-archived copy's versionPublisher`s PDF


Abstract

Accurate harmonization of food names across heterogeneous and multilingual datasets remains a major challenge in food informatics, dietary assessment systems, and data-driven public health research. Modern AI-based food recognition models such as LogMeal, FoodSAM, and OpenAI Vision can identify multiple components within complex dishes, but they frequently produce inconsistent, culturally specific, and multilingual labels. These inconsistencies complicate downstream tasks including nutritional analysis and cross-dataset integration. In this study, we evaluated practical methods for mapping AI-generated food component names to Finnish menu-based ground truth in a real-world restaurant setting. We collected 320 meal images using an integrated camera–scale system; 167 images containing multi-component dishes were selected for detailed evaluation against Finnish lunch-line menu labels. We compared (i) a segment-aware, menu-constrained mapping approach that uses LogMeal segmentations and prompts OpenAI Vision to select the best-matching item from the daily menu for each segment, and (ii) a hybrid manually curated canonical dictionary and fuzzy string matching pipeline applied separately to labels from different AI sources. Mapping performance is measured using precision, recall, and F1-score. The segment-aware OpenAI Vision approach achieved the best overall results (Precision = 0.90, Recall = 0.70, F1 = 0.79), while the hybrid dictionary+fuzzy method also improved consistency over direct label matching. These results indicate that menu-aware segment-level reasoning and lightweight lexical normalization are effective for food-name harmonization and can support scalable dietary monitoring and menu analytics.



Keywords:
data integrationFood Name MappingFood Name Standardizationfuzzy matchingString Similarity

Downloadable publication

This is an electronic reprint of the original article.
This reprint may differ from the original in pagination and typographic detail. Please cite the original version.




Funding information in the publication
This research was supported by Business Finland (2022/31/2023). We gratefully acknowledge the Flavoria® multidisciplinary research platform and our colleagues at the Nutrition and Food Research Center of the University of Turku for their continued support.


Last updated on 25/08/2026 08:25:51 AM