A1 Vertaisarvioitu alkuperäisartikkeli tieteellisessä lehdessä
Evaluating firearm examiner testimony using large language models: a comparison of standard and knowledge-enhanced AI systems
Tekijät: Pompedda; Francesco; Santtila, Pekka; Di Maso, Eleonora; Nyman; Thomas J.; Yongjie, Sun; Zappala, Angelo
Kustantaja: Taylor & Francis online
Julkaisuvuosi: 2025
Journal: Journal of Psychology and AI
Artikkelin numero: 2503343
Vuosikerta: 1
Numero: 1
eISSN: 2997-4100
DOI: https://doi.org/10.1080/29974100.2025.2503343
Verkko-osoite: https://doi.org/10.1080/29974100.2025.2503343
Rinnakkaistallenteen osoite: https://research.utu.fi/converis/portal/detail/Publication/492228205
This study evaluated the decision-making of Large Language Models (LLMs) in interpreting firearm examiner testimony by comparing a standard LLM to one enhanced with forensic science knowledge. The present study is a replication study. We assessed whether LLMs mirrored human decision patterns and if specialised knowledge led to more critical evaluations of forensic claims. We employed a 2 × 2 × 7 between-subjects design with three independent variables: LLM configuration (standard vs. knowledge-enhanced), cross-examination presence (yes vs. no), and conclusion language (seven variations). Each model condition performed 200 repetitions per scenario. This yielded a total of 5,600 measures of binary verdicts, guilt probability ratings, and credibility assessments. LLMs showed low conviction rates (9.4%) across conditions, with logical variations as a function of the way in which the firearm expert’s conclusion was formulated. Cross-examination produced lower guilt assessments and scientific credibility ratings. Importantly, knowledge-enhanced LLMs demonstrated significantly more conservative evaluations of firearm evidence across all match conditions compared to standard LLMs. LLMs, particularly when enhanced with domain-specific knowledge, showed advantages in evaluating complex scientific evidence compared to human jurors in Garrett et al. (2020), suggesting potential applications for AI systems in supporting legal decision-making.
Ladattava julkaisu This is an electronic reprint of the original article. |
Julkaisussa olevat rahoitustiedot:
The work was supported by the New York University Shanghai [10109_Academic Research]; Spring Crocus s.r.l., managing body of CBT. ACADEMY.