A1 Vertaisarvioitu alkuperäisartikkeli tieteellisessä lehdessä

Evaluating firearm examiner testimony using large language models: a comparison of standard and knowledge-enhanced AI systems




TekijätPompedda; Francesco; Santtila, Pekka; Di Maso, Eleonora; Nyman; Thomas J.; Yongjie, Sun; Zappala, Angelo

KustantajaTaylor & Francis online

Julkaisuvuosi2025

JournalJournal of Psychology and AI

Artikkelin numero2503343

Vuosikerta1

Numero1

eISSN2997-4100

DOIhttps://doi.org/10.1080/29974100.2025.2503343

Verkko-osoitehttps://doi.org/10.1080/29974100.2025.2503343

Rinnakkaistallenteen osoitehttps://research.utu.fi/converis/portal/detail/Publication/492228205


Tiivistelmä

This study evaluated the decision-making of Large Language Models (LLMs) in interpreting firearm examiner testimony by comparing a standard LLM to one enhanced with forensic science knowledge. The present study is a replication study. We assessed whether LLMs mirrored human decision patterns and if specialised knowledge led to more critical evaluations of forensic claims. We employed a 2 × 2 × 7 between-subjects design with three independent variables: LLM configuration (standard vs. knowledge-enhanced), cross-examination presence (yes vs. no), and conclusion language (seven variations). Each model condition performed 200 repetitions per scenario. This yielded a total of 5,600 measures of binary verdicts, guilt probability ratings, and credibility assessments. LLMs showed low conviction rates (9.4%) across conditions, with logical variations as a function of the way in which the firearm expert’s conclusion was formulated. Cross-examination produced lower guilt assessments and scientific credibility ratings. Importantly, knowledge-enhanced LLMs demonstrated significantly more conservative evaluations of firearm evidence across all match conditions compared to standard LLMs. LLMs, particularly when enhanced with domain-specific knowledge, showed advantages in evaluating complex scientific evidence compared to human jurors in Garrett et al. (2020), suggesting potential applications for AI systems in supporting legal decision-making.


Ladattava julkaisu

This is an electronic reprint of the original article.
This reprint may differ from the original in pagination and typographic detail. Please cite the original version.




Julkaisussa olevat rahoitustiedot
The work was supported by the New York University Shanghai [10109_Academic Research]; Spring Crocus s.r.l., managing body of CBT. ACADEMY.


Last updated on 2025-06-06 at 09:52