A1 Refereed original research article in a scientific journal
The ‘fourth wall’ and other usability issues in AI-generated personas : comparing chat-based and profile personas
Authors: Kaate, Ilkka; Salminen, Joni; Jung, Soon-Gyo; Santos, Joao M.; Hayhanen, Essi; Xuan, Trang; Azem, Jinan Y.; Jansen, Bernard
Publisher: Taylor & Francis
Publishing place: ABINGDON
Publication year: 2025
Journal: Behaviour and Information Technology
Journal name in source: Behaviour & Information Technology
Journal acronym: BEHAV INFORM TECHNOL
Number of pages: 17
ISSN: 0144-929X
eISSN: 1362-3001
DOI: https://doi.org/10.1080/0144929X.2025.2469659
Web address : https://doi.org/10.1080/0144929x.2025.2469659
Self-archived copy’s web address: https://research.utu.fi/converis/portal/detail/Publication/491326502
Large Language Models (LLMs) are emerging as a powerful tool for AI-generated personas. This study evaluates the usability of AI-generated personas, comparing chat and profile formats. The findings indicate chat personas tend to be perceived more favourably, and profile personas exhibit greater variability in user perception. The increased difficulty and longer dwell time experienced by users with the profile persona, despite negative usability metrics, paradoxically resulted in better task performance. Usability issues indicate that many current limitations of AI, including verbosity, hallucinations, and empty rhetoric which was described as the persona having 'no soul', are inherited in AI-generated chat personas. However, there are also new issues. For one, the risk of information overload in an AI-generated profile persona implies that the AI does not consider human users' cognitive limitations when designing the persona (but usability scores for profile personas increase with dwell time, implying that users get used to the longer format the more time they spend). Another is the 'fourth wall' effect of AI-generated chat personas in which the user feels they are talking to someone describing the persona rather than the persona itself. Future work could address the usability paradox and the fourth wall effect of using personas.CCS CONCEPTSHuman-centered computing Human computer interaction (HCI)
Downloadable publication This is an electronic reprint of the original article. |
Funding information in the publication:
Open Access funding provided by the Qatar National Library.
This work was supported by Liikesivistysrahasto.