A1 Refereed original research article in a scientific journal

The ‘fourth wall’ and other usability issues in AI-generated personas : comparing chat-based and profile personas




AuthorsKaate, Ilkka; Salminen, Joni; Jung, Soon-Gyo; Santos, Joao M.; Hayhanen, Essi; Xuan, Trang; Azem, Jinan Y.; Jansen, Bernard

PublisherTaylor & Francis

Publishing placeABINGDON

Publication year2025

JournalBehaviour and Information Technology

Journal name in sourceBehaviour & Information Technology

Journal acronymBEHAV INFORM TECHNOL

Number of pages17

ISSN0144-929X

eISSN1362-3001

DOIhttps://doi.org/10.1080/0144929X.2025.2469659

Web address https://doi.org/10.1080/0144929x.2025.2469659

Self-archived copy’s web addresshttps://research.utu.fi/converis/portal/detail/Publication/491326502


Abstract

Large Language Models (LLMs) are emerging as a powerful tool for AI-generated personas. This study evaluates the usability of AI-generated personas, comparing chat and profile formats. The findings indicate chat personas tend to be perceived more favourably, and profile personas exhibit greater variability in user perception. The increased difficulty and longer dwell time experienced by users with the profile persona, despite negative usability metrics, paradoxically resulted in better task performance. Usability issues indicate that many current limitations of AI, including verbosity, hallucinations, and empty rhetoric which was described as the persona having 'no soul', are inherited in AI-generated chat personas. However, there are also new issues. For one, the risk of information overload in an AI-generated profile persona implies that the AI does not consider human users' cognitive limitations when designing the persona (but usability scores for profile personas increase with dwell time, implying that users get used to the longer format the more time they spend). Another is the 'fourth wall' effect of AI-generated chat personas in which the user feels they are talking to someone describing the persona rather than the persona itself. Future work could address the usability paradox and the fourth wall effect of using personas.CCS CONCEPTSHuman-centered computing Human computer interaction (HCI)


Downloadable publication

This is an electronic reprint of the original article.
This reprint may differ from the original in pagination and typographic detail. Please cite the original version.




Funding information in the publication
Open Access funding provided by the Qatar National Library.
This work was supported by Liikesivistysrahasto.


Last updated on 2025-01-04 at 14:42