A4 Refereed article in a conference publication

ChatGPT as tutor? A case study on competitive programming




AuthorsRytilahti, Juuso; Lokkila, Erno

EditorsCarmo, Mafalda

Conference nameInternational Conference on Education and New Developments

Publication year2024

JournalEducation and New Developments

Book title Education and New Developments 2024: Volume 2

First page 91

Last page95

ISBN978-989-35728-0-1

ISSN2184-044X

eISSN2184-1489

DOIhttps://doi.org/10.36315/2024v2end019

Web address https://doi.org/10.36315/2024v2end019


Abstract

In this paper, we present a case study on how students utilize ChatGPT as a tutor for a short competitive programming course. The students were divided into two groups, one led by a teacher, and the other tutored by ChatGPT. The course was an intensive five-day course and both groups studied concurrently. Students could freely choose which group they participated in. The ChatGPT group was provided a guide on the basics of prompting, including approaches on how to generate the study material. Both groups were allowed to use any learning material, and only the teacher-led group excluded the use of generative AI tools. Research questions: In this study, we focus on the following questions: (1) How did students approach using ChatGPT as a tutor?; (2) Are there significant differences between students led by a teacher or by ChatGPT?; (3) How did students in both groups experience the teaching and did it correlate to the achieved learning results (exam)? Methodology: The data consists of survey data, and final exam given to students (N=11). We also collected the discussion history of the ChatGPT group. The discussion history was divided into prompt-message pairs (N=340) and analyzed. The data was analyzed using mixed methods. The discussion history was analyzed using grounded theory. Statistical methods were used to find any correlation between initial skill level and learning as well as the tag distribution of the ChatGPT discussions. Results: Differences were found between the two groups. Those with a higher initial skill level seemed to favor the ChatGPT group, whereas the less experienced chose the in-person teaching. Analysis of the ChatGPT discussion history showed mostly similar usage patterns across students. We present the distribution of tags used by the ChatGPT group. Additionally, we give insight on how to approach similar research settings in the future. Impact: All around the world, students are already utilizing ChatGPT as a substitute for a teacher or a tutor. Our pilot study provides insight into how students approach utilizing ChatGPT as a tutor in a programming teaching setting. These preliminary results can be used to guide future research settings.



Last updated on 2025-04-06 at 14:24