G4 Monografiaväitöskirja
Documents multilingues pour logiciels et utilisabilité
Tekijät: Salmi Leena
Kustantaja: University of Turku
Julkaisuvuosi: 2003
ISBN: 951-29-2620-2
Tiivistelmä
The purpose of this study is to investigate the problems that users of a wordprocessing program have with software documentation. Videotape data consisting of 33 user sessions of an average length of 45 minutes were collected by using the usability testing method.
The users were Finnish (18 sessions) and French (15 sessions) university students, neither novices nor experts with computers, without any previous experience of using the program. They were students of either linguistic or other humanist subjects. In the tests, they worked either individually (N=14) or in pairs (N=19). They did the same tasks using two language versions of the same program (English and Finnish or English and French). While working on the tasks, they could use both the online help and the printed manual shipped with the software whenever they wanted.
For the analysis, the videotape data were transcribed into user protocols containing
information on what the users did and said. Each task by each user was also visualized as a “user path”, i.e. a flow chart showing how they used the software and what help topics or manual chapters they consulted.
The analysis of the videotape data was both quantitative and qualitative. In the quantitative analysis, completion times and the number of consultations of the documentation were compared between the different user groups using statistical
analyses. In the qualitative analysis, problems that occurred were retrieved from the “user paths” and the protocols and classified into 6 groups: 1) problems related to the use of the wordprocessing program specifically (N=86), 2), problems related to the use of computers in general (N=59), 3) problems related to the text in the user interface (terminology) (N=16), 4) problems related to the structure of the documentation (N=20), 5) problems related to the contents of the documentation (N=39), and 6) problems related to the task (N=80). In 6), a series of problems emerged that could have formed a group of their own: problems related to the English language (N=24), such as finding how to say “table of contents” in English.
The results seem to support the hypothesis that using a wordprocessing program in one’s mother tongue takes less time than using the software in English. The differences in completion times were, however, statistically significant only in task 6 (especially for the Finnish users) (p=0.0104). As for the number of problems, the use of the software in one’s mother tongue seems to facilitate the use, especially in a complex task like making a table of contents, but not if the terms used are unfamiliar or incorrect.
As for the method of doing the task, individually versus in pairs, there were no statistically significant differences in the completion times, but the pairs consulted the
documentation more often than the individuals with consultation of the manual being significantly different (p=0.0396). More problems occurred in the sessions with pairs than with individuals, 171 versus 129, but this difference was not statistically significant. However, more research is needed to gain additional information on the differences in discourse (whether the discussion by pairs reveals more about their ways of problem solving than the thinking aloud by individuals).
The more experienced users had seemingly less problems than the less experienced (although the aim of the study was to use “intermediate” users, there were some differences in the backgrounds of the users). A hypothesis based on cultural differences found in previous studies supposed that the French users would consult the documentation less than the Finnish users. There was only a slight difference in the use: 147 consultations for the Finns, 136 for the French. The hypothesis that students of linguistic subjects would accomplish tasks faster and with less problems than students of other humanist subjects was not corroborated. As hypothesized, problems related to the documentation did not occur only in the translated documentation (in French and Finnish), but in the original as well (in English). Most of the problems were not related to the language version, although there were some problems related to translated terms (cases where the translation of a term was mistakenly replaced by a wrong term).
Many of the problems related to the documentation in general could be explained by
differing background knowledge. The user did not have all the information the writer of
the original text had supposed they would have - or the writer had not correctly anticipated the variety of users’ needs.
The purpose of this study is to investigate the problems that users of a wordprocessing program have with software documentation. Videotape data consisting of 33 user sessions of an average length of 45 minutes were collected by using the usability testing method.
The users were Finnish (18 sessions) and French (15 sessions) university students, neither novices nor experts with computers, without any previous experience of using the program. They were students of either linguistic or other humanist subjects. In the tests, they worked either individually (N=14) or in pairs (N=19). They did the same tasks using two language versions of the same program (English and Finnish or English and French). While working on the tasks, they could use both the online help and the printed manual shipped with the software whenever they wanted.
For the analysis, the videotape data were transcribed into user protocols containing
information on what the users did and said. Each task by each user was also visualized as a “user path”, i.e. a flow chart showing how they used the software and what help topics or manual chapters they consulted.
The analysis of the videotape data was both quantitative and qualitative. In the quantitative analysis, completion times and the number of consultations of the documentation were compared between the different user groups using statistical
analyses. In the qualitative analysis, problems that occurred were retrieved from the “user paths” and the protocols and classified into 6 groups: 1) problems related to the use of the wordprocessing program specifically (N=86), 2), problems related to the use of computers in general (N=59), 3) problems related to the text in the user interface (terminology) (N=16), 4) problems related to the structure of the documentation (N=20), 5) problems related to the contents of the documentation (N=39), and 6) problems related to the task (N=80). In 6), a series of problems emerged that could have formed a group of their own: problems related to the English language (N=24), such as finding how to say “table of contents” in English.
The results seem to support the hypothesis that using a wordprocessing program in one’s mother tongue takes less time than using the software in English. The differences in completion times were, however, statistically significant only in task 6 (especially for the Finnish users) (p=0.0104). As for the number of problems, the use of the software in one’s mother tongue seems to facilitate the use, especially in a complex task like making a table of contents, but not if the terms used are unfamiliar or incorrect.
As for the method of doing the task, individually versus in pairs, there were no statistically significant differences in the completion times, but the pairs consulted the
documentation more often than the individuals with consultation of the manual being significantly different (p=0.0396). More problems occurred in the sessions with pairs than with individuals, 171 versus 129, but this difference was not statistically significant. However, more research is needed to gain additional information on the differences in discourse (whether the discussion by pairs reveals more about their ways of problem solving than the thinking aloud by individuals).
The more experienced users had seemingly less problems than the less experienced (although the aim of the study was to use “intermediate” users, there were some differences in the backgrounds of the users). A hypothesis based on cultural differences found in previous studies supposed that the French users would consult the documentation less than the Finnish users. There was only a slight difference in the use: 147 consultations for the Finns, 136 for the French. The hypothesis that students of linguistic subjects would accomplish tasks faster and with less problems than students of other humanist subjects was not corroborated. As hypothesized, problems related to the documentation did not occur only in the translated documentation (in French and Finnish), but in the original as well (in English). Most of the problems were not related to the language version, although there were some problems related to translated terms (cases where the translation of a term was mistakenly replaced by a wrong term).
Many of the problems related to the documentation in general could be explained by
differing background knowledge. The user did not have all the information the writer of
the original text had supposed they would have - or the writer had not correctly anticipated the variety of users’ needs.
Turun yliopiston Tutkimustietojärjestelmän portaali