Despite being a common, established concept in wide usage, usability tests can vary greatly in goals,
techniques and results. A usability test purchased and performed for a specific software product, may
result in either minor user interface improvements or radical U-turns in the development. Such variation
has been discussed as a problem of the scientific reliability and validity of the testing method. In practice
it is more important what ‘kind of data’ one can expect of the selected method than whether it is reliably
always the same data. This expectation of information content or ‘scope’ is of importance for evaluators,
who select and conduct usability tests for a specific purpose. However, the scope is not explicitly stated
or even discussed: Too often the premise is that, because a usability test involves users, it brings the
(necessary) user-centeredness to the design i.e. takes socio-technical fundamentals as inherently given.
Through a literature review of testing practices and analytical considerations, we search for the scope
of a usability test, which could deliberately approach the socio-technical tradition and equally develop
both the system and the user organization. A case example represents a possible realization of the extended scope of usability test.
Keywords: Usability testing, Scope, IS evaluation and development, Socio-technical approach.