Translated Abstract
The purpose of this study is to establish a set of user-centered usability test method, which enables test results with higher validity and more reliability. Four issues are the major focuses in this research: First, based on the found defects of prevailed western usability theories, a new theoretical framework of usability test factors is built up from user models. Second, aimed at the inadequacy of Nielsen’s test methodology, an improved usability test method is suggested on following aspects in particular: how to select subjects for testing from various end user groups, task design, test duration, etc. Third, by examine the mathematical model of the usability test sample size of the theory in representation of Lewis, Virzi and Nielsen, it is argued that the sampling can not be simply categorized as a mathematic issue, but should be given an integrated thinking of the comprehensive test methods, subjects from various groups, and tasks. Fourth, Contrary to the concerns of the cost of testing in representation of Dumas, Sorce, Virzi and Nielsen, it’s claimed that both the method of increasing the number of the subjects and increasing the number of tasks can result in an incremental number of usability findings. However, within the fixed budget, it’s more effective to increase tasks number than to increase the number of subjects.In this study, an experiment is designed on the basis of a usability test on the desktop-based software Word 2007. The experimentis consisted of 244 task chains, covering full functions of Word 2007 (there’s no overlap feature or functionality among these task chains). In this experiment, the data of testing findings are collected from 35 novice users, 33 normal users and 30 expert users (98 in total). The overall cumulated test duration exceeds 800 hours. Via the analysis and study on these data, 5 fundamental hypothesisof this research are verified and validated in this research. It also points out and provides guidance on such questions as how to conduct usability test for a specific product with higher validity and more reliability, how to increase the numbers of usability findings within a fixed budget, how to calculate reasonable and necessary sampling size of subjects for the test.This study can contribute in 5 major aspects as follows: Initially, building up the user-centered usability test methodology, in which a series of topics are elaborated: how to establish a theoretical framework of usability test factors according to the characteristics of users’ task, cognition, learning and making errors, how to improve the usability test by taking into consideration of the validity and reliability, to which less attention are paid in past tests, and consequently how to improve the human-computer interface design based on these usability findings. Second, by comparing and analyzing the task difference and mental difference among users from groups of novice, normal and expert, it is proclaimed that users from all group have to be taken into account during the process of usability test or user interface design. Third, by supported by the data demonstration of limited usability findings can be obtained in limited testing time, it is also justified that limited or short duration of test resulting in less reliable findings. Forth, it claims that the theory of small sample size: in the representation of Nielson, is not reliable, moreover, the current mathematical model for estimation of user sample size is inaccurate. Based on studying the relationship between the number of subjects and usability findings, a new method, namely method of matrix, is proposed and provided for proper estimation of subject sample size. Fifth, the last one, within fixed budget, to increase the number of tasks should always be taken in higher priority, even if increasing both tasks and subjects have affects on number of usability findings.KEY WORDS: User-centered Usability test User model Sample size Cost analysisTYPE OF THESIS: Applied Research
Corresponding authors email
