Affordable Access

Publisher Website

A cost-effective evaluation method for use by designers

International Journal of Man-Machine Studies
Publication Date
DOI: 10.1016/s0020-7373(05)80167-1
  • Design


A strong case has been made for iterative design, that is, progressing through several versions of a user interface design using feedback from users to improve each prototype. One obstacle to wider adoption of this approach is the perceived difficulty of obtaining useful data from users. This paper argues that quantitative experimental methods may not be practical at early stages of design, but a behavioural record used in conjunction with think-aloud protocols can provide a designer with the information needed to evaluate an early prototype in a cost-effective manner. Further, it is proposed that a method for obtaining this data can be specified which is straightforward enough to be used by people with little or no training in human factors. Two studies are reported in which trainee designers evaluated a user interface by observing a user working through some set tasks. These users were instructed to think aloud as they worked in a procedure described as “cooperative evaluation”. The instruction received by the designers took the form of a brief how-to-do-it manual. Study 1 examines the effectiveness of the trainee designers as evaluators of an existing bibliographic database. The problems detected by each team were compared with the complete set of problems detected by all the teams and the problems detected by the authors in a previous and more extensive evaluation. Study 2 examined the question of whether being the designer of a system makes one better or worse at evaluating it and whether designers can predict the problems users will experience in advance of user testing.

There are no comments yet on this publication. Be the first to share your thoughts.


Seen <100 times