CAN YOU COMBINE USABILITY METRICS INTO SINGLE SCORES?
Throughout the history of statistics, there has been an initial reluctance to combine measurements in any way, typically followed by empirical and theoretical work that supports the combination. For example, before the mid-17th century, astronomers would not average their observations—“the idea that accuracy could be increased by combining measurements made under different conditions was slow to come”. We are now so used to the arithmetic mean that we often don’t give a second thought to computing it (and in some situations we really should). But what about combining similar measurements from different sources into a composite metric? That’s exactly what we do when we compute a stock index such as the Dow-Jones Industrial Average. We are comfortable with this type of combined score, especially given its successful use for over 100 years, but that level of comfort was not always in place. When William Stanley Jevons published analyses in which he combined the prices of different commodities into an index to study the global variation in the price of gold in the mid-19th century, he met with significant criticism. Stock and commodity indices at least have the common metric of price. What about the combination of different metrics, for example, the standard usability metrics of successful completion rates, completion times, and satisfaction? The statistical methods for accomplishing this task, based on the concepts of correlation and regression, appeared in the early 20th century and underwent an explosion of development in its first half (Cowles, 1989), producing principal components analysis, factor analysis, discriminant analysis, and multivariate analysis of variance (MANOVA). Lewis (1991) used nonparametric rank-based methods to combine and analyze time-on-task, number of errors, and task-level satisfaction in summative usability tests. Conversion to ranks puts the different usability metrics on a common...
Please join StudyMode to read the full document