User Experience Evaluation Methods in Academic
User Experience Evaluation Methods in Academic and
Industrial Contexts
Virpi Roto1,2, Marianna Obrist3, Kaisa Väänänen-Vainio-Mattila1,2
1
2
3
Tampere University of Technology, Human-Centered Technology, Korkeakoulunkatu 6,
33720 Tampere, Finland. [virpi.roto, kaisa.vaananen-vainio-mattila]@tut.fi
Nokia Research Center, P.O.Box 407, 00045 Nokia Group, Finland. virpi.roto@nokia.com
ICT&S Center, University of Salzburg, Sigmund-Haffner-Gasse 18, 5020 Salzburg, Austria. marianna.obrist@sbg.ac.at ABSTRACT
In this paper, we investigate 30 user experience (UX) evaluation methods that were collected during a special interest group session at the CHI2009 Conference. We present a categorization of the collected UX evaluation methods and discuss the range of methods from both academic and industrial perspectives.
Keywords
User experience, Evaluation methods
INTRODUCTION & BACKGROUND
As particular industry sectors mature, usability and technical reliability of products is taken for granted and users start to look for products that provide engaging user experience
(UX). Although the term user experience originated from industry and is a widely used term also in academia, the tools for managing UX in product development are still inadequate. UX evaluation methods play a key role in ensuring that product development is going to the right direction. Many methods exist for doing traditional usability evaluations, but user experience (UX) evaluation differs clearly from usability evaluation. Whereas usability emphasizes effectiveness and efficiency [5], UX includes hedonic characteristics in addition to the pragmatic ones
[6], and is thus subjective [3,12]. Therefore, UX cannot be evaluated with stopwatches or logging. The objective measures such as task execution time and the number of clicks or errors are not valid measures for UX, but we need to understand how the user feels about the system. User’s motivation and
Industrial Contexts
Virpi Roto1,2, Marianna Obrist3, Kaisa Väänänen-Vainio-Mattila1,2
1
2
3
Tampere University of Technology, Human-Centered Technology, Korkeakoulunkatu 6,
33720 Tampere, Finland. [virpi.roto, kaisa.vaananen-vainio-mattila]@tut.fi
Nokia Research Center, P.O.Box 407, 00045 Nokia Group, Finland. virpi.roto@nokia.com
ICT&S Center, University of Salzburg, Sigmund-Haffner-Gasse 18, 5020 Salzburg, Austria. marianna.obrist@sbg.ac.at ABSTRACT
In this paper, we investigate 30 user experience (UX) evaluation methods that were collected during a special interest group session at the CHI2009 Conference. We present a categorization of the collected UX evaluation methods and discuss the range of methods from both academic and industrial perspectives.
Keywords
User experience, Evaluation methods
INTRODUCTION & BACKGROUND
As particular industry sectors mature, usability and technical reliability of products is taken for granted and users start to look for products that provide engaging user experience
(UX). Although the term user experience originated from industry and is a widely used term also in academia, the tools for managing UX in product development are still inadequate. UX evaluation methods play a key role in ensuring that product development is going to the right direction. Many methods exist for doing traditional usability evaluations, but user experience (UX) evaluation differs clearly from usability evaluation. Whereas usability emphasizes effectiveness and efficiency [5], UX includes hedonic characteristics in addition to the pragmatic ones
[6], and is thus subjective [3,12]. Therefore, UX cannot be evaluated with stopwatches or logging. The objective measures such as task execution time and the number of clicks or errors are not valid measures for UX, but we need to understand how the user feels about the system. User’s motivation and
References: 1. Amaldi, P., Gill, S., G., Fields, B., Wong, W., (Eds.), (2005) ceedings.doc (accessed: 26.06. 2009). (2008). Instant Card Technique: How and Why to apply in User-Centered Design 3. Desmet, P.M.A. (2002). Designing Emotions. Doctoral dissertation, Technical University of Delft. interactive systems (formerly known as 13407). 6. Hassenzahl, M. 2004. The interplay of beauty, goodness, and usability in interactive products 7. Hazlett, R., L., (2006). Measuring emotional valence during interactive experiences: boys at video game play, 8. Fields, B., Amaldi, P., Wong, W., Gill, S., (2007). Methods in Product Development workshop. Uppsala, Sweden, August 25, 2009. Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008)