Training Methods

Developments in Business Simulations and Experiential Learning, Volume 32, 2005

DEVELOPING MANAGERIAL EFFECTIVENESS: ASSESSING AND COMPARING THE IMPACT OF DEVELOPMENT PROGRAMMES USING A MANAGEMENT SIMULATION OR A MANAGEMENT GAME
John Kenworthy Managing Director, Corporate Edge Asia [e-mail] Annie Wong Director Corporate Edge Asia [e-mail]
ABSTRACT
This research evaluates the effectiveness of using a management simulation, a management game or case studies within a strategic management training programme. The literature suggests that there is anecdotal evidence that both simulations and games surpass the use of case studies, but there is much criticism of the lack of robust research models used to validate the claims. Using a quasi-experimental design with a reliable managerial competency assessment instrument, the authors assess the impact of different programme groups, the assessed change in workplace behaviour on a 180° basis and participant learning as demonstrated to their own senior managers.

BACKGROUND AND CONTEXT
A large amount of business gaming literature has dealt with how its method fared against the traditional methods for delivering course material (Keys & Wolfe, 1990). For example, the studies by Kaufman (1976), McKenney (1962, 1963), Raia (1966) and Wolfe and Guth (1975) found superior results for game-based groups versus case groups either in course grades, performance on concepts, examinations, or goal-setting exercises. Although anecdotal evidence suggests that students seem to prefer games over other, more traditional methods of instruction, reviews have reported mixed results. Despite the extensive literature, many of the claims and counterclaims for the teaching power of business games and simulations rest on anecdotal materials or inadequate or poorly implemented research (Gredler, 1996). As reviewed by Keys and Wolfe (1990), these research defects have clouded the business gaming literature and hampered the

References: Aldrich, C. (2002). A field guide to educational simulations. Alexandria, VA.: American Society for Training & Development. Anderson, P. H., & Lawton, L. (1997). Demonstrating the learning effectiveness of simulations: Where we are and where we need to go, Developments in Business Simulation & Experiential Exercises (Vol. 24, pp. 6873). Argyris, C. (1980). Some limitations of the case method: Experiences in a management development program. Academy of Management Review, 5(2), 291-298. Bligh, D. (1971). What 's the use of lectures? Harmondsworth: Penguin. Brenenstuhl, D. C., & Catalanello, R. F. (1977). An analysis of the impact upon the learning effectiveness of traditional instruction, simulation, gaming and experiential teaching methodologies: An experimental design. Computer Simulation and Learning Theory, 3, 463-473. Burgoyne, J., & Cooper, C. L. (1975). Evaluation methodology. Journal of Occupational Psychology, 48, 53-62. Butler, R. J., Markulis, P. M., & Strang, D. R. (1988). Where are we? An analysis of the methods and focus of the research om simulation gaming. Simulation & Games, 19, 3-26. Byrne, E. T., & Wolfe, D. E. (1974). The design, conduct and evaluation of a computerized management game as a form of experiential learning. Simulations, Gaming and Experiential Learning Techniques, 1, 22-30. Callanhan, M. R. (1999). Simulation and role play. Alexandria, VA.: American Society for Training & Development. Certo, S. C. (1976). The experiential exercise situation: A comment on instructional role and pedagogy evaluation. Academy of Management Review, 1(3), 113116. Easterby-Smith, M. (1994). Evaluating management development, training and education (Second ed.). Aldershot: Gower. Easterby-Smith, M., & Ashton, D. J. L. (1975). Using repertory grid technique to evaluate management training. Personnel Review, 4(4), 15-21. Easterby-Smith, M., & Thorpe, R. (1997). Research traditions in management learning. In J. Burgoyne & M. Reynolds (Eds.), Management learning: Integrating perspectives in theory and practice (pp. 38-53). London: Sage Publications Ltd. 173 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Easterby-Smith, M., Thorpe, R., & Lowe, A. (1991). Management research: An introduction. London: Sage. Feinstein, A. H., & Cannon, H. M. (2001). Fidelity, verifiability, and validity of simulation: Constructs for evaluation. Detroit: Wayne State University. Feinstein, A. H., & Cannon, H. M. (2002). Constructs of simulation evaluation. Simulation and Gaming, 33(4), 425,440. Feinstein, A. H., Mann, S., & Corsun, D. L. (2002). Charting the experiential territory: Clarifying definitions and uses of computer simulation, games, and role play. The Journal of Management Development, 21(10), 732-744. Freedman, R. D., & Stumpf, S. A. (1980). Learning style theory: Less than meets the eye. Academy of Management Review, 5(3), 445-447. Garris, R., Ahlers, R., & Driskell, J. E. (2002). Games, motivation, and learning: A research and practice model. Simulation & Gaming, 33(4), 441-467. Gibbs, G. (1988). Learning by doing: A guide to teaching and learning methods.London: Kogan Page. Gosen, J., & Washbush, J. (2004). A review of scholarship on assessing experiential learning effectiveness. Simulations & Gaming, 35(2), 270-293. Gosenpud, J. (Ed.). (1990). Evaluation of experiential learning. London: Kogan Page. Gredler, M. E. (1996). Educational games and simulations: A technology in search of a (research) paradigm. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology. New York: Simon & Schuster Macmillan. Hamblin, A. C. (1974). Evaluation and control of training. Maidenhead: McGraw Hill. Hays, R. T., & Singer, M. J. (1989). Simulation fidelity in training systems design: Bridging the gap between reality and training. New York: Springer-Verlag. Higgs, M., & Rowland, D. (2001). Developing change leaders: Assessing the impact of a development programme. Journal of Change Management, 2(1), 4764. Hunsaker, J. S. (1981). The experiential learning model and the learning style inventory: An assessment of current findings. Journal of Experiential Learning and Simulation, 2, 145-152. Kaufman, F. L. (1976). An empirical study of the usefulness of a computer-based business game. Journal of Educational Data Processing, 13(1), 13-22. Kenworthy, J. M. (2003). Evaluating management development. Henley-on-Thames: Henley Management College. Keys, J. B. (1977). The management of learning grid for management development. Academy of Management Review, 2(2), 289-297. Keys, J. B., Wells, R. A., & Edge, A. G. (1994). The multinational management game: A simuworld. The Journal of Management Development, 13(8), 26-37. Keys, J. B., & Wolfe, J. (1990). The role of management games and simulations in education and research. Journal of Management, 16(2), 307-336. Kirkpatrick, D. (1959/60). Techniques for evaluating training programs: Parts 1 to 4. Journal of the American Society for Training and Development, November, December, January and February. Kirkpatrick, D. L. (1994). Evaluating training programs: The four levels.San Francisco: Berret-Koehler. Kolb, D. A. (1984). Experiential learning: Experience as a source of learning and development. Englewood Cliffs, NJ: Prentice Hall. Kolb, D. A. (1999). Learning style inventory (Version 3 ed.): HayGroup. Lamb, S. W., & Certo, S. C. (1978). The learning style inventory (lsi) and instrument bias. Academy of Management Proceedings, 28-33. Lundy, J. (2003, October). E-learning simulation: Putting knowledge to work. Paper presented at the Gartner U.S. Symposium/Itxpo. Luthans, F., Rosenkrantz, S. A., & Hennessey, H. W. (1985). What do successful managers really do? An observation study of managerial activities. The Journal of Applied Behavioural Science, 21(3), 255-270. May, T. (1993). Social research: Issues, methods and process. Buckingham: Open University Press. McBer. (1997). Managerial competency questionnaire. London: TRG Hay/McBer. McKenney, J. L. (1962). An evaluation of business game in an mba curriculum. Journal of Business, 35, 278-286. McKenney, J. L. (1963). An evaluation of a decision simulation as a learning environment. Management Technology, 3(1), 56-67. McKenney, J. L. (1967). Simulation gaming for management development.Boston, MA.: The Division of Business, Harvard College. Miles, W. G., Biggs, W. D., & Schubert, J. N. (1986). Students perceptions of skill acquisition through cases and a general management simulation: A comparison. Simulation & Games, 17, 7-24. Mitchell, R. C. (2004). Combining cases and computer simulations in strategic management courses. Journal of Education for Business, 79(4), 198-204. Morse, K. (2001). Assessing the efficacy of experiential learning in a multicultural environment. Developments in Business Simulation & Experiential Learning, 28, 153-159. Partridge, S. E., & Scully, D. (1979). Cases versus gaming. Management Education and Development, 10(3), 172180. Raia, A. P. (1966). A study of the educational value of management games. Journal of Business, 39, 339-352. Remenyi, D., Williams, B., Money, A., & Swartz, E. (1998). Doing research in business and management. London: Sage Publishing. 174 Developments in Business Simulations and Experiential Learning, Volume 32, 2005 Robotham, D., & Jubb, R. (1996). Competences: Measuring the unmeasurable. Management Development Review, 9(5), 25-29. Romme, A. G. L. (2003). Learning outcomes of microworlds for management education. Management Learning, 34(1), 51-61. Ross, S., & Morrison, G. R. (2003). Experimental research methods. In D. H. Jonassen (Ed.), Handbook of research methods in educational communications and technologies (Second ed., pp. 1021-1043): AECT. Russ-Eft, D., & Preskill, H. (2001). Evaluation in organizations a systematic approach to enhancing learning, performance, and change. Cambridge, MA.: Perseus Publishing. Sales, E., & Cannon-Bowers, J. A. (2001). The science of training: A decade of progress. Annual Review of Psychology, 52, 471-499. Spencer, L. M., & Spencer, S. (1993). Competence at work: Models for superior performance. New York: John Wiley & Sons. Warr, P. B., Bird, M. W., & Rackham, N. (1970). Evaluation of management training. Aldershot: Gower. Washbush, J., & Gosen, J. (2001). Learning in total enterprise simulations. Simulation & Gaming, 32, 281296. Wellington, W. J., & Faria, A. J. (1992). An investigation of the relationship between team cohesion, player attitude, performance attitude on player performance. Developments in Business Simulation & Experiential Exercises, 19, 184-189. Wimer, S. (2002). The dark side of 360-degree. Training & Development, 37-42. Wolfe, J. (Ed.). (1990). The evaluation of computer-based business games: Methodology, findings, and future needs. USA: Association for Business Simulation and Experiential Learning (ABSEL). Wolfe, J., & Crookall, D. (1998). Developing a scientific knowledge of simulation/gaming. Simulation & Gaming, 29, 7-19. Wolfe, J., & Guth, G. (1975). The case approach vs. Gaming in evaluation. Journal of Business, 48(3), 349364. 175