Knowing there is a definitive need to measure the impacts of a large corporate cost like learning it is fitting to have an industry acceptable model for doing so. This model is actually one that has been in existence since the 1950’s but continues to be accepted today using technology and creativity to maximize its benefits for the modern corporation. In 1959, Donald L. Kirkpatrick, PhD, published a series of four articles called “Techniques for Evaluating Training Programs.” The articles described the four levels of evaluation that he had formulated based on his work for his PhD dissertation at the University of Wisconsin, Madison. Kirkpatrick’s goal was to clarify what evaluation meant. The model clearly defined evaluation as meaning “measuring changes in behavior that occur as a result of training programs.” The model itself is composed of four Levels of training evaluation. A fifth level, ROI has been added since then. The fifth level was the brainchild of Dr. Jack J. Phillips, PhD. The illustration below and subsequent commentary summarize Kirkpatrick’s Four Levels and Phillips’ Fifth Level.
Representation of Kirkpatrick's and Phillips' model of learning analytics showing level-wise measurement objectives
Level 1 - Satisfaction
Level One is a satisfaction survey. Per Kirkpatrick, “evaluating reaction is the same thing as measuring customer satisfaction. If training is going to be effective, it is important that students react favorably to it.” The guidelines for Level One are as follows:
Determine what you want to find out Design a form that will quantify the reactions Encourage written comments and suggestions Strive for 100% immediate response Get honest responses Develop acceptable standards/ benchmarks Measure reactions against standards, and recommend appropriate action Communicate reactions and actions as appropriate
The benefits to conducting Level One evaluations are:
A proxy for customer satisfaction Immediate and real-time feedback on an investment A mechanism to measure and manage learning providers, instructors, courses, locations, and learning methodologies A way to control costs and strategically spend your budget dollars If done properly, a way to gauge a perceived return on learning investment
The instruments for conducting Level One valuations are:
“Happy sheets”, feedback form Verbal reaction, post-training surveys, or questionnaires
Level 2 - Learning
Level Two is a “test” to determine if the learning transfer occurred. States Kirkpatrick, “It is important to measure learning because no change in behavior can be expected unless one or more of these learning objectives have been accomplished. Measuring learning means determining one or more of the following.”
What knowledge was learned? What skills were developed or improved? What attitudes were changed?
The guidelines for Level Two are as follows:
Use a control group, if practical
Evaluate knowledge, skills, and or attitudes both before and after the program Use a ‘test’ to measure knowledge and attitudes Strive for 100% response Use the results to take corrective actions
The benefits to conducting Level Two evaluations are:
Learner demonstrates the transfer of learning Provides training managers with more conclusive evidence of training effectiveness
The instruments for conducting Level Two valuations are:
Typically assessments or tests before and after the training Interview or observation
Level 3 - Impact
Level Three evaluates the job impact of training. “What happens when trainees leave the classroom and return to their jobs? How much transfer of knowledge, skill, and attitudes occurs?” Kirkpatrick questions, “In other words, what change in job behavior occurred because people attended a training program?” The guidelines for Level Three are as follows:
Use a control...