Evaluating the Effectiveness of Safety Training
Have the participants perform, and you will have a better understanding of the actual learning that took place.
AS safety professionals, all of us have a basic understanding of the effectiveness of our safety program--but what about our safety training specifically? Many people, unfortunately, see safety training as a compliance issue: something that has to get done, not necessarily something that is really going to change or impact safety performance.
Training employees about safe forklift operation and actual safe forklift operation may be miles apart. Evaluating the effectiveness of our training not only will allow us to see areas where the training needs to be improved, but may provide insight on ways to improve it, as well.
Using Reaction Surveys Wisely
Most of us are familiar with perception or reaction surveys typically given at the end of a class. These are also known as level-one evaluations by most trainers. They evaluate the training based on the participants' perception. Often administered at the end of a program, they typically ask questions such as: Was the training effective? Was the instructor knowledgeable? Were the handouts appropriate? Was the room comfortable?
While perception surveys can provide good information, you want to be sure the questions are relevant. Think about it--if the evaluation is given at the end of the class, will knowing temperature comfort levels benefit you after the class is finished? This should be determined before the class begins, or at least toward the beginning. Many times, there is nothing the instructor can do to change or adjust certain things. Unless the information is going to be used immediately or for the next class, why ask?
While asking whether the instructor appeared prepared or had good delivery may be appropriate, whether the instructor was knowledgeable really is not something that should be asked of the participants. Another content expert should determine the knowledge level of the instructor, not the students. If the students were in the position to be able to judge, then they probably don't need to be in the class.
Another common mistake when thinking about evaluation is not considering whether the class is going to be repeated. In other words, certain aspects in a one-time class may not be worth evaluating. Remember, one of the main reasons for evaluation is to be able to make adjustments or improvements on the next class. If there isn't a next class, you may want to reconsider the need to evaluate, or at least the need to perform a level-one reaction survey. Perhaps a higher-level evaluation (level three or four) would be more appropriate.
Higher-Level Evaluations
A level two evaluation tries to determine whether the participants learned anything. This is frequently done in the form of a quiz at the end of a class, often just before the level one perception survey. You may hear the instructor say, "After you complete the quiz (level 2), please complete the evaluation (level 1)."
Level two evaluation is the most common evaluation tool used. Most trainers see it as the way to demonstrate the training was effective. Just ask, "How do you know the training was effective?" and you probably will be handed the stack of quizzes. This level of evaluation can do a pretty good job for certain things. It can show that knowledge was gained and retained--at least, until the end of the class. The problem is that knowledge does not necessarily mean "learning."
If the goal of the training is the retention of information, for instance the flashpoint of a certain chemical, then a quiz may be the best evaluation. If a skill or task is being taught, however, such as the use of a gas detection meter, then a more effective measurement of learning would be to set up scenarios, drills, demonstrations, role-playing, or some similar evaluation that would demonstrate learning took place. If you can create a situation that is as similar to the expected real-world performance as possible, you increase the likelihood of getting a more effective evaluation. Think about the performance requirement in a cardiopulmonary resuscitation class. A student may be able to remember and select the correct answer on the multiple-choice quiz for the compressions-to-breaths ratio or the sequence of events when activating the EMS, but if you have her actually perform CPR on the manikin, you get a better indication of her skill level. The same is true of forklift training, lockout/tagout, or any number of training sessions: Have the participants perform, and you will have a better understanding of the actual learning that took place.
You will want to use a form that allows you to document performance so you can keep records for your training. Documenting the higher-level evaluations is unfortunately what sometimes makes a quiz so attractive. A quiz may be quick and easy, but you will gain a great deal more by documenting performance.
This article originally appeared in the June 2003 issue of Occupational Health & Safety.