Training Evaluation Levels

Why Training Evaluation Levels Matter for the ASP Exam

Safety professionals spend enormous time and money delivering training—hazard communication, lockout/tagout, confined space entry, fall protection, forklift operation, and more. But delivering training is not the same as achieving results. The BCSP ASP examination blueprint includes Training, Education and Communication as a domain, and within it you are expected to know how to evaluate the effectiveness of training, not merely how to deliver it. The dominant framework for this is the Kirkpatrick Four-Level Model (often extended to a fifth level, Return on Investment, by Jack Phillips). Questions on this topic appear regularly because they test whether you understand the difference between activity (people attended a class) and outcomes (injuries decreased because behavior changed). Understanding the levels also supports compliance with ANSI/ASSP Z490.1 (Criteria for Accepted Practices in Safety, Health and Environmental Training), which requires that training programs include an evaluation component.

What Is the Training Evaluation Levels Model?

Donald Kirkpatrick introduced his model in the 1950s as a way to assess training programs in a structured, progressive manner. Each level builds on the one before it, and each becomes more difficult, more time-consuming, and more expensive to measure—but also more meaningful to the organization.

Level 1 – Reaction
Measures how participants felt about the training. Did they find it relevant, engaging, well-organized, and useful? Did they like the instructor, the materials, the facility, the pace?
• Typical tools: end-of-course evaluation forms, "smile sheets," satisfaction surveys, verbal feedback, focus groups.
• Timing: immediately at the end of the session.
• Limitation: A positive reaction does not prove anyone learned anything. Trainees can enjoy a class and retain nothing; conversely, a demanding class may be rated poorly yet produce strong learning.

Level 2 – Learning
Measures the extent to which participants acquired the intended knowledge, skills, and attitudes. Did they actually learn what the objectives said they would learn?
• Typical tools: written pre-tests and post-tests, quizzes, skill demonstrations, practical exercises, simulations, role plays, return demonstrations (e.g., properly donning a respirator or applying a lock and tag).
• Timing: during or immediately after training; a pre-test/post-test design isolates the gain attributable to the course.
• Limitation: Demonstrating knowledge in the classroom does not guarantee the person will apply it on the job.

Level 3 – Behavior (Transfer)
Measures whether participants apply what they learned back on the job. Has performance changed? Is the worker now using the fall arrest system correctly, following the confined space permit procedure, or wearing hearing protection consistently?
• Typical tools: direct workplace observations, behavior-based safety sampling, supervisor and peer interviews, job safety audits, inspections, follow-up surveys, review of work records.
• Timing: weeks to months after training (commonly 30, 60, or 90 days) to allow habits to form and be observed.
• Key insight: Behavior change depends not only on training but on the work environment—supervisor support, availability of equipment, time pressure, reinforcement, and consequences. If behavior does not change, the cause may be a system problem rather than a training failure.

Level 4 – Results
Measures the organizational impact of the training. Did the business outcomes that justified the training actually improve?
• Typical metrics: injury and illness rates (TRIR, DART), lost workdays, workers' compensation costs, property damage, near-miss trends, regulatory citations, audit scores, productivity, quality, absenteeism, turnover.
• Timing: months to a year or more after training.
• Limitation: Results are influenced by many variables beyond training (new equipment, engineering controls, staffing changes, economic conditions). Isolating the training effect is difficult; control groups and trend analysis help.

Level 5 – Return on Investment (Phillips Extension)
Jack Phillips added a fifth level that converts Level 4 results into monetary value and compares them to the fully loaded cost of the training.
• Formula: ROI (%) = (Net Program Benefits ÷ Program Costs) × 100, where Net Benefits = Program Benefits − Program Costs.
• Example: A training program costs $20,000 and reduces injury costs by $80,000. Net benefit = $60,000. ROI = ($60,000 ÷ $20,000) × 100 = 300%.
• Also related: Benefit-Cost Ratio (BCR) = Program Benefits ÷ Program Costs = $80,000 ÷ $20,000 = 4:1.

How the Model Works in Practice

1. Plan evaluation before training is designed. Kirkpatrick recommended working backward: define the desired Results (Level 4), identify the Behaviors needed to achieve them (Level 3), determine the Learning required to produce those behaviors (Level 2), and then design an engaging course (Level 1). Evaluation criteria should align with the learning objectives written during needs assessment.

2. Use the ADDIE connection. In the instructional systems design cycle (Analyze, Design, Develop, Implement, Evaluate), the Kirkpatrick levels are the structure for the Evaluate phase, and evaluation data feed back into Analysis for continuous improvement.

3. Match the level to the decision you need to make. Level 1 data help improve delivery and instructors. Level 2 data verify competency (critical for regulatory training such as OSHA 1910.147 or 1910.146, which require the employer to verify that employees have the required knowledge and skills). Level 3 data reveal transfer barriers. Level 4 and 5 data justify the program to management.

4. Recognize the cost/difficulty gradient. Nearly every organization evaluates at Level 1; fewer at Level 2; far fewer at Levels 3 and 4; very few at Level 5. The higher the level, the more resources and time required, but the greater the value of the information.

5. Remember that each level provides different, not redundant, information. Success at a lower level does not guarantee success at a higher one, and failure at a higher level does not always mean the training itself was deficient.

Common Misconceptions to Avoid
• Attendance records, sign-in sheets, and certificates are documentation, not evaluation of effectiveness.
• Post-training quizzes measure Learning (Level 2), not Behavior (Level 3). Watching someone perform the task on the floor weeks later is Level 3.
• A drop in injury rates is Level 4 (Results), not Level 3, even though it may be caused by behavior change.
• Reaction sheets are not useless—they identify problems with delivery—but they are the weakest evidence of effectiveness.
• ROI is only valid when benefits are converted to dollars; a simple reduction in incident counts remains Level 4.

How to Answer Exam Questions on Training Evaluation Levels

Most ASP questions on this topic fall into a few predictable patterns:

Pattern 1 – Identify the level from a described activity.
Read the scenario and ask: What is being measured, and when?
• Opinions, satisfaction, comfort, enjoyment, "rated the course" → Level 1 Reaction.
• Tests, quizzes, demonstrations, "scored", "knowledge gain", pre/post comparison → Level 2 Learning.
• Observation on the job, supervisors report workers now use..., audits of work practices, "applied", "transfer" → Level 3 Behavior.
• Injury rates, costs, productivity, citations, organizational metrics → Level 4 Results.
• Dollar comparison of benefits to costs, percentage return → Level 5 ROI.

Pattern 2 – Identify the best method for a given level.
Choose the tool that fits the level: survey for reaction, test or skill check for learning, workplace observation for behavior, incident/cost data for results.

Pattern 3 – Which level is most/least difficult, costly, or valuable?
Level 1 is easiest, cheapest, most commonly done, least indicative of effectiveness. Level 4 (or 5) is hardest, most expensive, least commonly done, most meaningful to management.

Pattern 4 – Which level answers a specific management question?
"Did they like it?" → 1. "Did they learn it?" → 2. "Are they doing it?" → 3. "Did it matter to the organization?" → 4. "Was it worth the money?" → 5.

Pattern 5 – Calculation questions.
Know ROI and BCR formulas and the difference between them. ROI uses net benefits (benefits minus costs) divided by costs, expressed as a percentage. BCR uses total benefits divided by costs, expressed as a ratio.

Pattern 6 – Best practice/design questions.
Correct answers favor: establishing evaluation criteria during needs analysis and objective-writing; using measurable, observable objectives; using pre-tests to establish a baseline; following up on the job after a delay; and using evaluation findings to revise the program.

Exam Tips: Answering Questions on Training Evaluation Levels

• Memorize the sequence with a mnemonic: R-L-B-R — "Really Learn, Behave Right" (Reaction, Learning, Behavior, Results), plus ROI as the optional fifth.
• Look for timing cues. "At the end of class" usually points to Level 1 or 2; "three months later" points to Level 3; "over the following year" points to Level 4.
• Look for location cues. In the classroom → Levels 1 and 2. On the job site → Level 3. In organizational records → Levels 4 and 5.
• Distinguish "can do" from "does do." A skill demonstration in training proves the worker can perform the task (Level 2). Observing the worker actually performing it during normal work proves they do perform it (Level 3).
• Do not confuse behavior change with results. If the question mentions injury frequency, severity, cost, or compliance metrics, it is Level 4 even if behavior is implied.
• When asked for the "most effective" or "best" indicator of training effectiveness, the higher level is almost always the correct answer (Level 3 or 4 over Level 1 or 2).
• When asked for the "most commonly used" or "easiest" method, the answer is Level 1 reaction surveys.
• When the question concerns OSHA verification requirements (e.g., lockout/tagout, powered industrial trucks, confined space), Level 2 learning assessment—and ideally Level 3 observation—is the minimum needed to demonstrate competency.
• Read calculation questions carefully: determine whether they ask for ROI (percentage, net benefits) or BCR (ratio, gross benefits). Watch for distractor answers that use the wrong numerator.
• Remember the limitation of each level when a question asks what a result does not prove—e.g., high reaction scores do not prove learning; passing a test does not prove on-the-job application.
• Eliminate documentation-only answers. Sign-in sheets, certificates, and training matrices show that training occurred, not that it was effective.
• Consider the full system. If a scenario says employees passed the test but still do not follow the procedure, the best answer usually points to workplace barriers (supervision, reinforcement, equipment, culture), not to redoing the same training.

Practice Questions

1. After a hazard communication course, the safety manager reviews completed participant questionnaires rating the instructor's clarity and the relevance of the material. Which evaluation level is this?
Answer: Level 1 – Reaction. The data reflect participant opinions about the training experience.

2. Six weeks after forklift training, supervisors conduct unannounced observations and record whether operators sound the horn at intersections and perform pre-shift inspections. This is an example of which level?
Answer: Level 3 – Behavior. The evaluation measures on-the-job application after a time delay.

3. A company spent $15,000 on ergonomics training and documents a $60,000 reduction in musculoskeletal disorder claims the following year. What is the ROI?
Answer: Net benefits = $60,000 − $15,000 = $45,000. ROI = ($45,000 ÷ $15,000) × 100 = 300%. (The BCR would be 4:1.)

4. Which evaluation level is generally the most difficult and costly to measure but provides the most convincing evidence of training value to senior management?
Answer: Level 4 – Results (or Level 5 – ROI if listed), because organizational outcomes are affected by many variables and require long-term data collection.

5. Employees score an average of 95% on a post-training lockout/tagout exam, yet audits reveal continued failure to verify zero energy state. What does this most likely indicate?
Answer: Learning (Level 2) occurred, but transfer to Behavior (Level 3) did not. The safety professional should investigate workplace barriers such as supervision, time pressure, equipment availability, and reinforcement rather than assuming the training content was deficient.

Summary

The Kirkpatrick model gives safety professionals a logical, progressive framework for answering the critical question: Did the training work? Level 1 asks whether trainees liked it, Level 2 whether they learned it, Level 3 whether they use it, Level 4 whether the organization benefited, and Level 5 whether it paid for itself. For the ASP exam, master the definitions, the measurement tools and timing for each level, the increasing difficulty and value as you move up the hierarchy, the ROI and BCR formulas, and the ability to classify any described scenario into the correct level. With these fundamentals, questions on training evaluation become some of the most predictable—and most answerable—items on the exam.