# data[].attributes.headline
The Kirkpatrick Model of Training Evaluation

# data[].attributes.subject
The Kirkpatrick Model of Training Evaluation

# data[].attributes.attachment_link.title
The Kirkpatrick Model of Training Evaluation

# data[].attributes.headline
How to Use the Kirkpatrick Model for Effective L&D Assessment

# data[].attributes.subject
How to Use the Kirkpatrick Model for Effective L&D Assessment

# data[].attributes.attachment_link.title
How to Use the Kirkpatrick’s 4 Levels of Evaluation Model for Effective L&D Assessment

# data[].attributes.attachment_link.readable_text
Why Evaluate Training Impact? The Kirkpatrick Model Can Help You

In short, so that you can improve the learner experience and achieve the goals of your training. Good Instructional Design never really comes to an end; your content can always be refined and improved. If you simply finish a project and move on to the next one, you will never learn; if there are problems with the content you're producing, they will persist through the content you produce in the future. As designers and developers, it is our duty to be learning ourselves and to be continually improving what we create. We owe it to our clients and to the people learning from our courses.

As we go through these levels, ask yourself where you would place your current evaluation of your training. And what could you do to move it on to the next level?

The Levels Of Kirkpatrick

Level 1: Reaction

This is on the level of 'which face best represents how you feel about this'. It's about how participants respond to your training; did they enjoy it? Did they think it was valuable? Did they feel good about the instructor or the venue or the design? It tells you nothing about whether your course actually fulfilled its objective and taught something; the later levels do that, but it does give you an idea how your learning was received, and how you could improve the User Experience.
How do I move up to this level?
If you aren't currently evaluating your training at all, this is the place to start. Why not implement a post-course questionnaire, asking participants how they found the course? Gather the responses and see if there are any recurring themes. Do people struggle with the navigation? Find the voiceover irritating? Do you need to rethink accessibility? There's a potential wealth of information here, and a questionnaire is a simple thing to set up.

Level 2: Learning

The point of learning is-unsurprisingly-that people learn! Did the participants actually learn from your material? How much has their knowledge increased?
How do I move up to this level?
A good way of assessing this is with 2 quizzes-one at the beginning and one at the end of the course. Ask questions about the same topics, and see if learners are answering more questions correctly after their training. If they are, it would suggest that they did learn; if not, then something about your learning material is clearly not doing its job.

This can be a helpful method of evaluation, as it can give you specific information. If all your learners are getting questions about a specific topic wrong, it might be time to look again at how you're teaching that topic. What about it is unclear? How could you better present it so your learners take the knowledge on board?

Another way of assessing this is, again, with a post-course questionnaire. But in addition to the basic questions of level 1, you could ask the learners to tell you what they learned on this course. In some cases, this might even be more insightful than a quiz. Asking people to articulate something in their own words shows how much they truly understand about it.

One final suggestion; you could send out a follow-up quiz, say a week after the training. Quizzing them straight after they've taken the course, when it's all fresh in their minds, doesn't tell you what they'll remember over the long term. What do they recall a week later? A month later? And if the key points aren't being remembered; how can you improve your learning to ensure that they are? Should you be providing refresher training or job aids in addition to your training course? Would it help to build a mobile app that sends daily tips and reminders to your learners after the course?

Level 3: Behavior

We've all been on those compliance courses where we learn about the correct procedure for doing something, and then go back into the office the next day and continue doing things exactly how we used to. The issue here is not lack of knowledge; we know the correct procedure. It's that the knowledge is not being applied. Reaching Kirkpatrick's level 3 means asking, are participants using what they learned?
How do I move up to this level?
This is usually something you have to assess a little while after the course has been taken; you need to leave time for the new behaviors to settle in.

The best way of gaining this kind of insight is through 360º feedback.

360º feedback comes from the participant themselves, their colleagues, and their superiors. Asking the participant and everyone around them if their behavior has changed after taking the learning will give you a 360º view of the situation! If your training has had the desired effect, it will be noticeable to everyone involved.

Sometimes, the feedback will say that no changes have occurred. In those instances, it's important to ask why people think this is the case. Behavior can only change if the conditions for it are favorable. Will the boss actually let your participant apply their new knowledge? Is there a tool or a system that has not been put in place? Does the learner have any desire or incentive to apply the learning? And what can be done to remedy these situations?

Level 4: Results

Kirkpatrick's final level of evaluation looks at whether training positively impacted the organization.

This relies on goals being set before the development of the training. What changes were managers looking for? How is success defined? Otherwise, you won't know what results you're hoping to see.
How do I move up to this level?
The way you evaluate this will be determined by the results you're looking to see. Typically, it will involve analyzing data. If it's improvement in ROI you're aiming for, you need to be assessing financial statements. If it's a lessening of health and safety incidents in the office, you need the data on how many incidents there have been.

When evaluating the impact of your training, it's important to know about all 4 of these levels. For example, if the behavior hasn't changed, you need to look at the previous levels to understand why-did people actually learn what they needed to? And if not, is that because the design was so confusing and unhelpful that they mentally checked out? Plan your training evaluations to cover all 4 levels; that way, you'll have the whole perspective on how effective your training is, and how it can be improved.

# data[].attributes.attachment_link.description
Wondering what The Kirkpatrick Model really is? Check all about The Kirkpatrick Model and how it can help you evaluate your training impact.

# data[].attributes.headline
The Kirkpatrick Evaluation Model: How to measure training effectiveness

# data[].attributes.subject
The Kirkpatrick Evaluation Model: How to measure training effectiveness

# data[].attributes.attachment_link.readable_text
Are your employee training programs effective? How can you even tell?

The first step is measurement. You need to understand what is working (and what isn't) so you can design and develop programs that meet the needs of your employees and the business.

The Kirkpatrick Evaluation Model helps businesses do just that.

Through this simple four-step process, you can measure training effectiveness and improve your instructional design for future initiatives.

What is the Kirkpatrick Model?

The Kirkpatrick Evaluation Model was first introduced in 1959 by Donald Kirkpatrick, a professor emeritus at the University of Wisconsin. It has since undergone several iterations over the years by Kirkpatrick and, after his death, his family-the most recent update, called the New World Kirkpatrick Model, was released in 2016.

The model outlines four levels of training evaluation:

The Kirkpatrick Model remains a popular and widely used evaluation strategy because of its simplicity and relevancy across industries and organizations.

4 levels of the Kirkpatrick Evaluation Model

Benefits and limitations of the Kirkpatrick Model

The Kirkpatrick Model's advantages make it an attractive choice for trainers and other business leaders. It:

Provides clear evaluative steps to follow

Works with traditional and digital learning programs

Gives HR and business leaders valuable insight into their overall training programs and their impact on business outcomes

Additionally, its simple approach is highly flexible and adaptable across industries and applications, making it easy for trainers to implement the model.

However, there are limitations and tradeoffs. The system can be time-consuming and requires costly investment to conduct all the pre-tests, post-tests, and analyses of learning outcomes.

It is also difficult to directly link business results to specific training. Proving efficacy and return on investment (ROI) is hard since the causes are often complex and multifaceted. To do so accurately requires further resources and expertise.

Kirkpatrick's 4 levels of training evaluation

Level 1: Reaction

The first step is documenting your employees' reactions to the training. Employee engagement (at work or during training) is crucial for running a productive business-but it's not so easy to achieve. Understanding how employees perceived the training can give you insight into what is working well or needs improvement.

To learn what your employees think about your training program, you can send out a survey immediately following the session. Consider asking questions like the following:

Was the presentation engaging?

Did you feel the training was worth your time?

Can you apply what you learned to your job?

Did you like the style or method of training?

What would you change or improve for future training?

What resources or support do you need to apply what you learned?

You can also track metrics such as participation rate, completion rate, and time spent on training (useful for self-directed online training courses).

Use the feedback to help you identify areas for improvement and consider possible changes for future iterations of your training program.

Level 2: Learning

Here you measure exactly what the employees learned (or didn't learn) in the training. To do this effectively, you should test your employees before and after the training to measure progress. You can use skills-based evaluations or interviews to record your findings.

Unlike simple or subjective reactions to the training, during this stage, you'll want to measure specific outcomes. Start by identifying what learning outcomes or results you're aiming for in the training, then test for those outcomes before and after.

Level 3: Behavior

The next step is to gauge whether your training has impacted behavior. In other words, are your employees applying what they learned?

This process takes time-after all, it can take weeks or months for employees to build confidence or have the opportunity to apply their knowledge.

To measure behavior, conduct interviews, record observations on the ground, and provide opportunities (e.g., assign projects) for team members to apply the skills they learned in the training.

However, keep in mind that a lack of behavioral change doesn't necessarily mean the training was ineffective. Evaluate your current processes and cultural conditions in the business to ensure you're building an environment that encourages learning and positive change.

Level 4: Results

The final step of the Kirkpatrick Model is measuring results. This is the stage where you evaluate how the behavioral changes impacted the business and whether your training investment resulted in a good ROI.

Using the Kirkpatrick Evaluation Model to design training

While the Kirkpatrick Evaluation Model is meant to help you improve existing training, you can also use it to start from scratch. Many trainers use the Kirkpatrick Evaluation Model backward as a guide for developing effective training. They begin by identifying the outcomes they hope to achieve from the training and use those desired results to design a program to achieve them.

However, when you identify the outcomes and metrics of success you're aiming for in your training, you can design more effective programs with measurable results.

Kirkpatrick Evaluation Model (Click on image to modify online)

The takeaway

Despite its limitations, the Kirkpatrick Evaluation Model is still one of the most common training evaluation methods today.

The key to using it effectively is to make training evaluation an integral part of your training design from the beginning. By working backward on the Kirkpatrick 4 levels, you can develop training initiatives that are effective and impactful-and directly tied to measurable outcomes.

Use Lucidchart to keep your training processes on track

As you design your training programs, use Lucidchart to outline your processes as a flowchart and use swimlanes to make sure you hit each target in the evaluation model.

If you use a learning management system, you can even link data from the LMS reports (such as participation rates) directly to Lucidchart to visualize your training progress and make connections to ROI.

You can also bring Lucidchart into your corporate classroom as a learning tool. Visuals are a great way to increase engagement and improve knowledge retention (particularly for visual learners).

Learn more about the importance of employee training and development programs, and how Lucid can help.

# data[].attributes.attachment_link.title
How to Use the Kirkpatrick Evaluation Model

# data[].attributes.attachment_link.description
Are your employee training programs effective? See how the Kirkpatrick Evaluation Model can help you design and measure programs to meet your employees' needs.

# data[].attributes.attachment_link.readable_text
The Kirkpatrick Model

Level 1: Reaction

The degree to which participants find the training favorable, engaging, and relevant to their jobs

Level 2: Learning

The degree to which participants acquire the intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training

Level 3: Behavior

The degree to which participants apply what they learned during training when they are back on the job

Level 4: Results

The degree to which targeted outcomes occur as a result of the training and the support and accountability package

# data[].attributes.attachment_link.description
The Kirkpatrick Model, or four levels of training evaluation, consists of Reaction, Learning, Behavior and Results.

# data[].attributes.headline
The Kirpatrick Model of Training Evaluation 

# data[].attributes.subject
The Kirpatrick Model of Training Evaluation 

# data[].attributes.attachment_link.readable_text
Updated

This guide will introduce the Kirkpatrick Model and the benefits of using this model in your training program. After reading this guide, you will be able to effectively use it to evaluate training in your organization.

Discover:

What is the Kirkpatrick Model?

The Kirkpatrick model, also known as Kirkpatrick's Four Levels of Training Evaluation, is a key tool for evaluating the efficacy of training within an organization. This model is globally recognized as one of the most effective evaluations of training.

The Kirkpatrick model consists of 4 levels: Reaction, learning, behavior, and results.

It can be used to evaluate either formal or informal learning and can be used with any style of training.

The Kirkpatrick Model has been widely used since Donald Kirkpatrick first published the model in the 1950s and has been revised and updated 3 times since its introduction. In 2016, it was updated into what is called the New World Kirkpatrick Model, which emphasized how important it is to make training relevant to people's everyday jobs.

The Kirkpatrick Model

Level 1: Reaction
The first level is learner-focused. It measures if the learners have found the training to be relevant to their role, engaging, and useful.
There are three parts to this:

1. Satisfaction: Is the learner happy with what they have learned during their training?

2. Engagement: How much did the learner get involved in and contribute to the learning experience?

3. Relevance: How much of this information will learners be able to apply on the job?

Reaction is generally measured with a survey, completed after the training has been delivered. This survey is often called a 'smile sheet' and it asks the learners to rate their experience within the training and offer feedback.

Some of the areas that the survey might focus on are:

Program objectives

Course materials

Content relevance

Facilitator knowledge

Tips for Implementing Level 1: Reaction

Use an online questionnaire.

Set aside time at the end of training for learners to fill out the survey.

Provide space for written answers, rather than multiple choice.

Pay attention to verbal responses given during training.

Create questions that focus on the learner's takeaways.

Use information from previous surveys to inform the questions that you ask.

Let learners know at the beginning of the session that they will be filling this out. This allows them to consider their answers throughout and give more detailed responses.

Reiterate the need for honesty in answers - you don't need learners giving polite responses rather than their true opinions!

Level 2: Learning
This level focuses on whether or not the learner has acquired the knowledge, skills, attitude, confidence, and commitment that the training program is focused on.
These 5 aspects can be measured either formally or informally.

For accuracy in results, pre and post-learning assessments should be used.

Tips for Implementing Level 2: Learning

Conduct assessments before and after for a more complete idea of how much was learned.

Questionnaires and surveys can be in a variety of formats, from exams, to interviews, to assessments.

In some cases, a control group can be helpful for comparing results.

The scoring process should be defined and clear and must be determined in advance in order to reduce inconsistencies.

Make sure that the assessment strategies are in line with the goals of the program.

Don't forget to include thoughts, observations, and critiques from both instructors and learners - there is a lot of valuable content there.

Level 3: Behavior

This step is crucial for understanding the true impact of the training.
It measures behavioral changes after learning and shows if the learners are taking what they learned in training and applying it as they do their job.
It also looks at the concept of required drivers. That is, "processes and systems that reinforce, encourage and reward the performance of critical behaviors on the job."

The results of this assessment will demonstrate not only if the learner has correctly understood the training, but it also will show if the training is applicable in that specific workplace.

This is because, often, when looking at behavior within the workplace, other issues are uncovered. If a person does not change their behavior after training, it does not necessarily mean that the training has failed.

It might simply mean that existing processes and conditions within the organization need to change before individuals can successfully bring in a new behavior.

Tips for Implementing Level 3: Behavior

The most effective time period for implementing this level is 3 - 6 months after the training is completed. Any evaluations done too soon will not provide reliable data.

Use a mix of observations and interviews to assess behavioral change.

Be aware that opinion-based observations should be minimized or avoided, so as not to bias the results.

To begin, use subtle evaluations and observations to evaluate change. Once the change is noticeable, more obvious evaluation tools, such as interviews or surveys, can be used.

Have a clear definition of what the desired change is - exactly what skills should be put into use by the learner? How is mastery of these skills demonstrated?

Other questions to keep in mind are the degree of change and how consistently the learner is implementing the new skills. Will this be a lasting change?

Evaluations are more successful when folded into present management and training methods.

Level 4: Results
This level focuses on whether or not the targeted outcomes resulted from the training program, alongside the support and accountability of organizational members.
For each organization, and indeed, each training program, these results will be different, but can be tracked using Key Performance Indicators. Some examples of common KPIs are increased sales, decreased workers comp claims, or a higher return on investments.

This level also includes looking at leading indicators. These are "short-term observations and measurements suggesting that critical behaviors are on track to create a positive impact on desired results."

Tips for Implementing Level 4: Results

Before starting this process, you should know exactly what is going to be measured throughout, and share that information with all participants.

If possible, use a control group.

Don't rush the final evaluation - it's important that you give participants enough time to effectively fold in the new skills.

It is key that observations are made properly, and that observers understand the training type and desired outcome.

You can ask participants for feedback, but this should be paired with observations for maximum efficacy.

Especially in the case of senior employees, yearly evaluations and consistent focus on key business targets are crucial to the accurate evaluation of training program results.

At all levels within the Kirkpatrick Model, you can clearly see results and measure areas of impact. This analysis gives organizations the ability to adjust the learning path when needed and to better understand the relationship between each level of training. The end result will be a stronger, more effective training program and better business results.

# data[].attributes.attachment_link.title
The Kirkpatrick Model of Training Evaluation: Guide

# data[].attributes.attachment_link.description
Updated This guide will introduce the Kirkpatrick Model and the benefits of using this model in your training program. After reading this guide, you will be able to effectively use it to evaluate training in your organization.

# data[].attributes.headline
How to use the Kirkpatrick Model in Evaluation

# data[].attributes.subject
How to use the Kirkpatrick Model in Evaluation

# data[].attributes.attachment_link.readable_text
LEVEL 2 - LEARNING

As the name says, level 2 is about assessing what the participant learned. I like to think: Knowledge, Skills, and Attitudes. Assessing learning can (and arguably should) touch on each of these. So, your questions might be:

I can explain our COVID-19 safety policies. [knowledge] I understand why our COVID-19 safety policies are important. [knowledge] I am confident that I can enforce our COVID-19 safety policies. [attitude]
I learned 3 ways to build buy-in about our COVID-19 safety policies. [skills]

As these examples imply, it's very common for these questions to be included on a post-workshop survey. I usually embed them with the Reaction survey, so participants fill out one survey after training. I try to keep the total question count under 20, usually Likert Scale, with some opportunity for qualitative feedback.

TIP: If it's important to you to be able to say that the training was the reason for the results you get, you'll want to consider a baseline survey - that is, a survey with the same learning questions as the post-training survey, but it's completed before training. That way, any change that you see can be more strongly linked to the training that they received, as opposed to pre-existing knowledge, skills, or attitudes.

If your training has learning objectives, this is a good place to look for what knowledge, skills or attitudes the program is intended to impact.

Personally, I've always lacked confidence in results that come from self-assessed ratings of knowledge, skills, and attitude. We know there are several biases in play - including a tendency to not use the full range of a scale, and to rate yourself positively. One way to mitigate these confounders is to test the learning. Instead of asking for an opinion about what they learned, the post-training survey could be formatted to actually test the learning:

List three COVID-19 safety protocols implemented at your workplace. Which of the following is a reason why these COVID-19 protocols were selected. [multiple choice]
Describe one way to build buy-in with your staff around COVID-19 policies.

The downside is that not only are these potentially more resource-intense to analyze and report on, but most programs and organizations are worried about the impression it gives to test participants or workshop attendees. We all hate tests, right!? I haven't had much luck convincing a program to use testing as opposed to a self-rated survey. Let me know if you've fared any better!

LEVEL 3 - BEHAVIOUR

Here is where it gets a bit tricky. To measure behaviour you need to have access to the participants after a set amount of time; your participants need an opportunity to put their learning into action.

Assuming you have email addresses or perhaps a reason to bring this group together again, I've assessed behaviour change through surveys (yes, surveys again). These surveys are usually sent over email after a pre-determined amount of time. The time interval depends on the learning you are hoping they achieved and the opportunities to implement it. I've used anywhere from 4 weeks to 6 months.

In the last [time interval] I have:

a) Explained why COVID-19 safety protocols are important.

b) Used techniques learned at training to deal with an individual not wishing to follow COVID-19 safety protocols.

It's possible that you could re-administer the baseline/post-training survey as a [time interval] follow-up. This would help you to assess retention of learning and may get at some behaviour change too. The difference though is that the assessment of learning was likely opinion-based or in-the-moment, while assessment of behaviour change is retrospective - it's not about what they have knowledge, attitude or skills to do, or what they intend to do, it's about what they did do.

Another option is to change your data collection strategy:

Observation: can you watch attendees implementing the training?

Interviews: can you gather data from people who didn't attend the training to understand what changes they have seen? Or interview the training attendees to understand what they've put into practice (and in what context)

Role playing: If you don't anticipate being able to reach the attendees after a time period, perhaps role playing (and observation) could be a part of the training curriculum. Can attendees demonstrate what they have learned?

Evidence of action: perhaps you can access evidence that proves the training resulted in action - maybe participants were asked to write business plans (how many were written?) or were asked to design and implement a communication strategy about COVID-19 policies (was it done?)

In our scenario, observation may look like observing building entrance processes and observing, counting or noting the number of times a COVID-19 policy is explained or enforced. Or, you could survey staff to see how well they understand the COVID-19 policies (assuming their knowledge is a result of the training to select staff). The key to Level 3 is the demonstration of the behaviour.

LEVEL 4: RESULTS

Results brings everything full circle. It drives at the purpose of the training to begin with - what is the impact of the training. In our scenario, the ultimate goal may have been to have staff and patrons compliant with COVID-19 safety protocols. Assessment of Results will be a reflection of this.

# of encounters with non-compliant staff # of encounters with non-compliant patrons
# of COVID-19 outbreaks at the workplace

Assessment of Results may start to blur the lines with the rest of your evaluation plan. Perhaps the training is part of a larger program that is designed to create a safe environment for staff and patrons, part of which was implementing COVID-19 policies.

Assessment of Results likely requires an even longer time frame than Level 3, Behaviour.

Step 3: Reporting

I don't think I've ever used the Kirkpatrick levels explicitly in my reporting. I think most audiences are not interested in the theory of evaluating a training program but are more interested in answering "did the training work?" As I mentioned, Kirkpatrick has usually been only a part of my evaluation planning, so, similarly, reporting on the effectiveness of training is usually only one part of my reporting.

Because Level 1 of Kirkpatrick assesses formative questions about training - things that you could change or adapt before running the training session again, I have often produced formative reports or briefs that summarize just Levels 1 and 2 of Kirkpatrick. This promotes utilization of evaluation results!

Using surveys (or a test!) likely gives you some quantitative data for you to employ your data viz skills on. But keep in mind that it's not necessary for you to report all of the data you gather. You likely don't need a graph showing your reader that the participants thought the room was the right temperature. Sometimes less is more and a couple of statements like "Participants found the training environment to be conducive to learning and found the training to be engaging. However, the days were long, and they recommended more breaks." can cover a lot of your Reaction results. We've got lots of resources to help you with your report design .

Sometimes reports, particularly interim reports, are due before you can get to behaviour change or results. This is actually one of the criticisms of Kirkpatrick - that many evaluations will cover Levels 1 and 2 thinking them sufficient but fail to invest the time and resources into ensuring the behaviour changes and outcomes are captured. This is one of the reasons that planning an evaluation concurrently with project design is helpful and can prevent these shortcuts.

# data[].attributes.attachment_link.title
How do I use the Kirkpatrick Model in Evaluation? - Eval Academy

# data[].attributes.attachment_link.description
Kirkpatrick is probably one of those names/methods you've heard about in your evaluation career, but have you ever used it? In this article, we explain how to use the Kirkpatrick model in your evaluation planning, implementation, and reporting.

# data[].attributes.attachment_link.readable_text
Did you think that it was successful?

What were the biggest strengths and weaknesses of the training?

Did you like the venue and presentation style?

Did the training session accommodate your personal needs?

Were the training activities engaging?

What are the three most important things that you learned from this training?

From what you learned, what do you plan to apply in your job?

What support might you need to apply what you learned?

Identify how you want to measure people's reactions. Many people use employee satisfaction surveys to do this, but you can also watch trainees' body language during the session, or ask for verbal feedback.

Analyze the feedback, and consider the changes that you could make in response.

Kirkpatrick's Level 2: Learning

Level 2 focuses on measuring what your trainees have and haven't learned. In the New World version of the tool, Level 2 also measures what they think they'll be able to do differently as a result, how confident they are that they can do it, and how motivated they are to make changes.

This demonstrates how training has developed their skills, attitudes and knowledge, as well as their confidence and commitment.

To measure how much your trainees have learned, start by identifying what you want to evaluate. Training sessions should have specific learning objectives, so make those your starting point.

You can measure learning in different ways, depending on the objectives. But it's helpful to measure these areas both before and after training.

Before the training begins, test your trainees to determine their knowledge, skill levels and attitudes. Then, when the training is finished, test your trainees a second time to measure what they have learned, or measure their learning with interviews or verbal assessments.

Kirkpatrick's Level 3: Behavior

This level helps you to understand how well people apply their training. It can also reveal where people might need help. But behavior can only change when conditions are favorable.

Imagine that you're assessing your team members after a training session. You can see little change, and you conclude that they learned nothing, and that the training was ineffective.

It's possible, however, that they actually learned a lot, but that the organizational or team culture obstructs behavioral change. Perhaps existing processes mean that there's little scope to apply new thinking, for example.

As a result, your people don't feel confident in applying new knowledge, or see few opportunities to do so. Or, they may not have had enough time to put it into practice.

Be sure to develop processes that encourage, reinforce and reward positive changes in behavior. The New World Kirkpatrick Model calls these processes "required drivers." If a team member uses a new skill effectively, highlight this and praise him or her for it.

Effectively measuring behavior is a longer-term process that should take place over weeks or months following the initial training.
Questions to ask include:

Did the trainees put any of their learning to use?

Are trainees able to teach their new knowledge, skills or attitudes to other people?

Are trainees aware that they've changed their behavior?

One of the best ways to measure behavior is to conduct observations and interviews. Another is to integrate the use of new skills into the tasks that you set your team, so that people have the chance to demonstrate what they know.

Kirkpatrick's Level 4: Results

At this level, you analyze the final results of your training. This includes outcomes that you or your organization have decided are good for business and good for your team members, and which demonstrate a good return on investment (ROI). (Some adapted versions of the model actually have a Level 5, dedicated to working out ROI.)

Level 4 will likely be the most costly and time-consuming. Your biggest challenge will be to identify which outcomes, benefits, or final results are most closely linked to the training, and to come up with an effective way to measure these outcomes in the long term.

Modern trainers often use the Kirkpatrick model backward, by first stating the results that they want to see, and then developing the training that is most likely to deliver them. This helps to prioritize the goals of the training and make it more effective.
Here are some outcomes to consider, depending on the objectives of your training:

Increased employee retention.

Increased production.

Higher morale.

Reduced waste.

Increased sales.

Higher quality ratings.

Increased customer satisfaction.

Fewer staff complaints.

Make a series of short-term observations and measurements to check that changes in behavior due to training are making a worthwhile difference to your team's performance. The New World Kirkpatrick Model calls these "leading indicators."

Reprinted with permission of Berrett-Koehler Publishers, Inc., San Francisco, CA. From Evaluating Training Programs, © 1996 by Donald L.Kirkpatrick & James D Kirkpatrick. All rights reserved. www.bkconnection.com. Terms reproduced from The New World Kirkpatrick Model with permission from Kirkpatrick Partners.

Limitations of Kirkpatrick's Model

Kirkpatrick's model remains popular, but it should be used with care. The basic structure is now more than 60 years old (despite its many updates), and the ways that people learn and organizations operate has changed radically in this time. Even the term "training" has been largely replaced by "learning and development."

Today, other, non-formal methods of workplace training are often more popular and effective (as shown by the 70:20:10 model). And, with the rise of personalized, user-directed learning, formal training is becoming less prominent. Kirkpatrick's model is not necessarily suited to this new approach to learning.

Another drawback is that Levels 3 and 4, which arguably yield the most useful information for the business, are time-consuming, resource-intensive, and expensive to implement. So the model may not be practical for all organizations, especially if you don't have a dedicated training or HR department to conduct the analysis. And, it's not ideal for all situations, such as one-off training.

Most importantly, organizations change in many ways, and these changes affect behaviors and results, as well as training. For example, measurable improvements in retention and productivity could result from the arrival of a new boss, or from a new computer system, rather than training. Or it could be a combination of these.

Kirkpatrick's model is great for evaluating training in a "scientific" way, but with so many possible variables, Level 4 may be limited in its usefulness.

# data[].attributes.attachment_link.title
Kirkpatrick's Model - Analyzing Learning Effectiveness

# data[].attributes.attachment_link.description
The Kirkpatrick Four-Level Training Evaluation Model uses Reaction, Learning, Behavior, and Results to measure the effectiveness of learning programs.

# data[].attributes.headline
The Kirpatrick Model of Training Evaluation 

# data[].attributes.subject
The Kirpatrick Model of Training Evaluation 

# data[].attributes.attachment_link.readable_text
The Kirkpatrick Model of Evaluation, first developed by Donald Kirkpatrick in 1959, is the most popular model for evaluating the effectiveness of a training program. The model includes four levels of evaluation, and as such, is sometimes referred to as 'Kirkpatrick's levels" or the "four levels."

This article explores each level of Kirkpatrick's model and includes real-world examples so that you can see how the model is applied.

If at any point you have questions or would like to discuss the model with practitioners, then feel free to join my eLearning + instructional design Slack channel and ask away.

What is the Kirkpatrick Model of Evaluation?

The Kirkpatrick Model of Evaluation is a popular approach to evaluating training programs. However, despite the model focusing on training programs specifically, it's broad enough to encompass any type of program evaluation.

For all practical purposes, though, training practitioners use the model to evaluate training programs and instructional design initiatives. It covers four distinct levels of evaluation:

Level 1: Reaction

Level 2: Learning

Level 3: Behavior

Level 4: Results

As you move from levels 1 through 4, the evaluation techniques become increasingly complex and the data generated becomes increasingly valuable.

Due to this increasing complexity as you get to levels 3 and 4 in the Kirkpatrick model, many training professionals and departments confine their evaluation efforts to levels 1 and 2. This leaves the most valuable data off of the table, which can derail many well intended evaluation efforts.

Finally, if you are a training professional, you may want to memorize each level of the model and what it entails; many practitioners will refer to evaluation activities by their level in the Kirkpatrick model.

If you're in the position where you need to evaluate a training program, you should also familiarize yourself with the techniques that we'll discuss throughout the article.

Kirkpatrick's Four Levels of Training Evaluation

Now it's time to dive into the specifics of each level in the Kirkpatrick Model.

We move from level 1 to level 4 in this section, but it's important to note that these levels should be considered in reverse as you're developing your evaluation strategy. We address this further in the 'How to Use the Kirkpatrick Model' section.

Level 1: Reaction

Reaction data captures the participants' reaction to the training experience. Specifically, it refers to how satisfying, engaging, and relevant they find the experience.

This is the most common type of evaluation that departments carry out today. Training practitioners often hand out 'smile sheets' (or 'happy sheets') to participants at the end of a workshop or eLearning experience. Participants rate, on a scale of 1-5, how satisfying, relevant, and engaging they found the experience.

Level 1 data tells you how the participants feel about the experience, but this data is the least useful for maximizing the impact of the training program.

The purpose of corporate training is to improve employee performance, so while an indication that employees are enjoying the training experience may be nice, it does not tell us whether or not we are achieving our performance goal or helping the business.

With that being said, efforts to create a satisfying, enjoyable, and relevant training experience are worthwhile, but this level of evaluation strategy requires the least amount of time and budget. The bulk of the effort should be devoted to levels 2, 3, and 4.

Kirkpatrick Level 1 Evaluation Techniques

As discussed above, the most common way to conduct level 1 evaluation is to administer a short survey at the conclusion of a training experience. If it's an in-person experience, then this may be conducted via a paper handout, a short interview with the facilitator, or an online survey via an email follow-up.

If the training experience is online, then you can deliver the survey via email, build it directly into the eLearning experience, or create the survey in the Learning Management System (LMS) itself.

Common survey tools for training evaluation are Questionmark and SurveyMonkey.

Kirkpatrick Level 1 Evaluation Examples

Let's consider two real-life scenarios where evaluation would be necessary:

A large technical support call center rolled out new screen sharing software for agents to use with the customers. They're providing training to teach the agents how to use the new software.

An industrial coffee roastery company sells its roasters to regional roasteries, and they offer follow-up training on how to properly use and clean the machines.

In the call center example, imagine a facilitator hosting a one-hour webinar that teaches the agents when to use screen sharing, how to initiate a screen sharing session, and how to explain the legal disclaimers. They split the group into breakout sessions at the end to practice.

At the conclusion of the experience, participants are given an online survey and asked to rate, on a scale of 1 to 5, how relevant they found the training to their jobs, how engaging they found the training, and how satisfied they are with what they learned. There's also a question or two about whether they would recommend the training to a colleague and whether they're confident that they can use screen sharing on calls with live customers.

In the coffee roasting example, imagine a facilitator delivering a live workshop on-site at a regional coffee roastery. He teaches the staff how to clean the machine, showing each step of the cleaning process and providing hands-on practice opportunities.

Once the workshop is complete and the facilitator leaves, the manager at the roastery asks his employees how satisfied they were with the training, whether they were engaged, and whether they're confident that they can apply what they learned to their jobs. He records some of the responses and follows up with the facilitator to provide feedback.

In both of these examples, efforts are made to collect data about how the participants initially react to the training event; this data can be used to make decisions about how to best deliver the training, but it is the least valuable data when it comes to making important decisions about how to revise the training.

For example, if you find that the call center agents do not find the screen sharing training relevant to their jobs, you would want to ask additional questions to determine why this is the case. Addressing concerns such as this in the training experience itself may provide a much better experience to the participants.

Level 2: Learning

Learning data tells us whether or not the people who take the training have learned anything. Specifically, it helps you answer the question: "Did the training program help participants learn the desired knowledge, skills, or attitudes?".

Level-two evaluation is an integral part of most training experiences. Assessment is a cornerstone of training design: think multiple choice quizzes and final exams.

This data is often used to make a decision about whether or not the participant should receive credit for the course; for example, many eLearning assessments require the person taking it to score an 80% or above to receive credit, and many licensing programs have a final test that you are required to pass.

Finally, while not always practical or cost-efficient, pre-tests are the best way to establish a baseline for your training participants. When you assess people's knowledge and skills both before and after a training experience, you are able to see much more clearly which improvements were due to the training experience.

Kirkpatrick Level 2 Evaluation Techniques

While written or computer-based assessments are the most common approach to collecting learning data, you can also measure learning by conducting interviews or observation.

For example, if you are teaching new drivers how to change a tire, you can measure learning by asking them to change a tire in front of you; if they are able to do so successfully, then that speaks to the success of the program; if they are not able to change the tire, then you may ask follow-up questions to uncover roadblocks and improve your training program as needed.

However, if you are measuring knowledge or a cognitive skill, then a multiple choice quiz or written assessment may be sufficient. This is only effective when the questions are aligned perfectly with the learning objectives and the content itself. If the questions are faulty, then the data generated from them may cause you to make unnecessary or counter-intuitive changes to the program.

Kirkpatrick Level 2 Evaluation Examples

Carrying the examples from the previous section forward, let's consider what level 2 evaluation would look like for each of them.

For the screen sharing example, imagine a role play practice activity. Groups are in their breakout rooms and a facilitator is observing to conduct level 2 evaluation. He wants to determine if groups are following the screen-sharing process correctly.

A more formal level 2 evaluation may consist of each participant following up with their supervisor; the supervisor asks them to correctly demonstrate the screen sharing process and then proceeds to role play as a customer. This would measure whether the agents have the necessary skills.

The trainers may also deliver a formal, 10-question multiple choice assessment to measure the knowledge associated with the new screen sharing process. They may even require that the agents score an 80% on this quiz to receive their screen sharing certification, and the agents are not allowed to screen share with customers until passing this assessment successfully.

In the industrial coffee roasting example, a strong level 2 assessment would be to ask each participant to properly clean the machine while being observed by the facilitator or a supervisor. Again, a written assessment can be used to assess the knowledge or cognitive skills, but physical skills are best measured via observation.

Level 3: Behavior

As we move into Kirkpatrick's third level of evaluation, we move into the high-value evaluation data that helps us make informed improvements to the training program.

Level 3 evaluation data tells us whether or not people are behaving differently on the job as a consequence of the training program. Since the purpose of corporate training is to improve performance and produce measurable results for a business, this is the first level where we are seeing whether or not our training efforts are successful.

While this data is valuable, it is also more difficult to collect than that in the first two levels of the model. On-the-job measures are necessary for determining whether or not behavior has changed as a result of the training.

Kirkpatrick Level 3 Evaluation Techniques

Reviewing performance metrics, observing employees directly, and conducting performance reviews are the most common ways to determine whether on-the-job performance has improved.

As far as metrics are concerned, it's best to use a metric that's already being tracked automatically (for example, customer satisfaction rating, sales numbers, etc.). If no relevant metrics are being tracked, then it may be worth the effort to institute software or a system that can track them.

However, if no metrics are being tracked and there is no budget available to do so, supervisor reviews or annual performance reports may be used to measure the on-the-job performance changes that result from a training experience.

Since these reviews are usually general in nature and only conducted a handful of times per year, they are not particularly effective at measuring on-the-job behavior change as a result of a specific training intervention. Therefore, intentional observation tied to the desired results of the training program should be conducted in these cases to adequately measure performance improvement.

Therefore, when level 3 evaluation is given proper consideration, the approach may include regular on-the-job observation, review of relevant metrics, and performance review data.

Kirkpatrick Level 3 Evaluation Examples

Bringing our previous examples into a level 3 evaluation, let's begin with the call center. With the roll-out of the new system, the software developers integrated the screen sharing software with the performance management software; this tracks whether a screen sharing session was initiated on each call.

Now, after taking the screen sharing training and passing the final test, call center agents begin initiating screen sharing sessions with customers. Every time this is done, a record is available for the supervisor to review.

On-the-job behavior change can now be viewed as a simple metric: the percentage of calls that an agent initiates a screen sharing session on. If this percentage is high for the participants who completed the training, then training designers can judge the success of their initiative accordingly. If the percentage is low, then follow-up conversations can be had to identify difficulties and modify the training program as needed.

In the coffee roasting example, the training provider is most interested in whether or not their workshop on how to clean the machines is effective. Supervisors at the coffee roasteries check the machines every day to determine how clean they are, and they send weekly reports to the training providers.

When the machines are not clean, the supervisors follow up with the staff members who were supposed to clean them; this identifies potential road blocks and helps the training providers better address them during the training experience.

Level 4: Results

Level 4 data is the most valuable data covered by the Kirkpatrick model; it measures how the training program contributes to the success of the organization as a whole. This refers to the organizational results themselves, such as sales, customer satisfaction ratings, and even return on investment (ROI). (In some spinoffs of the Kirkpatrick model, ROI is included as a fifth level, but there is no reason why level 4 cannot include this organizational result as well).

Many training practitioners skip level 4 evaluation. Organizations do not devote the time or budget necessary to measure these results, and as a consequence, decisions about training design and delivery are made without all of the information necessary to know whether it's a good investment.

By devoting the necessary time and energy to a level 4 evaluation, you can make informed decisions about whether the training budget is working for or against the organization you support.

Kirkpatrick Level 4 Evaluation Techniques

Similar to level 3 evaluation, metrics play an important part in level 4, too. At this level, however, you want to look at metrics that are important to the organization as a whole (such as sales numbers, customer satisfaction rating, and turnover rate).

If you find that people who complete a training initiative produce better metrics more than their peers who have not completed the training, then you can draw powerful conclusions about the initiative's success.

A great way to generate valuable data at this level is to work with a control group. Take two groups who have as many factors in common as possible, then put one group through the training experience. Watch how the data generated by each group compares; use this to improve the training experience in a way that will be meaningful to the business.

Again, level 4 evaluation is the most demanding and complex - using control groups is expensive and not always feasible. There are also many ways to measure ROI, and the best models will still require a high degree of effort without a high degree of certainty (depending on the situation).

Despite this complexity, level 4 data is by far the most valuable. This level of data tells you whether your training initiatives are doing anything for the business. If the training initiatives are contributing to measurable results, then the value produced by the efforts will be clear. If they are not, then the business may be better off without the training.

Kirkpatrick Level 4 Evaluation Examples

In our call center example, the primary metric the training evaluators look to is customer satisfaction rating. They decided to focus on this screen sharing initiative because they wanted to provide a better customer experience.

If they see that the customer satisfaction rating is higher on calls with agents who have successfully passed the screen sharing training, then they may draw conclusions about how the training program contributes to the organization's success.

For the coffee roastery example, managers at the regional roasteries are keeping a close eye on their yields from the new machines. When the machines are clean, less coffee beans are burnt.

As managers see higher yields from the roast masters who have completed the training, they can draw conclusions about the return that the training is producing for their business.

How to Use the Kirkpatrick Model

Now that we've explored each level of the Kirkpatrick's model and carried through a couple of examples, we can take a big-picture approach to a training evaluation need.

Consider this: a large telecommunications company is rolling out a new product nationwide. They want to ensure that their sales teams can speak to the product's features and match them to customer's needs - key tasks associated with selling the product effectively.

An average instructional designer may jump directly into designing and developing a training program. However, one who is well-versed in training evaluation and accountable for the initiative's success would take a step back.

From the outset of an initiative like this, it is worthwhile to consider training evaluation. Always start at level 4: what organizational results are we trying to produce with this initiative?

In this example, the organization is likely trying to drive sales. They have a new product and they want to sell it. Let's say that they have a specific sales goal: sell 800,000 units of this product within the first year of its launch.

Now the training team or department knows what to hold itself accountable to.

From there, we consider level 3. What on-the-job behaviors do sales representatives need to demonstrate in order to contribute to the sales goals? Working with a subject matter expert (SME) and key business stakeholders, we identify a list of behaviors that representatives would need to exhibit.

Now we move down to level 2. What knowledge and skills do employees need to learn to ensure that they can perform as desired on-the-job? We can assess their current knowledge and skill using surveys and pre-tests, and then we can work with our SMEs to narrow down the learning objectives even further.

Finally, we consider level 1. How should we design and deliver this training to ensure that the participants enjoy it, find it relevant to their jobs, and feel confident once the training is complete?

You can also identify the evaluation techniques that you will use at each level during this planning phase. You can map exactly how you will evaluate the program's success before doing any design or development, and doing so will help you stay focused and accountable on the highest-level goals.

Kirkpatrick Model of Evaluation Wrap-up

When it comes down to it, Kirkpatrick helps us do two things: understand our people and understand our business. What do our employees want? What are their anxieties? What's holding them back from performing as well as they could?

As far as the business is concerned, Kirkpatrick's model helps us identify how training efforts are contributing to the business's success. This is an imperative and too-often overlooked part of training design. If the training initiatives do not help the business, then there may not be sufficient reason for them to exist in the first place.

If you'd like to discuss evaluation strategy further or dive deeper into Kirkpatrick's model with other practitioners, then feel free to join the ID community.
View the Full Guide to Become an Instructional Designer.

# data[].attributes.attachment_link.title
The Kirkpatrick Model of Training Evaluation (with Examples) | Devlin Peck

# data[].attributes.attachment_link.description
This article explores the Kirkpatrick model of training evaluation, which is the most well-known and widely-implemented evaluation model in the field of instructional design. Plenty of real-world examples included!

# data[].attributes.headline
Your Complete Guide to the Kirpatrick Evaluation Model

# data[].attributes.subject
Your Complete Guide to the Kirpatrick Evaluation Model

# data[].attributes.attachment_link.readable_text
Analyzing the results of training programs is extremely difficult. L&D teams are challenged to demonstrate the value that programs deliver to both individual employees and the organization as a whole. Unfortunately, few learning leaders are able to do this.

"92% of senior leaders and stakeholders want to see quantifiable results from their training functions. Yet only 9% of internal L&D organizations show them anything," notes Paul Leone, Ph.D., an industrial and organizational psychologist and training measurement expert.

So then, how should L&D teams go about going beyond the smile sheet to measure ROI?

Enter the Kirkpatrick Evaluation Model, perhaps the best-known model of evaluating the results of training programs. The Kirkpatrick Evaluation Model is widely used because it measures training outcomes at multiple levels. So, organizations can use the model according to their own definition of the word "evaluation." What might be important to one team in one organization might not be to another.

Are you showing your senior leadership team how much ROI your training initiatives are delivering to the bottom line?
Watch this on-demand webinar, with author and learning measurement expert Paul Leone, Ph.D., to create your "story of impact" and show robust results to your business partners.

The Kirkpatrick Model's Four Levels of Training Evaluation

The Kirkpatrick Evaluation Model was created by Donald Kirkpatrick and measures training through four levels, each of increasing depth or complexity:

Level 1 - Reaction

This basic level asks learners about how they felt about the training. Questions might be as simple as:

Did you enjoy it?

Did you feel it was valuable?

Did you feel that the instructor did a good job?

These questions are usually delivered via a simple post-course questionnaire. While self-reported answers from learners might seem superficial, there is still value in providing Level 1 evaluation.

Level 1 can be helpful for instructional design teams to broadly understand whether the course resonated with learners. For example, overwhelmingly negative or even neutral reviews are a clear review that things could have been better.

Level 2 - Learning

This next level of Kirkpatrick seeks to help L&D managers determine whether learning occurred or any knowledge/skills were acquired. It gives a chance for learners to apply what they learned.

To do this, instructional design teams will usually deliver several assessments. There can be the expected one at the end of the training, but this doesn't help managers understand how much they learned.

For a deeper look at how much knowledge or skills were gained, the L&D team can give multiple assessments. For example, one assessment at the beginning and then one similar one at the end of the course-or even months after the training has passed. This insight helps to determine how much the learner already knew and how useful or impactful the training really was.

Level 3 - Behavior

Kirkpatrick's Level 3 goes even deeper than Level 2 measurement. It seeks to determine just how much of the training is impacting the employee's day-to-day work. It also looks at whether the training is relevant to their jobs.

This type of behavior measurement requires quite a big leap from Level 2 of the model. It does not rely simply on answers to post-training assessments. To carry out Level 3, learning leaders will need to garner much more feedback, including that from co-workers, managers, and others who work regularly alongside the learner.

If the training course has had the desired effect, it will be noticeable-both qualitatively and quantitatively-to everyone involved. Additionally, a lack of feedback is also itself feedback. Subtle clues can help L&D better understand whether the training had any influence on the employee's work.

Level 4 - Results

The highest level of the Kirkpatrick Evaluation Model has L&D determine whether the training results in a positive impact on the overall organization and key business performance indicators.

For some departments or domains, the desired results might be clear: sales training led to more sales, which affected revenue. Or, stronger customer care training led to fewer returns or chargebacks.

Beyond scenarios like these, it can be challenging for L&D to determine the organizational impact of training. For example, measuring the impact of an empathy or leadership training course. To measure results and the impact of these types of training, L&D need to work with managers to determine metrics that can serve as the signals of positive employee performance.

You and your team put a lot of time and effort into creating training. Prove your value to the rest of the organization by using this free Training ROI Calculator.

How do you use the Kirkpatrick Evaluation Model?

The beauty of the Kirkpatrick Model is that any level can be used anytime in any learning setting. Level 1 might be used immediately just to garner immediate learner feedback. But Levels 2 and 3 might not be implemented for 1-2 months after the training has passed.

Kirkpatrick also helps teams better understand where instructional design needs to improve. For example, say that in Levels 1 and 2 there is consistent feedback that the courses did not meet expectations or assessments showed poor retention. This demonstrates an issue not be with learners, but with the courses or course design.

Is the Kirkpatrick Evaluation Model the best way to measure learning effectiveness?

Measuring the impact of your training not only earns you more respect and partnership from business leaders. But you will also receive more recognition and rewards for your hard work-including more budget.

This budget could be used for hiring more designers and developers or outside subject matter experts. These extra resources can build courses based on training requests that you, unfortunately, had to turn down previously.

For some organizations, the Kirkpatrick Model doesn't go high enough. Level 4 might seem the highest, as it seeks to measure the impact to the organization as a whole. However, it might still be too vague for some organizations.

As a result, some L&D teams have added two additional levels: Level 5 and Level 6.

Level 5 - Return on Investment

In some organizations, Level 4 is pushed to Level 5, a Return on Investment (ROI). This measures any financial impact the training initiative has on the organization, such as increased revenues and lowered costs.

So Level 4 should measure, as a whole, the positive performance and behaviors studied in Level 3.

Dr. Leone has Level 4 include a control group and attribution technique. L&D should track both trained and untrained participants, in order to measure learning impact by groups. In this way, an organization can measure the attribution of training to large groups of employees.

Beyond this level, an organization can go deeper to seek the financial impact, which would comprise Level 5.

Level 6 - Maximizing Impact

At this level, Dr. Leone suggests that organizations focus on the factors that are influencing the impact of a training program. Many factors beyond obvious training outcomes can easily influence the results at each level of impact. Without understanding and addressing these factors, the organization will never improve, no matter the quality of the training.

For instance, Dr. Leone suggests examining which "climate factors" will influence the impact and ROI of a training initiative. For example, superior manager support may lead to behavioral improvements (Level 3), greater business impact (Level 4), and a higher ROI (Level 5).

Still struggling to measure L&D's impact?

While tools to collaborate more effectively and scale course development more rapidly are important for efficiency, the ability to measure whether those courses made an impact is critical for the L&D function to thrive.

If you're looking for ways to tell the story of your L&D team's impact back to stakeholders in the organization, check out how Cognota provides operational insights from your L&D processes such as training intake, capacity and project planning, and course creation. See for yourself with a 14 day free trial!

A well planned and allocated training budget enables your L&D department to operate as efficiently as possible. Download this free eBook, "How to Effectively Calculate and Allocate Your Training Budget."

# data[].attributes.attachment_link.title
Your Complete Guide to the Kirkpatrick Evaluation Model - Cognota

# data[].attributes.attachment_link.description
Curious about the Kirkpatrick Evaluation Model? This guide will explain it all for you so you can better measure your L&D efforts.

# data[].attributes.headline
The Kirpatrick Model: a Comprehensive Guide to Measuring Learning Effectiveness in Organizations

# data[].attributes.subject
The Kirpatrick Model: a Comprehensive Guide to Measuring Learning Effectiveness in Organizations

# data[].attributes.attachment_link.readable_text
Training and development are essential components of any organization's success. The ability to provide employees with the knowledge and skills necessary to perform their roles is critical to achieving business objectives. However, it can be challenging to measure the effectiveness of training programs. This is where the Kirkpatrick Training Model comes in.

Developed by Donald Kirkpatrick in the 1950s, the model has become one of the most widely used methods for evaluating the impact of training programs. In this article, we'll take a deep dive into the Kirkpatrick Model and explore its history, constituents, and importance. This is a part of our comprehensive coverage of 13 Learning and Development Models. So if you are a L&D Manager or an Instructional Designer, please read on

History of the Kirkpatrick Model

Donald Kirkpatrick, an American educator, and consultant, first introduced his training evaluation model in a series of articles in the 1950s. He later expanded on the model in his 1975 book, "Evaluating Training Programs." Kirkpatrick's model is based on the premise that there are four levels at which training programs can be evaluated.

These levels are:

Level 1: Reaction - Measures participants' immediate response to the training, such as their satisfaction with the training experience.

Level 2: Learning - Measures the extent to which participants have acquired the knowledge, skills, and attitudes taught in the training.

Level 3: Behavior - Measures the extent to which participants apply what they learned in the training on the job.

Level 4: Results - Measures the impact of the training on the organization's performance, such as increased productivity or decreased costs.

The Four Stages of the Kirkpatrick Model

To understand the Kirkpatrick Model, it's important to break down the four stages. Let's explore each of them in detail:

The Kirkpatrick Model is based on four levels of evaluation, each with its own set of criteria and methods for measuring learning effectiveness. These four levels are:
Level 1: Reaction
The first level of the Kirkpatrick Model measures participants' reactions to the training program. This level focuses on participants' attitudes and opinions about the training program and how they perceive the program's value. The evaluation methods used at this level include surveys, questionnaires, and interviews.
Level 2: Learning
The second level of the Kirkpatrick Model measures the extent to which participants have learned the material covered in the training program. This level focuses on the knowledge, skills, and attitudes that participants have gained from the training program. The evaluation methods used at this level include tests, quizzes, and assessments.
Level 3: Behavior
The third level of the Kirkpatrick Model measures the extent to which participants have changed their behavior as a result of the training program. This level focuses on the application of knowledge and skills learned in the training program to the participants' work or personal lives. The evaluation methods used at this level include observation, self-reporting, and feedback from managers or peers.
Level 4: Results
The fourth level of the Kirkpatrick Model measures the impact of the training program on the organization as a whole. This level focuses on the tangible results that the organization has achieved as a result of the training program, such as increased productivity, improved quality, and reduced costs. The evaluation methods used at this level include performance metrics, financial data, and customer satisfaction surveys.

Implementing the Kirkpatrick Model in Your Organization

To implement the Kirkpatrick Model in your organization, follow these five steps:
Step 1: Identify the Learning Objectives
The first step in implementing the Kirkpatrick Model is to identify the learning objectives for your training program. These objectives should be specific, measurable, achievable, relevant, and time-bound (SMART)
Step 2: Develop Evaluation Methods
Once you have identified the learning objectives, the next step is to develop evaluation methods for each level of the Kirkpatrick Model. These methods should be aligned with the learning objectives and should be designed to measure the effectiveness of the training program.
Step 3: Collect Data
After you have developed your evaluation methods, the next step is to collect data from your participants. This data can be collected through surveys, tests, observations, and feedback from managers or peers.
Step 4: Analyze the Data
Once you have collected your data, the next step is to analyze it to determine the effectiveness of your training program. This analysis should be done at each level of the Kirkpatrick Model, and the results should be used to identify areas for improvement.
Step 5: Report Results
Finally, once you have analyzed your data, the last step is to report your results to stakeholders in your organization. This report should include a summary of your findings, recommendations for improvement, and an action plan for implementing those recommendations.

Why is the Kirkpatrick Model used in training?

The Kirkpatrick Model is used in training because it provides a systematic approach to evaluating the effectiveness of training programs. By breaking down the evaluation process into four stages, the model enables organizations to measure the impact of training on participants, their behavior, and the organization's performance. The model is also flexible, allowing organizations to tailor the evaluation process to their specific needs.

Kirkpatrick Model Example

To illustrate how the Kirkpatrick Model works in practice, let's consider an example. Suppose an organization conducts a training program to improve the customer service skills of its call center employees. The organization can use the Kirkpatrick Model to evaluate the effectiveness of the training as follows:

Levels

Level 1: Reaction - The organization can collect feedback from the participants about their satisfaction with the training program. This feedback can be collected through surveys, questionnaires, or interviews. For example, the organization may ask participants to rate the quality of the training materials or the expertise of the trainers.

Level 2: Learning - The organization can measure the extent to which participants have acquired the knowledge, skills, and attitudes taught in the training program. This can be done through pre-and post-training assessments or tests. For example, the organization may ask participants to take a knowledge test before and after the training program to measure the improvement in their understanding of customer service.

Level 3: Behavior - The organization can observe the participants on the job and collect data on their performance. This can be done through employee feedback or performance metrics. For example, the organization may ask customers to rate the quality of service they received from the call center employees before and after the training program to measure the improvement in customer satisfaction.

Level 4: Results - The organization can measure the impact of the training program on the organization's performance. This can be done through key performance indicators (KPIs) such as customer satisfaction, call resolution rates, or sales figures. For example, the organization may compare customer satisfaction scores or sales figures before and after the training program to measure the impact of the training on the organization's performance.

Why is Kirkpatrick Model Important?

The Kirkpatrick model is an essential tool for evaluating the effectiveness of training programs in organizations. It provides a framework for measuring the effectiveness of training programs, which is essential for ensuring that the organization is getting a return on investment for its training efforts.

The Kirkpatrick model helps organizations identify the strengths and weaknesses of their training programs, enabling them to make data-driven decisions on how to improve their training programs. It also provides a common language for discussing training effectiveness, which can improve communication and collaboration between the different stakeholders involved in the training program.

Limitations of the Kirkpatrick Model

While the Kirkpatrick Model is widely used and respected, it does have some limitations. One of the main limitations is that it is focused on measuring the effectiveness of training programs, rather than the effectiveness of the overall learning and development strategy of an organization.

Another limitation of the Kirkpatrick Model is that it can be time-consuming and expensive to implement, particularly at the higher levels of evaluation. This can be a barrier for some organizations, particularly those with limited resources.

Why is the Kirkpatrick Model Important?

The Kirkpatrick Model is important because it provides a structured approach to evaluating the effectiveness of training programs. By breaking down the evaluation process into four stages, the model enables organizations to assess the impact of training on multiple levels, including participants, behavior, and organizational performance. This information can be used to make data-driven decisions about the effectiveness of training programs and to improve future training initiatives.

Best Practices for Using the Kirkpatrick Model

To get the most out of the Kirkpatrick Model, organizations should follow these best practices:

1. Align training programs with learning objectives.

2. Develop evaluation methods that are aligned with learning objectives and are appropriate for each level of the Kirkpatrick Model.

3. Collect data from multiple sources, including participants, managers, and peers.

4. Analyze data at each level of the Kirkpatrick Model to identify areas for improvement.

5. Report results to stakeholders in a clear and concise manner.

Is the Kirkpatrick Model the Best Way to Measure Learning Effectiveness?

While the Kirkpatrick model is widely used and recognized as a valuable tool for evaluating training programs, it is not the only way to measure learning effectiveness. There are several other models, such as the Phillips model and the Brinkerhoff model, which can be used to evaluate the effectiveness of training programs.

Organizations should choose the evaluation model that best suits their needs and objectives. The Kirkpatrick model is particularly effective for evaluating the effectiveness of training programs in terms of behavior change and business impact. However, other models may be more appropriate for evaluating other aspects of learning effectiveness, such as engagement or knowledge retention.

What are the Limitations of the Kirkpatrick Model?

Despite its widespread use, the Kirkpatrick model has some limitations that should be considered. One limitation is that it is a linear model, which assumes that the training program is a sequential process that moves from one level to the next. In reality, training programs can be more complex, with different levels of evaluation occurring simultaneously.

Another limitation is that the Kirkpatrick model focuses on the evaluation of the training program itself, rather than the broader context in which it operates. It does not take into account external factors, such as changes in the market or organizational culture, which may impact the effectiveness of the training program.

Finally, the Kirkpatrick model can be time-consuming and expensive to implement, particularly at the results level, which requires the use of business metrics to evaluate the training program's effectiveness.

Can the Kirkpatrick Model be used to evaluate non-training-related programs or initiatives?

Yes, the Kirkpatrick Model can be adapted and used to evaluate non-training-related programs or initiatives. While the model was originally designed for evaluating training programs, the principles of the model can be applied to other types of programs or initiatives.

For example, the four levels of evaluation can be adapted to evaluate the effectiveness of an organizational change initiative:

Level 1: Reaction - Participants' reactions to the change initiative

Level 2: Learning - The extent to which participants have learned about the change initiative

Level 3: Behavior - The extent to which participants have changed their behavior as a result of the change initiative

Level 4: Results - The impact of the change initiative on the organization

By adopting the Kirkpatrick Model to evaluate non-training-related programs or initiatives, organizations can ensure that they are using a comprehensive framework for evaluating their programs and initiatives, and can make data-driven decisions about how to improve them.

Conclusion

The Kirkpatrick model is a valuable tool for evaluating the effectiveness of training programs in organizations. It provides a framework for measuring the impact of training programs on behavior change and business outcomes. However, it is not the only way to measure learning effectiveness, and organizations should choose the evaluation model that best suits their needs and objectives.

Despite its limitations, the Kirkpatrick model remains a popular and effective tool for evaluating the effectiveness of training programs. By using the model to evaluate their training programs, organizations can make data-driven decisions on how to improve their training programs and ensure

# data[].attributes.attachment_link.title
The Kirkpatrick Model: A Comprehensive Guide to Measuring Learning Effectiveness in Organizations - SimpliMBA

# data[].attributes.attachment_link.description
Are you looking to revolutionize your training and development programs? Look no further than the Kirkpatrick Model! Discover the industry's leading method for evaluating program effectiveness and improving ROI. Boost your career and transform your organization with the Kirkpatrick Model. Learn more now!

# data[].attributes.attachment_link.readable_text
The fields of Instructional Design (ID) and Learning Design (LD) are built around models and theories to ensure that educational content is able to address the needs of learners. Instructional design models like Analysis, Design, Development, Implementation and Evaluation (ADDIE), the Successive Approximation model (SAM) and Backwards Design are used to create learning experiences, and are then followed by an evaluation and review to refine the process and create deeper engagement with learners.

In this post, we explore one of the most popular evaluation models - the Kirkpatrick Evaluation model - and how it can be used to measure outcomes and improve learning results.
What Is the Kirkpatrick Evaluation Model? The Four Levels of the Kirkpatrick Model Level 1: Reaction Level 2: Learning Level 3: Behavior Level 4: Results Benefits & Challenges of the Kirkpatrick Model How to Use the Kirkpatrick Model How Does Instructional Design Employ Evaluation Models?
What Is the Kirkpatrick Evaluation Model?

The Kirkpatrick Model of evaluation was developed by Donald Kirkpatrick in 1959 as a means of understanding how an organization's training or educational process is functioning, and how the people within it are performing. It works by providing a model to capture and then analyze data to measure both immediate impacts and longer-term results.

Even after 60+ years, the Kirkpatrick training model continues to be the most well-known and widely used means of evaluating the effectiveness of training programs. A study published in The Institute for Employment Studies noted that the model "remains useful for framing approaches to training and development evaluation" and can be applied in the evaluation of any type of educational or interactive program.

The Kirkpatrick method works by breaking a training/education process into four distinct levels to identify the effects of the training and how well they are contributing to successful learning outcomes.
The 9 Reasons You Should Consider a Career in Instructional Design (Plus Salary Info) - Get The Guide >>
The Four Levels of the Kirkpatrick Model

There are four levels that come in increasing order of complexity, as shown in the following image:

Data gained from the earlier levels tends to be more immediate and personal to learners, while the higher levels focus on long-term benefits that can be applied across an organization. These later levels require more complex data gathering and analysis, which makes the results more informative, but also more time-consuming and complicated to measure.

As the International Society for Educational Technology notes, this means those who use the Kirkpatrick Model of training evaluation may not progress beyond the second level due to the required time investment or budgetary restrictions.

Here's what the levels cover and how they build on each other.

Level 1: Reaction

Kirkpatrick defines the first and most immediate evaluation as capturing how participants find a training or instruction to be "favorable, engaging, and relevant." Educators, trainers and administrators collect reactions through short surveys provided at the end of a session, sometimes through paper "smile sheet" handouts or through one-on-one interviews. Modern evaluations often use email invitations to online surveys and digital assessment tools such as Questionmark, Qualtrics and SurveyMonkey.

A survey will ask how the participants feel about the program, whether they found it useful, and their impressions of the experience. Sometimes this data is combined with additional learner feedback provided through comment forms or direct verbal responses from participants.

This initial level requires the least amount of time and budget to administer and is helpful for determining how informative the content was and how effectively it was delivered. On the other hand, this level also provides the least useful insights, as it doesn't offer any data on how effectively the information was synthesized or how problematic areas could be improved.

For those kinds of insights, we'll need to progress to the higher levels.

Level 2: Learning

This level is defined as where instructors determine whether the learners managed to acquire the "intended knowledge, skills, attitude, confidence, and commitment based on their participation in the training." If there's a pattern in what learners struggled to understand or retain, that could point to a problem in an area of the instruction or to specific material that might be more challenging for the learners than intended.

Level 2 learning assessments can be administered via written or computer-based quizzes or tests or through direct interviews and observations. For more detailed assessment, participants may be asked to perform a task relating to the training and then judged on their level of success. Pre-tests can also be administered prior to training to more accurately measure the degree of "before and after" improvements.

Aside from evaluating the efficacy of the training, this data can also be used to determine whether participants should receive credit for the course, or if they're eligible for licenses or certificates. While measuring learning outcomes provides an understanding of learners' immediate and short-term gains, we need to move up another level to determine any long-term value.

Level 3: Behavior

The third level of Kirkpatrick's training evaluation seeks to determine whether or not learners' behaviors have changed as a result of the program. Beyond the initial impressions or immediate knowledge retention, Kirkpatrick describes this level of evaluation as offering a real opportunity for instructors to see if a training has been successful and participants can "apply what they learned during training when they are back on the job."

Accurately measuring behavior can be challenging, as different participants are likely to manifest changes over extended periods of time. Initial successes might wane as some learners return to bad habits, while other learners may steadily improve over time as they practice their technique and refine their knowledge through repetition. For those reasons, the most effective performance metrics are the ones that can be used in occasional direct on-the-job observations and regular performance reviews - such as a system that could automatically track relevant variables, like daily sales numbers or reviews from satisfied customers.

The most comprehensive measurement of learners' behavior comes from combining relevant metrics with regular performance reviews, as well as peer- and self-reviews. When used in coordination, these measurements provide a comprehensive "360-degree feedback" to measure all areas of a learner's growth.

Though, if we want to measure a training's effects on the larger organization, we'll need to advance to the fourth and final level of the Kirkpatrick evaluation model.

Level 4: Results

According to Kirkpatrick, this level measures how the training program contributes to the success of the organization "as a result of the training and the support and accountability package." Specifically, it attempts to measure the changes in a specific area, which can include sales, spending, customer satisfaction and overall efficiency in order to determine the return on investment (ROI).*

This final level is the most challenging one to realize, as it can require a significant investment of time and budget to accurately measure results. Making a direct correlation between a training program and changes in performance can be hard to accurately determine, as improper or incorrect data can skew results. One way to effectively measure the impact of the training would be to use a control group, and then compare performance over time between it and the training group.

When done correctly, this level of evaluation offers the most valuable data for an organization, providing insights into both how a training program has created benefits, and also how it can be improved.

* Note that some applications of the Kirkpatrick Model are designed so that ROI is separated out as its own fifth level of evaluation. For simplicity's sake, we're keeping our review of the model to four levels.
The 9 Reasons You Should Consider a Career in Instructional Design (Plus Salary Info) - Get The Guide >>
Benefits & Challenges of the Kirkpatrick Model

The reason the Kirkpatrick training model is still widely used is due to the clear benefits that it can provide for instructors and learning designers:

It outlines a clear, simple-to-follow process that breaks up an evaluation into manageable models.

It works with both traditional and digital learning programs, whether in-person or online.

It is a highly flexible and adaptable model, making it easy to implement across different environments and fields.

It provides instructors with a means of gathering feedback to continually improve their training programs and methods.

It gives managers and leaders insight into the effectiveness of training programs and their overall impact on an organization.

However, there are also some challenges and concerns to be careful of when employing the Kirkpatrick Model:

The third and fourth levels are more time-consuming to employ correctly and can be costly to implement; organizations will need to plan accordingly to take advantage of these higher levels.

It can be difficult to directly link specific results to the effects of the training. IDs and their organizations will need to carefully design evaluation metrics to avoid drawing the wrong conclusions.

Even with carefully developed metrics, it may not be possible to directly prove the efficacy of the program or a positive ROI without a long-term and consistent measurement of results.

The Kirkpatrick Model is only employed after a training or instructional program is finished, which means it's harder to adjust or improve a training that is in-progress.

How to Use the Kirkpatrick Model

To address some of the challenges with utilizing the Kirkpatrick Model and to maximize its benefits, it's recommended to start at the highest level and work backwards. That means the first step should be to ask questions about your higher-level, long-term goals:

What are the results that the organization is looking for?

Which areas need to see the results?

What are the most important benefits to achieve?

After determining the long-term goals, consider which behaviors need to be developed or encouraged in order to achieve them. As an ID or LD you should consult with subject matter experts (SMEs) and stakeholders to curate a list of what's possible:

What will positive changes in behavior look like?

How will learning outcomes be accurately measured?

What "level" of success is acceptable?

From there, determine what are the knowledge and skills that learners will need to develop to enable those positive behaviors. This will require assessing their current knowledge, experience and skill levels to determine which areas are lacking.

Finally, the training program itself will need to be evaluated, tested and revised in order to ensure that it's accessible, relevant and enjoyable for all learners. This will help to ensure a more positive reaction to the material and that the information is understandable and easier to synthesize.

Following these instructional design principles can aid in the development of a program that is effective for your learners and beneficial for any organization.
The 9 Reasons You Should Consider a Career in Instructional Design (Plus Salary Info) - Get The Guide >>
How Does Instructional Design Employ Evaluation Models?

Instructional design seeks to provide best-practice solutions in how to bridge knowledge, skill and attitude gaps between different learners. If you're interested in the most effective means of employing the Kirkpatrick Model - or other important evaluation models - to train and educate people in today's digital age, consider the University of San Diego's online Master of Science in Learning Design and Technology program.

You'll be introduced to a very high-level overview of Kirkpatrick's Model in our foundational course, followed by a more detailed review in our Program Design, Assessment and Evaluation course. Regardless of the evaluation model you choose to employ when designing and revising an instructional solution, mastering ID principles will help you create effective training and educational materials. Learn more about how you can master these skills and techniques at USD LDT.

Kirkpatrick Model FAQs

# data[].attributes.attachment_link.title
The Kirkpatrick Training Evaluation Model [+ Benefits & FAQs]

# data[].attributes.attachment_link.description
What makes the Kirkpatrick training evaluation model so popular? Discover why this model is such an essential part of #LearningDesign in our review of its benefits, its limitations and how it can inform your own training evaluation.

# data[].attributes.attachment_link.description
Presentation on Kirkpatrick's Training Evaluation Model

# data[].attributes.headline
How to Implement Kirkpatrick's Model of Training

# data[].attributes.subject
How to Implement Kirkpatrick's Model of Training

# data[].attributes.attachment_link.title
How to Master the Kirkpatrick Model of Training Evaluation in 6 Simple Steps

# data[].attributes.attachment_link.description
Learn exactly how to master the Kirkpatrick Model of Training Evaluation in 6 simple steps.