Training is important. But how do you know if it has worked? That was the question that Donald Kirkpatrick tackled in his 1954 PhD dissertation. He looked at how industrial training can be evaluated and produced what is still the most widely-used training evaluation model in the world.
And, although Kirkpatrick Partners, under the leadership of his son and daughter-in-law, has modified the model to create the ‘New World Kirkpatrick Model’, it remains remarkably unchanged. It is a sign of the precision with which Kirkpatrick defined four levels against which we can evaluate training.
And, if you say ‘what has training evaluation got to do with management?’ – consider this. As a manager, you’ll be constantly training your team members informally And you’ll often be securing them places on external training – or at least signing-off on it. How can you know whether that training is effective? Simple: by evaluating it against Kirkpatrick’s four levels.
Training is expensive. I should know: I provide it.
And the cost is mostly not in paying the trainer. Nor is it in any travel, accommodation, or subsistence for the participants. It’s the time that six, 12, or even more of your staff are doing the training and therefore not doing their jobs.
So, shouldn’t you know whether you are getting good value for the money and time you commit to it?
More than that, how confident are you that the skills you are training your people are actually the right ones to move your organisation and its performance in the right direction? That’s why we need training evaluation. And it’s what the four levels of the Kirkpatrick training evaluation model measure.
What are the Four Levels of the Kirkpatrick Training Evaluation Model?
The Four levels of the Kirkpatrick model are:
Let’s look at them each.
How the student reacts to the training.
Do they enjoy it, and find it engaging and relevant to their jobs? This is the kind of ‘customer satisfaction’ level that standard course feedback measures.
Assessed by: feedback, conversation, interview
This is quick, easy, and inexpensive to do, and nearly every training course has this. Typically done immediately after the training.
How much the student’s knowledge, skills, capability, and confidence increase as a result of the training.
And, crucially, does the training increase their commitment to making positive changes to their work?
Working from a framework that Kirkpatrick Partners suggests, we can ask five questions:
- Knowledge: ‘Do you know it?’
- Skill: ‘Can you do it?’
- Attitude: ‘Do you believe this will be worthwhile for you to do on the job?’
- Confidence: ‘Do you think you can do it on the job?’
- Commitment: ‘Do you intend to do it on the job?’
Assessed by: testing, interview, observation
This is relatively simple for basic skills, but even at level 2, evaluation is tricky for complex skills – particularly in the social arena. Typically done immediately after the training.
The extent to which participants apply what they learned in their training when they return to their job.
Does this result in improved performance that demonstrates greater ability and improved behaviours?
Assessed by: observation, performance evaluation, 360 degree feedback
Line managers should be doing this as a matter of course, but relating behaviour changes to specific training is not straightforward. Typically happens in the months following the training.
The effects that the training has on the organisation’s performance.
Do you get the outcomes you expected when commissioning the training? These may include customer satisfaction, process efficiency, cost reduction, increased sales, or retention and absenteeism improvements.
Assessed by: management reporting, team performance measures and KPIs, ROI
Once again, the challenge is to correlate organisational changes to individual training. Sufficiently robustly. Even then, correlation does not necessarily imply causation. Typically evaluated between three and 12 months after training, depending on business cycle.
How Did the Current Kirkpatrick Model Come about?
Donald Kirkpatrick studied whether training was making a difference to the people he trained, the people they managed, and the organisations they worked in. He used a framework that Dr. Raymond Katzell had published. It has four steps:
- Determine how trainees feel about the program.
- Determine how much the trainees learn in the form of increased knowledge and understanding.
- Measure the changes in the on-the-job behavior of the trainees.
- Determine the effects of these behavioral changes on objective criteria such as production, turnover, absenteeism, and waste.
Notice how similar these are to the four levels that are now synonymous with Kirkpatrick himself. They did not appear in his 1954 dissertation.
Two years after receiving his PhD, Kirkpatrick wrote an influential article in The Journal of the American Society of Training Directors called ‘How to Start an Objective Evaluation of Your Training Program’. In this, he fully acknowledges Katzell, in his opening remarks. But far more influential has been his 1959 series of articles, which used the four terms, Reaction, Learning, Behavior, and Results.
Controversy: Kirkpatrick or Katzell?
It was these articles that trainers read and valued. It was these articles that led them to coining the term, ‘The Kirkpatrick Model’. And in these articles, Kirkpatrick neglected to credit Katzell’s work on the four levels, and he continued to omit it in subsequent publications.
Will Thalheimer has researched this extensively. And, in 2018, he put his evidence to the founders of Kirkpatrick Partners, James Kirkpatrick and Wendy Kayser Kirkpatrick. Their account of the story of the four levels now includes Katzell’s contribution.
What is Your experience of Training Evaluation?
We’d love to hear your experiences, ideas, and questions. Please leave them in the comments below.
Books to Read
Whilst you can still get copies of Donald Kirkpatrick’s original book, and also later editions updated by James Kirkpatrick, the most recent work is by James and Wendy Kirkpatrick. Their most up-to-date is the 2016 book, ‘Kirkpatrick’s Four Levels of Training Evaluation’ (US|UK), so that’s the one I’d recommend.
To Learn More and go Beyond Four Levels
The Training Evaluation Pocketbook expands on the Kirkpatrick model and offers a model with nine outcomes. Four of these track back to the work of Katzell and Kirkpatrick. If you are a manager responsible for commissioning or delivering training, this is a hugely valuable little book.
Note: I am indebted to the Work-Learning Research website for the history of Kirkpatrick’s/Katzell’s four levels.