Recently I stumbled across the Kirkpatrick model for evaluation of training. Having not seen this model before, my curiosity got the better of me and I researched it a little further.
The following is an extract from http://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
The Kirkpatrick Model is the worldwide standard for evaluating the effectiveness of training. It considers the value of any type of training, formal or informal, across four levels. Created by Dr. Don Kirkpatrick in the 1950s, the model is applied before, during and after training to both maximize and demonstrate training’s value to the organization.
The Kirkpatrick model is based on:
Level 1 Reaction – The degree to which participants find the training favourable, engaging and relevant to their jobs
Level 2 Learning – The degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment based on their participation in the training
Level 3 Behaviour – The degree to which participants apply what they learned during training when they are back on the job
Level 4 Results – The degree to which targeted outcomes occur as a result of the training and the support and accountability package
Reading this got me thinking about how this can be applied to Adult Learning and in particular, how it can be used when adopting technology to capture the information without introducing more barriers.
Below I’ve looked at each Level and applied these to adult learning with technology.
Level 1 – Reaction
This is by far the most relevant for adult learners. We know that adult learners have a higher motivation to want to learn and it will always have some form of reward at the end. This level is very important as based on the outcomes of the evaluation you will be able to gauge how devoted the student is, especially when it comes to the proceeding levels.
But how would you capture these results? Understanding the barriers to learning with technology that some adult learners face (Bias, Literacy, Experience / Exposure, Fear and Social / Culture) we need to think outside the box. I understand that every situation is different, and each situation will present its own challenges. Finding a happy medium will always be difficult.
If we were to focus purely on online delivery. We could use assessments, forums, chat or even video conferencing to gauge reaction. You will know the literacy and experience of your students very early in the delivery of the course. Be prepared to spend more time with these students to bridge the gap, making them comfortable with the technology and where / what the data they are providing is going and used for.
Level 2 – Learning
This level is where we start to look at the learning taking place, is it effective? Is it being absorbed?
Your adult learners will be very diverse with their learning abilities, styles and rate of retention. It is this level where you will start to gauge if your learners are actually ‘learning’ by using different evaluation techniques.
For me, if you use the word ‘exam’ or ‘test’, some adult learners will freak out, stress and have increased levels of anxiety. This is not a true measurement of what a student has learnt. The fear associated with exams will stem back to their early education and the amount of pressure that was applied during their formative learning lives.
So how would / could you capture what is being learnt without the added pressure?
Observations by you as their trainer is the best way to evaluate the learning of your adult learners. You generally get to know your students strengths and weaknesses. Couple these with external responsibilities (read life of an adult) you will be able to effectively monitor and measure the learning of your students.
You could use online interviews or even phone / video interviews (dependant on the student’s digital literacy) to bridge the gap in understanding the level of learning. Human engagement (albeit using technology) will always reduce the gap when dealing with adult learners.
Level 3 – Behaviour
This level is where we can start to observe the behaviour of our adult learners on what they’ve learnt. This would usually be in a workplace, but how can you measure this? What if you are delivering in a face to face environment, how can you validate their behaviour?
You could provide observation or work sheets or even task books that the adult learners must have completed by their supervisors. There are online surveys that could be conducted, but be aware of the ‘fear’ of older adult learners entering data into an anonymous form.
Or call them! As the trainer, make an appointment in your calendar for 3 – 6 months after training and see how they’re going. Take and record notes against the student’s records. Have a predefined set of questions ready. You’ll be able to gauge very quickly how they’re going.
Level 4 – Results
This looks at the actual results of the training that was applied. In some circumstances, this would be easy to define and capture, but in my opinion, not always. You could use the same techniques as Level 3 to capture the evaluation. In order the results to be successful, there needs to be some form of positive change and outlook for the adult learner, something we are all aiming to achieve.
References:
http://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/
http://www.kirkpatrickpartners.com/Our-Philosophy/The-Kirkpatrick-Model
Hi Julian! I definitely think that the model does and can apply to online training, even though at the time of creation, there was no e-learning going on. I completely agree that the “reaction” level is the most relevant and probably the most important to understand before going forward. Gaining a positive reaction from an online training software is obviously what employers want. If a learner is not interested, then engagement will not happen. And if engagement does not happen, then less learning and retention will take place. I think reaction could be assessed through a simple survey after a lesson. The learners could also be asked how they initially thought it would be like before logging on in relation to how their experience actually was after the first lesson. As far as the learning goes—I agree that knowing there is a test or quiz at the end freaks people out and may even distract them from learning. I’ve been through online training sessions that will ask a few questions here or there as I learn. Making answering more interactive than just choosing a multiple choice answer helps too. There are so many ways online training can be more successful! Thanks again for the read.