Using the Phillips ROI Model to Analyze Library Training

In my last post, I introduced the Kirkpatrick Model of evaluation. It is useful for all training, but may be particularly helpful as many of you are new to engaging in virtual instruction. The Phillips ROI Model builds on the Kirkpatrick Model by adding a level that considers the return on investment.

 For virtual instruction, training evaluation can help you determine:

• Is the format I am using working?
• Are people happy with the presentation style?
• Do attendees learn what they need to to be successful?
• Are people using what I teach?

If you have this data from in-person trainings it would be particularly beneficial to compare the data to see where improvements can be made in your virtual instruction.

As mentioned, the Phillips ROI Model builds on the Kirkpatrick Model by adding a level that considers the return on investment. To do this, you must collect data before, during, and after your training.

The Phillips ROI Model has five levels:

Level 1: Reaction

This level remains the same as the Kirkpatrick Model. At this stage, you want to determine how participants reacted to the training.

Level 2: Learning

This level also remains the same as the Kirkpatrick model. At this level, you want to understand if the audience learned the information you presented in your training.

Level 3: Application and Implementation

While this level is called Behavior in the Kirkpatrick model, the underlying goals of evaluation at this stage remain the same. At this level, you want to determine if the audience applies the knowledge they learned.

However, Phillips goes farther to help trainers understand how to improve their work by identifying whether the reason training wasn’t successful was because individuals do not understand how to apply the learning, or if there are barriers to implementation.

If the issue is with application, then learners do not understand how to apply what they learned.

However, implementation can also be a barrier. You should gather enough data to understand if there is something specific that is keeping individuals from implementing the learning. For example, is there a link that doesn’t work for stakeholders the way you expected it to? Does a system have barriers that are keeping stakeholders from implementing the learning even though they know how to apply it? (Additional examples and further explanation can be found in Deller [2019] in the reference list).

Level 4: Impact

In the Kirkpatrick model, the focus is on determining if the learning objectives for your training were met. With the Phillips Model, you should take a broader view and determine “the impact of the training on the organization” (Deller, 2019, para. 18).

Level 5: Return on Investment

This level is an addition from the Phillips ROI Model. “This level uses cost-benefit analysis to determine the value of training programs. It helps companies measure whether the money they invested in the training has produced measurable returns, and if so, what they are” (Deller, 2019, para. 19).

Additional Resources

Downes, A. (2017). An overview: Phillips Model for learning evaluation. Watershed. https://www.watershedlrs.com/blog/learning-evaluation/phillips-model/

*If you want additional information on how to calculate ROI, I recommend this source. 

Herrholtz, K. (2020). Extend your training evaluation to include the Phillips ROI model. eLearning Industry. https://elearningindustry.com/extend-training-evaluation-include-phillips-roi-model

Phillips, J. (2003). Return on investment in training and performance improvement programs (2nd ed.). Butterworth-Heinemann.

*This resource is the original work on the Phillips ROI Model.

References

Deller, J. (2019). The complete Philips ROI Model tutorial for beginners. KudoSurvey. https://kodosurvey.com/blog/complete-philips-roi-model-tutorial-beginners