In my last post, I introduced the Kirkpatrick Model of evaluation. It is useful for all training, but may be particularly helpful as many of you are new to engaging in virtual instruction. The Phillips ROI Model builds on the Kirkpatrick Model by adding a level that considers the return on investment.
For virtual instruction, training evaluation can help you determine:
- Is the format I am using working?
- Are people happy with the presentation style?
- Do attendees learn what they need to to be successful?
- Are people using what I teach?
If you have this data from in-person trainings it would be particularly beneficial to compare the data to see where improvements can be made in your virtual instruction.
As mentioned, the Phillips ROI Model builds on the Kirkpatrick Model by adding a level that considers the return on investment. To do this, you must collect data before, during, and after your training.
The Phillips ROI Model has five levels:
Level 1: Reaction
This level remains the same as the Kirkpatrick Model. At this stage, you want to determine how participants reacted to the training.
Level 2: Learning
This level also remains the same as the Kirkpatrick model. At this level, you want to understand if the audience learned the information you presented in your training.
Level 3: Application and Implementation
While this level is called Behavior in the Kirkpatrick model, the underlying goals of evaluation at this stage remain the same. At this level, you want to determine if the audience applies the knowledge they learned.
However, Phillips goes farther to help trainers understand how to improve their work by identifying whether the reason training wasn’t successful was because individuals do not understand how to apply the learning, or if there are barriers to implementation.
If the issue is with application, then learners do not understand how to apply what they learned.
However, implementation can also be a barrier. You should gather enough data to understand if there is something specific that is keeping individuals from implementing the learning. For example, is there a link that doesn’t work for stakeholders the way you expected it to? Does a system have barriers that are keeping stakeholders from implementing the learning even though they know how to apply it? (Additional examples and further explanation can be found in Deller  in the reference list).
Level 4: Impact
In the Kirkpatrick model, the focus is on determining if the learning objectives for your training were met. With the Phillips Model, you should take a broader view and determine “the impact of the training on the organization” (Deller, 2019, para. 18).
Level 5: Return on Investment
This level is an addition from the Phillips ROI Model. “This level uses cost-benefit analysis to determine the value of training programs. It helps companies measure whether the money they invested in the training has produced measurable returns, and if so, what they are” (Deller, 2019, para. 19).
Downes, A. (2017). An overview: Phillips Model for learning evaluation. Watershed. https://www.watershedlrs.com/blog/learning-evaluation/phillips-model/
*If you want additional information on how to calculate ROI, I recommend this source.
Herrholtz, K. (2020). Extend your training evaluation to include the Phillips ROI model. eLearning Industry. https://elearningindustry.com/extend-training-evaluation-include-phillips-roi-model
Phillips, J. (2003). Return on investment in training and performance improvement programs (2nd ed.). Butterworth-Heinemann.
*This resource is the original work on the Phillips ROI Model.
Deller, J. (2019). The complete Philips ROI Model tutorial for beginners. KudoSurvey. https://kodosurvey.com/blog/complete-philips-roi-model-tutorial-beginners
Lauren Hays, PhD, is an Assistant Professor of Instructional Technology at the University of Central Missouri. Please read her other posts about skills for special librarians. And take a look at Lucidea’s powerful integrated library systems, SydneyEnterprise, and GeniePlus, used daily by innovative special librarians in libraries of all sizes and budgets.
Librarians, archivists and museum professionals can learn and improve our organizations by seeing good practices LAM colleagues are developing.
Special librarians delivering training should know what doesn’t work, as well as what does. The myth of learning styles is an example.
Slack offers a common communication platform with colleagues for quick questions, common challenges, and projects; practical tips for using it.
Interview with librarian and consultant Miriam Kahn with her perspective on trends in special librarianship and the future of the profession.