Assessment and Evaluation in Special Libraries, Part 2

In my last post, I shared about assessment and evaluation in special libraries. In this post, I am going to continue that conversation, but focus on how to determine your assessment questions and what methods are best to answer those questions.

Determining Assessment Questions

Start by figuring out what you want to know.

Questions you may want answers to are:

• What works?
• What does not work?
• What can be improved?
• How can something be improved?
• What things are occurring?
• What challenges exist?

You may also want to know:

• What resources are being used?
• What resources are not being used?
• What gaps exist in services?
• What gaps exist in resources?
• How many stakeholders are using the library?
• What departments primarily use the library?
• Who is participating in research consultations?
• What type of reference questions do we primarily get?

If you do not have a clear understanding of what you are assessing the data will not be as helpful as it could be.

Once you know what you need to assess and evaluate, you must determine the best methods to use so as to get the necessary data to fully answer the question. To determine the best methods, you should clearly articulate the assessment questions, identify what data will answer those questions, and then select the best method to collect the needed data.  Methods could be a survey, user counts, interviews, database accesses, or observational data of how students use space.

Common data collection methods in the library include surveys, usage/data mining, and qualitative methods including focus groups, interviews, observations, and usability studies. You can also use a combination of these. An in-depth explanation of each type of data collection method is outside the scope of this post, but I recommend spending time reading about each method to determine what type is most appropriate for your assessment question. 

Once you have decided what you want to know and the best methods for collecting the data, you require a plan for implementing the assessment and evaluation. This includes first identifying who is responsible for the work.  Individuals need to know what they are assigned and how their assessment tasks fits into the bigger picture. Then, create a timeline. Work with the individuals you identified as being responsible for part of the assessment work.  Get their input on the timeline.  Ask what is reasonable.  Also keep in mind any external stakeholders impacting the timeline within which the assessment cycle must be completed.  For example, you may have to submit a report on a certain day, or you may need to collect data over a specific period of time.  

Next, determine where you will keep the data.  The data should be private and secure.  It also should only be accessible by individuals with a need to know—until you are ready to repackage it for a broader audience. Then, start thinking about how you will package the data for stakeholders.  Consider whether different stakeholders need different things, and if this impacts how you will collect and store the data for later use. 

Once you have gathered the data spend time reflecting on your plan for presenting and packaging the data.  Is it still valid?  Has anything changed?  Does the collected data fit the presentation model you had in mind? 

Then, spend time reviewing the data individually and as a team of librarians.  Others may have additional insights into the data.  

Finally, it is time for the next stage: evaluate what needs to change based on the data.

• Do you need to consider adding more resources in a certain department? 
• Do you need to consider shifting your hours? 
• Do you need to remove some resources from the collection in order to make room for new material?
• What patterns, categories or trends are emerging?
• What is unexpected?
• What did we learn from the data?

The feedback loop with the assessment and evaluation process is very important. Too often data is gathered and not used in a strategic way to make decisions. Implementing a feedback loop in your process will help ensure the assessment and evaluation process is worthwhile. To create a feedback loop, share the data with others to get their insights, and explain what has been done (or will be done) based on the data. Then, take time to identify new assessment questions and start the process of assessment and evaluation again.