Reliability and Validity in Library Assessment

Lauren Hays

Lauren Hays

May 17, 2022

We all know the importance of assessment. We need to gather assessment data in order to determine how well our programs are meeting needs, how our collection is being used, what gaps we may need to fill, etc.

Gathering data can feel fairly straightforward, especially if the data is compiled from a computer system that runs a report. However, if we need to gather data from individuals on their knowledge (perhaps to see what they learned in an instruction session), or from individuals about their experiences with the library, the process can be more challenging. When we create assessments we need to consider both validity and reliability.

Validity refers “to the extent that a test measures what it is supposed to measure” (Center for Innovation in Teaching and Learning, 2021, para. 1). The Standards for Educational and Psychological Testing (2014) defines validity as the “degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (p. 11).

Reliability refers to the extent that an assessment produces consistent results.

In order to gather accurate data it is necessary to determine how you will ensure the validity and reliability of your assessment tools. To do this I suggest the following:

Validity:

  • Clearly define the goals for the assessment
  • Match assessment questions to each goal
  • Ask a group of individuals to review the assessment and explain to you what they think each question means. This will help identify anything that is not clear. 
  • Triangulate data–compare the data gathered in the assessment to other data that may be available

Reliability:

  • Ask 4-6 questions for each assessment goal (Shrock & Coscarelli, 2008)
  • Use a tool to gather assessment data that participants are familiar with (i.e. paper and pencil, Qualtrics, etc.)
  • Give participants a similar environment (i.e. the same length of time to take the assessment, a comfortable place, etc.)
  • If you include short-answer questions or other types of questions that are not multiple choice or true-false, you will need to ensure that all reviewers use the same scoring system and are trained in what they should look for.

As you embark on library assessment, I hope you consider the validity and reliability of your measures. Taking time to plan for both validity and reliability will help ensure that the data you gather is accurate data you can use.

Many papers and articles have been written about assessment in libraries. I listed some resources below that may be beneficial if you want to explore library assessment in more detail. 

Library Assessment Resources

Library Assessment Conference (Virtual, November)

Assessment and Evaluation in Special Libraries (previous Lucidea post from me)

Library Assessment Fundamentals (from CORE, part of ALA)

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.apa.org/science/programs/testing/standards

Center for Innovation in Teaching and Learning. (2021). Assessment validity and alignment. University of Illinois at Urbana-Champaign. https://citl.illinois.edu/citl-101/teaching-learning/resources/teaching-across-modalities/teaching-tips-articles/teaching-tips/2021/08/11/assessment-validity-and-alignment

Phelan, C., & Wren, J. (2006). Reliability and validity. UNI Office of Assessment. https://chfasoa.uni.edu/reliabilityandvalidity.htm

Shrock, Sa. A., & Coscarelli, W. C. (2008). Criterion-referenced test development: Technical and legal guidelines for corporate training (3rd ed.). Wiley. 

Lauren Hays

Lauren Hays

Lauren Hays, PhD, is an Assistant Professor of Instructional Technology at the University of Central Missouri, and a frequent presenter on topics related to libraries and librarianship. Her expertise includes information literacy, educational technology, library and information science education, teacher identity, and academic development. Please read Lauren’s other posts about skills relevant to special librarians who provide instruction. And take a look at Lucidea’s powerful integrated library systems, SydneyEnterprise, and GeniePlus, used daily by innovative special librarians in libraries of all types, sizes and budgets.

Similar Posts

Leave a Comment

Comments are reviewed and must adhere to our comments policy.

0 Comments
  • Canada: +1 604 278 6717
  • United States: +1 781 938 4444
  • Europe: +44 115 955 5936
  • Australia: +61 3 8526 0143
  • support@lucidea.com
  • supportUS@lucidea.com

Pin It on Pinterest

Share This