We all know the importance of assessment. We need to gather assessment data in order to determine how well our programs are meeting needs, how our collection is being used, what gaps we may need to fill, etc.
Gathering data can feel fairly straightforward, especially if the data is compiled from a computer system that runs a report. However, if we need to gather data from individuals on their knowledge (perhaps to see what they learned in an instruction session), or from individuals about their experiences with the library, the process can be more challenging. When we create assessments we need to consider both validity and reliability.
Validity refers “to the extent that a test measures what it is supposed to measure” (Center for Innovation in Teaching and Learning, 2021, para. 1). The Standards for Educational and Psychological Testing (2014) defines validity as the “degree to which evidence and theory support the interpretations of test scores for proposed uses of tests” (p. 11).
Reliability refers to the extent that an assessment produces consistent results.
In order to gather accurate data it is necessary to determine how you will ensure the validity and reliability of your assessment tools. To do this I suggest the following:
- Clearly define the goals for the assessment
- Match assessment questions to each goal
- Ask a group of individuals to review the assessment and explain to you what they think each question means. This will help identify anything that is not clear.
- Triangulate data–compare the data gathered in the assessment to other data that may be available
- Ask 4-6 questions for each assessment goal (Shrock & Coscarelli, 2008)
- Use a tool to gather assessment data that participants are familiar with (i.e. paper and pencil, Qualtrics, etc.)
- Give participants a similar environment (i.e. the same length of time to take the assessment, a comfortable place, etc.)
- If you include short-answer questions or other types of questions that are not multiple choice or true-false, you will need to ensure that all reviewers use the same scoring system and are trained in what they should look for.
As you embark on library assessment, I hope you consider the validity and reliability of your measures. Taking time to plan for both validity and reliability will help ensure that the data you gather is accurate data you can use.
Many papers and articles have been written about assessment in libraries. I listed some resources below that may be beneficial if you want to explore library assessment in more detail.
Library Assessment Resources
Library Assessment Conference (Virtual, November)
Assessment and Evaluation in Special Libraries (previous Lucidea post from me)
Library Assessment Fundamentals (from CORE, part of ALA)
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). (2014). Standards for educational and psychological testing. American Educational Research Association. https://www.apa.org/science/programs/testing/standards
Center for Innovation in Teaching and Learning. (2021). Assessment validity and alignment. University of Illinois at Urbana-Champaign. https://citl.illinois.edu/citl-101/teaching-learning/resources/teaching-across-modalities/teaching-tips-articles/teaching-tips/2021/08/11/assessment-validity-and-alignment
Phelan, C., & Wren, J. (2006). Reliability and validity. UNI Office of Assessment. https://chfasoa.uni.edu/reliabilityandvalidity.htm
Shrock, Sa. A., & Coscarelli, W. C. (2008). Criterion-referenced test development: Technical and legal guidelines for corporate training (3rd ed.). Wiley.
Lauren Hays, PhD, is an Assistant Professor of Instructional Technology at the University of Central Missouri, and a frequent presenter on topics related to libraries and librarianship. Her expertise includes information literacy, educational technology, library and information science education, teacher identity, and academic development. Please read Lauren’s other posts about skills relevant to special librarians who provide instruction. And take a look at Lucidea’s powerful integrated library systems, SydneyEnterprise, and GeniePlus, used daily by innovative special librarians in libraries of all types, sizes and budgets.
Never miss another post. Subscribe today!
Librarians need to consider whom their collections represent and develop collection management policies to create equitable and inclusive collections
Author interview: Cathryn Copper on The Experimental Library: A Guide to Taking Risks, Failing Forward, and Creating Change
Editor interview: Harnett and Cantwell, Finding Your Seat at the Table: Roles for Librarians on Institutional Regulatory Boards and Committees
Dr. Lauren Hays shares some of her Fun Productivity and Educational Tools for Librarians as summer fades and the academic year begins