creating an architecture of assessment: using benchmarks to measure library instruction progress and...
Post on 28-Dec-2015
219 Views
Preview:
TRANSCRIPT
Creating An Architecture of Assessment: using benchmarks
to measure library instruction progress and success
Candice Benjes-Small
Eric Ackermann
Radford University
Looking for alternatives
• Number of sessions plateau
• Scoured literature
• Attended conferences
• Networked with colleagues
Our environment
• Public university
• 9000+ students
• Courses not sequenced
• Instruction built on one-shots
Macro look at program
• Focus on us, not students
• Search for improvements over time
• Student evaluations as basis
Goals
• Provide data to satisfy three constituents
– Instruction librarians: immediate feedback
– Instruction team leader: annual evaluations
– Library Admin: justify instruction program
Development
• 4-point Likert scale
• Originally had a comment box at end
• Major concern: linking comments to scale responses
Solution: Linked score and comment responses
• Q1. I learned something useful from this workshop.
• Q2. I think this librarian was a good teacher.
Inspiration for benchmarks
•University of Virginia library system use of metrics to determine success
•Targets outlined
•We would do one department rather than entire library
To learn more about UVA’s efforts, visit http://www.lib.virginia.edu/bsc/
Benchmark baby steps
• Look at just one small part of instruction program
• Begin with a single benchmark
• Identify one area to assess
• Decided to do one particular class
Introduction to Psychology
•Teach fall and spring, beginning 2006
•14 sections of 60+ students
•Shared script and PPT
•Everyone teaches over 2 days
To see our shared PPT, visit http://lib.radford.edu/instruction/intropsych.ppt
Developing benchmarks
• Selected a comment based metric for Instruction Team
• Chose class of comments: “What did you dislike about the teaching?” (Question #2)
Current benchmarks
• Partial success: 5 < 10% total comments for Question 2 are negative
• Total success: < 5% total comments for Question 2 are negative
Success?
• Reached our desired benchmark for partial success- never quite went below 5%
• Tweaking the script again
• Continuous improvement
Scaling for your program
• Adjust the benchmark levels
• Only look at score responses (quantitative) instead of comments (qualitative)
• Adjust the number of benchmarks used
Sharing with administrators
• Team annual reports
• Stress evidence-based nature
• Use percentages, not a 4-point scale
Advantages
• Flexiblity to measure what you want to know
• Provides structured goal
• Evidence-based results more convincing
More advantages
• Continuous evaluation results over time
• Data-driven decisions about instruction program
• Do-able
top related