1. Rebecca Jones, MLS rebecca@dysartjones.com www.dysartjones.comRebecca Jones, MLS rebecca@dysartjones.com www.dysartjones.com

2. Underlying Assumptions Few libraries have a culture of assessment Difficult and complex Most measures indicate pastperformance No cause-and-effect relationship between measures Performance measures quantitative, but library outcomes are largely qualitativeMeasuring for Results: The Dimensions of Public Library Effectiveness Joe Matthews, 2004 3. Underlying Assumptions Identifying & illustrating value depends on conversations with yourstakeholders The first conversation shouldnt be when measures are presented More isnt necessarily better In fact, more clouds the issue and the messageNot everything that counts can be counted & not everything that can be counted counts. - Einstein 4. Three types of measuresPerformance or ValueOperationalSatisfaction 5. Measures are, by definition, based on a beginning or monitor results against anagreed-to objective or value 6. Questions driving measures Whats the library or information centre doing? How much is it doing? How well is it doing it? Who is it doing it for? What positive impact is the servicehaving for them?What action will you take as a result of knowing this? 7. Usage Statistics Weve long tracked how much We also need to look at differences: Peaks? Dips? Switches? What are these statistics really telling us? And not? Are they highlighting where we should dig deeper? 8. Customer Satisfaction Tends to focus on existing products and/or services Usually measured through surveys 9. The performance question 10. Framework Page 1What measures do you keep? 11. Meaningful performance measures Matter to you & to your stakeholders Are conveyed in the stakeholders language Demonstrate that the service makes a difference Focus attention on what is most important for the organization Are critical for managing, planning & decision-making Are organization-dependent Connect strategic directives to employees 12. No one magic measure Successful organizations: clarity of purpose understand their culture performance measurement system that fits that culture Value is dynamic, economic , psychological & relative to alternativesLearning from others 13. Framework Page 2Understand the Context Align Strategies & ObjectivesCommunicate ResultsTranslate Data into Outcomes & ImpactsIdentify Services & ProgramsManage Measurement DataDefine Measures 14. Understand the Context 15. Know the context and contextorsHow is value defined, measu red, and communicated?Name your stakeholdersHow do they measure their progress? 16. Healthy living advocateCouncilor Good Cause Endorses physical activities Spend for bikelanes, pools, rinks, etc The library is just a community centre anyway Says more people are impacted by his stuff than by the libraryBoard Member: Ms. Expert Does reconnaissance work on other library systems andreports back to Board May not have a good grasp on what libraries do Feels money is no object when it comes to something she sees as a trend Wants to compete in the library industry 17. The Provost of Doom Libraries days are numbered Libraries arent relevant Everyone has computers and internet need information? Just google it Starbucks has wireless With eBooks and research online believes facilities canbe made smallerDean of Humanities Dr. Kee Paper Influential in university & broader academic circles Focus on research Has deep seated beliefs in traditional library services Quiet No technology Libraries are institutions for reading & research 18. Framework Page 3Start with Stakeholders 1. Critical that you understand the context in which the measures are considered. In your organization, how is value defined, measured and conveyed? 2. Who are the stakeholders who must understand the value of your services? NamePositionHow do they Goals & measure objectives their progress?What do we know about them? What type of stakeholder are they?What do they know about our services? 19. Logic Model Popular in non-profit sector & increasingly in libraries and culture Based on logical links cause & effect Logical links: IF teens believe that the Library sees them as valuable, equal community members & takes their needs seriously THEN teens are more likely to engage in discussions & programs in the Library that will positively impact their studies & their lives IF the information professional ensures current, relevant content is easily available on employees desktops THEN the organization will save $X in lost hunt time 20. Framework Page 4Its logical thatINPUTOUTPUTOUTCOMEIMPACTresource perspectiveoperational perspectiveuser perspectivestakeholderperspective 21. INPUT Resource PerspectiveOUTPUT Operational PerspectiveEquipment Space Supplies Funding StaffTraining module Program Report # of Program Attendees # of Programs Held 22. OUTCOME User Perspective Increased skill Know-how or knowthat Change in status Change in behaviour or attitudeIMPACT Stakeholder Perspective Lower dropout rates Increased employment levels Increased recovery rates 23. Start with the end in mindRefocus from the activity to the impact 24. Align Strategies & Objectives 25. Align Strategies and Objectives Stakeholders Goals & ObjectivesYour Goals & Objectives: How they ContributeProvide excellent clinical careSupport the development of procedures and policies relating to clinical careIncreased research grantsSupport the research processNewcomers contributing to the community economicallyEquip newcomers with links to language, literacy, employment and social supports 26. Framework Page 5 TopAlign objectives Articulate & document your goals & objectivesWhat are you trying to accomplish? Clarify how these contribute towards your stakeholdersobjectives & desired outcomes 27. Identify Services & Programs 28. Framework Page 5 BottomIdentify Services What services, programs or products contribute to your realizing anobjective?Stakeholder Goals & Objectives Your Goals & ObjectivesYour Services 29. Academic Example InputOne liaison librarianOutputFaculty research support for a Grant ApplicationOutcomeCompleted application submitted on timeImpactGrant dollars received by faculty. Faculty member is able to hire 2 grad students to conduct research: increased university reputation, a number of articles are submitted and accepted by academic journals, increased citations for faculty member, university increases visibility. 30. Public Library Example Input-Funding -Staffing -Supplies =$10,000 to develop learning place programsOutput10 new programs each offered 5 times per yearOutcomeNumber of program registrations increased by 25% year to year. 1,600 students ages 6-10 completed learning programImpactChildren attending programs are able to maintain the skills taught in school verified by parent follow-up. Library recognized for helping children develop and maintain literacy skills. Board of Education partnership & recognition based on improvements in standardized testing results. 31. Government Example Input$100K resource budgetOutput2500 resources utilizedOutcomeResources used by 75% of employees in policy developmentImpactUsage enabled policy development to: 1. Eliminate department-specific resource purchases saving the organization $35K 2. Reduce policy duplication or blind-sides by at least 25% (as reported during policy debriefings) 32. GoalObjectivesObjectivesActivities or servicesActivities or servicesOutputsLogical flowOutputsImmediate OutcomesLong-term OutcomesImmediate OutcomesObjectives 33. Define Measures 34. Define Success What will success look like? How will you track progress towards meeting goals? What indicators or measures will you use? Qualitative? Quantitative? What data needs to be collected? How? Keep it key Keep it simple Be practical Dont get mired oroverwork frontline staff 35. Framework Page 6 LeftService Impact Inputs Person power, money, content, etc.Activity What you doOutputs What the activity producesOutcomes How the client uses the outputImpact What changes for the clientOutcome MeasuresImpact Measures 36. Define Outcomes and Impact What will success look like? And how will you get there?INPUTOUTPUTOUTCOMEIMPACT 37. Define Outcomes and Impact OUTCOME User Perspective Increased skill Know-how or know-that Change in status Change in behaviour or attitudeIMPACT Stakeholder PerspectiveLower dropout rates Increased employment levels Increased recovery rates 38. Define measures by success being sought Inputs Person power, money, content, etc.Activity What you doOutcomes How the client uses the outputOutputs What the activity producesImpact What changes for the client 39. Framework Page 6 BottomSo what data will you need to collect? What data or measures do you What data or measures do you to collect to collect about Outcomes? about Impact? 40. Manage Measurement Data 41. Framework Page 7Manage the data Do it Determine a collection schedule Work out agreements with other departments that have needed data Are you currently collecting data or indicators that are no longer relevant? How long do you need to keep data? Who is responsible? 42. Translate Data into Outcomes & Impacts 43. Framework Page 8 BottomInterpret data So... What? What does the data say? What doesnt it say? Examine it from various angles What is the progress towards the goals & objectives? 44. Communicate Results 45. Framework Page 8Communicate results Actually begins when you are understanding the context Your message: we have contributed towards your goals by If your goals & objectives are meaningful for your planning & decision-making, your measures will be indispensable 46. http://www.shapingoutcomes.org 47. The real question How have we impacted your project, work or decisions today? 48. Thanks! Rebecca Jones With special thanks to...Moe Hosseini-Ara @ Markham Public Library