determining and demonstrating value rebecca jones and moe hosseini

Determining &

Demonstrating Value -

with the Logic Model

Moe Hosseini-Ara, City of Markham

[email protected]

Rebecca Jones, Brampton Public Library

[email protected]

TO WHOM

?

Ivanna U.R. Mony

• Socially connected

• Lobbying to have library

budget reallocated to other

projects

• Has never used the library

• Buys all her books from

Amazon

• Why does the library need

so many staff ? And why are

they paid so much?

Dr. Ural Doomed

• Libraries days are numbered

•

– Everyone has computers and internet

–

– Research happens in faculties not

libraries

• With eBooks and research online,

believes facilities can be made

smaller no need for print material

CFO Roi (Roy) Bottomline, MBA

• All about the bottomline

• Revenue and investments

• How do your services support revenue generation?

•revenue are a drain on the system

•it!

Kee Paiper

• Influential in many circles

• Has deep seated beliefs in

traditional library services and

importance of paper for legal

profession

• Libraries are institutions for

archives & research

Understand the Context

Know the Context & the Contexters

How do they

measure their

progress?

Know your

Stakeholders who

are they?

How is value

defined,

measured &

communicated?

Symptoms

Underlying causes

the water

causing the

symptoms?

•

•

performance

• Difficult and complex

•

decision-makers

• Identifying & illustrating value depends on conversations with your

stakeholders

•

& mental

models

How should

we frame

this issue?

Know the

language of

your context

MetricsValue

Customer Constituent

Student

Start with stakeholders

Name Position How do they

measure their

progress?

Goals &

objectives

What do we know about them?

What type of stakeholder are

they?

What do they

know about

our services?

Critical that you understand the context in which the measures are considered.

1. In your organization, how is value defined, measured and conveyed?

2. Who are the stakeholders who must understand the value of your services?

EXERCISE

The Logic Model

The Logic Model

goes beyond documenting what you did and

measures what difference you made in the life of

changed

Measuring Impact/Value

Change in the human condition?

• Behavior

• Skill

• Knowledge

• Attitude

• Circumstances

• Awareness

• Motivation

• Condition

• Status

INPUT

resource perspective

OUTPUT

operational perspective

OUTCOME

user perspective

IMPACT

stakeholder perspective

IMPACT

Equipment

INPUTResource

Perspective

Equipment

Space

Supplies

Funding

Staff

IMPACT

Equipment

OUTPUTOperational Perspective

Training module

Program

Report

# of Program Attendees

# of Programs Held

IMPACT

OUTCOMEUser

Perspective

Increased skill

Know-how or know-that

Change in status

Change in behavior or

attitude

IMPACT

IMPACTStakeholder Perspective

Long term results

Lower dropout rates

Increased employment

levels

Increased recovery rates

Job Skills Training

Input(resources)

• 10 hours of staff time

• $100 budget required to create the program (supplies and

handouts)

Output(operations)

• 2 hour job skills program created for student job seekers

• 4 programs offered over a period of a month

• total of 107 participants in attendance

Outcome(user)

• Students gained resume and cover letter writing skills

• Knowledge of where to look for jobs and submit applications

• Ability to give interviews, proficiency in answering interview

questions

Impact(stakeholder)

• 80% of the participants who attended the program reported that

they were either able to get interviews or actual job placements

• University can claim that students are able to get jobs following

graduation

BEGINWITH

THEIN MIND

ENDCovey 1989

Public Library Example

Input(resources)

• Staffing; Program Design; Marketing Material, Facilities; Equipment

• Speakers = 6 female professionals working in STEM careers scientists, astronauts,

engineers, doctors, programmers

• $1,500 to pay for honorariums and supplies

Output(operations)

• 6 week program offered twice a year

• 25 girls ages 10-12 complete program each session (50 participants per year)

• Pre and Post-session questionnaire; (permission to follow-up in 3 years)

Outcome(user)

• Girls ages 10-12 have a better understanding of STEM careers

• Increased interest in sciences & maths as potential course options

• Positive response to questionnaire that the girls want to be scientists, astronauts,

engineers, programmers, etc.

Impact(stakeholder)

• Girls ages 10-12 are interested in pursing STEM careers (based on questionnaire

response (short-term)

• These girls take maths, sciences, computers in high school and pursue degrees in

STEM in university (mid-term)

• Girls pursue careers in STEM and become scientists, programmers, doctors, etc. (long-

term)


Input(resources)





Output(operations)




Outcome(user)



• Positive response to questionnaire about what the girls want to be - scientists,

astronauts, engineers, programmers, etc.

Impact(stakeholder)






term)


Input(resources)





Output(operations)




Outcome(user)



• Positive response to questionnaire that the girls want to be scientists, astronauts,

engineers, programmers, etc.

Impact(stakeholder)






term)

Academic Library Example

Input(resources)

• One liaison librarian

Output(operations)

• Faculty research support for a Grant Application

• Completed application submitted on time

Outcome(user)

• Grant dollars received by faculty

• Faculty member is able to hire 2 grad students to conduct

research

Impact(stakeholder)

• Increased university reputation

• A number of articles are submitted and accepted by academic

journals, increased citations for faculty member

• University increases visibility.

Government Library Example

Input(resources)

• $100K resource budget

• Staff to advise on development of policies

Output(operations)

• 2500 resources utilized by 75% of department employees

• Library has contributed to the development of 6 policies in past

12 months

Outcome(user)

• 6 policies approved and in place

• Eliminate department-specific resource purchases saving the

organization $35K

Impact(stakeholder)

• Credible policies

• -

by at least 25% (as reported during policy debriefings)

• Reduced costs to run department

Corporate Library Example

Input(resources)

• $80K electronic content contract

•

departments applications

• Skilled info pro to develop research portfolio for business development

department

Output(operations)

• Customized research portfolio in response to request for background on

deal between XYZ company & ABC government agency, including specific

data on terms how ABC likes deal structured

• Library resources are integrated into business development applications

Outcome(user)

• Client extrapolated how ABC agency likes to see deals structured and

structured deal appropriately

Impact(stakeholder)

• Organization successfully completed a deal with ABC government agency

for $10M, the 1st in 5 years

Measurement

Framework

Understand the Context

Align Strategies &

Objectives

Identify Services & Programs

Define Measures

Manage Measurement

Data

Translate Data into

Outcomes & Impacts

Communicate Results

Align Strategies &

Objectives

Align Strategies & Objectives

Your Goals & Objectives: How they

Contribute

Accelerate newcomers contribution to

the community & economy

Offer programs and services that

connect newcomers with relevant

agencies and services

Increased research grants Support the research process and

assist with submission and completion

of grants

Newcomers contributing to community

economically

Welcome newcomers with links to

language, literacy, employment &

social supports

Align objectives

• Articulate & document your goals &

objectives

What are you trying to accomplish?

• Clarify how these contribute towards

desired outcomes

EXERCISE

Identify Services & Programs

Identify Services

What services, programs or products contribute to your realizing

an objective?

Stakeholder Goals & Objectives

Your Goals & Objectives

Your Services

EXERCISE

Define Measures

What measures do

you keep now?

• No one magic measure

• Successful organizations:

• clarity of purpose

• understand their culture

• performance measurement system that fits that

culture

•

alternatives

• Identifying outcomes & especially impact takes a long time

& heavy duty analysis

Value

SatisfactionOperational

Three Types of Measures

Operational/Usage Measures

•

• We also need to look at differences:

• Peaks? Dips? Switches?

• What are these statistics really telling us? Or, not?

• Who needs to see these statistics?

• Are they telling a story that you want to tell?

Operational

Customer Satisfaction Measures

• Tends to focus on existing

products and/or services

• Usually measured through

surveys

• Are we also measuring

importance?

• Survey burnout, plus the

expectation is to score all

Satisfaction

• Are we adding value?

• What difference are we making?

• What is the impact of our

services and/or programs?

• How are we contributing to the

Value

What are meaningful measures?

Matter to you AND your

stakeholders

language

Focus attention on what is most

important for the organization

Are critical for managing,

planning & decision-making

Are organization-dependent

Measure: info about what is

Define success

• What will success look like?

• How will you track progress towards meeting goals?

• What indicators or measures will you use?

• Qualitative? Quantitative?

• What data needs to be collected?

• Keep it key

• Keep it simple

• Be practical

• or

overwork frontline staff

Service Impact

InputsPerson power, money, content, etc.

ActivityWhat you do

OutputsWhat the activity produces

OutcomesHow the client uses the output

ImpactWhat changes for the client

Outcome Measures Impact Measures

EXERCISE

Manage Measurement

Data

Manage collection

• Determine a collection schedule• Work out agreements with other departments that

have needed data• Are you currently collecting data or indicators that

are no longer relevant?• How long do you need to keep data?• Who is responsible?

EXERCISE

Translate Data into

Outcomes & Impacts

Define outcomes & impact

• What will success look like?

•

OUTCOME IMPACT

INPUT OUTPUT

Define outcomes & impact

OUTCOME

User Perspective

Increased skill

Know-how or know-that

Change in status

Change in behaviour or

attitude

IMPACT

Stakeholder Perspective

Longer term results

Lower dropout rates

Increased employment

levels

Increased recovery rates

Measuring Impact/Value

Change in the human condition?

• Behavior

• Skill

• Knowledge

• Attitude

• Circumstances

• Awareness

• Motivation

• Condition

• Status

Interpret data

•

•

• Examine it from various angles

• What is the progress towards the goals &

objectives?

EXERCISE

Communicate the Results

Remember us

?

Communicate results

• Actually begins when you are understanding the context

• Your message: e have

contributed towards your goals

• If your goals & objectives are meaningful for your planning & decision-making, your measures will be indispensable

EXERCISE

c

Value

SatisfactionOperational

Sweet Spot

What do you need to stop doing?

Resources

IMLS Webography

http://www.imls.gov/applicants/webography.aspx

Shaping Outcomes

http://www.shapingoutcomes.org/index.htm

Overcoming Our Habits and Learning to Measure Impact by Moe Hosseini-Ara and Rebecca Jones.

Information Today, Inc. June 2013.

http://www.infotoday.com/cilmag/jun13/Hosseini-Ara_Jones--Overcoming-Our-Habits-and-Learning-to-

Measure-Impact.shtml

Seven Easy Steps to Help You Measure Up by Moe Hosseini-Ara and Rebecca Jones. Information Today, Inc.

July/August 2013.

http://www.infotoday.com/cilmag/jul13/Hosseini-Ara_Jones--Seven-Easy-Steps-to-Help-You-Measure-Up.shtml

REBECCAJONES

MOEHOSSEINI-ARA

&

[email protected]

[email protected]

determining and demonstrating value rebecca jones and moe hosseini

Government & Nonprofit