Transcript
Page 1: Measuring Success in Patient Advocacy Initiatives

Measuring Success in Patient Advocacy InitiativesJuly 9, 2014

Page 2: Measuring Success in Patient Advocacy Initiatives

Welcome!

Lori Melançon, Senior Director,

Corporate Affairs, Onyx Pharmaceuticals

Ken Berger, President & CEO, Charity Navigator

Carmen Perez, Manager, Measurement

and Standards, CECP

Page 3: Measuring Success in Patient Advocacy Initiatives

Progress in Cancer: Measuring Success in Patient Advocacy InitiativesA Metrics Study

Lori MelançonSenior Director, Corporate AffairsOnyx Pharmaceuticals

Page 4: Measuring Success in Patient Advocacy Initiatives

Measuring Impact in the Evolving World of Nonprofit Organizations (NPOs)

Increased pressure to measure impact

Fierce competition for funding

Measuring success a daunting challenge

Page 5: Measuring Success in Patient Advocacy Initiatives

Progress in Cancer Study Overview

Objectives• Evaluate how diverse NPOs in cancer measure impact against goals to

assess trends and best practices

Rx4good MethodologyPrimary Research: Group Self-Assessment

▪ 24 NPOs participated in self-assessment of metrics best practices via 8-item Survey Monkey questionnaire

Secondary Research: Assessment of Websites and Annual Reports▪ 45-question survey evaluated how 35 NPOs report on diverse measures of

progress against goals

1

2

Page 6: Measuring Success in Patient Advocacy Initiatives

Study Sample

Websites and annual reports of 35 NPOs assessed:

Self-assessments completed by 24 (69%) of the 35 organizations

Page 7: Measuring Success in Patient Advocacy Initiatives

Group Demographics

The majority of the 35 assessed organizations have been operating for 20+ years with annual revenue of $1M–$5M

Age of Organization Annual Revenue

<$1M 20% (n=7)

$1M - $5M 31% (n=11)$5M – $10M 23% (n=8)

$10M - $30M 11% (n=4)

>$30M 11% (n=4)

N/A 3% (n=1)

20+ Years 49% (n=17)

10-20 Years 43% (n=15)

5-10 Years 6% (n=2)

1-5 Years 2% (n=1)

Page 8: Measuring Success in Patient Advocacy Initiatives

Primary Research FindingsNPO Self-Assessment of Metrics Best Practices

Page 9: Measuring Success in Patient Advocacy Initiatives

Survey Revealed NPO’s Perspective on Metrics Reporting…

75% of NPOs have changed their view about metrics during last three years• increased

understanding of value of metrics

• funder expectations

96%Metrics are essential for evaluating success against goals

33%The work we do is difficult to measure

25%Metrics are important but we don’t have the resources to measure our programs8%

We would rather invest in programs than in measuring results

Page 10: Measuring Success in Patient Advocacy Initiatives

Key Obstacles to Measuring Impact

Lack of Resources Determining Outcomes ChangeWe have limited time, staff and funding to do all the things we want to do…And it's hard to increase staff size without an increase in funding.

Measuring impact takes so much staff and time that the programs do not get the attention they deserve. It is the wrong way around.

Not all of our programs are easy to track, especially grassroots and advocacy initiatives.

It can be can be a challenge to determine the best way to measure impact. It can also be challenging to determine if behavior has really changed due to a program or an event.

“ ““ “

“ ““ “

Page 11: Measuring Success in Patient Advocacy Initiatives

NPOs Measure Impact in Many Ways, Led by Progress Achieved Against Goals

96%

92%

92%

92%

79%

75%

75%

67%

67%

54%

Progress Achieved Against Goals

Numbers Reached, Attended Events

Actions Taken by Stakeholders as a Result of Programs

Social Media or Web Engagement

Media Coverage

Funding Received

Perceptions Changed or Reinforced

Sustainability of Programs or Initiatives

Message Delivery

Funding Given

Q: What are the ways in which you measure your impact now? (Check all that apply)

Page 12: Measuring Success in Patient Advocacy Initiatives

Metrics Ranked Most and Least Valued by NPOs Mirror Those Attributed to Funders

Q: Please rank the following metrics options in order of importance to your ORGANIZATION/FUNDERS, with one being the most important, 10 least important.

NPO Both Funders

Numbers Reached, Attended Events

Actions Taken By Stakeholders as a Result of Programs

Progress Achieved Against Goals

Perceptions Changed or Reinforced

NPO Both Funders

Funding Received

Funding Given

Message Delivery

Media Coverage

Sustainability of Programs

MOST IMPORTANT*

*Ranked 1 or 2 as “most important” by the majority of survey respondents

*Ranked 9 or 10 as “least important” by the majority of survey respondents

LEAST IMPORTANT*

Page 13: Measuring Success in Patient Advocacy Initiatives

Organizations Revealed Best Practices for Measurement

Ongoing review of progress against goals

Broad program evaluation & follow-up

Digital reporting

Tracking research advances

Page 14: Measuring Success in Patient Advocacy Initiatives

Secondary Research FindingsHow NPOs Publicly Report Progress Against Goals via Websites and Annual Reports

Page 15: Measuring Success in Patient Advocacy Initiatives

Cancer NPOs Have a Strong Commitment to Reporting Results…

91%Publish an annual

report

89%Report annual revenue &

operating expenses

60%Clearly state

organizational goals

46%Report progress against

each goal

Page 16: Measuring Success in Patient Advocacy Initiatives

…and a Strong Commitment to Transparency

Approximately 80% identify corporate and individual sources of grants

Majority report funding given and funding received (65% and 70%, respectively)

Dollar size, number and impact of grants both given and received are often disclosed

Page 17: Measuring Success in Patient Advocacy Initiatives

Strong Social Media Presence, Yet Less Often Measured or Reported

97%

97%

91%91%

60%49%

46%Measure impact on progress based on social media or web engagement

29%Quantify growth of social media engagement

Page 18: Measuring Success in Patient Advocacy Initiatives

Website Research Shed Light on NPO Priority Metrics

Audience reach/event attendance (91%)

Legislative/policy work (74%)

Scientific progress (63%)

Survey patients about impact of programs

(23%)

Media coverage (43%) Message delivery (9%)

Website hits (29%) Unique visitors (31%)

MORE COMMON LESS COMMON

Page 19: Measuring Success in Patient Advocacy Initiatives

Key Takeaways and Study Implications

Metrics increasingly valued by NPOs, driven largely by funder expectations

▪ Tasked with finding more immediate measures of success▪ Limited resources and knowledge on how to best convey impact

Variety of reporting techniques; more “outputs” than “outcomes”▪ Funders and NPOs largely aligned in metrics they deem most/least important▪ More focused on reporting results than websites/annual reports indicate

NPO demographics have marginal bearing on metrics activities▪ Effective metrics reporting not necessarily linked to size or revenues

Diverse stakeholders and funding obstacles will continue to challenge metrics reporting, but best practices and cost-effective strategies exist to showcase impact

Page 20: Measuring Success in Patient Advocacy Initiatives

Your Guide to Intelligent Giving

Where the Heart

Meets the Mind

Critical Friend to Charities

Page 21: Measuring Success in Patient Advocacy Initiatives

The Core Challenge As I See It*“… there is virtually no

credible evidence that most nonprofit organizations actually produce any social value.”

*“The End of Charity” by David Hunter – Philadelphia Social Innovations Journal

Page 22: Measuring Success in Patient Advocacy Initiatives

The Nonprofit Marketplace Hewlett Fdn

THE ROADMAP TO A SOLUTION TO THE CHALLENGESINSTITUTIONALFUNDERSsupply $ & technical assistance formeaningful information

BENEFICIARIESare engaged& empoweredto providemeaningfulinformation

InstitutionalFunders usea set of standardizedNP reports of meaningful information

Beneficiaries receive feedback on how their meaningful information was utilized

Page 23: Measuring Success in Patient Advocacy Initiatives

WHAT IS MEANINGFUL INFORMATION FOR MEASURING A CHARITY’S SUCCESS?

RESULTSGOVERNANCE

FINANCIAL HEALTH

Page 24: Measuring Success in Patient Advocacy Initiatives

How Does This Fit into the Evolution of our Rating System?

• CN 1.0 – Financial HealthLaunched in 2002 with 1,100 charities

• CN 2.0 – GovernanceLaunched Sept 20, 2011 with 5,500 charities

• CN 3.0 – Results ReportingMethodology released and data collection begun Jan 2013, with a goal of 10,000 charities rated by end of 2016

Page 25: Measuring Success in Patient Advocacy Initiatives

Results Reporting Is…• An assessment of how charities use their results

internally and share them with stakeholders, including donors

• We are looking to see that you are reporting on results measures, and showing how your organization learns and improves based on those measures (i.e. learning and improving over time is more important than a ‘snapshot’ of results)

Page 26: Measuring Success in Patient Advocacy Initiatives

How Charity Navigator Evaluates Results Reporting

FIVE ELEMENTS:1. Alignment of Mission, Solicitations

and Resources

2. Results Logic and Measures

3. Validators

4. Constituent Voice

5. Published Evaluation Reports

Page 27: Measuring Success in Patient Advocacy Initiatives

One Element Will Impact Ratings Next Year

Element One: Alignment of Mission, Solicitations and Resources

Page 28: Measuring Success in Patient Advocacy Initiatives

Element Two: Results Logic and Measures

• Is the organization’s causal logic, theory of change, plausible?

• If applicable, is there an indication of how much of the action is required to produce the pre-defined outputs and outcomes?

• Is this logic based on reasonable evidence?• Are there specified measures (indicators) to be collected

and a plan to do so?

Page 29: Measuring Success in Patient Advocacy Initiatives

Element Three: Validators

• Have your charity’s results been vetted by another organization?

• Not every charity and cause area will have a validator. That will not diminish your rating.

Page 30: Measuring Success in Patient Advocacy Initiatives

Element Four: Constituent Voice

• How well does your charity collect and publish feedback from your primary constituents (the people who are meant to be the direct recipients of benefits created by the organization’s actions).

• May not apply to every cause area, but will apply to most.

Page 31: Measuring Success in Patient Advocacy Initiatives

Element Five: Published Evaluation Reports

• Does your charity publish evaluation reports that cover the results of its programs at least every five years?

• Are those reports based on recognized techniques to understand your results?

• Does you charity explain what, if anything, it is changing as a result of the findings in the evaluation report?

Page 32: Measuring Success in Patient Advocacy Initiatives

WHAT WE HAVE LEARNED SO FAR

Out of Approximately 1,000 Human Service Charities Evaluated So Far:

• ~ 70% of Charities have a clearly defined program

• ~ 15% of Charities have a clear definition of how much of an action is required to produce the pre-defined outputs and outcomes

• ~2% of Charities have at least 1 of the other 3 elements of Results Reporting (Validators, Constituent Voice or Published Evaluations)

Page 33: Measuring Success in Patient Advocacy Initiatives

A DREAM THAT YOU CAN PLAY A ROLE IN MAKING A REALITY

RESULTS STATEMENTS,

TODAY TOMORROW?????

Page 34: Measuring Success in Patient Advocacy Initiatives

To Learn More About How We Measure Success:

Page 35: Measuring Success in Patient Advocacy Initiatives

To Learn More In General About Results Measurement:Books /Articles/ Studies

1. The Battle for the Soul of the Nonprofit Sector, Berger, Penna and Goldberg, Philadelphia Social

Innovation Journal

2. Money Well Spent by Paul Brest, et. Al

3. Working Hard and Working Well, by David E. K. Hunter

4. Billions of Drops in Millions of Buckets by Goldberg

5. Leap of Reason by Mario Marino

6. The Nonprofit Outcome Toolbox by Dr. Robert Penna

7. Charity Navigator’s webinar on how to use our site

8. Saving Philanthropy Video

9. Independent Sector’s Charting Impact

10. PerformWell Web Site

Page 36: Measuring Success in Patient Advocacy Initiatives

Email: [email protected]

Blogs:

www.kenscommentary.org

blog.charitynavigator.org

Twitter: kenscommentary

LinkedIn: http://lnkd.in/jKeK5U

Register for Newsletter, Etc.: https://www.charitynavigator.org/index.cfm?bay=my.login

KEEPING IN TOUCH

Page 38: Measuring Success in Patient Advocacy Initiatives

THE CEO FORCE FOR GOOD

Page 39: Measuring Success in Patient Advocacy Initiatives

Utilities (n=18)

Technology (n=25)

Materials (n=16)

Industrials (n=25)

Health Care (n=24)

Financials (n=48)

Energy (n=11)

Consumer Staples (n=20)

Consumer Discretionary (n=26)

Communications (n=10)

1.23%

1.06%

0.98%

0.76%

1.58%

0.96%

0.76%

1.12%

1.25%

1.10%

2013 Industry Giving Comparisons – Median Giving as a % of Pre-Tax Profit

For Pharmas: 16.1%

Page 40: Measuring Success in Patient Advocacy Initiatives

Pharmaceuticals Giving at a Glance, 2013

79%of all companies run a corporate foundation

86%of all companies match

employee gifts

62% of all companies give

internationally

64%of all companies make

non-cash gifts

78% of Pharma

companies

89% of Pharma

companies

100% of Pharma

companies

78% of Pharma

companies

N=261 for full sample. Pharma n=9.

Page 41: Measuring Success in Patient Advocacy Initiatives

A Majority of Companies are Measuring the Societal Value of their Contributions

Very Experienced (5 Years or More)

Moderately Experienced (3-4 Years)

Slightly Experienced (2 Years or Less)

18%

35%

47%

76% Measure Outcomes and/or Im-

pacts

N=160. N=119.

24% Do Not Measure

Outcomes and/or Impacts

Page 42: Measuring Success in Patient Advocacy Initiatives

Nearly a Third of Companies Focused On Outcome/Impact Measurement in a Strategic Philanthropic Program

42

• For companies only evaluating grants larger than a specific threshold, the average threshold was approximately $105,000

• “Other” descriptions include companies that: • Only evaluate foundation grants• Evaluate for a combination of strategic programs, grant size, and cause area(s)• Evaluate for as many grants as possible without any specific direction

Description of Impact Measurement Scope % of Companies

All grants, regardless of grant size 23%

Only grants made for a strategic philanthropic program(s) 31%Only grants larger than a specific threshold 16%Only grants larger than a specific threshold AND made to a specific cause area(s) 12%Only grants made to a specific cause area (or cause areas) 8%

Other 10%

Page 43: Measuring Success in Patient Advocacy Initiatives

How much do companies report spending on evaluation?

43

48%of companies spent

money in 2013 to evaluate grants

5%of 2013 contributions were

dedicated to measuring the outcomes and/or

impacts associated of their grants

Note: Spending money includes contributions to grantees (earmarked for evaluation) and payments to consultants, contractors, and evaluation specialists. N=77.

Note: Median. Includes companies spending money on evaluation. N=37.

Page 44: Measuring Success in Patient Advocacy Initiatives

Companies Use a Variety of Resources for Measurement Activities in Strategic Giving Programs

Internal R

esource

(Entire

ly In-

House)

Consu

lting

Firm(s)

Internal R

esourc

e (Infor

med by E

xterna

l Reso

urces)

Universi

ties

Publicly

Ava

ilable

Metrics

Resear

ch In

stitutio

n(s)

Other

26% 26%

20%15%

11% 9% 7%

N=105.

Page 45: Measuring Success in Patient Advocacy Initiatives

66% of Companies Partner with Grantees to Measure the Outcomes and/or Impacts of their Grants

45

% of Strategic Programs Focused on:

Common impact metrics for Education included:

• Literacy Rates• Graduation Rates• College Readiness Scores• Bachelor’s Degree Attainment

Rates• Racial Disparity Changes• Attendance Rates• Teacher Retention Rates• # of girls/women supported• # of people obtaining access to

technology

Common impact metrics for Health & Social Services

included:

• Emergency Room Visits• Hospital Recidivism• Immunizations Administered• Patient Engagement• Number of children receiving

immunizations• Increased farmer yields• Other donors leveraged for

marginalized community• Improved health resiliency

Common impact metrics for Community & Economic Development included:

• Number of Women Impacted• Credit Score Changes• Net Worth Changes• Number of Loans Administered• Number of Jobs Created• Business Growth Metrics• Business Profit Changes

Education: 41% Health: 26% Econ Dev:16%

Page 46: Measuring Success in Patient Advocacy Initiatives

There’s A Lot Out There To Help!

46

PSI – Impact Magazine Urban Institute: PerformWellHealth and Safety Section

Center for Disease Control and Prevention

Steps CommunitiesCHANGE Communities

RAND Getting to Outcomes 2004ISBN 0-8330-35285-2

Nurse-Family Partnership Theory of Change Logic Model

Foundation CenterTools and Resources for

Assessing Impact

Page 48: Measuring Success in Patient Advocacy Initiatives

Questions & Answers

Page 49: Measuring Success in Patient Advocacy Initiatives

Grants Announcement

• In partnership with a major university specializing in social innovation and metrics

• Call for entries open late summer• NPOs with a healthcare focus and a demonstrated commitment to

metrics reporting eligible • Submit your email address via post-webinar survey for more

information

The Onyx Advancing Innovation in Measuring Advocacy Outcomes Award

Page 50: Measuring Success in Patient Advocacy Initiatives

Thank you!


Top Related