plan, evaluate and improve your health literacy interventions and activities
DESCRIPTION
Plan, Evaluate and Improve Your Health Literacy Interventions and Activities. Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant April 10, 2013. Objectives. After attending the session, participants will be able to: Conceptualize a health literacy program evaluation - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/1.jpg)
Plan, Evaluate and Improve Your Health Literacy Interventions and Activities
Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant
April 10, 2013
![Page 2: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/2.jpg)
Objectives
• After attending the session, participants will be able to:– Conceptualize a health literacy program evaluation
– Write measurable health literacy outcome objectives
– Identify indicators to assess both process and outcomes
– Identify user-friendly evaluation resources for future use
Olney, C. and Barnes, Susan (2006). Planning and Evaluating Health Information Outreach. NN/LM Outreach Evaluation Resource Center. Seattle, WA.
![Page 3: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/3.jpg)
Ten Attributes of Health Literate Health Care Organizations
–
Brach C, Keller D, Hernandez LM, et al. Ten Attributes of Health Literate Health Care Organizations. Washington DC: National Academy of Sciences. 2012
![Page 4: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/4.jpg)
Case Study
Providence, RI
Dorcas Place Adult and Family Learning Center Computer Lab
Western Maine
Auburn Public Library Computer Training Class
![Page 5: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/5.jpg)
Research vs. Evaluation
• Research: To draw causal links between observed phenomena and to add to the knowledge base of the field
• Evaluation: To demonstrate effectiveness, and determine what works and what does not for the purpose of program improvement
Boulmetis, J and Dutwin, P (2005). The ABCs of Evaluation. Jossey-Bass. San Francisco, CA
![Page 6: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/6.jpg)
Health Literacy Program / Evaluation Planning
• Step 1: Know Your Audience
• Step 2: Identify Goals and Objectives
• Step 3: Develop Program (and evaluation) Plan
• Step 4: Implement and Evaluate
• Step 5: Apply and Share Results
![Page 7: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/7.jpg)
Step 1: Know Your Audience
• Start with an Assessment – Identify needs in context – Guide your program – Build on already existing capacity– Identify champions within your organization
• Methods– Informal – discussion, observation– Formal – Key informant interviews, focused group
discussions, survey
![Page 8: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/8.jpg)
Attribute 1:
In a Health Literate Organization leadership makes health literacy integral to its mission,
structure, and operations. –
![Page 9: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/9.jpg)
Case Study
• Methodology: Leaders at community-based organizations interviewed. Semi-structured interview protocol designed to explore community confidence in using the Internet to find health information and local resources to support access. All interviews conducted in person. Data were coded and analyzed into themes.
• Key Findings:• Maine has a growing senior population and community-based organizations
support public health efforts. Public schools, adult education, and libraries offer computer access and support (n=9).
• Latinos get their health information from sources they know and trust, including the radio. A network of minority health programs offer computers access, training and support (n=10).
![Page 10: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/10.jpg)
Step 2: Identify Goals & Objectives
• Goal -- Why you are doing what you are doing and for whom
• Example: – Increase use of health literacy interventions
– Improve community access to reliable online health information
![Page 11: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/11.jpg)
Attribute 2:
A Health Literate Organization integrates health literacy into planning, evaluation,
patient safety, and quality improvement. –
![Page 12: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/12.jpg)
Case Study
Focused Health Information Outreach Project
Goal: Increase access to health information among communities experiencing health disparities
National Network of Libraries of Medicine – New England Region (NN/LM NER) http://nnlm.gov/ner/
![Page 13: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/13.jpg)
Step 3: Develop Program(and Evaluation) Plan
Logic Model: A planning tool connecting activities to outcomes
If you have these resources
Resources Activities/Outputs Outcomes
and conduct these activities
you will get these outcomes
![Page 14: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/14.jpg)
Why a Logic Model?
• Provides a framework for planning• Helps organize your thoughts• Clarifies intended outcomes• Links activities to outcomes • Acts as a communication tool
ActivitiesResources
OutputsOutcomes
![Page 15: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/15.jpg)
Attribute 3:
A Health Literate Organization prepares its workforce.
![Page 16: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/16.jpg)
Case StudyGoal: Increase access to health information among communities experiencing health disparities
Resources Activities Outputs Outcomes
Increase knowledge of NLM resources
Increase use of NLM resources
Increase community experts to help others
Number of trainings held
Number trained
Types of materials developed
Number of materials distributed
Develop training materials
Distribute materials at community events
Consultant
Outreach coordinators
NLM materials
NLM funding
Conduct trainings
Working student
Librarians in communities
![Page 17: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/17.jpg)
Activity
• Choose a goal (or use your own)– To increase staff awareness and use of health literacy interventions
– To improve community access to reliable online health information
– To improve health information literacy skills among patients and their families
• Complete the logic model worksheet - identify outcomes, activities, outputs, and resources
![Page 18: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/18.jpg)
Attribute 4:
A Health Literate Organization involves the population served
in evaluation of health information and services.
![Page 19: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/19.jpg)
Case Study
• Community Assessment– Key informant interviews
• Program– Tailored efforts based on findings – Community-based partnerships
• Evaluation – Outcome assessment: pre-post training evaluations,
follow-up assessment– Process assessment: story-based evaluation,
community partner exit interviews
Western Maine (rural)
Providence, RI(Latino)
![Page 20: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/20.jpg)
Evaluation Plan
• Process– To determine what worked and what didn’t– For program improvement
• Outcome– To demonstrate effectiveness, value, and influence– For comparison purposes, to determine which programs
are more effective
![Page 21: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/21.jpg)
Attribute 5:
A Health Literate Organization meets the needs of the populations
and avoids stigmatization.
![Page 22: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/22.jpg)
Process Assessment
• Methods – Activity logs– Teacher feedback forms– Focused group discussion– Interviews
![Page 23: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/23.jpg)
Outcome Assessment
• Methods– Existing records– Pre/post tests– Follow-up surveys– Observation – Interview – Story-based
![Page 24: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/24.jpg)
Attribute 6:
In a Health Literate Organization providers and staff use health literacy strategies in verbal communication
and confirm understanding.
![Page 25: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/25.jpg)
Case Study
Outcomes (what you want) Indicators (how you will know)
Increased knowledge of NLM resources
Increased use of NLM resources
Increased community experts to help others
Will report use two weeks later of follow up
Will name resources on post-surveys
Will document helping others on story-based evaluation forms
Outcomes & Indicators
![Page 26: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/26.jpg)
Measurable Objectives
Four components of a measurable objective1. Condition
• Following the training…• After watching the video…
2. Who• …the learner…• …the patient…• …the caregiver
![Page 27: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/27.jpg)
Measurable Objectives
3. Performance (knowledge, attitudes, behavior)
• …will name…• … will agree…• … will use…
4. Criterion (quality/quantity) • …5 out of 6…• … at least 2…• … the correct way…
![Page 28: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/28.jpg)
Characteristics of a Measurable Objective
• Is audience focused, not intervention focused (the participant will…)
• Includes action verbs
• Targets one expectation
• Includes observable criteria for evaluation
• Matches goals to activities/strategies
![Page 29: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/29.jpg)
Attribute 7:
In a Health Literate Organization consumers have easy access to health
information and services and navigation assistance.
![Page 30: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/30.jpg)
Bloom’s Taxonomy
• Cognitive – explain, recall, list, predict, select, describe, define
• Affective – agree, avoid, help, join
• Psychomotor – use, develop, design, repair, taste, use, move
![Page 31: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/31.jpg)
Verbs to Avoid
KnowComprehendUnderstandAppreciate
StudyGain knowledge of
Learn
![Page 32: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/32.jpg)
Attribute 8:
A Health Literate Organization develops plain language materials that
are easy to understand and act on.
![Page 33: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/33.jpg)
Case Study
Condition Who Performance Criterion
Immediately after training
participants will name
Two weeks after training participants will report using
Three months training
participants
NLM resources on post-survey
NLM resources on follow up interview
will write a story Describing how they helped others find information on story-based evaluation form
Writing Measurable Objectives
![Page 34: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/34.jpg)
Activity
• Choose an outcome (or use your own)– Increase confidence in using plain language/teach-back– Increased awareness of available online health
information resources– Increased interaction with diabetes educators
• Complete the writing measurable objectives worksheet - identify condition, audience, performance, and criterion
![Page 35: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/35.jpg)
Step 4: Implement and Evaluate
• Expect the Unexpected
– Be flexible
– Adjust to the environment
– Address learners’ needs
– Tweak your notion of success
![Page 36: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/36.jpg)
Attribute 9:
A Health Literate Organization addresses health literacy in high-risk
situations, including transitions and about medicines.
![Page 37: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/37.jpg)
Case Example
Rhode Island: Pre/Post: Confidence in Ability to Find Health Information on the Internet
Pre N= 520
Post N=503
Maine: Pre/Post: Ever Used NLM Resources
Pre N=72
Post N=55
![Page 38: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/38.jpg)
Case Example
• Successes– Training participants shared resources with friends and
neighbors– Partners will continue to promote and distribute resources– Providers used resources to find accurate information to
use in their work
• Challenges – Pre- and post-evaluations were hard to administer
![Page 39: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/39.jpg)
Step 5: Apply and Share Results
• Apply Lessons Learned
– Make changes based on what you learned
– Replicate with similar or different audiences
• Share Your Findings
– Partners
– Colleagues
![Page 40: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/40.jpg)
Attribute 10:
At a Health Literate Organization they clearly communicates about health
plans and health care costs.
![Page 41: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/41.jpg)
Evaluation Resources
• Planning and Evaluating Health Information Outreach Projects
• Measuring the Difference: Guide to Planning and Evaluating Health Information Outreach
http://nnlm.gov/evaluation/guide/index.html
![Page 42: Plan, Evaluate and Improve Your Health Literacy Interventions and Activities](https://reader035.vdocuments.us/reader035/viewer/2022070404/56813ae7550346895da34779/html5/thumbnails/42.jpg)
Questions & Comments
Sabrina Kurtz-Rossi, M.Ed.Principal | Health Literacy Consultant [email protected]
Kurtz-Rossi & Associates Health*Literacy*Education*Evaluationwww.kurtz-rossi.com
Member, Clear Language Groupwww.clearlanguagegroup.com
Health Literacy Leadership Institute, June 10-14, 2013 http://healthliteracyleadreship.com