tball acsm lecture 2016 - healthcare pa evaluation methods_linked video
TRANSCRIPT
1
Methods for Evaluating Effectiveness of Promoting Physical Activity in Healthcare
Trever J. Ball, PhD, MPH Senior Outcomes AnalystHealth Promotion & Wellness Institute for Healthcare Delivery Research Intermountain [email protected]
World Congress on Exercise is MedicineMay 31, 2016Boston, MA
2
Background
Intermountain Healthcare• Not-for-profit health system• Based in Salt Lake City, UT• 22 hospitals• 185 clinics
3
4
QI: Role in Driving Evidence Base
Outcomes Research
Program Evaluation
Clinical Effectiveness
Implementation Science
Quasi-Experimental
Intervention Studies
Quality Improvement
Courtesy of L. SavitzInstitute of Medicine (IOM). “Health System Leaders Working Toward High-Value Care through Integration of Care and Research,” The National Academies, Workshop in Brief, June, 2014.
Driving the science of change/innovation…
5
Role of Research at Intermountain
Priorities:
1. High operational utility2. Of interest to clinical investigators3. Of interest to close partners4. Of interest to external investigators/entities
6
Quality Improvement Foundational Leaders
Homer Warner, MD, PhD– Medical informatics founder– 1950s – computer assisted CV
decision support– 1970s – HELP system
developed
Dr. Brent James, MD, MStat– CQI - Standardization of clinical
care through data analysis
7
Institute for Healthcare Delivery Research
Clinical Programs
Data Analysts
Delivery System Science
Data Analysts
Health Promotion & WellnessCommunity Health
ImprovementOutcomes Analyst
8
VP Community Benefit
Community Health
Improvement
Health Promotion &
Wellness
Behavioral Health Reporting
Outcomes Analyst
9
Health Promotion & Wellness
Insurance Members
Patients & Providers
Intermountain Employees Community Comm. Business
Development
10
Physical Activity “Vital Sign” (PAVS)
1. Ball TJ, Joy EA, Gren LH, Shaw JM. Concurrent Validity of a Self-Reported Physical Activity "Vital Sign" Questionnaire With Adult Primary Care Patients. Prev Chronic Dis. 2016;13:E16.
2. Ball TJ, Joy EA, Gren LH, Cunningham R, Shaw JM. Predictive Validity of an Adult Physical Activity "Vital Sign" Recorded in Electronic Health Records. J Phys Act Health. 2015.
3. Ball T, Joy E, Goh T, Hannon J, Gren L, Shaw J. Validity of two brief primary care physical activity questionnaires with accelerometry in clinic staff. Prim Health Care Res Dev. 2014;28:1-9.
4. Greenwood JLJ, Joy EA, Stanford JB. The Physical Activity Vital Sign: A Primary Care Tool to Guide Counseling for Obesity. Journal of Physical Activity & Health. 2010;7(5):571-576.
11
Physical Activity “Vital Sign”
12
13
14
Reasons to Evaluate
• Ensure efforts produce desired results
• Justifies need• Monitor goals• Facilitates accountability
15
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
16
Team Structure
Guidance Team
Project Team
Front Line Workers
Process
Health Promotion & WellnessPatient & Provider Development Team
Lead: Trever Ball, PhD, MPH
Clinic Receptionists MAs Physicians
Scott Hansen, MD Locke Ettinger, PT, PhD Tamara Sheffield, MD Roger Peterson, PT
Laurel Newman, RN Liz Joy MD, MPH Clinic Manager
Direction, control
Participation, ownership
Reports monthly
Fundamental knowledge
17
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
18
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
19
Engage Stakeholders
Describe projects
Focus evaluation
plans
Gather credible evidence
Justify conclusions
Ensure use and
disseminate
RE-AIM Framework: www.re-aim.org CDC Evaluation Framework. http://www.cdc.gov/eval/framework/index.htm
StandardsUtility
FeasibilityProprietyAccuracy
RE-AIM
20
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
21
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
22
Mission
Increase proportion of primary care visits for which physical activity levels of patients are asked
advised
23
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
24
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
25
Logic Modeling (“Outcomes Roadmap”)
26
Purposes of Logic Models
• Graphic depiction of relationship between activities and intended outcomes.
• Helps focus evaluation on most central and important questions.
CDC Evaluation Framework. http://www.cdc.gov/eval/framework/index.htm
27
Activities Outcomes
Early Later
28
Activities Outcomes
Early Later
Contact with Clinical Program Leads
Identify Champions Identify areas that health promotion fits into their workflow
Engage Intermountain Clinicians with tools Engage Clinician in their own health:
Nursing Study (Liz) LiVe Well for Employees (see Employee) Design engagement opportunities for physicians, nurses and other clinicians to improve their personal wellness habits.
Execute care processes for healthy behaviors
PAVS DPP Hypertension CHF – With CV Clinical Program Weigh to Health Falls – Stepping On (this may be in CB community) Potential Partners that are being considered
DATU Omada Takada
Determine a Behavior Change Framework Carry out research and development on “best practices” for achieving healthy lifestyles. Study and expand application of integrative wellness services.
Identify CP LiVe Well champion(s) Identify Health Pathway(s) to include tools from the Lifestyle CPM Build healthy lifestyles into High Blood Pressure CPM Nursing intern student study results (Liz) Health pathways include assessment and management of prevention and wellness Meet with each CP to prioritize efforts aimed at prevention and wellness Clinical wellness champions identified and trained on LWM CPM in relevant CPs Dissemination & implementation of DPP to relevant CPs Priorities identified within each CP for implementation in 2015-2016 # of CPs with identified and trained clinical champion # of CP that have implemented
Orient and engage the champion to LiVe Well programs and tools Education event, or online resource for clinicians Measure patient participation in exercise a d wt loss. BMI CPM include assessment and management of prevention and wellness strategies Number of patients not converting to Diabetes Number of patients not progressing to CHF Number of patients increasing or maintaining PA. Number of patients with controlled BP Obesity Rates in adults and children PA rates in Children and adults Other population health measures
29
Activities Outputs Outcomes
Early Later
Early Later
Contact with Clinical Program Leads
Identify Champions
Identify areas that health promotion fits into their workflow Engage Intermountain Clinicians with tools
Identify clinical wellness champions and train on LWM CPM in relevant CPs Meet with each CP to prioritize efforts aimed at prevention and wellness
Identify Health Pathway(s) to include tools from the Lifestyle CPM Build healthy lifestyles into High Blood Pressure CPM Dissemination & implementation of DPP to relevant CPs Identify priorities within each CP for implementation in 2015-2016 Orient and engage the champion to LiVe Well programs and tools
Education event, or online resource for clinicians
Execute care processes for healthy behaviors
PAVS
DPP
Hypertension
CPM include assessment and management of prevention and wellness strategies Patient participation in exercise and wt loss # of CPs with identified and trained clinical champion # of CP that have implemented operational workflow to refer patients into DPP # of new CPMs with measureable and evidence based wellness and prevention strategies
Culture of Wellness Health pathways include assessment and management of prevention and wellness # of patients adhering to recommendation for CHF lifestyle Clinical Values Lifestyle measures Rate of physicians recording and counseling on PAVS Rate of physicians counseling on BP compliance that includes lifestyle
BMI Patient rate not converting to Diabetes Patient rate not progressing to CHF Patient rate increasing or maintaining PA. Patients rate with controlled BP Obesity rates in adults and children PA rates in adults and children Other population health measures Lifestyle measures
30
ReachAdoptionImplementationMaintenance Effectiveness
Glasgow, Vogt et al. 1999; Gaglio, Shoup et al. 2013; Yeh, Chang et al. 2014; Thomas, Krevers et al. 2015
RE-AIMFramework
31
Example Central Evaluation Question:
Do efforts to adopt and implement care
processes related to the Physical Activity Vital Sign improve patient
physical activity?
32
Example Central Evaluation Question:
Do efforts to adopt and implement care
processes related to the Physical Activity Vital Sign improve patient
physical activity?
33
34
Detailed health promotion
strategies for patients and
providers
35
Physical Activity “Value Chain”
Activities Outcomes
Adapted from: Program Measurement & Evaluation Guide: Core Metrics for Employee Health Management, 2015. HERO & PHA
36
Physical Activity “Value Chain”
Activities Outcomes
Ask AssessAdvise
AssistArrange
PA Behavior
Clinical Outcomes
Financial Outcomes
Adapted from: Program Measurement & Evaluation Guide: Core Metrics for Employee Health Management, 2015. HERO & PHA
37
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
38
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
39
Aim
Increase the proportion of patient visits at the (choose clinic) that have mins/week of physical activity recorded from __% to __% by (date).
40
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
41
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
42
Quality Improvement Tools
• RE-AIM framework• Cause & effect diagrams• Driver diagrams• Pareto charts• Flow charts/logic modeling• Run charts• SPC charts
43
Why is physical activity not asked and recorded?
Staff Factors
Health ITOperations
Patient Factors
Healthy staff
Care Manager
MAs
Clinicians
Research Findings
Age disparity
BMI disparity
Data Architecture
Decision Support
New EHR integration
Activating team
Educating teams benefits of PA
44
Odds of Treatment for Physical Activity by Patient Demographic
Female Male
18-29
30-39
40-49
50-64 >6
4<1
8.5
18.5-
24.9
25.0-
29.9
30.0-
34.9
35.0-
39.9
≥40.0
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
Standard PAVS Only PA Advice Only PAVS + PA Advice
n=257,869
45
Odds of Treatment for Physical Activity by Patient Demographic
Female Male
18-29
30-39
40-49
50-64 >6
4<1
8.5
18.5-
24.9
25.0-
29.9
30.0-
34.9
35.0-
39.9
≥40.0
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
Standard PAVS Only PA Advice Only PAVS + PA Advice
n=257,869
46
Odds of Treatment for Physical Activity by Patient Demographic
Female Male
18-29
30-39
40-49
50-64 >6
4<1
8.5
18.5-
24.9
25.0-
29.9
30.0-
34.9
35.0-
39.9
≥40.0
0.00
1.00
2.00
3.00
4.00
5.00
6.00
7.00
8.00
1.33
3.30
5.23
7.24
1.84 1.831.32
2.58
Standard PAVS Only PA Advice Only PAVS + PA Advice
n=257,869
47
Why is physical activity not asked and recorded?
Staff Factors
Health ITOperations
Patient Factors
Self-complete formsHealthy staff
Care Manager
MAs
Clinicians
Research Findings
Age disparity
BMI disparity
Tools
Clinic support for process
Assessment & operations of clinic education needs
Support for feedback/reporting
Data Architecture
Decision Support
New EHR integration
Explanation of terms
Activating team
Educating teams benefits of PA
48
Quality Improvement Tools
• RE-AIM framework• Cause & effect diagrams• Driver diagrams• Pareto charts• Flow charts/logic modeling• Run charts• SPC charts
49
Quality Improvement Tools
• RE-AIM framework• Cause & effect diagrams• Driver diagrams• Pareto charts• Flow charts/logic modeling• Run charts• SPC charts
50
Aim Key Drivers Strategies
Increase proportion of patient adult visits at the _____ clinic that have PA recorded from __ % to __ % by [date]
Clinical Champion
Clinical excellence goal part of value recognition program
Health IT
Academic detailing our LWM CPM; presentations
Maintain work with PCCP
Work with new EHR team to ensure PAVS is there and easy to complete
51
Framework Aim Key Drivers Strategies
Ask
Increase the proportion of patient visits at the _____ (clinic/region) that have physical activity recorded (minutes/week) from __% to __% by (date).
Clinical championAcademic detailing our LWM CPMs to cliniciansPresentations
Clinical excellence goal Maintain work with PCCPWork with IMG
Health ITWork with iCentra team to ensure PAVS is there, is easy to complete, and is visible to clinicians
Assess & Advise
Increase the proportion of patient visits at the ______ (clinic/region) that have physical activity advice recorded from __% to __% by (date).
iCentraWork with iCentra development team to make sure product is placed well for iCentra workflow for clinician
Clinical excellence goal Maintain work with PCCP Work with IMG
iCentra +Manipulate patient activity graph to serve as a prompt for clinician, including if patient meets or does not meet PA guidelines
Assist & Arrange
Increase the proportion of SH MA members that access the fitness benefit from __% to __% by (date).
Referral networkSee Greenville, SC model (EIM slides)Member educationDevelop referral network
Increase the reimbursement/member for the fitness benefit (mean & median) from $__ to $__ by (date).
Payer factors for referral network
Member education
Develop referral network
52
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
53
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
54
DATA ANALYTICS 101
55
56
Intermountain Healthcare EDW
57
Query & Track Outcomes
TrackAnalyze
Disseminate
Interpret
Graphical presentation
Stratifications
VisualizeWrite & QA
query
QueryAgree on
evaluation question
Identify data elements
Consult
58
Query & Track OutcomesPatient_id
Patient_sex
Appt_st_dt
Ob_NCID
Encounter_num
001002003004005
Facility_ncid
Encounter_facility_id
Encounter_num
Primary_provider
Unit_num
Unit_num
Encounter_facility_id
Encounter_num
Primary_provider
Unit_num
59
Query & Track Outcomes
Patient_id
APPOINTMENT_START_DTTM
APPOINTMENT_STATUS_NCID
APPOINTMENT_STOP_DTTM
APPOINTMENT_TYPE_NCID
ENCOUNTER_FACILITY_NCID
ENCOUNTER_NUMBER
LAST_UPDATE_DT
MED_RECORD_FACILITY_NCID
MEDICAL_RECORD_NUMBER NOTE
PATIENT_BIRTHDATE
PATIENT_HOME_PHONE
PATIENT_NAME
PATIENT_SEX
PATIENT_WORK_PHONE
PROVIDER_NAME
PROVIDER_NCID
REASON
REFERRING_PROVIDER_NAME
REFERRING_PROVIDER_NCID
RESPONSIBLE_PARTY
SCHEDULED_DTTM
SCHEDULED_FACILITY_NCID
SCHEDULED_LOCATION_NCID
SCHEDULER_NCID
UNIT_NUMBER
00001
00002
00003
00004
00005
00006
00007
00008
00009
00010
60
Query & Track Outcomes
• 25,422 tables• 2,197,993,362 Queries/month• 157,797,141,557 Rows• 467,789 Columns
Courtesy of Dallin Rogers, Intermountain EDW Team
62
Query & Track Outcomes
64
Physical Activity “Value Chain”
Activities Outcomes
Ask AssessAdvise
AssistArrange
PA Behavior
Clinical Outcomes
Financial Outcomes
Adapted from: Program Measurement & Evaluation Guide: Core Metrics for Employee Health Management, 2015. HERO & PHA
65
Example Central Evaluation Question:
Do efforts to adopt and implement care
processes related to the Physical Activity Vital Sign improve patient
physical activity?
66
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
67
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
68
QI: Role in Driving Evidence Base
Outcomes Research
Program Evaluation
Clinical Effectiveness
Implementation Science
Quasi-Experimental
Intervention Studies
Quality Improvement
Courtesy of L. SavitzInstitute of Medicine (IOM). “Health System Leaders Working Toward High-Value Care through Integration of Care and Research,” The National Academies, Workshop in Brief, June, 2014.
Driving the science of change/innovation…
69
QI: Role in Driving Evidence Base
Outcomes Research
Program Evaluation
Clinical Effectiveness
Implementation Science
Quasi-Experimental
Intervention Studies
Quality Improvement
Courtesy of L. SavitzInstitute of Medicine (IOM). “Health System Leaders Working Toward High-Value Care through Integration of Care and Research,” The National Academies, Workshop in Brief, June, 2014.
Driving the science of change/innovation…
70
QI: Role in Driving Evidence Base
Outcomes Research
Program Evaluation
Clinical Effectiveness
Implementation Science
Quasi-Experimental
Intervention Studies
Quality Improvement
Courtesy of L. SavitzInstitute of Medicine (IOM). “Health System Leaders Working Toward High-Value Care through Integration of Care and Research,” The National Academies, Workshop in Brief, June, 2014.
Driving the science of change/innovation…
71
Designs & Methods
• Knowing what works requires that we design a study and use those methods that allow us to answer key questions.
• Design Approaches:– Intervention– Quasi-experimental– Observational– Time Series
• Methods:– Quantitative– Qualitative– Mixed
Courtesy of L. Savitz
72
Design: observational
Are there differences in level of treatment for physical inactivity by patient demographics?
• Logistic regression
𝑦 (𝑃𝐴𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 𝑙𝑒𝑣𝑒𝑙 )=𝛽0+𝛾 𝐹 ( h𝑑𝑒𝑚𝑜𝑔𝑟𝑎𝑝 𝑖𝑐𝑠 )+… 𝛽𝑘 ( covariates )+ μ
73
Design: observational
How much impact on patients do levels of treatment for physical inactivity have?
• Mixed effects regression
𝑦 (𝑃𝐴 ; h h𝑒𝑎𝑙𝑡 𝑜𝑢𝑡𝑐𝑜𝑚𝑒𝑠 )=𝛽0𝑖+𝛾 𝐹 (𝑃𝐴𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 )+𝛽𝑖𝑗 (𝑟𝑎𝑛𝑑𝑜𝑚)… 𝛽𝑖𝑗 ( 𝑓𝑖𝑥𝑒𝑑)+𝜇𝑖𝑗
74
Method: mixed
What are behavioral and other differences in providers using and not using the PAVS?
• Descriptive statistics• Survey constructs of Theory of Planned Behavior
Adapted from: https://commons.wikimedia.org/w/index.php?curid=42261999
Attitude
Subjective Norm
Perceived Behavioral Control
Intention Behavior
75
Design: quasi-experimental
Nonrandomized observational comparisons
Example:Does a program designed to increase ascertaining and recording patients’ self-reported physical activity as part of the initial visit intake improve glycemic control among patients with diabetes?
Grant RW. (2014). J Gen Intern Med, 2014;29(2):341-348
76
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4EVS Clinic HbA1c Change Non-EVS Clinic HbA1c Change
Per
cent
Cha
nge
Changes in HbA1c Over Time Among Patients with Diabetes Attending EVS Versus Non-EVS
Medical Centers
Adapted from: Grant RW. (2014). J Gen Intern Med, 2014;29(2):341-348 EVS: Exercise Vital Sign
Al patients w/diabetes (n=70,083)
Patients w/HbA1c>7 (n=30,487)
Patients w/HbA1c>8 (n=13,440)
77
Series1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4EVS Clinic HbA1c Change Non-EVS Clinic HbA1c Change
Per
cent
Cha
nge
Changes in HbA1c Over Time Among Patients with Diabetes Attending EVS Versus Non-EVS
Medical Centers
Adapted from: Grant RW. (2014). J Gen Intern Med, 2014;29(2):341-348 EVS: Exercise Vital Sign
Al patients w/diabetes (n=70,083)
Patients w/HbA1c>7 (n=30,487)
Patients w/HbA1c>8 (n=13,440)
78
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation
framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
79
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation
framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
80
Steps for PA Evaluation & Quality Improvement
1. Organize Team2. Identify evaluation
framework/s3. Mission Statement4. Conceptual models5. Aim Statement6. QI modeling7. Query & track outcomes8. Design research
Thank youQuestions?