b1 - embedding evidence in context: approaches used by an embedded research and evaluation unit -...
DESCRIPTION
TRANSCRIPT
Embedding Evidence in Context:
Approaches Used by an Embedded
Research and Evaluation UnitColleen J. Metge, BSc(Pharm), PhD
Director, Research & Evaluation Unit
Winnipeg Regional Health Authority
Evidence in Context
Flower class corvettes left from St.
John’s NFLD 1941-1945 during
the Battle of the Atlantic escorting
25,000 merchant ships during
WW2.
They were ill-suited as anti-
submarine, open-ocean vessels
but were easy to build. They were
largely served by men from the
Prairies.
HMCS Trillium on the open seas with an American crew
(row boat) returning to their ship after a U-boat attack
K141:
HMCS Summerside
Creds for the Session
The Winnipeg Regional Health Authority’s
(WRHA) Research & Evaluation Unit:
• Produces HTAs on request
• Is an embedded unit established for Mode
2 (II) knowledge production
• It’s key value: collaborative research &
evaluation in specific health care contexts
The Forks at Sunset, Winnipeg
What is the WRHA?
• comprised of six hospitals (two tertiary), 35
personal care homes, and 12 community
areas
Hospital T
Hospital C
Communities
CEO
VP VP VP VP
Programs Programs Programs Programs
The art of policy and practice involves
reconciling the strength of published evidence
with its relevance based on the experience of
those who know, live and work with the
problem that the evidence is designed to
solve.Lawrence W. Green, 2001
20 January 2009 8
Mode 2 (II) Knowledge Production
• Mode 1 (I): traditional, academic norms of
scholarship in the disciplines & institutions
in which researchers work
• Mode 2 (II): non-hierarchical relationships
with stakeholders to collaborate on a
research issue situated in a specific health
care context
Mode 1 Mode 2
Academic context
Disciplinary
Homogeneity
Autonomy
Traditional quality
control (peer review)
Context of application
Transdisciplinary
Heterogeneity
Reflexivity / social
accountability
Novel quality control
Knowledge Production
Gibbons M et al. (1994) New Production of Knowledge
Nowotny H et al. (2001) Re-thinking Science: Knowledge & the Public in an Age of Uncertainty
Mode 2 research includes a wider,
more temporary and heterogeneous
set of practitioners, collaborating on
a problem defined in a specific and
localized context
Gibbons 2005
…for those who have never heard of Mode 2
knowledge production, “think” integrated KT and the
KTA or Knowledge to Action cycle
12
and, we are back to discussing how
we embed evidence in context
WRHA’s Research & Evaluation Unit’s
experience, including their expertise in
critical appraisal, proposal development and
evaluation and knowledge translation
activities.
em·bed ( ěm-běd ) v. em·bed·ded also v.tr.
1. To fix firmly in a surrounding mass: embed a post
in concrete; fossils embedded in shale.
2. To enclose snugly or firmly.
3. To cause to be an integral part of a surrounding
whole:
4. To assign (a journalist) to travel with a military unit
during an armed conflict.
WRHA’s Research & Evaluation Unit
Critical Literature Synthesis
ResearchEvaluation
Capacity
Building
WRHA
Board & Senior
Management
Knowledge
Translation
(External)CHA
Hypotheses generated
Ongoing scan of evidence
important to SNR MGT
Knowledge Translation
(internal)
Examples:
• MH-CRC
• BGSC
• CareLinks
• ACSS
• Pre-hab
Leverage
Funding
Opportunities
WRHA’s
Research & Evaluation
Unit: A functionalapproach (2009)
The most precious gift we can offer
others is our presence
Thich Nhat Hanh
WRHA R&E TEAM members
Research Credentials
Disciplinary Area
Liping Zhang MD, MSc Data analystStatistics
Sara Kreindler D.Phil. (Oxford)Rhodes Scholar
Social PsychologyResearcher/KT/KS
Ashley StruthersShannon Winters
MSc (OT)MSc BA (Honors)
EvaluationKnowledge Synthesis
Paul Beaudin MSc PhDSpeech Pathology
QualitativeQuantitative
Catherine Charette PhDMultidisciplinary
QualitativeEvaluation
Yang Cui MSc (Econ)Doctoral candidate
Data analystEconomics
Sunita Bapuji RN, MSc(Nursing) QualitativeKnowledge Synthesis
Ola Norrie BSc(Pharm), MSc, PhD(Economics)
QuantitativeSurvey Developer
Xibiao Ye MD, PhD(Epidemiology)
QuantitativeKnowledge Synthesis
Critical Appraisal
& Synthesis of
Evidence
Community
Health
Assessment
Research Evaluation Capacity-
building
Consultations
Evidence scans (ongoing):
Physician Assistants
(SB/SW/CJM)
Focused Report
**Promoting Health Equity
(XY/PB/CJM)
Predictive Hospitalization
Using MCHP Remote Access
System (YC/XY/CJM)
Birth Centre
**Consulting (SW/CJM)
MPH Students:
Tracey Dowson (Bell Hotel
+ Food Access) CC/CJM
Enhancing Collaborative Pt-
Centred Care
(WCIHC: SK/CJM)
eMental Health Knowledge
Synthesis (also research)
CIHR funded (XY +++)
Community Engagement
(meeting with CADs)
Producing Profiles (ON)
Patient-flow initiatives
(SK/CJM)dia with pregnant
women who smoke (HSFM)
MACHs (Prenatal
Connect)
Needs Assessment &
Evaluation (SW/CJM
Forecasting, Projections &
Predictive Modelling Group
CJM/XY/YC/LZ
Free First Visit (LZ)
Acute Care Surgical Services:
Cochrane protocol developed &
underway
**Promoting Health Equity
Describing the Problem (XY,LZ,YC,CJM)
Recommendation Task Team (PB)
Communication
HSFM: ENCOURAGE
project (T.Duhamel)
CJM
Cardiac Services:
Developing projection
models for future service
use (XY/CJM)
SMD Peer-led groups integrated
with the HC system (AS)
Crowdsourcing (CJM) CHA-Indicator Review
Committee (XY/CJM)
PATHS:
Teen pregnancy(CC/CJM)
Suicide
Advanced Care Planning
Dignity in Care
(PB/CC/SB)
Briefing Note advisory
function (see PBMA)
CJM
After Hours Access
(Outpatient Tx of Latent & Active
TB (ON)
evidence-review)
CHAN: Change in format of
CHA (XY/CJM/CC)
Bell Hotel Supportive
Housing Project
(CC/CJM)
Web-site development
(KC/CJM)
Latent TB data collection issues
in 2-WRHA clinics
- ACCESS for decision support
(ON/CJM)
Patient-reported Outcome
Measures (PROMs) CJM “Silos”
paper in revision
CHAN: Evaluation committee
(CC)
CIHR application:Est Effect of
Neighbor Deprivation on Mental
Health Disorder Disparity
(Child/Adolesc):
Life-course Epidemiology
XYe, C.Metge (resubmit?)
Orthopedic Surgery
Consolidation
SK/LZ/CJM
Priority setting
Program-based marginal
analysis (PBMA)
CJM (possible Unit in
future)
Integration of Tuberculosis
Services &
TB Services Stakeholder
EngagementCHAN Engagement processes
(CJM)
P.I.E.C.E.S. Evaluation
(SB/SW)
Social Identity & Physician
Participation in Primary Care (SK)
CHA-Advisory Committee
(reconvene Fall 2012)
Lawson Fdn: Pre-cardiac surg
diabetes care (CJM/ON)
PPHU Evaluation Planning
(SB/PB/CJM)
Birth projections for
Women’s program
Patient-centred care discourse
document
SK
RLEI: Lang/Ethnic Ques
+ language/ethnicity Indicators
(development & evaluation)
CJM/CC
MMF Regional Adaptation
Strategy (PB)
Community Mental Health
Crisis Response Centre
(CRC)
CC/ON/CJM
Happy Child Happy Smile
(LZ)
Ante-natal Home care program
Food Security Profile Downtown
Community Area (TD; CC)
Cancer Patient Journey (WRHA/CCMB)
CJM
Regional Research Policy Review
(PB)
Rehab (AS)
CIHR OT Instrument Devl
French Language services
(PB)
CCSD Census Data
Consortium (LZ; CC)
PATHS: Case study evaluation of interventions for child health
with an equity lens (CJM/CC)
FSAS-ED(OT: linked to ER
projects)
• Creating unity
Creating a group mandala is a unifying
experience in which people can
express themselves individually within
a unified structure.
Bamboo forest on the way to
Waimoku Falls near Hana, Maui,
Hawaii
Outline of eight steps informing Knowledge to Action evidence summary approach knowledge
users. Khangura et al. Systematic Reviews 2012, 1:10
Rapid Reviews: Evidence Summaries
Knowledge to
Action step
Task
Step 1
Step 2
Step 3
Step 4
Step 5
Step 6
Step 7
Step 8
Needs assessment
Question development and refinement
Proposal development and approval
Systematic literature search
Screening and selection of studies
Narrative synthesis of included studies
Report production
Ongoing follow-up and dialogue with knowledge users
Pulmonary Rehabilitation (PR)
Evidence Scan
What is the effectiveness of outpatient & community-based
pulmonary rehabilitation (PR) programs for persons with chronic
obstructive pulmonary disease (COPD)?
SUMMARY
PR is an effective intervention for patients with COPD and improves
exercise tolerance, health related quality of life and symptoms of
dyspnea. There is also some evidence that PR is cost-effective and
reduces health care utilization. However, the benefits of PR tend to
diminish over time.
BACKGROUNDPulmonary rehabilitation has been defined as “a multidisciplinary program of care
for patients with chronic respiratory impairment that is individually tailored and
designed to optimize …
My friend, drop all your
preconceived and fixed ideas and
be neutral. Do you know why this
cup is useful? Because it is empty.
Bruce Lee
Healthcare Quarterly 2009 v12(2)
Br Med Bull 2010
95: 7-32.
Critical Literature Synthesis
ResearchEvaluation
Capacity
Building
WRHA
Board & Senior
Management
Knowledge
Translation
(External)CHA
Hypotheses generated
Ongoing scan of evidence
important to SNR MGT
Knowledge Translation
(internal)
Examples:
• MH-CRC
• BGSC
• CareLinks
• ACSS
• Pre-hab
Leverage
Funding
Opportunities
WRHA’s
Research & Evaluation
Unit: A functionalapproach (2009)
A planning tool for the
WRHA
Being able to
compare
Critical Literature Synthesis
ResearchEvaluation
Capacity
Building
WRHA
Board & Senior
Management
Knowledge
Translation
(External)CHA
Hypotheses generated
Ongoing scan of evidence
important to SNR MGT
Knowledge Translation
(internal)
Examples:
• MH-CRC
• BGSC
• CareLinks
• ACSS
• Pre-hab
Leverage
Funding
Opportunities
WRHA’s
Research & Evaluation
Unit: A functionalapproach (2009)
Strategies for Patient-oriented
Research (SPOR) Networks
• a collaboration of patients, health service providers,
policy/decision makers and health researchers across
Canada (CIHR)
– Conduct research that produces information that responds to the
needs of patients
• TWO foci at the moment:
– Patient-oriented network in adolescent and youth mental health
• KS (CIHR-funded):E-mental health in youth and adolescents
(X.Ye et al.) …a start
– Community-based Primary Health Care
• More research in this area in Manitoba would support our current
and changing environment
George & Fay Yee Centre for Healthcare Innovation lead by
Dr. Terry Klassen: Vision for a SUPPORT Unit (to support SPOR)
Critical Literature Synthesis
ResearchEvaluation
Capacity
Building
WRHA
Board & Senior
Management
Knowledge
Translation
(External)CHA
Hypotheses generated
Ongoing scan of evidence
important to SNR MGT
Knowledge Translation
(internal)
Examples:
• MH-CRC
• BGSC
• CareLinks
• ACSS
• Pre-hab
Leverage
Funding
Opportunities
WRHA’s
Research & Evaluation
Unit: A functionalapproach (2009)
Importance of Evaluation
• Evaluation helps to identify, describe, and understand the
context in which an initiative is implemented in order to
understand (and, sustain) outcomes
• Take a “realist approach”…one that is
1. Comprehensive in identifying how, and for whom, a change
in process might work and,
2. Understanding the implementation of the program as
proposed
• Conduct evaluation assessments (utilization focused) early on
– Allows for modifications prior to full-scale implementation
Evaluations span the scope of
healthcare delivery
• Community Mental Health Crisis Response Centre
• The Bell Hotel
• Women travelling for birth (Pre-natal connections)
• Birth Centre
• P.I.E.C.E.S.: skills development for working with
persons with dementia
• Evaluating how program planning is accomplished
within a WRHA agency
= new / innovative
What is RE-AIM?…Reach, Effectiveness, Adoption, Implementation, Maintenance
• Originally posited by Russell E. Glasgow in 1999.
• The premise?
– “Multilevel interventions that incorporate policy,
institutions & individual components should be evaluated
with measurements suited to their settings, goals &
purpose”
• The RE-AIM Framework uses 5 dimensions (which
occur at multiple levels) to determine & describe
the impact of a program or policy.
FIVE (5) dimensions…help to describe a successful impact & to design
interventions that:
Reach: …reach a broad and representative proportion of
the target population
Effectiveness: …leads to positive changes that are robust across
diverse groups
Adoption: …be adopted across a broad & representative
proportion of settings
Implementation: …lead to consistent implementation of strategies
at a reasonable cost
Maintenance: …lead to sustained delivery of intervention/change
Environmental ImpetusPersons seeking or
needing services
System Vision, Culture, Strategy & Leadership
Functional
Integration
Physician
Integration
Integration(Clinical/System/Person-based)
Horizontal
Vertical
Relational
Coordination
Relational
Coordination
Performance
(Outcomes)
Integration: Precursors Processes Outcomes
Use of RE-AIM Planning &
Evaluation Framework
for Working Groups
Model for Studying
INTEGRATION
Measures of Functional (FI) & Physician Integration
(PI), Relational Coordination (RC) & Integration are
available for use Before-After in the CPJI
60-days fromHigh suspicion to
1st Treatment
Knowing measures
can help in planning
Embedding Evidence in Context: Approaches
Used by an Embedded Research & Evaluation Unit
As the great sub-systems of modernity (state, market,
culture and science), became more transgressive, the
resulting fuzziness helped to create the transaction
spaces in which Mode 2 knowledge production
developed…Nowotny H, 2003
..we believe that our Unit was created to be a
transaction space for Mode 2 knowledge production
…or, using a bamboo forest as a metaphor for Mode 2 knowledge
production by a unit embedded in a health authority