Implementation of the Ontario Public Health Standards in 6 Ontario Health Units: Key Results and Responses
from Provincial and Health Unit Stakeholders
Heather Manson; Ruta Valaitis; Gayle Bursey; Dobbins, Maureen; Vera Etches; John Garcia; Betty-Ann Horbul; Anita Kothari; Daina
Mueller; O'Mara, Linda; Nancy Peroff-Johnston; Jennifer Pritchard; Sandra Regan; Doaa Saddek; Carol Timmings; Deanna White
Panelists:
George Pasut; Sylvia Shedden; Carol Timmings
The Ontario Public Health Convention - April 3rd, 2012 1
The RePHS Team
• Principal Investigators: Marjorie MacDonald, Trevor Hancock and Bernie Pauly
• ON Academic Researchers: Ruta Valaitis (lead), Linda O’Mara, Anita Kothari, Sandra Regan, John Garcia, Maureen Dobbins
• ON Decision-maker Researchers: Heather Manson (lead), Gayle Bursey, Vera Etches, Betty Ann Horbul, Doaa Saddek, Nancy Peroff-Johnston, Jenifer Pritchard, Renée St. Onge, Carol Timmings, Deanna White
• BC Academic Researchers: Marjorie MacDonald (lead), Allan Best, Anne George, Trevor Hancock, Esther Sangster Gormley, Joan Wharf Higgins, Craig Mitton, Bernie Pauly, Roger Wheeler, Sabrina Wong
• BC Decision-maker Researchers: Warren O’Briain (lead), Ted Bruce, Veronica Clair, Karen Dickenson, Lydia Drasic, Amanda Parks, Michael Pennock, Jennifer Scarr, Lorna Storbakken, Peggy Strass
• Research Coordinators: Diane Allan (BC), Danielle Hunter (ON)
• Funder: Canadian Institutes of Health Research (CIHR)
2
Overview of RePHS
Funder: CIHR
Time Frame: January 2009 – January 2014
Purpose: To examine the implementation of public health renewal processes using 2 public health programs across BC and ON
– chronic disease prevention/healthy living (CDP)
– sexually transmitted infection prevention (STIP)
3
Research Questions and Cross-cutting Themes
Question 1: What factors/contexts influence or affect the implementation of these policy interventions?
Question 2: What have been the impacts/effects of these policy interventions on: staff, the organization, the populations served, other organizations, and communities?
Cross-cutting themes:
a) equity
b) public health human resources
c) primary care/public health collaboration
4
RePHS Contributions
• To inform public health systems renewal in Canada and, in turn, contribute to improving population health and reducing health inequities;
• To advance the field of public health services research in Canada by implementing a consensus-based research agenda and applying/ developing innovative research methodologies;
• To inform integration and linkage of public/ population health and primary care services;
• To train expert public health services and population health researchers.
5
Ontario Academic Researchers
• Ruta Valaitis (ON Academic lead), Linda O’Mara & Maureen Dobbins (McMaster University)
• Anita Kothari & Sandra Regan (University of Western Ontario)
• John Garcia (University of Waterloo)
• Danielle Hunter (ON Research Coordinator)
6
Interviews and Focus Groups – Phase 1
• Focus groups (n=21) and Interviews (n=5) were conducted with HU managers, directors and front line staff throughout October-December 2010 and August 2011.
• A total of 120 people participated, from 6 participating health units. (including: rural/ remote -rural/ urban)
• All interviews and focus groups were transcribed and coded in NVivo 9.0.
7
Data Collection Timeline
8
Interview Question Themes
• Implementation
• Changes in activities since OPHS*
• Evidence*
• Evaluation
• Leadership*
• Partnerships
• *Marked topics covered by front line staff only.
• All above topics covered by managers and directors
9
Reporting
• Front line staff focus groups = 12
• Manager focus groups/interviews = 14
10
High Level Overview of Key Themes from Phase 1 Data
11
Implementation
12
Awareness of OPHS by
Managers
• Knowledge of OPHS prior to recruitment to this study was reported in most manager focus groups – knew about them because of position
– used policies as a planning tool
– no change in knowledge
We did have high awareness [of OPHS] because people from the
[HU] were part of the writing with the pre-implementation, so I
think we had a very good understanding in general. (Manager)
Staff Awareness of OPHS
• Front line staff awareness varied
• In over half of the focus groups, staff reported being unaware of OPHS
• In half of the focus groups, some staff reported being aware of OPHS or elements of OPHS
• In a couple of focus groups staff reported not paying any attention to OPHS
I’m familiar with the Ontario Public Health Standards but the old ones. I haven’t seen the new, updated ones. They have not been, in any way that I’m aware of, been shared with me. (STIP staff)
14
I don’t think our focus is much – like I don’t think about the Standards an awful lot. In the clinic here you are focused on hands-on work, so it’s not something that we dwell on too much. Like, we know what we have to do and we do it, like, we just do our job... And it’s not the Standard that we even care about, sorry. It’s the protocol... And the protocol didn’t change. (STIP staff)
How Staff Informed about OPHS
• Most frequently through formal communication –program discussions, new staff orientation,
presentations, emails, workshops • Often through involvement in development
–involved in providing feedback, or writing process
• Sometimes informed by communication from others outside Hus e.g., external networks, Ministry updates, webinars
• Reviewed individually in a few cases • Rarely through Internal involvement to strategize about
implementation in a few cases 15
What influences
implementation of the OPHS?
• Reported in about half of the focus groups
– Governance model
– Economic influences
• Reported in a few focus groups
– Policy influences
– Provincial level influences
– Staffing
– Community influences
16
Impacts of OPHS on Staff Generally
• Positive impacts reported in about half of focus groups
– generally positive; capacity building for staff; engagement with other programs; supports previous work
• Challenges reported in a few focus groups
– resource challenges (time/money/staff); conflict regarding tasks and interests; lack of skills re: use of evidence
17
Impacts of OPHS on Staff Roles
• Most staff and managers reported no change in role
• Half of focus groups reported some change in roles resulting from OPHS: – generalist to specialist and vice versa; tasks done differently, aligned
skills to tasks, eliminated tasks
• In about a third of focus groups, content focus changed – evidence based focus; more research and evaluation awareness and
activity
• In a few focus groups, greater accountability for evidence-based practice was reported
18
Impacts of OPHS on Staff Roles cont.
In half of the focus groups:
• Positive role impacts were reported, such as:
– policies reaffirm work being done, greater role definition
• Challenges were reported, such as:
– staff working beyond role, unsure of value of role, titles don’t represent actual work, lack of role clarity
19
General Program Impacts
In more than half focus groups participants reported:
• Meeting Standards anyway ; more flexible; less prescriptive programs
In about a third of focus groups participants reported :
• No programming changes; working to meet Standards/align with OPHS
In a few focus groups, participants reported:
• Increased accountability; less innovation focus; more prescriptive programs
20
Impacts on Program Planning
In about half of focus groups: • the quality of planning processes increased
– more comprehensive, focused, transparent, efficient
In a few focus groups: • Stronger use of evidence based programming • Community needs assessments emphasized more • Changes in program planning outputs
– logic models, program descriptions, building case for programs
Impacts on Program Planning
• Many focus groups reported positive comments about the planning process:
• Overall
• improving , evolving , works well , concerted effort
• Quality improving
• training for consistency, staff engagement, becoming more streamlined, transparent/open, accessible, support system in place, staff allowed to provide feedback on planning
22
Impacts on Program Evaluation
In a few focus groups:
• A change in what is evaluated
– more outcomes v.s. process; more quality assurance
• Limitations in evaluation skills; loss of PHRED support
• Evaluations conducted within more formal structures and tools 23
Impacts on Health Unit
• In a few focus groups, participants reported:
– Changes to program planning approaches
• consistent across HU; new tools; planning policy changes; more risk management
– Structural changes to HU
• more collaboration between branches, teams based on programs in Standards
– New opportunities due to OPHS
24
Impacts on Health Unit (cont.)
In a few focus groups, participants reported:
- Emphasis on evaluation
- Difficult to assess if OPHS are driver for changes
there’s been a change in the upper layer of management. And even at our
level, it’s been quite a transformation of management. So, I’m not sure if it’s
the Standards that have brought the changes, or the new management, but I
have seen an increase in the need for us to focus more on the epi data,
supporting everything that we do-- the evaluation, and all that stuff…(STIP
Staff)
25
Impact on Populations Positive
• A few focus groups reported:
– Focus on health inequities/equities
– Focus on priority populations
– Increased community understanding of PH issues
– Increased consideration of community issues
26
Impact on Populations Challenges
• A few focus groups reported:
– Unequal distribution of services for some populations
– Difficult to meet all community needs
27
Evidence
28
What guides or informs your practice?
• More than half of the focus groups reported:
– Research
– People
– All practice guidelines
– Information from the community
– Internal processes and or resources
– OPHS
29
… you know we’re not just guided, I
think, by literature and the formal
evidence, we’re guided a lot by
what our clients say (STIP Mgr)
• Half or just less than half of focus groups reported:
– Provincial level resources or organizations
– Legality
– Manuals or Guidelines
30
What guides or informs your practice? (cont.)
What Kinds of Mechanisms Foster the Use of Evidence?
• Over half of focus groups reported:
– Internal mechanisms (access to resources, specific position within HU)
• A few focus groups reported:
– Going to conferences
– Networks
– Structures or organizations external to the health unit
31
Barriers Regarding Evidence
• Over half of focus groups reported: – Intra-organizational barriers (staff and work issues)
– Barriers pertaining to evidence itself
– Community issues
– Resources (financial) and or provincial funding
• Many focus groups also reported:
– Issues related to time
– Political barriers
– Related to specific programs within health unit
– Values and beliefs
32
Evaluation
Monitoring processes
Definition of successful implementation
33
How is monitoring happening?
• A few focus groups reported:
– Operational planning determines what is monitored
– Evaluation more related to programming, not meeting OPHS
– No specific way HU is monitoring OPHS
– Activity reports to the BOH, reports to Ministry funded programs
– Unsure if they are monitoring
34
Who is doing the monitoring?
• In a few focus groups, participants reported:
– Managers and or coordinators
– Epidemiologists
– Research and evaluation division within health unit
35
What are the
accountability mechanisms?
• A few focus groups reported:
– Managers report to MOH
– MOH reports to BOH
– Unaware of any accountability mechanisms that exist, not fully developed internally
– Built into overall planning, implementation, & evaluation structure
36
Definition of successful implementation of OPHS
• Improved monitoring
• Client satisfaction data
• Performance measurements and outcomes
• Fulfilling the community needs
• Meet the protocols within OPHS
• Improved community health outcomes
• Improved program reach to target populations
• Improved quality of programs
37
Leadership and Partnerships
38
Leadership
• Almost half of the focus groups reported that the work environment is supportive:
• Management Style ― management is supportive, accessible, supports education/updates/skill
building, open door policy, supports innovation and creativity
• Setting Direction and Expectations ― organizational expectations, OPHS directions, job descriptions; code of
ethics up to date; internal planning, new policies; documentation template
• Organizational/Team development ― supporting professional education and skill building, staff development,
learning opportunities; team, peer-to-peer
39
Leadership Cont’d
• A few focus groups reported that the work environment is unsupportive:
• Lacking supports for work expectations ― Voice not being heard, little room for innovation, staffing patterns
between main and branch offices, caught in day-to-day micro management
• Transparency re: organizational and priority setting
• Lack of guidance re application of OPHS to practice
40
Partnerships
• A few focus groups reported some new partnerships since OPHS (e.g., police services, acute care centres)
• A few focus groups reported no change in partnerships since OPHS or unable to associate new partnerships with OPHS
• Internal Partnerships (other teams/divisions)
• External partnerships (community groups, other health units, schools, physicians)
41
Partnerships
• Over half of focus groups reported a sense of community involvement in planning – Community consultations
– Collaborations, community groups
– Coalitions, advisory committees – range of involvement/participation
– Other - surveys, assessments, forums, stakeholder meetings
– Peer projects/hiring of community peers
– Roles
• community health research
• advocacy
• Range of challenges – funding, trust, resources, PH perceived as government, cultural diversity, power
42
Data Collection Timeline
43
Situational Analysis
Social Network Analysis
Concept Mapping