using participatory action research to build a priority setting process in a canadian regional...

ARTICLE IN PRESS

0277-9536/$ - se

doi:10.1016/j.so

�Correspondfax: +1403 283

E-mail addr

cmitton@excha

Cam.Donaldso

Social Science & Medicine 63 (2006) 1121–1134

www.elsevier.com/locate/socscimed

Using participatory action research to build a priority settingprocess in a Canadian Regional Health Authority

San Pattena,�, Craig Mittonb,c, Cam Donaldsond

aCentre for Health and Policy Studies, University of Calgary, 3330 Hospital Drive N.W., Calgary, Alta., Canada T2N 4N1bCentre for Healthcare Innovation and Improvement, British Columbia Research Institute for Children’s and Women’s Health,

4480 Oak Street, E414A, Vancouver, BC, Canada V6 H 3V4cDepartment of Health Care and Epidemiology, University of British Columbia, Vancouver, Canada

dCentre for Health Services Research, School of Population & Health Sciences and Business School (Economics),

University of Newcastle upon Tyne, 21 Claremont Place, Newcastle upon Tyne NE2 4AA UK

Available online 15 March 2006

Abstract

Due to resource scarcity, every health system worldwide must decide what services to fund, and conversely, what services

not to fund. In order to institute and refine a macro-level priority setting framework within a large, urban health authority

in Alberta, Canada, researchers and decision makers together embarked on a participatory action research (PAR)

project.

The focus of this paper is the PAR process in this context, including reflections from PAR participants about the

contribution of the research methodology to their own practice as health care managers and clinicians. The use of

qualitative research in health economics—in this case, to refine the application of a macro-level priority setting model—is a

relatively new advancement.

PAR proved to be an appropriate and helpful approach to introducing a theoretically driven model of macro-level

priority setting within a large, complex health organization. However, it is important that support for the change is

sustained as long as necessary to embed the new practices into the organization.

r 2006 Elsevier Ltd. All rights reserved.

Keywords: Priority setting; Participatory action research; Health economics; Canada

Purpose

This article describes the unique application ofparticipatory action research (PAR) methods as ameans of building a macro-level priority setting

e front matter r 2006 Elsevier Ltd. All rights reserved

cscimed.2006.01.033

ing author. Tel.: +1403 245 3465;

5897.

esses: [email protected] (S. Patten),

nge.ubc.ca (C. Mitton),

[email protected] (C. Donaldson).

framework within a large, complex health organiza-tion. The methods for the project as a whole aredescribed in detail and PAR is critiqued as a changemechanism in this context. The PAR projectmanifested as seven phases, the first four of whichhave been reported elsewhere (Mitton, Patten,Waldner, & Donaldson, 2003); this paper reportson phases five through seven, providing furtherinsights on PAR as a means of introducing theprogram budgeting and marginal analysis (PBMA)framework and facilitating its transfer and uptake.

.

www.elsevier.com/locate/socscimed

ARTICLE IN PRESSS. Patten et al. / Social Science & Medicine 63 (2006) 1121–11341122

Finally, in response to suggestions that PBMAsucceeds only when health economists are activelypushing its agenda (Mitton & Donaldson, 2004),data from decision makers is presented regardingthe role of the researchers in introducing prioritysetting practices in the Calgary Health Region(CHR).

The overall purpose of this paper is to highlightkey findings related to the application of PAR as ameans of introducing a theoretical priority settingmodel, and in particular to report on the sustain-ability of change enacted through PAR once therewas no direct researcher involvement. In so doing,this paper builds on previous work in which theinstitutional context is described as a key driver forthe ultimate success or failure of the application oftechnical and/or so-called rational approaches tohealth care priority setting (Jan, 2000; Mitton &Donaldson, 2004).

Background

The CHR is one of nine integrated regional healthauthorities in the province of Alberta, Canada,which provides services across the continuum ofcare. In the CHR, all clinical and preventive servicesare overseen by one of seven Executive Director/Medical Director pairs who comprise the seniormanagement team, and in turn report directly to theChief Operating Officer and the Chief MedicalOfficer who are two of seven vice-presidentscomprising the Executive level of the organization.In addition, Regional Clinical Department Headsoversee clinical specialty areas and work alongsidethe Executive Director/Medical Directors in settingclinical priorities. The total annual operating budgetof the CHR is approximately CAD$1.5B (CA-D$1EUSD$0.82), with the majority of physicianreimbursement falling outside this budget.

In Fall 2001, a need was identified within theCHR for an explicit, systematic process for settingpriorities and allocating resources across broadservice areas. In response to this, researchers atthe University of Calgary teamed with decisionmakers in the CHR in order to develop andimplement a macro-level approach to prioritysetting based on a recognized health economicframework that had previously been used at moremicro levels of care in health organizations (Do-naldson & Farrar, 1993; Halma, Mitton, Donald-son, & West, 2004; Madden, Hussey, Mooney, &Church, 1995; Peacock, 1998). The approach used

in the CHR, based on the PBMA framework, isdescribed in detail elsewhere (Mitton et al., 2003).

A seven-phase PAR project was initiated in theCHR late in 2001 to develop and implement amacro-level priority setting model. PAR is anapproach to develop knowledge from experienceand engage stakeholders in becoming more aware ofconditions, while learning to take actions to alterthe practices of the organization (Fals-Borda &Rahman, 1991). This project included the study ofexisting priority setting practices and structures,provision of education to senior decision makersand clinicians about economic principles relevant topriority setting activity, and involved action to-wards the development and implementation of amacro level approach to priority setting. The mainoutcome of the project was the development of anew process for priority setting, as conceptualizedby the organization, as well as a series ofrecommendations to the CHR for process refine-ment and long-term sustainability of explicit,evidence-based priority setting practices.

The priority setting process as developed not onlyaddressed the challenges identified by CHR decisionmakers with respect to PBMA, but also was animportant contribution to the health economics andpolicy literature, noting that previously an explicitresource re-allocation process across major servicesareas had not been reported within a healthauthority with jurisdiction across such service areas(Mitton et al., 2003). A further significant outcomeof this project was that senior managers andclinicians in the CHR gained a better understandingof priority setting at a macro level across majorprogram areas, and have developed a set of skillsand strategies that directly contribute to improvingthe organization’s priority setting practices. Thislatter outcome is particularly important as previouswork has shown that decision makers in healthauthorities often lack the necessary skills to setpriorities and allocate resources (Lomas, 1997;Mitton & Donaldson, 2002).

Application of PAR methods

When a PAR approach is taken, it is important toexamine not only the outcomes of the project but aswell the process used in achieving those outcomes.In this section, we describe how PAR was appliedwithin the context of efforts to build a morerigorous macro-level priority setting process in theCHR.

ARTICLE IN PRESSS. Patten et al. / Social Science & Medicine 63 (2006) 1121–1134 1123

PAR project

PAR is a form of social research that blendsknowledge generation with organizational actionand change (Greenwood & Levin, 1998). PAR isdifferent from conventional health research in thatit focuses on ‘knowledge for action’ rather than‘knowledge for understanding’ (Cornwall & Jewkes,1995). In our case, the intent was to foster change inthe direction of a more systematic, evidence-basedpriority setting process at a macro-level within theCHR. Before engaging in a change process, it wasimportant for the researchers to understand theorganization and its current priority setting prac-tices. PAR became the method of choice once theresearch objectives were established, and once theresearchers appreciated the complexity of PBMAfrom the perspective of those applying it. PARinherently takes account of local priorities, pro-cesses and perspectives (Cornwall & Jewkes, 1995).Together, the researchers and senior decisionmakers within the CHR established the researchagenda, generated the knowledge necessary totransform the organization’s practices, and incor-porated the resulting approach into practice.Through the use of qualitative methods, decisionmakers in the CHR were able to understand macro-level priority setting within their own context, andthus, together, decision makers and researchers wereable to develop the PBMA model inductivelythrough immersion in the practice setting (Lincoln,1992).

The PAR project was carried out by a team oftwo health economists, a qualitative researcher andsenior managers and clinicians. Throughout theproject, there were close interactions between theinvestigators and the investigated, with explicitefforts by the researchers to understand the contextwithin which the macro-level priority settingapproach was being developed. The project fol-lowed general principles of qualitative research(Miles & Huberman, 1994) in that: (1) the research-ers maintained intense and prolonged contact withsenior decision makers within their own worksetting; (2) the researchers sought to gain a holisticoverview of the context under study (i.e., theorganization’s structure and culture, its historyand procedures with respect to priority setting, itsexplicit and implicit rules, and its guiding princi-ples); and (3) the researchers attempted to capturedata from the perspective of those individuals whowould have to implement PBMA on an ongoing

basis. The PAR project was a naturalistic explora-tion of how the CHR applied PBMA, revolvingaround the CHR’s priority setting processes.

Project phases

In initial discussions between the CHR and theresearch team in Summer 2001, it came to light thatthe organization was not only facing tough budget-ary decisions (i.e., a deficit of approximatelyCAD$40M) for the 2002/2003 fiscal year, but alsothat senior decision makers wanted to develop andimplement a longer term, sustainable approach tomacro-level priority setting. The PAR approachallowed the researchers to ‘get inside’ the organiza-tion and discover from within the barriers andfacilitators encountered in developing a macro-levelpriority setting model. The researchers took the leadin writing the proposal but significant input was alsoprovided by the decision makers. The researcherscirculated the research proposal to the decisionmakers, reached agreement on the aim to develop amacro-level priority setting model, and built rela-tionships with key participants who would likelyinfluence the processes from within. Ethics approvalwas received from the University of CalgaryConjoint Health Research Ethics Board in October2001.

The methodology of the PBMA study wasreflexive, flexible and iterative (Cornwall & Jewkes,1995). While there were preliminary plans for theproject, phases changed from their original concep-tion or emerged as new phases during the imple-mentation of the PAR project. In the end, the PARproject emerged as seven phases comprising devel-opment of the PBMA model, data collection, andapplication of the PBMA model (both with andwithout researchers’ involvement) (see Tables 1and 2).

Data collection

The research components of the PAR projectcombined qualitative data collection methods ofdocument review, participant observation, in-depthinterviews and focus groups. Throughout the PARprocess, the health economists participated in thepriority setting exercise not only as researchers, butalso as consultants. Participant observation noteswere taken during all priority setting meetingsand training presentations, as well as during thefocus groups and interviews. The researchers joined

ARTICLE IN PRESS

Table

1

Participatory

actionresearchphasesandtimetable

S. Patten et al. / Social Science & Medicine 63 (2006) 1121–11341124

ARTICLE IN PRESS

Table 2

Phases of the PAR project

1. Examination of recent and current priority setting practices

Researchers attended priority setting meetings of senior managers and clinicians between October 2001 and September 2002 to document

decision-making processes, sources of evidence, group dynamics and roles, and organizational culture. This allowed the researchers to

develop a holistic understanding of priority setting practices in the CHR, and make note of both explicit and tacit aspects of the CHR’s

organizational culture. As well, a social analysis of the decision-making structure of the organization was conducted in order to ensure that

all relevant members were included in the PAR process

2. Reflection upon recent and current priority setting practices

Reflections were gathered from the senior managers and clinicians involved in macro-level priority setting about their current practices and

processes, through participation in a focus group, to provide information about their involvement, influences, needs for greater

understanding, concerns, and suggestions for improvement with respect to the CHR’s recent and current macro-level priority setting

practices, and one-on-one qualitative interviews, to gather more in-depth and personal reflections on group dynamics, political and

interpersonal influences, and the role of personal values in priority setting practices

3. Introduction of priority setting economic principles

CM and CD provided explicit training to members of the senior management team to generate the knowledge necessary to transform the

organization’s priority setting practices. While the researchers provided ongoing training throughout the priority setting meetings attended

between October 2001 and September 2002, a number of specific presentations to senior management team and other decision makers

within the organization were also held during this period. Training included introduction to fundamental economic principles of

opportunity cost and marginal analysis, and introduced the PBMA framework for priority setting activity. During key presentations,

participant observation notes were taken (by SP) to document group dynamics, improvements in understanding, acquisition of new

concepts and skills, mobilization for action (e.g., levels of motivation), and potential challenges in implementation

4. Development and implementation of the priority setting model

Researchers and stakeholders collectively developed and refined a macro level priority setting approach within the organization in an

iterative manner. That is, the researchers proposed information and processes from the literature and their previous experience with

priority setting in other contexts, and the decision makers rejected or accepted this information and added to it from their own experience.

The initial priority setting process was developed and implemented (to inform budgetary decisions for the 2002/2003 fiscal year) over a

three-month period (January–April 2002) through bi-monthly meetings

5. Framework refinement

Reflections and suggestions for refinement were obtained from the decision makers who had just completed implementation of a novel

approach to priority setting at a macro-level across major service portfolios in the CHR. A second focus group (same participants as phase

two) and a further round of one-on-one interviews (new individuals) provided information pertaining to specific challenges encountered

during framework implementation and suggestions for process improvement. The data also included reflections on the PAR project itself,

its processes and outcomes, as well as prospects for sustaining the framework

6. Independent application of priority setting model

Researchers departed from the process. CHR senior managers implemented the priority setting model independent of the health

economics researchers for development of the 03/04 fiscal year budget

7. PAR follow-up

Interviews with senior managers to gather reflections on their first independent application of the priority setting model (without

researcher involvement)

S. Patten et al. / Social Science & Medicine 63 (2006) 1121–1134 1125

the priority setting meetings to conduct partici-pant observation and develop a contextual under-standing of the group dynamics, roles and salientissues to be explored in interviews and focusgroups, as well as in the longer-term follow-up.Current priority setting practices were also exam-ined through a review of relevant written docu-ments such as decision-making tools, internalcriteria on which priority setting decisions wouldbe based, and survey results from previous researchwith CHR decision makers (Mitton & Donaldson,2002).

In this paper, we report information gatheredfrom the interviews and focus groups conducted inphases five and seven of the PAR project. In phasefive, members of an internally struck priority settingcommittee (n ¼ 8) participated in a focus group toassess the processes and outcomes of the implemen-tation of the PBMA approach and to providesuggestions for improvement. In addition, one-on-one qualitative interviews were conducted with eightother senior managers to gather more in-depth andpersonal reflections on specific challenges encoun-tered during the implementation of the PBMA

ARTICLE IN PRESS

Table 3

PAR phase five focus group and interview guide

Focus group questions Interview questions

1. Overall, was the priority setting process fair and

transparent?

2. How well is the publicity condition met in this region?

3. How well is the relevance condition met in this region?

4. How well is the appeals condition met in this region?

5. How well is the enforcement condition met in this region?

6. Were the criteria used in the decision making process clear

and appropriate? What would you change?

7. What have you learned over the last four months about

communication and roll-out (feasibility) with respect to the

priority setting process?

8. How best should cross-portfolio priorities (disparate patient

groups) be compared?

9. What would you do differently for future priority setting

exercises in the RHA?

1. Was the priority setting process fair and transparent? How

do we ensure it is fair in the future?

2. How did the input of the researchers (health economists)

influence your understanding of MMA?

3. Who should be making priority setting decisions; is it the

right mix? How inclusive should the process be? At what

stage(s) should the Directors and Clinical Department

Heads be involved in the priority setting process?

4. How should we educate the public and build their

engagement in decision-making?

5. How best should cross-portfolio priorities (disparate patient

groups) be compared?

6. In what ways should or could PBMA be used more broadly

in the organization (i.e., beyond the operations portfolio)?

7. What would you do differently for future priority setting

exercises in the RHA?

Table 4

PAR phase seven interview guide

1. Please describe the macro-level priority setting process conducted this past year in the Calgary Health Region

2. In your opinion, and thinking specifically in comparison to the previous year, what were the key strengths of the process this year?

3. What were the key weaknesses of the process this year?

4. Do you think the process should be continued in future years? If so, how specifically do you think the process could be improved or

adapted in future years?

5. Do you think that the public could play a more explicit role in the process as it stands currently? What role might this be?

6. Do you think that evidence could play a more central role in the process? How exactly do you see this?

7. Based on the amount of resources freed up for re-investment, it would seem that there was little or no detriment in not having

researchers or health economists directly involved in the process. Would you agree with this? Why or why not?

S. Patten et al. / Social Science & Medicine 63 (2006) 1121–11341126

framework, as well as strategies to address thosechallenges. The focus group and interview guidesused in phase five are included in Table 3.

Phase seven was a final data collection period inthe spring of 2003 to gather follow-up reflectionsfrom CHR managers after they had independentlyapplied PBMA (in phase six) without input from theresearchers, to guide the 2003/2004 budget. It isimportant to note that all three researchers wereentirely absent from the application of PBMA in the2003/2004-budget cycle. There were no externalfunds remaining to support further priority settingresearch with the Region and two of the researchersleft Canada, thus leading to an opportunity for a‘natural experiment’ of sustainability and develop-ment of the PBMA process in their absence. Duringphase seven, approximately 1 year after the re-

searchers removed themselves from the CHR’spriority setting processes, one-on-one qualitativeinterviews were conducted with 17 senior managersand clinicians to gather the reflections on specificchallenges encountered during the independentimplementation of the process, as well as to identifystrategies to address those challenges. This interviewguide is found in Table 4.

Sampling strategy

Purposive sampling was used to select keyinformants based on the perspective and role ofthe individuals within the organization, as well asinformation that had already been gathered inprevious phases and remaining gaps in understand-ing (Stringer, 1999). The sampling comprised of


two simultaneous purposive sampling techniques(Patton, 1990): (1) intensity sampling for selectionof experiential experts; and (2) maximum varietysampling for selection of heterogeneous participantsrepresenting a wide range of clinical and communityhealth services, experience and involvement inmacro-level priority setting, and level of under-standing of economic principles. In practice, thismeant that the researchers and a senior CHRrepresentative discussed the roles of various man-agers and clinicians on the senior management teamand identified the relevant samples for the inter-views in phases five and seven.

Data analysis

With informed consent from participants, theinterviews and focus groups were audio taped andthen transcribed verbatim. Participant observationnotes and interview and focus group transcriptswere entered into QSR N5 software for storage,coding, text search and retrieval, and theme map-ping. Through thematic analysis, the researchersidentified and defined salient themes and recurringideas or concepts. The data were coded inductivelywith a thematic coding scheme that evolved using aconstant comparative method of analysis (Glaser &Strauss, 1967). Each iteration of comparing andcontrasting themes and concepts as they emergedinvolved both data reduction as the volumes ofwritten data were organized into manageablechunks, and interpretation, as the researcherbrought meaning and insight to the textual data(Marshall & Rossman, 1989). As categories ofmeaning emerged, the researcher searched for thosethat had internal convergence and external diver-gence (Guba, 1978).

Participants’ reflections on the PAR project

Participant reflections allowed the researchers toassess the extent to which the PAR approach was aneffective means of introducing, customizing andbuilding capacity to implement the new macro-levelPBMA approach. The key features of the PARapproach in facilitating the introduction andimplementation of PBMA are summarized below.

Need for change identified by management

A prerequisite for PAR is that the action orchange is deemed as important to the people most

affected. The PAR project was initiated in order tointroduce one possible priority setting approach,PBMA, in a time of major fiscal challenge and offerassistance in adapting and applying it to the contextof the CHR.

Fluidity of researchers’ roles

The health researchers were seen to serve the rolesof educator, process facilitator, health economist,content expert, management consultant and healthresearcher. This fluidity of roles was advantageousin that the senior managers perceived the research-ers as members of the group and did not act for thebenefit of the researchers, or for the benefit of theresearch project.

Recognizing change as an incremental process

The PAR project was initiated with the expecta-tion that change, both in terms of individualprofessional development and organizational devel-opment, would occur in the direction of a morerigorous macro-level priority setting model. Eventhough the CHR, upon completion of the PARproject, had tools to assist in priority setting, theorganization’s macro-level priority setting model isonly in the early stages of becoming integrated as apart of its regular practice.

Capacity building

In the interviews following the 2003/2004 budgetplanning process, decision makers attributed thesustained and independent application of thePBMA process to the participatory approach takenin the initial development in which CHR leadersbuilt a thorough understanding of its principles andprocesses. Overall, capacity was built throughintense collaboration in the first year of PBMA,then, once the basic concepts were embedded,decision makers were able to continue with sub-sequent application on their own.

Merging theory and practice

A key component of PAR is that the researchprocess brings theory and practice into closeralignment. In this case, the PAR project broughtan ‘objective’ theoretical perspective to the difficulttask of macro-level priority setting. One seniormanager described the principal contribution of the


PAR project as the introduction of economicprinciples combined with health economists’ prac-tical suggestions.

Recognizing the theory– practice divide

One Executive Director felt that the CHR needsto use the PBMA process pragmatically and apply itin appropriate contexts. In the follow-up interviews,one senior manager stated that the health econo-mists provided the CHR with a solid theoretical‘frame of reference.’ Despite this learning by seniormanagers, ‘‘there was a bit of frustration that thetheory is difficult to apply in reality.’’

Internal commitment

In the interviews following the 2003/2004 appli-cation, some of the senior managers expressed anoverall commitment to implementing the newprocess and felt the rationale for the new approachto priority setting was firmly implanted. In phasefive, one of the Vice-presidents attributed thesuccessful integration of the PBMA model to strongproponents who would remind their colleagues insenior management of the rules and the processes.Internal commitment was seen as key to movingforward with PBMA over time.

External objectivity

The decision makers felt that external guidance ofthe priority setting process was very valuable inbuilding a fair and rigorous process. One ExecutiveDirector felt that external researchers are more ableto fill an objective monitoring and mentorship roleto the PBMA process than staff members would be:‘‘As a researcher you don’t have any turf to protectother than the process, the integrity of the processand we need somebody to protect the integrity ofthe process.’’

Strengthening partnerships

Some of the CHR participants felt that the PARproject encouraged merging of different areas ofhealth expertise, building mutual understandingboth within the CHR and between the CHR andits academic partners. Another positive outcome ofthe collaboration built through the PAR project isthat it helped to overcome previous negativeperceptions of health economists.

Discussion

We examined the application of PAR methodsthat were used to introduce, adapt and implement anew priority setting approach in the CHR. Theapplication of PAR contributed two major findings:(1) there are several key features of PAR thatfacilitate the merging of theory and practice insetting health services priorities; and (2) a complexand new priority setting model with unfamiliarconcepts for many decision makers can be success-fully and independently implemented when PAR isused to introduce the model in a manner thataccounts for the unique context of the hostorganization. Overall, the use of a PAR approach,with its inherent requirement of researcher immer-sion and context-specific understanding, can beviewed as an effective means for promotingorganizational change. PAR was a suitable methodfor introducing PBMA to the CHR, as it helped theresearchers and decision makers to discover moreabout the applicability of economics-based techni-ques in complex organizational settings. This studyadds to a developing literature in which qualitativemethods have been applied to health economicsresearch (Coast, 1999; Coast, McDonald, & Baker,2004), in research on priority setting to examinecitizen involvement in rationing decisions (Coast,2001), to describe decision making processes and therole of health economics in health authorities(McDonald, 2002), and in research on allocatinghealth care resources on the basis of the size of thehealth improvement (Dolan & Cookson, 2000). Thecurrent focus of using PAR as a vehicle for applyingan economics-based approach to priority setting is,to our knowledge, novel.

This project has demonstrated the value of PARas an approach to bridge the gap between healtheconomics theory and the practice of macro-levelpriority setting in a health region. A number of theelements of this project that contributed to itssuccess in initiating an organizational changeprocess within the context of macro-level prioritysetting are highlighted in Table 5. While somedecision makers were confident that the organiza-tion has the capacity to independently implementthe PBMA model without further assistance fromexternal parties, most participants felt that evenwith the positive change towards a fair and rigorouspriority setting process, the change is best sustainedwith continued support from external health econ-omists. The CHR participants gained a better

ARTICLE IN PRESS

Table 5

Elements contributing to organizational change

Element Result

d Real pressures were faced by CHR decision makers in

dealing with a large budget deficit

d Provided strong internal impetus for developing a more

rigorous and fair priority setting model

d Dual management consultant/researcher role served by the

health economists

d Lended credibility to the research project and provided

professional development opportunities for the CHR

participants

d Researchers were permitted to immerse themselves in the

context within which the priority setting model would be

implemented in order to fully understand the complexities and

pressures faced by the CHR participants

d Allowed a customized and contextually appropriate priority

setting model to be developed and implemented

Researchers were willing to allow the PAR project to proceed

naturally through the various phases of action, research and

change interventions

Avoided the feeling of imposition upon the senior managers

who were facing their own realities

S. Patten et al. / Social Science & Medicine 63 (2006) 1121–1134 1129

understanding of health economics principles and acloser alignment between theoretical and practicalpriority setting models. Overall, the decision makersfelt that the researchers played a central role notonly in introducing the health economics theory,but also in stimulating professional self-reflectionand creating advancement towards improved orga-nizational practice.

About achieving change

The PAR methodology served as an importantmechanism for initiating and sustaining change ofpractice. Change occurred in the CHR at both thelevel of individual professional practice and organi-zational structures and processes with respect topriority setting and consciousness of health eco-nomics principles. The researchers served as cata-lysts to help senior decision makers define the needsfor macro-level priority setting and examine andimprove current practices. There were severalelements of action throughout the research process:(1) the meetings and focus groups strengthenedrelationships and understanding between indivi-duals and groups from a variety of positions inthe organization; (2) the exploratory phases (oneand two) put issues on the agenda for discussionand raised awareness about specific challenges, suchas identifying different viewpoints between man-agers and clinicians; and (3) the development andimplementation of a novel approach to prioritysetting in the CHR.

PAR served in this case both to alleviate aproblem (i.e., the need to develop a rigorous process

for macro-level priority setting in the CHR), and togenerate new knowledge about the application of aspecific approach to priority setting under theseconditions. Thus, the PAR project simultaneouslyachieved both problem solving and theory building.The PAR project facilitated a critically reflectiveprocess in which theory about the application of aframework for macro-level priority setting emergedfrom the practice of applying the model in the real-life setting of the CHR. This research project alignswith the four main characteristics of action researchdefined by Hart and Bond (1995): (1) collaborationbetween researchers and practitioners; (2) solutionof practical problems; (3) change in practice; and(4) development of theory. This project demon-strated that qualitative methods applied to healtheconomics creates models and practices that aregrounded in the real world of health care provision.

The main intervention that created the impetusfor change occurred through phase three, i.e.,building decision makers’ understanding of healtheconomic principles. This change intervention lar-gely occurred as experiential learning and con-sciousness-raising about relevant economicsprinciples. The change intervention was discreetand perhaps not always recognized by the partici-pants, and included subtle outcomes such asbuilding relationships, opening up lines of commu-nication, reframing issues, and changing the ways inwhich problems are discussed. Change was facili-tated by the willingness of both researchers andmanagers to approach the PBMA model in a spiritof inquiry. The structure and values of theorganization also facilitated the success of the


PAR project. Because group decision-making isessential for authentic commitment to organiza-tional change, senior management team membersworked together as an expert panel of seniormanagers and clinicians to exert self-direction inthe application of the new framework to theirpriority setting practices. Introducing the frame-work through the PAR process avoided a ‘top-down’ approach of imposing a model from theExecutive level of the organization, and fosteredboth empowerment and engagement.

One of the major lessons learned through thisPAR project was that even if members of a groupwelcome, value and are committed to change, itshould not be assumed that the change is sustain-able. While many of the senior management teammembers felt that organizational change had beeninitiated in the CHR through the introduction,customization and one budget cycle’s application ofthe PBMA framework, some felt that this organiza-tional change was not yet self-sustaining withoutcontinued support from external health economists.This perceived need for sustained consultation andassistance indicates that organizational change is aprocess that may take more time than typicallyallocated from a research project’s perspective.Moreover, in the case of this PAR project, theorganizational change involved a theoretical field oflearning (i.e., health economics) that was outside theknowledge domain of some of the group members.It should be noted, however, that even though thegroup members felt that continued support from anoutside expert would be necessary to sustain thechange, they did successfully implement PBMAthrough a second budget cycle without any assis-tance from health economists, and have just recentlycompleted their third planning process with thePBMA approach.

It is thus reasonable to assume that the PBMAprocess is becoming entrenched in the institutionalethos of the CHR. Despite numerous personnelchanges over the last year in even the most seniorpositions, PBMA continues to be referred to as thepriority setting and budget planning process ofchoice. While it is not possible to place a causalrelationship on the use of PAR necessarily resultingin successful longer-term application of PBMA, inour case it is clear that PAR was a vehicle thatenabled organizational change away from historicaland political resource allocation towards the use ofan explicit, evidence-based approach to prioritysetting. ‘Sustained use’ of PBMA has been shown in

a recent international survey to occur in roughlyhalf of the organizations in which it has beenapplied, but almost all of these applications havebeen within specific programs of care (as opposed toacross major service areas) and the time periodof ‘sustained use’ was not specified (Mitton &Donaldson, 2001). As such, additional follow-up inthe CHR is planned for 2006 by a UK-basedresearcher who was not involved in the initialresearch to examine the longer-term role and effectsof PBMA in the organization.

PAR enables decision makers to be engaged intheir own realm, recognizing the messiness andcomplexity of health care decision-making whileproviding a real vehicle for change. Knowledgetransfer is often viewed as an external process ofgetting evidence into the hands of decision makers,but a PAR perspective takes a more in depth andengaging view where researchers and decisionmakers over time can develop relationships andstruggle to come up with real-world solutions thatmay seem far from the ‘technical solutions’ so oftenderived within the health economics paradigm.Indeed, it has been argued that health care decisionmaker attitudes are at odds with rational models fordecision-making (McDonald, 2002). PAR allowsthe decision-making context to be grappled with,and it is setting PBMA in context that in our view isthe major contribution of this work vis-a-visprevious studies.

Roles and relationships

To ensure a successful PAR project, it isimportant that the processes of defining the problemand formulating the research topic/question arecollaborative. The researchers wished to advancethe body of knowledge on macro-level prioritysetting, while the decision maker partners requiredan explicit process to set priorities and allocateresources in their organization. It was serendipitousthat the research questions of interest to theresearchers happened to match the real-world fiscalconstraints faced by the decision makers. However,previous research in Alberta and elsewhere hasshown a lack of knowledge about explicit ap-proaches to priority setting in health authorities,but, nevertheless, a desire to develop one (Mitton &Donaldson, 2002), so it was not a surprise to theresearchers to find such willing partners.

Participation by the CHR decision makers wasvariable, with deeper and more central participation


at some stages than others. Participation by CHRdecision makers manifested itself in two forms:(1) consultative participation in which CHR man-agers had discussions with researchers before,during and after the PBMA model was introducedand implemented; and (2) collegial participationduring which researchers and CHR managerscollaborated as colleagues with different skills orexpertise to offer in a process of mutual learning.The PAR project aligned with many of theprinciples of participatory research (IHPR, 2000)with respect to the nature of participants’ involve-ment, origin of the research question, purpose of theresearch, process and contextual implications, op-portunities to address the issue of interest, andnature of the research outcomes.

In general, the researcher’s role in PAR is to actas a catalyst in helping members of the organizationto define or think differently about the problem, andto implement and monitor change that occurs(Springer, 1999). Specifically, the researchers servedas professional experts, working with key membersof the organization to design the project, gatherdata, interpret findings, and recommend action tothe sponsor organization. The researchers facilitatedthe engagement and understanding of the usersof the developed framework, and helped to pave theway for the implementation of the improved model.The researchers played an active role in the seniormanagement team meetings by providing inputabout economic principles or giving input into thedevelopment of priority setting processes. CHRdecision makers jointly inquired into a new processfor macro-level priority setting, gained professionaldevelopment, and engaged in self-reflection (both atan individual and group level) about their prioritysetting practices. At the same time, the CHRdecision makers built a commitment to investigateand improve their work, and clarify their own rolesin establishing an improved priority setting exercise.Overall, the senior management team membersworked alongside the action researchers to analyzeand solve their own issues, devise action plans toimprove practice, and evaluate such plans.

Transferability

The current study was carried out in a singleCanadian jurisdiction. This arose out of thenecessity to ‘get inside’ a health organization in anattempt to build the bridge between rationaldecision-making models and the real world. Never-

theless, there are a number of reasons to suggestthat the results reported herein (i.e., not the specificallocation decisions made but rather the applicationof PBMA using a PAR approach) are in facttransferable. First, all health care organizationsaround the globe, including the CHR, are facedwith limited resources and thus the need to makechoices about what to fund and what not to fund.Surveys across countries have shown that resourceallocation typically follows historical patterns, withlimited re-allocation across program areas despitedecision maker desires to become more proactive insetting priorities (Miller, 1997; Mitton & Donald-son, 2002; Mitton & Prout, 2004). Second, many ofthe barriers to adopting a change process moregenerally (e.g., perceived information deficits, un-certainty in decision-making) (McDonald, 2002)were most certainly apparent in the CHR. Thus,while any given organization may claim that PBMAthrough PAR is not possible, there is no overtlyobvious difference between the CHR and otherhealth authorities elsewhere responsible for meetingthe needs of the population. Third, the lessonsabout PAR from this study—in terms of relation-ships, recognition of barriers, collaborative devel-opment and implementation—fall within thebroader PAR literature. The novelty has been inapplying these concepts, and identifying relevantlessons within a health care organization challengedwith moving towards an explicit approach to settingpriorities. What remains is to pursue action researchon priority setting in other countries to test many ofthe lessons from this study. Such activity is currentlybeing undertaken in the UK by researchers at theUniversity of Newcastle upon Tyne and has alsobeen proposed in other areas of the UK.

Challenges and study limitations

The unpredictable nature of action researchnecessitates a flexible rather than a rigid researchdesign. The PAR process was not linear and did notfollow discrete stages as the researchers initiallyenvisioned. Action, research and change interven-tions interacted throughout the PAR project asdynamic and overlapping processes. In each phase,one aspect (action, research or change intervention)dominated, although other components were stillhappening. This fluidity in the process of PARmakes it difficult to actually measure the impactthat the health economists exerted upon the changeprocess. While the researchers envisioned a certain


orderly progression through the various phases ofthe PAR project, moving from an examination ofcurrent practices, to delivery of training andconsultation, to implementation of a new model,and finally to reflection, there were times during theproject that the CHR participants exerted their ownsequence of events, as appropriate for the reality oftheir work. Rather than providing discrete andsequential series of training sessions in healtheconomics principles, for example, the researcherswere asked to provide pieces of training (the changeintervention) throughout the PAR project as needsfor more information arose.

Another significant challenge was the pressure oflimited timelines. PAR requires adequate time toallow change to happen without participants feelingthat it is imposed. The PAR process was con-strained by externally imposed deadlines from theprovincial health ministry, in that a budget for theCHR had to be formulated. Acknowledging that itis not always possible or advisable to try to ‘hurryup’ the process of PAR, the researchers nonethelesswere also working under the constraints of researchfunding timelines. Ideally, a PAR project progressesto the point that ownership for the change inpractice is shifted to the extent that researchers areremoved from the process, and the change or actionis fully self-sustaining.

Finally, the follow-up interviews with CHRsenior managers and clinicians were conducted bymembers of the research team who the informantsidentified as those proposing the method. It isacknowledged that this may, potentially, have ledthe informants to be unwilling to be completelyhonest in their reflections–particularly if the infor-mants had negative feelings about the researchapproach. On one hand, the informants’ feedbackon the overall research approach may have beperceived as more reliable if the follow-up inter-views had been conducted by an interviewercompletely external to the PAR project. On theother hand, the richness of the information from thefollow-up interviews and focus groups was madepossible by the in-depth understanding and obser-vations of the qualitative researcher who had beeninvolved through the entire project.

Further study

It was stated in a survey of authors of PBMAstudies that this approach will only work if there’sdirect involvement of health economists or aca-

demic health researchers (Mitton & Donaldson,2001). However, in priority setting experiences withother health authorities within specific programareas, we have learned that managers do not alwaysfeel the need to be supported by researchers (Halmaet al., 2004). In essence, it seems that decisionmakers need to have reinforcement of the theore-tical principles behind PBMA, whether it is from anexternal health economist or an internal supportpersonnel within the given organization. The role ofsuch an internal consultant would be to ensure thatthe PBMA process is sustainable and to build broadcross-organizational commitment. Rather than re-lying on external ‘experts’, it would be moresustainable for PBMA processes to be mentoredfrom within. Thus, health organizations interestedin developing an explicit approach to prioritysetting at the macro level may best be advised tostrike a dedicated ‘priority setting team’ to coordi-nate PBMA activities and provide both practicaland technical support to decision makers.

In addition, this qualitative inquiry has led tofurther research proposals, to evaluate the pilotedmacro-level priority setting framework in otherhealth organizations in both Canada and the UK.Themes generated through the interviews and focusgroups, such as factors for success of using anexplicit approach to priority setting, the implemen-tation of incentive systems to foster stakeholderengagement, the use of evidence to support deci-sions to improve patient outcomes, and the affect ofa framework on changes in actual health outcomes,will be explicitly tested in these further contexts.

Conclusions

In summary, PAR proved to be an appropriateand helpful approach to introducing a theoreticallydriven model of macro-level priority setting within alarge, complex health organization. The CHRmanagers and academic researchers engaged incooperative inquiry from different perspectives butwith the common goal of implementing PBMAwithin the CHR. The PAR process achieved twomajor outcomes: (1) the merging of theory andpractice with respect to a specific framework as amacro-level priority setting model within the CHR;and (2) the development of skills and understandingamongst CHR senior managers and cliniciansenabling them to apply a novel priority settingframework within their own work reality. Nextsteps would include consideration of striking a


priority setting team to guide future processes, andoutside the CHR, further investigation of macro-level practices in other contexts.

This application of PAR highlighted its value infacilitating the transfer of theory to practice, which,if conducted in a truly participatory manner, cancreate significant change in practice within arelatively short period of time. We would recom-mend the application of PAR in similar contextswhere a theoretical model must be introduced to agroup of decision makers with the aim of aiding andimproving their professional practice. However, it isimportant that support for the change is sustainedas long as necessary to engrain the new practicesinto the organization. We would encourage otherhealth organizations to apply PAR in building notonly fair and rigorous processes for making difficultfunding decisions, but in other areas of managementpractice as well. Not only can insights of decisionmakers improve the quality of research and ensureface validity, their involvement has importantimplications for the sustainability and appropriate-ness of theoretical models being transferred topractice.

Acknowledgments

The authors thank the members of the seniormanagement team in the Calgary Health Region fortheir participation in this project, as well as theCanadian Health Services Research Foundation forfunding this work. Craig Mitton receives salarysupport from the Canadian Priority Setting Re-search Network, and during 2003/2004, was anESRC Advanced Institute of Management Re-search (AIM) International Fellow. Cam Donald-son holds the Health Foundation Chair in HealthEconomics, and during 2003/2004, was an AIMPublic Services Fellow. The views expressed arethose of the authors, not the Calgary Health Regionor the funders.

References

Coast, J. (1999). The appropriate uses of qualitative methods in

health economics. Health Economics, 8, 345–353.

Coast, J. (2001). Citizens, their agents and health care rationing:

An exploratory study using qualitative methods. Health

Economics, 10, 159–174.

Coast, J., McDonald, R., & Baker, R. (2004). Issues arising from

the use of qualitative methods in health economics. Journal of

Health Services Research and Policy, 9(3), 171–176.

Cornwall, A., & Jewkes, R. (1995). What is participatory

research? Social Science & Medicine, 41(12), 1667–1676.

Dolan, P., & Cookson, R. (2000). A qualitative study of the

extent to which health gain matters when choosing between

groups of patients. Health Policy, 51, 19–30.

Donaldson, C., & Farrar, S. (1993). Needs assessment: Develop-

ing an economic approach. Health Policy, 25, 95–108.

Fals-Borda, O., & Rahman, M. A. (1991). Action and knowledge:

Breaking the monopoly with participatory action-research.

New York: Apex Press.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded

theory: Strategies for qualitative research. Chicago: Aldine.

Greenwood, D. J., & Levin, M. (1998). Introduction to action

research. Thousand Oaks: Sage Publications Inc.

Guba, E. G. (1978). Toward a methodology of naturalistic inquiry

in educational evaluation. Los Angeles: Center for the Study of

Evaluation.

Halma, L., Mitton, C., Donaldson, C., & West, B. (2004). A case

study on priority setting in rural southern Alberta: Keeping

the house from blowing in. Canadian Journal of Rural

Medicine, 9(1), 26–36.

Hart, E., & Bond, M. (1995). Action research for health and social

care: A guide to practice. Philadelphia: Open University Press.

Institute of Health Promotion Research (IHPR) (2000). Guide-

lines and categories for classifying participatory research

projects in health promotion. www.ihpr.ubc.ca.guidelines.htm

Jan, S. (2000). Institutional considerations in priority setting:

Transactions cost perspective on PBMA. Health Economics,

9(7), 631–641.

Lincoln, Y. S. (1992). Sympathetic connections between qualita-

tive methods and health research. Qualitative Health Re-

search, 2, 375–391.

Lomas, J. (1997). Reluctant rationers: Public input to health care

priorities. Journal of Health Services Research and Policy,

2(2), 103–111.

Madden, L., Hussey, R., Mooney, G., & Church, E. (1995).

Public health and economics in tandem: Programme budget-

ing, marginal analysis and priority setting in practice. Health

Policy, 33, 161–168.

Marshall, C., & Rossman, G. B. (1989). Designing qualitative

research. Newbury Park, CA: Sage.

McDonald, R. (2002). Using health economics in health services.

Rationing rationally?. Buckingham: Open University Press.

Miles, M. B., & Huberman, A. (1994). Qualitative data analysis.

Newbury Park, CA: Sage.

Miller, P. (1997). Managing informed purchasing: A survey of

decision makers. Journal of Management in Medicine, 11(1),

35–42.

Mitton, C., & Donaldson, C (2001). Twenty-five years of

program budgeting and marginal analysis in the health sector,

1974–1999. Journal of Health Services Research and Policy,

6(4), 239–248.

Mitton, C., & Donaldson, C. (2002). Setting priorities in

Canadian regional health authorities: A survey of key

decision makers. Health Policy, 60(1), 39–58.

Mitton, C., & Donaldson, C. (2004). Priority setting toolkit: A

guide to the use of economics in health care decision making.

London: BMJ Books.

Mitton, C., Patten, S., Waldner, H., & Donaldson, C. (2003).

Priority setting in health authorities: A novel approach

to a historical activity. Social Science & Medicine, 57,

1653–1663.

http://www.ihpr.ubc.ca.guidelines.htm


Mitton, C., & Prout, S. (2004). Setting priorities in the south west

of western Australia: Where are we now? Australian Health

Review, 28(3), 301–310.

Patton (1990). Qualitative evaluation and research methods (2nd

ed). Newbury Park: Sage.

Peacock, S. (1998). An evaluation of program budgeting and marginal

analysis applied in South Australian hospitals. Melbourne: Center

for Health Program Evaluation, Monash University.

Stringer, E. T. (1999). Action research (2nd Ed). Thousand Oaks:

Sage.

using participatory action research to build a priority setting process in a canadian regional...

Documents