This may be the author’s version of a work that was submitted/acceptedfor publication in the following source:
Ash, Susan, O’Connor, Jackie, Anderson, Sarah, Ridgewell, Emily, &Clarke, Leigh(2015)A mixed-methods research approach to the review of competency stan-dards for orthotist/prosthetists in Australia.International Journal of Evidence-Based Healthcare, 13(2), pp. 93-103.
This file was downloaded from: https://eprints.qut.edu.au/91967/
c© Consult author(s) regarding copyright matters
This work is covered by copyright. Unless the document is being made available under aCreative Commons Licence, you must assume that re-use is limited to personal use andthat permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then referto the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe thatthis work infringes copyright please provide details by email to [email protected]
Notice: Please note that this document may not be the Version of Record(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) canbe identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.
https://doi.org/10.1097/XEB.0000000000000038
1
AMixedMethodsResearchApproachtotheReviewofCompetencyStandardsfor1
Orthotist/ProsthetistsinAustralia.2
34SusanAsh,Ph.D,M.H.P.,Dip.Nut.Diet.,B.Sc.,Consultant,QueenslandUniversityof5
Technology,Brisbane,Queensland,Australia.6
7
JackieO’Connor,B.P.O.(Hons),ProjectOfficer,AustralianOrthoticProsthetic8
Association;POBox1219Greythorn,Vic3104,Australia.9
10
SarahAnderson,M.P.H.,B.P.O.,Lecturer,NationalCentreforProstheticsand11
Orthotics,LaTrobeUniversity,KingsburyDrive,Bundoora,Vic3086,Australia.12
13
EmilyRidgewell,Ph.D.,B.P.O.(Hons),Registrar,AustralianOrthoticProsthetic14
Association;POBox1219Greythorn,Vic3104,Australia.15
16
LeighClarke,M.P.H.,B.P.O.(Hons),ExecutiveOfficer,AustralianOrthotic17
ProstheticAssociation;POBox1219Greythorn,Vic3104,Australia.18
19
20
21
Acknowledgements:Theworkundertakeninthismanuscriptwasfundedbya22
ProfessionalServicesDevelopmentProgramgrantfromtheAustralian23
Government.ThanksareextendedtoShaneGrant,projectofficerforStage1of24
theproject.25
2
AMixedMethodsResearchApproachtotheReviewofCompetencyStandardsfor26
Orthotist/ProsthetistsinAustralia.27
Aim:Therequirementforanalliedhealthworkforceisexpandingastheglobal28
burdenofdiseaseincreasesinternationally.Tosafelymeetthedemandforan29
expandedworkforceoforthotist/prosthetistsinAustralia,competencybased30
standards,whichareup‐to‐dateandevidencebased,arerequired.Theaimsof31
thisstudyweretodeterminetheminimumlevelforentryintothe32
orthotic/prostheticprofession;todevelopentrylevelcompetencystandardsfor33
theprofessionand;tovalidatethedevelopedentrylevelcompetencystandards34
withintheprofessionnationally,usinganevidencebasedapproach.35
Methods: A mixed methods research design was applied, using a three step36
sequentialexploratorydesign,wheresteponeinvolvedcollectingandanalyzing37
qualitative data from two focus groups; step two involved exploratory38
instrument development and testing, developing the draft competency39
standards; and step three involved quantitative data collection and analysis, a40
Delphisurvey.InStage1(steps1and2),thetwofocusgroups,anexpertanda41
recent graduate group of Australian orthotist/prosthetists, were led by an42
experiencedfacilitator,toidentifygapsinthecurrentcompetencystandardsand43
then to outline a key purpose, work roles and tasks for the profession. The44
resulting domains and activities of the first draft of the competency standards45
were synthesised using thematic analysis. In Stage 2 (step 3), the draft46
competencystandardswerecirculatedtoapurposivesampleofthemembership47
of theAustralianOrthotic Prosthetic Association, using three rounds of Delphi48
survey.Aprojectreferencegroupoforthotist/prosthetistsreviewedtheresults49
ofbothstages.50
3
Results: In Stage 1, the expert (n=10) and the new graduate (n=8) groups51
separatelyidentifiedworkrolesandtasks,whichformedtheinitialdraftofthe52
competency standards. Further drafts were refined and performance criteria53
added by the project reference group resulting in the final draft competency54
standards.InStage2,thefinaldraftcompetencystandardswerecirculatedto5655
members(n=44finalround)oftheAssociation,whoagreedonthekeypurpose,56
6 domains, 18 activities and 68 performance criteria of the final competency57
standards.58
Conclusion: This study outlines a rigorous and evidence‐basedmixedmethods59
approach for developing and endorsing professional competency standards,60
whichisrepresentativeoftheviewsoftheprofessionoforthotist/prosthetists.61
4
Background:62
TheWorldHealthOrganisation(WHO)WorldReportonDisabilityrecommends63
that rehabilitation services are multidisciplinary, effective and accessible to64
those with disabilities. Approximately 15% of the world’s population65
experiences some form of disability. The disabling barriers include lack of66
servicesforhealthcareandrehabilitationbutalsopoorcoordinationofservices,67
inadequatestaffingandweakstaffcompetencies(1).68
Orthotist/prosthetists are tertiary qualified allied health professionals who69
clinically assess the physical and functional attributes of individuals with70
mobility and functional limitations. These limitations may result from illness,71
injuryand/ordisability,includinglimbamputations.Orthotist/prosthetistsmay72
then prescribe and facilitate the provision of orthoses and prostheses to73
minimize the effect of these limitations. Australia currently has an74
orthotist/prosthetist workforce of approximately 400 practicing clinicians,75
whichrepresentsalowratiopercapita (1/56,552ofpopulation)(2)compared76
totheUKbenchmarkfororthotistsalone,of1/30,555ofpopulation(3).Health77
workforce shortages have been recognized around the globe, however simply78
trainingmoreof the same typeofhealthworkforcewillnotmeet theneedsof79
complex health environments. The WHO has recommended cooperation80
between educational institutions and health employing agencies to match81
professionaleducationtohealthservicedelivery(4).Oneofthekeydeliverables82
fromtheAustralianHealthWorkforceStrategicReformforActionistocreatean83
adaptablehealthworkforce that is equippedwith the competencies toprovide84
teambasedandcollaborativemodelsofcare(5).85
5
TheAustralianOrthoticProstheticAssociation(AOPA)developedtheiroriginal86
competencystandardsin1999,withsmallchangesamendedin2003(6),87
howeverneitherprocesshadinvolvedanendorsementprocessbythe88
membership.Trainingfortheprofessionisfocusedinoneuniversityandthe89
numberofgraduatesfromthissourceisnotlikelytomeetthecurrentorfuture90
workforcedemandscreatedbyanincreasinglyageingpopulationandincreased91
ratesofchronicdisease.Up‐to‐datecompetencystandardsarethusrequiredto92
describe21stcenturypracticeofAustralianorthotist/prosthetistsandtoensure93
anyinternationallytrainedorthotist/prosthetists,wishingtofillworkforcegaps94
andpracticeinAustralia,havemettransparentstandards.95
Mixed methods research can address exploratory and confirmatory questions96
usingbothqualitativeandquantitativeapproaches(7).Narrativeinterviewsand97
guided focus groups have been used in health professions to explore themes98
around professional competencies (7‐10). Thematic analysis of interview data99
canprovidearigorousbutflexiblemethodforundertakingqualitativeresearch100
thatismethodologicallysound(11).BraunandClarkesuggestthatusingwritten101
transcripts from initial data collection to assignmeanings or codes; clustering102
thesemeanings into themes; integrating the results into an exhaustivemap of103
theanalysis;andthenvalidatingtheresultswithextractsandanalysisfromthe104
relevantliterature;providessucharigorousapproach(11).105
TheDelphiprocessisusedtodeterminethelevelofconsensusonanissueand106
uses a quantitative approach. It attempts to overcome the disadvantage of107
domination of the group by individuals or those with a vested interest. The108
extent to which each respondent agrees with the issue is measured on a109
numerical or categorical scale. Surveys are generated to explore the issue110
6
involved over numerous rounds, usually three,with participants anonymously111
ratingtheirlevelofagreement(12).ThefirstroundoftheDelphiofteninvolves112
expertsprovidingtheiropinionsonaspecificmatterandthesubsequentrounds113
providingthelevelofagreement.Criticsofthismethodpointoutthatagreement114
doesnotnecessarilymeanthatthecorrectinterpretationhasoccurred.Despite115
these reservations, the Delphi process has been used globally in health116
professionalcompetencydevelopment,includingnursing(13)andnutritionand117
dietetics (14, 15). The reactive Delphi is a variation of the traditional Delphi118
whereinformationispresenteddirectlyintothefirstroundofthesurvey.Whilst119
the reactive Delphi may constrain initial opinion (16), this technique is often120
utilisedsubsequenttopreviousresearch(13,16,17).121
Theaimof thispaper is todescribe themixedmethods researchmethodology122
used by AOPA to determine the minimum level for entry into the123
orthotic/prosthetic profession in Australia; to develop entry level competency124
standards for the profession; and to validate the developed entry level125
competencystandardswithintheprofessionnationally. 126
Methods:127
Amixedmethodapproach,usingathreestepsequentialexploratorydesignwas128
used,wheresteponeinvolvedcollectingandanalyzingqualitativedatafromtwo129
focus groups; step two involved exploratory instrument development and130
testing, developing the draft Competency Standards; and step three involved131
quantitative data collection and analysis, a Delphi survey. The study was132
conductedintwostagesandwasoverseenbyaprojectreferencegroup,which133
included the representatives of the professional association executive, those134
involvedinresearchanduniversitytraining,aprojectofficerandanexperienced135
7
facilitator.Stage1(February–May,2013)involvedtheidentificationofgapsin136
the 2003 Competency Standards and development of the key purpose of the137
profession and the domains, activities and performance indicators of the draft138
competency standards. Stage 2 (November, 2013 – April 2014) involved the139
confirmationorvalidationofthesecompetencystandardsandtheirfinalization140
(seeFigure1).Ethicsapprovalwasgrantedbythe<deidentified>HumanEthics141
Committee.142
143
Figure1nearhere.144
145
In Stage 1, semi‐structured focus groupswere used to elicit themes about the146
gaps,keypurpose,workrolesandtasksofAustralianorthotist/prosthetistsfrom147
twogroups,anexpertgroupandarecentgraduategroup.Inthispaper,wehave148
defineddomainasaworkrole,sometimesreferredtointheliteratureasaunit149
of competencyor standard,activityasa taskperformedwithin thatworkrole,150
sometimes referred to as a core competency or element and performance151
indicatorasastatementofhowtheactivityortaskwouldbemeasured.152
A series of questions, shown in Table 1, were presented to both groups and153
discussion facilitated by the experienced facilitator with expertise in154
competency standardsdevelopment andapractitionerof anotheralliedhealth155
profession (SuA). The questions have been used previously in competency156
standardsdevelopment(9).157
158
Table1nearhere.159
8
Focus group participants were all members of AOPA or teaching into the160
academic course and actively employed in the profession. Expert group161
participants had aminimumof 10 years’ experience in the orthotic/prosthetic162
fieldand/orwereaknownexpert.Recruitmentoftheexpertgroupwasinitially163
through direct approach via email. The recent graduate group graduated in164
Australiabetween2009‐2011andwererecruitedthroughanadvertisementon165
theAOPAwebsite.Purposivesamplingwasusedtoensurerepresentationfrom166
different workplace settings within both groups, with a maximum number of167
twelveineachgroup. Focusgroupswereconductedface‐to‐facefortheexpert168
group and via teleconference/videoconference where available for the recent169
graduategroupandwererecordedandtranscribedverbatim.170
Tworesearchers,theprojectofficerandexperiencedfacilitator,undertook171
thematicanalysisofthetranscriptsindependently.Textinthetranscriptswas172
readtoidentifyinitialcodes,whichwereunderlinedandannotatedinmargins. 173
Majorthemeswereidentifiedfromthesecodesandcomparedforcongruence174
beforebeingcategorisedintomajorworkrolesandcounted.Relationships175
betweenworkrolesandthesubthemeofworktasks,wereestablished.These176
themeswerethentabulatedasworkrolesandtasks.177
Themesrelatingtothekeypurposeoftheprofessionweregroupedtoformfour178
mainstatements.Gapsinthecurrentcompetencystandardsweretabulated179
separatelytoinformthedomains,activitiesandperformanceindicators.180
Performance indicators were added to the domains and activities after the181
project reference group agreed on the domains and activities. The project182
reference group generated six iterations of this draft before the expert group183
9
met face‐to‐face to agree on the final draft, again facilitated by the same184
experiencedfacilitator(SuA).185
BetweenStage1and2,theprojectreferencegroupdefinedtermsandcontexts186
forassessment,viarangestatements.Rangestatements,sometimesreferredto187
as range variables, are defined as ‘the part of a unit of competency which188
specifiestherangeofcontextsandconditionstowhichtheperformancecriteria189
apply’ (18).Definingtherangestatementswasthefinalstageofdevelopmentof190
thedraftcompetencystandards,whichwereusedinStage2.191
Stage 2 involved the use of the reactiveDelphi technique to validate the draft192
competency standards. Participants were required to be full or part time193
financial members of AOPA who had consented to receive AOPA emails. The194
AOPA database was searched to determine membership demographics in195
relationtoage,location,genderandworktype.Fromthe271eligiblemembers,196
184 were identified as meeting the following purposive sample criteria for197
utilizingand/orbeinginterestedincompetencystandards;beinginternationally198
trained;teachingwithintheacademiccourse;beingfacilitymanagerspotentially199
employinggraduatepractitionersorsupervisingstudentsonpracticeplacement.200
Invitations to participatewere sent via email to a subset ofmembers (n=107)201
whosedemographicsreflectedthebroadermembership.202
Surveys were constructed using SurveyMonkey®(19) and were distributed203
alongwith supporting documentation via the AOPAwebsite. A personal email204
with completion instructions was sent to each participant in each round.205
Demographic and professional informationwas collected in Round 1. Prior to206
disseminationall surveyswerepilotedby theproject referencegroupand five207
otherswhowerenoteligibletoparticipate inthesurveybuthadknowledgeof208
10
andinterestintheproject.Surveyswerecompletedoveratwoweekperiodwith209
reminder emails sent at oneweek, 72hours and24hoursprior to closing the210
surveyinordertoreduceattritionrates.211
InRound1,thedomainsandactivitieswerepresentedwithouttheperformance212
indicatorsandrangestatements.Participantsexpressedtheirlevelofagreement213
thateachactivitywasrequiredofanentrylevelorthotist/prosthetistinorderto214
achieve safepractice andpositive client outcomes, using a5point Likert scale215
(strongly disagree, disagree, undecided, agree, strongly agree)with the option216
for open‐ended comment on all aspects of the document. The percentage of217
participantsnominatingeachoftheLikertcategorieswascalculated.Agreement218
wasdefinedas≥75%ofparticipantsnominatingeitherstronglyagreeoragree. 219
The competency standardswere adjusted in light of feedbackbetween rounds220
andparticipantswereprovidedwithasummaryofgroupresultsandanoutline221
ofanychanges.222
InRound2participantswereasked torate theiragreementwithanyactivities223
thatdidnotachieve75%agreementaswellasallperformanceindicators.Range224
statementswereincludedinRound2inordertoprovidefurthercontext.225
In Round 3, participants were asked to rate their agreement with any226
performanceindicators,whichdidnotachieveagreementinRound2aswellas227
their agreement with the key purpose of the profession. In this round228
participantswerealsoprovidedwithanindividualsummaryofRound2results.229
230
Results:231
SampleCharacteristics232
11
InStage1,10expertsand8graduatesparticipatedinthefocusgroups.InStage233
2,52%(n=56)of those invited toparticipate consented todo so. Seventynine234
percent(n=44)ofthosewhoconsentedcompletedallthreerounds,resultingin235
anattritionrateof21%.Ultimately,16%ofthemembershipparticipatedinthe236
threeroundsoftheDelphiprocess.Thedemographicdetailsoftheparticipants237
in Stage 1 and 2 are shown in Table 2. Demographics were similar in both238
samplesforStage1andStage2.239
240
Table2nearhere.241
242
DevelopmentofCompetencyStandards243
Stage1:244
Keythemesidentifiedasgapsinthecurrentcompetencystandardsweresimilar245
intheexpertandgraduatefocusgroups.Theseincluded;anemphasison246
evidence‐basedandethicalpracticewithaclient/patientfocus;theneedfor247
continuingprofessionaldevelopment;andmentoringtoremainprofessionally248
currentandidentificationofscopeofpractice.Graduatesfeltverystronglythat249
theirtechnicalskillswereessentialtotheirrole.250
Statementsaboutgapsincluded:251
Evidence:252
‘youshouldhavetheskillstoclinicallyjustifywhyyouareprescribingit,and253
yes, you should have the clinical skills to review it and see what the254
functionaloutcomesofyourdecisionsareforapatient.’ExpertB255
Continuingprofessionaldevelopment:256
12
‘anessential skillwouldbe to identify thegaps inyourknowledgeand in257
yourpractice,maybeitwouldbeworthincludingsomethinginhavingthat258
skill. And also secondly following on from that, being able to access259
research and have the methods to be able to gain knowledge of best260
practice.’GraduateB261
Graduateswereabletoclearlydefineakeypurpose,whichinvolvedtheclinical262
decisionmakingandtechnicalskillsrequiredtoperformtheroleofthe263
orthotist/prosthetist.264
Keypurposestatementsincluded:265
‘ItellpeopleImakeartificiallimbs’GraduateA266
‘weassessapatient,andthenwecomeupwithaprescriptionfortheirdevice,267
evenifit’srelatedtoaprosthesisandit’sprettyclearthattheyneedone268
belowtheknee…there’slotsmoreinvolvedwiththat,andperhapssayinga269
prescriptionisagoodwordtoinclude.Alsotoincludethefactthatwe270
clinicallyassessthepatientusingourclinicalknowledge.’GraduateB271
Theexpertgrouphadamorediverserangeofviewsaboutthewordingofthe272
keypurpose,perhapsreflectingtheirgreaterdiversityofemploymentroles,273
howevertheoverallthemesofclinicalcareandtechnicalskillweresimilar.274
‘weareabletoconductfullbio‐mechanicalassessmentsandlinkthosebio‐275
mechanical assessmentwith intimate knowledge ofmaterials, technology276
and gait goals and functional goal, tomatchmaterials, technology and277
patient presentation with that equipment, or thatmaterial tomeet the278
patientoutcomes.’ExpertA279
Fourkeypurposestatementsweredevelopedfromtheanalysisoftheinitial280
focusgroupsfordiscussionduringthesecondexpertfocusgroup.281
13
Thethemesaroundworkrolesandtasksfrombothgroupsweresimilarandare282
showninTable3.283
284
Table3nearhere.285
286
Thereweresomeareaswhichrecentgraduatesthoughtshouldbecoveredinthe287
competencystandardsandwhichtheexpertsfeltwerebeyondentrylevel.288
Intheservicemanagementarea,graduatesgaveexamplesofbeinginsituations289
wheretheymayhavebeenthesolepractitionerwithinmonthsofgraduation.290
‘theseniorclinicianhere….hadtobeawayforaboutnineweeksforahealth291
issue,soIwasleftwiththewholedepartmentwithoutanytechnicians,andby292
myself,managing…doingthetechworkandtheclinicalworkhere.’Graduate293
C.294
‘Ithinkinthefuture…we’llallbeinvolvedwithsettingupmoreoutreach295
clinics,anddevelopingservicedeliverytothebroaderpopulation.’Graduate296
B297
Expertsontheotherhand,focusedmoreonskillsthatwereminimumandhow298
youcouldconsistentlyassessthese.299
‘theyneedtohavetimemanagementskillsandprojectmanagementskills’300
ExpertB.301
‘those key kind of graduate management skills, self‐management, time302
managementarethecoreones.’ExpertC.303
The initial draft domains and activities were developed from Table 3, with304
reference to the 2003 Competency Standards and presented to the project305
referencegroup.Atthispoint,referencewasmadetointernationalorthoticand306
14
prosthetic andotherAustralianhealthprofessions’ competency standards.The307
processofaddingrangestatementsresultedinamendmentstothecompetency308
standards to avoid repetition and increase clarity. The final draft was further309
refinedtoinclude6domains,20activitiesand69performancecriteria.Figure2310
showsthe6domainsofpractice.311
312
Figure2nearhere313
314
Stage2:ValidationoftheCompetencyStandards315
Atotalof56participantscompletedthefirstroundoftheDelphisurvey.316
Participantsagreedthateighteen(18,or90%)ofthe20activitieswere317
appropriateforanentry‐levelorthotist/prosthetist(seeFig3).Agreement318
rangedfrom87.5to100%with16activitiesreceiving>90%agreement.319
320
Figure3nearhere.321
15
Twoactivities,bothwithinDomain4(ServiceManagementandImprovement),producedan
undecidedresult:
4.3 Managesclientfundingallocation(53.6%agree,33.9%undecided,12.5%
disagree)and
4.4 Participatesinfacilityresourcemanagement(73.2%agree,17.9%undecided,
8.9%disagree).
Commentsfromparticipantssuggestedthatwhiletheconceptwascorrect,thewording
indicatedactivitiesappropriateforaskilllevelhigherthanentrylevel.AsActivities4.3and
4.4hadanundecidedresultratherthandisagreement,thereferencegrouprevisedthe
activitieswithinthisdomaintoallowtheintendedconceptstoremain.Thisprocessresulted
intheconceptsofActivities4.3and4.4beingincorporatedintoperformancecriteria,rather
thanactivities.
DuetothehighlevelofagreementinRound1,Round2progressedtoratingtheperformance
indicators. A specific question asking for comments in relation to Domain 4 activitieswas
includedtoallowforfurtherfeedbackonthechangesmadefromRound1.
Forty‐nine(49)participants(87.5%)completedthesecondroundofthesurvey.
Participantswereaskedtoratewhethertheperformanceindicatorsdescribedan
observableand/orassessableactionwhichisexpectedoftheworkforcewhen
performingtherelevantactivity.Participantsagreedthat68outofthe69performance
indicatorswereassessabletasksoftherelevantactivity.Agreementlevelsrangedfrom
79.6to100%,with66performanceindicatorshavinganagreementlevel>90%(see
Table5).Participantsdidnotagreeonperformanceindicator5.3.3‐acquiresfurther
16
qualificationstopracticebeyondprofessionalscopeofpractice(54.2%agreement)‐in
Domain5(ProfessionalValuesandBehaviours).
Table5nearhere.
As inRound1, theproject reference group revised theperformance indicatorswith<75%
agreement,basedonthequalitativecommentsprovidedbyparticipants.Althoughonlyone
performance indicator did not achieve agreement, all three performance indicatorswithin
this activity (Activity5.3)werepresented to participants for re‐assessment inRound3. In
Round3,participants(n=44)agreedthatthesethreerevisedperformanceindicators(listed
below)wereallassessabletasksoftherelevantactivity:
5.3.1Workswithinprofessionalscopeofpracticeandauthorityprovidedbytheclient
andemployer(95.4%agreement)
5.3.2 Seeks assistance or refers on when beyond own level of competence (97.7%
agreement)
5.3.3 Recognises where further training is required to conduct independent practice.
(97.7agreement)
DISCUSSION:
Thisstudyisoneoffewoutliningthemixedmethodsresearchmethodology(13,16,20)used
todevelopor reviewcompetencystandards inhealthprofessions.Other studieshaveused
similar methodology however few have combined rigorous qualitative and quantitative
methodologiestodevelopandthenvalidateprofessionalstandards.
17
Sherbino et al (20) conducted a mixed methods study to define the key roles and
competenciesofClinician‐Educators.Expertsparticipatedinfivefocusgroups(finalsample,
n=22), to define attributes, domains of competence and core competencies. One national
surveyofkeyeducator stakeholders (n=1110)validated the results,usingamixof4‐point
categorical responses, such as agree/disagree and important/not important. Mixing
categorical responseswithouta secondroundof surveyingpotentiallyweakens theoverall
result.O’ConnellandGardner(8)usedcriterionsamplingofemergencypracticenurses(n=5)
to develop a competency framework for emergency nursing practice. The draft specialist
competenciesandperformanceindicatorsweresenttoanexpertpanel(n=12)viaaseriesof
Delphi surveys for agreement.Thenumberswere small andpossiblynot representativeof
thenursingprofession,howeverboththesestudiesusedsimilarmethodologyasthecurrent
study.
Oneof theunique featuresofour studywas theuseof separateexpertandgraduate focus
groups. The experts were chosen to represent key stakeholders, especially officers of the
professionalassociation,previousauthorsofthecompetencystandards,universityeducators
andmanagersof largedepartmentsemployingnewgraduateorthotist/prosthetists.This is
similar to other studies in allied health(21). The graduate focus group was chosen to
represent relatively recent graduates whomay be working in emerging areas of practice.
Interviewsof recentgraduates indieteticshaveshownthat theyoftendescribeworking in
emergingsituations,whichseniorpractitionerswouldnotconsiderentrylevel(9,22). Asa
stakeholder group, graduates are rarely approached as a separate group in the review of
competency standards in other allied health professions. Much discussion occurred in the
expert group about whether the standards were minimum on graduation or aspirational,
18
describingentry levelpractice in the first6 ‐12months.Theverydefinitionofcompetency
havingtobeassessedintheworkplacemakestheformer,ieminimumongraduation,difficult
to achieve, however for professional registration or recognition purposes, this is the
norm(21, 23). Despite this, the resulting domains or work roles, shown in Figure 2, were
grouped similarly to other health professions; collaboration/sharing/communicating;
therapeutic role; prioritizing workload; ethical care; promoting best practice;
professionalism(24) or Clinical Expert, Communicator, Collaborator, Manager, Health
Advocate,Scholar,andProfessional(25).
Several iterations of the initial draftwere considered first by the project reference group,
priortopresentationtothesecondexpertfocusgroupfordiscussion.Theprojectreference
groupagainrefinedthedraftcompetencystandardspriortothefinaldraftbeingusedinthe
Delphiprocess.Thisprocess,step2intheexploratorysequentialmodeloutlinedabove,and
outlined in Figure 1, allowed reflection and integration of views but also review of
appropriateliteratureandotherprofessionalcompetencystandards.
Thequantitativestep,step3orDelphiprocessallowedarankingofagreementfromawider
group of participants. Jones and Hunter(12) describe consensus methods as needing
anonymity, iteration, controlled feedback and statistical group response (only if scores are
sequentialandnotcategorical).OurresultsfromthefirstDelphiroundshowedremarkable
agreement, with 16 activities achieving >90% consensus. Revision of the Service
ManagementandImprovementdomain, theonlydomainwhereactivitiesshowed less than
90%agreement,resultedinsomeactivitiesbeingre‐writtenasperformanceindicators.This
was consistent with both focus groups perceiving management differently, at entry level.
Commentsfromparticipantsindicatedthatmuchofthelackofagreementinvolvedwording
19
of the activity rather than the activity itself. Therefore, in the second round of Delphi, all
revised activities were shown but participants rated their level of agreement with the 69
performance indicators,within those activities. ThismodifiedDelphimethod is similar to
that usedbypharmacists inprimaryhealth care (16)where anonline surveywasused to
agree on competency descriptions, organized into domains, elements and sub‐elements.
Participantswerethenaskedtorankona6pointLikertscalehowoftentheyperformedthe
sub‐element(performance indicator)andhowcritical theythoughtthesub‐elementwasto
achieve patient outcomes. The results expressed as importance rankings were seen as
validatingtheoriginalcompetencystatements.
Other studies have used subsequent rounds ofDelphi to improve agreement.Hughes et al
used two rounds of Delphi surveys to improve agreement on 143 competency elements
(activities) inpublic healthnutrition,howeveronly reported the shift in agreementon the
109elements forwhich therewas>67%agreement(14). Others inadvanceddieteticsand
nursing, have reported changed median (17) or mean score(8, 13) respectively between
rounds.
Thehigh levelof agreement inour study forall roundsof theDelphiwasnotunlikeother
studies, although other professional competency studies used differentmeanings for their
ratingscalesanddifferentmethods forcomputing therankings.Generallyconsensus levels
are high. Hughes et al only reported 33 (out of a possible 143) competencies rated as
essential by 100% of participants (14). O’Connell and Gardner reported mean agreement
scoresrangingfrom4.6‐4.9outofamaximumscoreof5(8).
Thestrengthofthisstudyisthatitusedastructuredexploratoryandconfirmatoryapproach
withacceptedqualitativeandquantitativemethods todevelopand thenvalidateminimum
20
entry level competency standards for orthotist/prosthetists in Australia. The level of
agreementwasveryhigh(>87%for18outof20activities,seeFigure3).Wheretherewasno
agreement in the first rounds of the Delphi survey re‐wording resulted in very high
agreementonsubsequentrounds(>80%forall18activitiesand68performancecriteria,see
Table3).
Limitations include thesmallnumbers,withonly16%of the totalmembership involved in
the validation, however these numbers and the attrition rate of 21% are consistent with
other studies of competency validation in other professions (8, 14, 16). Most participants
graduatedfromtheonlyprograminAustralia,whichcouldsuggestthatorthotist/prosthetist
experience is constrained by training in only one institution in Victoria. Every attempt
howeverwas used to ensure representativeness of the profession in Australia. Due to the
purposive sampling criteria, only184outof271memberswereeligible toparticipateand
only 107members invited to participate to reflect themembership profile. Every attempt
was made to include orthotist/prosthetists who were members of AOPA but had trained
outside Australia and participants were selected to be representative of the AOPA
membership,includingthosewhoworkedoutsidethemetropolitanarea.Table2showsthat
despite a large percentage of respondents being Victorian this is representative of the
membership and where the majority practice. Further research could apply these
competency standards to assess entry‐level practice for graduates exiting an education
program,oralternatively,orthotist/prosthetistsfromothercountries.
Overall this study has outlined an evidence‐based approach to the review of competency
standardsforahealthprofession,suchasorthotist/prosthetists.Themethodologyisrobust
andcouldbeappliedtootherdisciplines.
21
Table1.Focusgroupquestions1. WhatdoyouthinkarethegapsinthecurrentCompetencyStandards?2. Whatisthekeypurposeoftheprofession?3. Whatischangingorlikelytochangeintheprofessionthatmightaffect
thispurpose?4. Whatmusthappenforthekeypurposetobeachieved?5. Whydoestheprofessiondoit?6. Whatmajorthingswouldyouhavetodotoperformthatrole?7. Returningtothegapsidentifiedearlier,doyouwishtochangeoradd
anything?
22
Table2Mean(±SD)ageandproportion(%)byfemalegenderandworklocationofoverallOrthotist/Prosthetistprofession,focusgroupsandDelphiparticipants. Membership
Profilen=271
ExpertFocusGroupn=10Stage1
GraduateFocusGroupn=8Stage1
DelphiParticipants(3rounds)n=44Stage2
Mean(±SD)Age(yrs)
39(±11.6) 41(±9) 27(±5) 38(±11.8)
Female(%) 39
50 78 43
*Victoria(%) 52
100 56 48
ThesoletraininginstitutionforOrthotist/ProsthetistsinAustraliaisinVictoriaandthemajorityofgraduatesareemployedinthatstate.
23
Table3InitialthemesdevelopedfromfocusgroupsWorkroles TasksCollaborativeclientcare Managespatientassessmentand
treatmentaspartofahealthcareteamAssesses,prescribes,treats,plansandrefers,liaiseswithotherprofessionals
EthicalandSafePractice Usesevidence,justifiesinterventionFollowslegislation,occupationalhealthandsafetyregulationsKnowsscopeofpractice
Communicatesusinggoodwritten,oralandinterpersonalskills
Canusearangeofmodalitiesincludinge‐healthAdvocatesforpatientinavarietyofforumsAdvocatingforotherstaff
Managesresources ManagesselfManagesbudgets,fundingsystemsandadministrationofthese.
PrescribesmaterialsandtechnologywithinOrthotist/Prosthetistscopeofpractice
Manufacturesoroverseemanufactureofcustomdevicesandmodifications
Continuingprofessionaldevelopment Engagesinselfdevelopment
24Table4:PercentageAgreementforPerformanceIndicatorsinRound2and3DelphiSurveysPerformanceIndicator–finalwording Round
2Round3
1.1.1Ensuresallinteractionswiththeclientand/orcarerdemonstraterespect,honesty,empathyanddignity,andareconductedinaculturallyappropriatemanner
95.9
1.1.2Ensurestheclientisthefocusofthecarepathway 91.8 1.1.3Ensurestheclientand/orcarerisawareoftheirrightsandresponsibilities 93.9 1.1.4Obtainsinformedconsentfromtheclientand/orcarerpriortotheprovisionofcare 95.9 1.1.5Listenseffectivelytotheclientand/orcarer 100 1.1.6Encouragestheclientand/orcarertoparticipateandprovidefeedback 98 1.1.7Providesprompt,accurateandcomprehensiveinformationincleartermstoenableclientsand/orcarerstomakeinformeddecisions
95.9
1.2.1Receivesandformulatesclientreferrals,professionalhandovers,healthcareteamreportsandothertreatmentplans
98
1.2.2Respects,acknowledgesandutilisestheexpertiseofotherhealthprofessionals 100 1.2.3Establishesandmaintainseffectiveworkingrelationshipswithotherhealthprofessionalstoenhancecollaborativepracticeandaccesstocare
100
1.2.4Activelyparticipatesinhealthcareteamsandseeksopportunitiestodemonstrateprofessionalexcellence
89.8
1.3.1Providesclinicaljustificationandevidenceforprescribedorthotic/prostheticclientcare 93.9 1.3.2Providesrelevantinformationinordertofacilitateclientaccesstocare 79.6 2.1.1Identifiessubjectiveandobjectiveinformationtoenabledevelopmentofanappropriateorthotic/prostheticmanagementplan
98
2.1.2Selectsassessmenttechniques,outcomemeasuresandothertools/instruments,basedonevidencewhicharerelevanttotheclient’spresentation
95.9
2.1.3Performsassessmentprofessionally,safelyandeffectively 100 2.2.1Accessesandutilisesthebestavailableevidencetoguideclinicaldecisions 93.8 2.3.1Facilitateclientand/orcarertoestablishpersonalgoals 93.9 2.3.2Considerstheinformationobtained,theclientand/orcarer’sgoalsandavailableevidencewhenformulatingtreatmentoptions
100
2.3.3Discussestreatmentoptionswiththeclientand/orcarertosupportclientcentredcareandinformedchoice
98
2.3.4Discussesshortandlongtermtreatmentgoalswiththeclientand/orcarer 91.8
252.3.5Identifiesclientswhorequirecollaborativecareandliaiseswiththehealthprofessionalteamtoensureintegratedcareplanning
98
2.3.6Determinesandjustifiesthedesigndetailsoftheorthosisand/orprosthesisprescription 95.9 2.3.7Includesclient,carerand/orhealthprofessionalteameducationandfollow‐upwhenplanningtreatment
95.9
2.3.8Selectsappropriateoutcomemeasures 81.7 2.4.1Considersallrelevantcharacteristicsoftheclientduringorthosis/prosthesisfittingandreviewprocesses
100
2.4.2Usesappropriatetechniquestoensureoptimalfitandfunctionoftheorthosis/prosthesis 93.9 2.4.3Reviewstheclientatappropriateintervalstoevaluatefit,function,qualityandsafetyoftheorthosis/prosthesis
100
2.4.4Evaluatesandmonitorstreatmentoutcomesusingpatientfeedbackand/oroutcomemeasures 98 2.4.5Modifiestreatmenttoensurebestpossibleoutcomesaremaintained 95.9 2.4.6Discussesprogresstowardgoalswiththeclientand/orcarer 93.9 2.5.1Adherestolegislativeandorganisationalrequirementsforalldocumentation 98 2.5.2Maintainslegible,conciseandaccuratedocumentationusingcontemporarymethods 98 2.5.3Safelyandsecurelystoresinformationandactstomaintainconfidentialitywhilstensuringavailabilityofinformationtootherhealthprofessionalsinvolvedinthecarepathway
93.8
3.1.1Utilisesappropriatecasting,measuringand/orcastmodificationtechniquestofacilitatefabrication 100 3.1.2Fabricatesand/orcoordinatestheoptimalfabricationoforthoses/prostheses 96 3.1.3Performsand/orcoordinatesrequiredmodificationsoforthoses/prostheses 93.8 3.2.1Assessestheorthosis/prosthesisforstructuralsafetyatappropriateintervals 93.9 3.2.2Ensurestheorthosis/prosthesisiscompliantwithmanufacturerguidelinesandstandards 95.9 4.1.1Facilitatesappropriatecompletionofallsupportiveactivities 87.8 4.1.2Facilitatesappropriatecompletionoftreatmentprovision 93.9 4.1.3Demonstratesanabilitytotriageindividualclientcaseloadwithinbroaderfacilityworkload 95.9 4.2.1Determinesavailablefundingforprescribedcareplan 95.8 4.2.2Preparesand/orcoordinatessubmissionofdocumentationforclientfundingsupportasrequired 95.9 4.2.3Prescribesanddesignsorthosis/prosthesistoachieveoptimaloutcomeswithintheapprovedbudgetforclientcare
91.7
4.2.4Understandsandconformstofundingarrangements,budgetallocations,statisticalreportingandfinancialtransactionrequirementsrelevanttotheworkplace
91.6
4.3.1Strivestocontinuallyimproveefficiency 89.6
264.3.2Recognisesservicegapsorinefficienciesandworkscollaborativelytoidentifysolutions 91.7 4.3.3Participatesinauditprocessesandqualityimprovementinitiatives 95.8 5.1.1Adherestolegislationandworkplaceguidelinesrelatingtosafety 100 5.1.2Identifiesworkplacehazardsandactstoeliminateorreducerisks 100 5.2.1Complieswithrelevantlegallaws,regulations,policiesandguidelines 100 5.2.2Abidesbyapplicablecodesofethicsandconduct 100 5.2.3Recognisestheresponsibilitytodonoharm 97.9 5.2.4Recognisesandrespondsappropriatelyifclientand/orcarerisatrisk 100 5.3.1Workswithinprofessionalandpersonalscopeofpracticeandauthorityprovidedbytheclientandemployer(Round2)
97.9
5.3.1Workswithinprofessionalscopeofpracticeandauthorityprovidedbytheclientandemployer(Round3)
95.4
5.3.2Acquiresfurthertrainingandassessmenttodeveloppersonalscopeofpractice 91.7 5.3.2Seeksassistanceorrefersonwhenbeyondownlevelofcompetence 97.75.3.3Acquiresfurtherqualificationstopracticebeyondprofessionalscopeofpractice(Round2) 54.2 5.3.3Recogniseswherefurthertrainingisrequiredtoconductindependentpractice(Round3) 97.76.1.1Undertakesindependentlearningtofurtherownknowledgeandskillsonacontinuousbasis 95.8 6.1.2Sharesskillsandknowledgewithhealthprofessionalcolleaguesandstudents 100 6.1.3Participatesinhealthprofessionaltrainingandresearchasopportunitiesarise 89.6 6.1.4Seeksoutleadersintheprofessionforadviceandmentoring 97.9 6.1.5Offersconstructivefeedbackandassistancetootherhealthprofessionals 95.8 6.2.1Assessesandcriticallyanalysesresearchliteratureandothersourcesofevidencetoimprovepractice
91.6
6.2.2Demonstratesasystematicapproachtoanalysisanddecisionmaking 91.7 6.2.3Integratesthebestavailableevidenceandnewlearningintopracticetoimprovehealthoutcomesforclients
97.9
6.2.4Demonstratesknowledgeofnewtechniquesandtechnologyrelevanttoorthotics/prosthetics 81.3 6.2.5Criticallyandcontinuouslyevaluatespractice 93.7
27
1 2
28
FigureLegends3Figure1MixedMethodsApproachtoDevelopingCompetencyStandard4Figure2:DomainsofOrthotic/ProstheticCompetencyStandards 5Figure3DelphiRound1AnalysisofDomainsandActivities. 6
29
References7
1. WorldHealthOrganisation.SummaryWorldReportonDisability.:World8Health Organisation and World Bank.; 2011 [cited 2014 30 April, 2014].9Available from:10http://whqlibdoc.who.int/hq/2011/WHO_NMH_VIP_11.01_eng.pdf?ua=1.112. AustralianBureauofStatistics.Customisedreport:Censusofpopulation12andhousing;customizeddatareportcode251912;orthotistorprosthetist.2014.133. National Health Service Scotland. Scottish Orthotic Services Review14Edinburgh: National Health Service, Scotland; 2005. Available from:15http://www.sehd.scot.nhs.uk/publications/dc20050614orthotics.pdf.164. World Health Organisation. Transformative scale up of health17professional education. Geneva, Switzerland:World Health Organisation; 201118[cited 2014 30 April,]. Available from:19http://whqlibdoc.who.int/publications/2011/9789240685215_eng.pdf.205. HealthWorkforceAustralia.HealthWorkforceAustralia: 2011.National21HealthWorkforceInnovationandReformStrategicFrameworkforAction2011‐222015 Adelaide, South Australia: Health Workforce Australia; 2011. Available23from: https://http://www.hwa.gov.au/sites/uploads/hwa‐wir‐strategic‐24framework‐for‐action‐201110.pdf.256. The Australian Orthotist Prosthetist Association Inc. Entry Level26CompetencyStandardsforAustralianOrthotist/ProsthetistsMelbourne,Victoria272014.3rd:[Availablefrom:http://www.aopa.org.au/documents/item/27.287. Larkin PM, Begley CM, Devane D. Breaking from binaries‐‐using a29sequentialmixedmethodsdesign.NurseResearcher.2014;21(4):8‐12.308. O'Connell J, Gardner G. Development of clinical competencies for31emergency nurse practitioners: a pilot study. Australasian Emergercy Nursing32Journal.2012Nov;15(4):195‐201.eng.339. AshS,GoncziA,HagerP.Combiningresearchmethodologies todevelop34competency‐based standards for dietitians: a case study for the professions.35National Office of Overseas Skills Recognition Research paper no. 6. DEET.36Canberra:AustralianGovernmentPublishingService;1992.3710. Phillips S, Ash S, Tapsell L. Relevance of the competency standards to38entry level dietetic practice. Australian Journal of Nutrition & Dietetics.392000;57(4):198‐207. PubMed PMID: 2002151039. Language: English. Entry40Date:20021101.RevisionDate:20091218.PublicationType:journalarticle.4111. Braun V, Clarke V. Using thematic analysis in psychology. Qualitative42ResearchinPsychology.20062006/01/01;3(2):77‐101.4312. Jones J, Hunter D. Consensus methods for medical and health services44research. BMJ (Clinical Research Ed). 1995;311(7001):376‐80. PubMed PMID:457640549.4613. ChangAM,GardnerGE,DuffieldC,RamisM‐A.ADelphistudytovalidate47an advanced practice nursing tool. Journal Of Advanced Nursing.482010;66(10):2320‐30.4914. HughesR,BegleyA,YeatmanH.Aspirationalcompetencyexpectationsfor50publichealthnutritionistsinAustralia:Aconsensusstudy.Nutrition&Dietetics.512013.Epub13November,2013.52
30
15. Davies J, Hughes R, Margetts B. Towards an international system of53professionalrecognitionforpublichealthnutritionists:afeasibilitystudywithin54theEuropeanUnion.PublicHealthNutrition.2012;15(11):2005‐11.5516. Kennie‐KaulbachN,FarrellB,WardN,JohnstonS,GubbelsA,EgualeT,et56al.Pharmacistprovisionofprimaryhealthcare:amodifiedDelphivalidationof57pharmacists'competencies.BMCfamilypractice.2012;13:27.Epub2012/03/30.58eng.5917. BrodyRA,Byham‐Gray L, Touger‐DeckerR, PassannanteMR,O'Sullivan60Maillet J. IdentifyingComponentsofAdvanced‐LevelClinicalNutritionPractice:61A Delphi Study. Journal of the Academy of Nutrition and Dietetics.622012;112(6):859‐69.6318. Naidu R, Stanwick, J., Frazer, K. Glossary of Terms Adelaide, South64Australia: National Centre for Vocational Education Research; 2013. Available65from:http://www.voced.edu.au/content/glossary‐vet.6619. SurveyMonkeyInc.SurveyMonkey®.PaloAlto,California,USA2013.6720. SherbinoJ,FrankJR,SnellL.DefiningtheKeyRolesandCompetenciesof68the Clinician‐Educator of the 21st Century: A National Mixed‐Methods Study.69Academicmedicine : journal of the Association of American Medical Colleges.702014May89(5):783‐9.Epub2014/03/29.Eng.7121. RodgerS,ClarkM,BanksR,O'BrienM,MartinezK.Anationalevaluation72of the Australian Occupational Therapy Competency Standards (1994): a73multistakeholder perspective. Australian Occupational Therapy Journal.742009;56(6):384‐92.7522. DowdingK,AshS,Shakespeare‐FinchJ.Usingcriticalincidentinterviews76to identify the mental health knowledge, skills and attitudes of entry‐level77dietitians.Nutrition&Dietetics.2011;68(4):297‐304.7823. Australian New Zealand Podiatry Accreditation Council. Podiatry79Competency Standards for Australia and New Zealand 2012 [cited 2014 1280August]. Available from: http://www.anzpac.org.au/files/Podiatry Competency81StandardsforAustraliaandNewZealandV1.1211212(Final).pdf.8224. Dewing J, Traynor V. Admiral nursing competency project: practice83development and action research. Journal of Clinical Nursing. 2005;14(6):695‐84703.8525. FrankJR,DanoffD.TheCanMEDSinitiative: implementinganoutcomes‐86basedframeworkofphysiciancompetencies.MedicalTeacher.2007;29(7):642‐877.888990
91
92
31
93