05 mr graeme campbell - miiaa.com.au graeme campbell.pdf · session: revalidation in the australian...
Post on 08-Jul-2020
0 Views
Preview:
TRANSCRIPT
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 1
REVALIDATION
Graeme Campbell,Surgeon, Bendigo, Victoria
Chair, Professional Standards RACS
MIIAA SEP 2013
Revalidation
● Evolution of CPD programs
● Theory of lifelong learning
● Possible methods of performance assessment
● Evidence base, and gaps in evidence
● What is RACS currently doing?
● The way forward
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 2
Medical Professionalism
● A commitment by medical professionals to competence, integrity and morality, altruism, and the promotion of the public good forms the basis of a social contract between the profession and society, which in return grants the profession a monopoly of the use of its knowledge base, the right to considerable autonomy in practice and the privilege of self-regulation. Professions and their members are accountable to those served and to society.
RACP, RCPSC, RACS Consensus statement 2012
Public expectations
● Not clear, but likely expect medical practitioners to be
● Competent
● Fit
● Safe
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 3
Nine RACS Competencies
Competence and Performance
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 4
“Traditional” CPD
● Based around attendance at educational activities
● No evidence of learning required
● No attempt to measure performance
“Modern” CPD, dual emphasis
● Meetings
● Journals
● Specialist visits
● E-learning
● Teaching
● Research
● Etc.
● Peer reviewed audit
● Hospital credentialing
● Practice visits● Performance
review
Education Performance Assessment
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 5
Lifelong learning, principles
● Commitment to lifelong learning
● Best practice in learning
● Learning aligned to competence and performance (learning plan)
● Learning relevant to career stage and scope of practice
● Meaningful assessment
● Learning enabled by information technology and communications
Performance Assessment, what is the evidence base?
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 6
Performance Assessment, what is the evidence base?
● Multi-source feedback
● Audit & feedback
● Simulation
Multi-source feedback
● Minimum 6-8 reviewers
● Can assess multiple competencies, especially communication, collaboration, professionalism, management, ? not medical expertise
● Reliable & valid
● Can identify improvement opportunities
● Delivery of the feedback is the key**
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 7
Audit & feedback
● 140 randomized trials, Cochrane reviews
● Modest change in behaviour, especially:-
low baseline performancetrusted colleague multiple feedback episodes &
methods** explicit action plan developed
Simulation
● Positive impact on learning
● Allows repetitive practice
● Skills can translate into practice
● Can assess individual or team performance
● Can be low or high fidelity
● Feedback crucial**
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 8
Gaps in Evidence
● Many tools either not tested or no demonstrated efficacy
● Lack of understanding of how to translate assessment tool into real world practice
● Use tools in isolation or combination?
● Episodic vs. continuous assessment
● Appropriate, validated metrics
● Practicality of various systems
Assessment tools, CARVE criteria
● Cost-effectiveness
● Acceptability
● Reliability
● Validity
● Efficacy
Van Der Vieuten: Assessment of Professional Competence Advances in Health Sciences Education 1996, vol1, No 1
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 9
What is RACS doing?
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 10
Nine RACS Competencies
Behavioural Markers
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 11
Multisource Feedback
● Self
● Medical peers
● Trainees and juniors
● Nursing staff – wards, theatre, clinic, rooms
● Practice staff
● Managers and directors
● Patients (?)
● Aim for at least 8 sources
Multisource Feedback
● Needs to be collated by someone
● Appropriate feedback giventimelyspecificconstructiveappropriate and fairsensitiverespectful
Training in how to give constructive feedback is important***
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 12
Practice Visits NZOA Program
● Two reviewers, two days, seven year cycle
● Notes reviews
● Directly observeconsultationsoperating theatre
● Feedback
Practice Visits NZOA
● Positively received by reviewer & reviewed● Now mandated by NZOA, endorsed by
RACS● However
costlydifficult for very small or very large
groups little scientific validation● Not clear how underperformance will be
managed
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 13
Mortality Audit (ANZASM)
● Required for all engaged in operative practice
● Still some gaps
● Has provided some clear messagesDVT prophylaxisfluid managementcentralization of some procedurescommunication & transfer issuesavailability of ICU
● Has not identified “Dr Death”
Code of Conduct & CPD
● Code of conduct mandates CPD
● Failure to fulfil CPD requirements is a breach of the code of conduct
● This breach will be notified
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 14
What is the role of the Colleges?
● Set standards, specific, assessable
● Develop a specific performance assessment framework, with reporting system to support this
● Develop performance assessment tools, applicable across various stages of a career
● Provide ongoing learning & up skilling processes
● Professional support
● Addressing & remediating underperformers
Royal Australasian College of Surgeons
`
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 15
Evolution or revolution?
● Most doctors already undertake lifelong learning
● Performance appraisal now routine in public hospitals, becoming more common in private hospitals and large group practices
● Audit is well established in procedural specialties
● Mandatory CPD now accepted
However
● Performance review tools still to be developed in many specialties
● A mechanism for delivering performance review to individual practitioners still needs work
● Training in appropriate feedback needs much more attention
● Need for clearer remediation strategies
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 16
Reality check
● Still much more to be gained by system improvement than focussing on individuals
● Cost must be acceptable
● (Note recent suggestion limiting educational expenses to $2,000 p.a.)
● Is the system so broken?
Revalidation, critical considerations
● Transparent● Integrated into practice & work setting● Practical● Not overly resource intensive● Feasible for time poor professionals● Sensitive to the motivations that drive learning● Based on evidence● Include performance assessment● Add value to existing QI activities
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 17
Revalidation should NOT be
● Excessively burdensome and professionally disruptive
● Assessment disconnected from relevant scope of practice
● Perceived as punitive
● A legal risk for Colleges
What is the real game plan?
● Quality improvement for all?
● Locate serious underperformance?
MIIAA Annual Forum 2013 12 September 2013
Session: Revalidation in the Australian Context 18
Conclusions
● The profession must demonstrate a commitment to lifelong learning
● Performance assessment will increase
● Both of these need to align with the real world, and not be an added burden to busy practitioners
● The Colleges are well placed to drive these changes, and need to collaborate with each other and the regulators
Royal Australasian College of Surgeons
top related