05 mr graeme campbell - miiaa.com.au graeme campbell.pdf · session: revalidation in the australian...

MIIAA Annual Forum 2013 12 September 2013

Session: Revalidation in the Australian Context 1

REVALIDATION

Graeme Campbell,Surgeon, Bendigo, Victoria

Chair, Professional Standards RACS

MIIAA SEP 2013

Revalidation

● Evolution of CPD programs

● Theory of lifelong learning

● Possible methods of performance assessment

● Evidence base, and gaps in evidence

● What is RACS currently doing?

● The way forward



Medical Professionalism

● A commitment by medical professionals to competence, integrity and morality, altruism, and the promotion of the public good forms the basis of a social contract between the profession and society, which in return grants the profession a monopoly of the use of its knowledge base, the right to considerable autonomy in practice and the privilege of self-regulation. Professions and their members are accountable to those served and to society.

RACP, RCPSC, RACS Consensus statement 2012

Public expectations

● Not clear, but likely expect medical practitioners to be

● Competent

● Fit

● Safe



Nine RACS Competencies

Competence and Performance



“Traditional” CPD

● Based around attendance at educational activities

● No evidence of learning required

● No attempt to measure performance

“Modern” CPD, dual emphasis

● Meetings

● Journals

● Specialist visits

● E-learning

● Teaching

● Research

● Etc.

● Peer reviewed audit

● Hospital credentialing

● Practice visits● Performance

review

Education Performance Assessment



Lifelong learning, principles

● Commitment to lifelong learning

● Best practice in learning

● Learning aligned to competence and performance (learning plan)

● Learning relevant to career stage and scope of practice

● Meaningful assessment

● Learning enabled by information technology and communications

Performance Assessment, what is the evidence base?



Performance Assessment, what is the evidence base?

● Multi-source feedback

● Audit & feedback

● Simulation

Multi-source feedback

● Minimum 6-8 reviewers

● Can assess multiple competencies, especially communication, collaboration, professionalism, management, ? not medical expertise

● Reliable & valid

● Can identify improvement opportunities

● Delivery of the feedback is the key**



Audit & feedback

● 140 randomized trials, Cochrane reviews

● Modest change in behaviour, especially:-

low baseline performancetrusted colleague multiple feedback episodes &

methods** explicit action plan developed

Simulation

● Positive impact on learning

● Allows repetitive practice

● Skills can translate into practice

● Can assess individual or team performance

● Can be low or high fidelity

● Feedback crucial**



Gaps in Evidence

● Many tools either not tested or no demonstrated efficacy

● Lack of understanding of how to translate assessment tool into real world practice

● Use tools in isolation or combination?

● Episodic vs. continuous assessment

● Appropriate, validated metrics

● Practicality of various systems

Assessment tools, CARVE criteria

● Cost-effectiveness

● Acceptability

● Reliability

● Validity

● Efficacy

Van Der Vieuten: Assessment of Professional Competence Advances in Health Sciences Education 1996, vol1, No 1



What is RACS doing?



Nine RACS Competencies

Behavioural Markers



Multisource Feedback

● Self

● Medical peers

● Trainees and juniors

● Nursing staff – wards, theatre, clinic, rooms

● Practice staff

● Managers and directors

● Patients (?)

● Aim for at least 8 sources

Multisource Feedback

● Needs to be collated by someone

● Appropriate feedback giventimelyspecificconstructiveappropriate and fairsensitiverespectful

Training in how to give constructive feedback is important***



Practice Visits NZOA Program

● Two reviewers, two days, seven year cycle

● Notes reviews

● Directly observeconsultationsoperating theatre

● Feedback

Practice Visits NZOA

● Positively received by reviewer & reviewed● Now mandated by NZOA, endorsed by

RACS● However

costlydifficult for very small or very large

groups little scientific validation● Not clear how underperformance will be

managed



Mortality Audit (ANZASM)

● Required for all engaged in operative practice

● Still some gaps

● Has provided some clear messagesDVT prophylaxisfluid managementcentralization of some procedurescommunication & transfer issuesavailability of ICU

● Has not identified “Dr Death”

Code of Conduct & CPD

● Code of conduct mandates CPD

● Failure to fulfil CPD requirements is a breach of the code of conduct

● This breach will be notified



What is the role of the Colleges?

● Set standards, specific, assessable

● Develop a specific performance assessment framework, with reporting system to support this

● Develop performance assessment tools, applicable across various stages of a career

● Provide ongoing learning & up skilling processes

● Professional support

● Addressing & remediating underperformers

Royal Australasian College of Surgeons

`



Evolution or revolution?

● Most doctors already undertake lifelong learning

● Performance appraisal now routine in public hospitals, becoming more common in private hospitals and large group practices

● Audit is well established in procedural specialties

● Mandatory CPD now accepted

However

● Performance review tools still to be developed in many specialties

● A mechanism for delivering performance review to individual practitioners still needs work

● Training in appropriate feedback needs much more attention

● Need for clearer remediation strategies



Reality check

● Still much more to be gained by system improvement than focussing on individuals

● Cost must be acceptable

● (Note recent suggestion limiting educational expenses to $2,000 p.a.)

● Is the system so broken?

Revalidation, critical considerations

● Transparent● Integrated into practice & work setting● Practical● Not overly resource intensive● Feasible for time poor professionals● Sensitive to the motivations that drive learning● Based on evidence● Include performance assessment● Add value to existing QI activities



Revalidation should NOT be

● Excessively burdensome and professionally disruptive

● Assessment disconnected from relevant scope of practice

● Perceived as punitive

● A legal risk for Colleges

What is the real game plan?

● Quality improvement for all?

● Locate serious underperformance?



Conclusions

● The profession must demonstrate a commitment to lifelong learning

● Performance assessment will increase

● Both of these need to align with the real world, and not be an added burden to busy practitioners

● The Colleges are well placed to drive these changes, and need to collaborate with each other and the regulators

Royal Australasian College of Surgeons

05 mr graeme campbell - miiaa.com.au graeme campbell.pdf · session: revalidation in the australian...

Documents