® scoring the clinical proficiency examination dr. beth sabin, assistant director avma education...
TRANSCRIPT
®
Scoring the Clinical Proficiency
Examination
Dr. Beth Sabin, Assistant DirectorAVMA Education and Research Division
CLEAR Annual ConferenceSeptember 16, 2005
®
Clinical Proficiency Examination
3.5-day, 7-section, hands-on assessment of clinical veterinary medical skillsMust pass all sections
Administered at multiple ECFVG-approved sites according to Manual of Administration
Also administered by the Canadian National Examining Board (NEB)
®
Educational Commission for Foreign
Veterinary Graduates
Seven-member AVMA committee Oversees educational equivalency
assessment program Candidates are graduates of international
non–AVMA-accredited veterinary schools Certificate holders meet educational
prerequisite for licensure or employment
®
ECFVG Program Steps
1. Enroll; proof of graduation2. Assessment of English language
competency (ETS, CAEL, IELTS)3. Assessment of basic and clinical
sciences knowledge (NAVLE)4. Assessment of hands on clinical skills
(CPE, postgraduate ECY)
®
Clinical Proficiency Examination
3.5-day, 7-section, hands-on assessment of clinical veterinary medical skillsAnesthesiologyClinical and Anatomic PathologyEquine PracticeFood Animal PracticeRadiologySmall Animal PracticeSurgery
®
Clinical Proficiency Skills
CPE Manual of Administration defines as:“…combination of knowledge and technical
skills.”
Examiner assesses knowledge by directly observing examinee behaviors as examinee performs basic skills required of a minimally competent entry-level practitioner.
®
CPE Audit
Audit conducted in Spring 2001 Findings provided to ECFVG Fall 2001
CPE accomplishes the intended goal - it assesses the minimal clinical competency of a graduate of a non-accredited veterinary program
Suggestions for enhancements
®
Post-audit CPE Activities
Examiners’ Training Review of CPE Manual of Administration Examiners’ Master Manual Review of CPE scoring parameters Survey of faculty and students
®
CPE History
Experts selected most pertinent skills to evaluate performance competency
Scoring process defined Goal is to separate “masters” from “non-
masters” Interested in evaluating for minimal
competency
®
Performance Benchmarks
Consider the behaviors to be observed and what constitutes minimal levels of performance at each CPE scoring level:SuperiorGoodSatisfactoryUnsatisfactory
®
CPE Rating Scale(for six of seven sections)
100% - 86% Superior Superior performance of required clinical skills
85% - 71% Good Good performance of required clinical skills
70% - 61% Satisfactory Satisfactory performance of required clinical skills
< 60% Unsatisfactory Failed to demonstrate the required clinical skills
®
Development of Assessment
Methods Competencies identified for each section Example—Clinical and Anatomic Pathology
Recognize and interpret visual images of gross pathological lesions
Perform a necropsy and take tissue samples for histopathology
Perform common laboratory tests and interpret results of those tests in the areas of clinical microbiology, clinical parasitology, hematology, and urinalysis
®
Assessment of Skills
Skills/activities identified for each competency
Example—Perform necropsy Thoroughly examine intact carcass Opens carcass, examine major cavities, remove
viscera and head Examine major organs Examine muscles, joints, bone marrowExamine endocrine glands, lymph nodes
®
Behavioral Anchors
Observable behaviors identified for each skill/activity
Example—Thoroughly examine intact carcass (5 pts)
1 pt: Assesses general body condition/hydration 1 pt: Examines skin/hair coat 1 pt: Examines head 1 pt: Examines limbs including feet 1 pt: Examines perianal and genital region, mammary
glands
®
Calculating Section Scores
Determine weighting of each competency Example—Clinical and Anatomic Pathology
Recognize and interpret visual images: 16%Perform necropsy: 20%Perform lab tests and interpret results: 64%
16% for each of four areas
Sum weighted station scores = section score Pass = 60
®
CPE Rating Scale(for surgery)
2004 review by subject matter experts and testing consultants
Minimal competency best assessed via pass/fail
Experts developed:Check list of passing behaviorsCheck list of failing behaviorsAppendix of fatal flaws
®
Scoring Surgery
CPE Manual states: A “pass” = at least a minimally competent
performance of entry-level surgical skills Examiner to use the surgical skills
assessment sheet as a checklist and to record positive and negative behaviors
Only feedback provided is in the case of a fatal flaw
®
Observable Behaviors
Tissue trauma Passing behaviors
Demonstrates atraumatic tissue handling throughoutSkills do not result in cumulative traumatic handling of
tissue Failing behaviors
Causes significant damage to other tissues/organs (fatal)
Skills result in cumulative traumatic handling of tissue
®
Failing Surgery
CPE Manual states: Should a candidate exhibit any one behavior
in the “Failing Behaviors” column of the assessment sheet, he/she will be allowed to complete the Surgery section but will receive a failing grade
®
Fatal Flaws
CPE Manual States: Should the candidate commit a fatal flaw, the
examiner will interrupt the examination, correct the error to ensure safety of the animal and personnel, and award the candidate a failing grade.
The candidate will not be allowed to complete the Surgery section of the CPE.
®
Five Fatal Flaws
Fails to control significant hemorrhage Fails to create a secure closure of the
abdominal wall Unable to achieve and maintain an aseptic
field Causes significant damage to other
tissues/organs Exhibits any other behavior that puts the
animal’s life at risk
®
Radiology
Ongoing review by subject matter experts Development of consensus on types of
radiographs and radiographic lesions Asking experts: “What types of lesions
should a minimally competent entry-level veterinarian be able to identify and in what species?”
®
Goals of Radiology Review
Long-termDevelop a central bank of radiographs
Short-termUpdate competencies, skills/activities,
and observable behaviors for assessment of minimal entry-level competency in radiology
®
Additional Information
ECFVG Web sitewww.avma.org/education/ecfvg/default.asp
Contact ECFVG [email protected], ext 6675