![Page 1: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/1.jpg)
SELF EVALUATION POORLY SELF EVALUATION POORLY CORRELATED WITH CORRELATED WITH
RESIDENCY ADMISSION RESIDENCY ADMISSION GRADES ON OSCE-BASED GRADES ON OSCE-BASED
TESTTEST Júlio César de Oliveira Júlio César de Oliveira
Maria Patrocínio Tenório NunesMaria Patrocínio Tenório NunesLeonardo Borges de Barros e SilvaLeonardo Borges de Barros e SilvaCristina Helena Fonseca-GuedesCristina Helena Fonseca-Guedes
Itamar de Souza SantosItamar de Souza SantosJoaquim Edson VieiraJoaquim Edson Vieira
UNIVERSITY OF SÃO PAULOUNIVERSITY OF SÃO PAULO SCHOOL OF MEDICINE SCHOOL OF MEDICINE
São Paulo São Paulo BRAZILBRAZIL
![Page 2: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/2.jpg)
UNIVERSITY OF SÃO PAULOUNIVERSITY OF SÃO PAULO SCHOOL OF MEDICINESCHOOL OF MEDICINE
São Paulo São Paulo BRAZILBRAZIL
OSCE Performance as a Residency Admission Test
![Page 3: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/3.jpg)
-600 Graduated Doctors around the country
-264 positions as a Resident
-21 Residency Training Programs
From: 60 different Schools 21 different States
Phase 1 - Writing test 3.000 candidates
Phase 2 – OSCE Based Test 600 candidates
![Page 4: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/4.jpg)
PHASE 2 - OSCE Selection ProcessPHASE 2 - OSCE Selection Process
Actress patient
Candidate
Observer
![Page 5: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/5.jpg)
OBJECTIVE
• Correlation between Self Evaluation score and OSCE grade in a Residency Admission Test
![Page 6: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/6.jpg)
CASUISTIC and METHODSCASUISTIC and METHODS
• Self-evaluation questionnaire
• 314 recently graduated physicians
• Performance on Objective Structured Clinical Examination – OSCE
• Correlation between OSCE final grade and self-evaluation of skill levels
![Page 7: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/7.jpg)
CASUISTIC and METHODSCASUISTIC and METHODSSelf evaluation questionnaire
0 You do not know the procedure
1 You have read about or been told how the procedure is done
2 You have seen another person perform the procedure
3 You have done the procedure with guidance of another person
4 You have done the procedure correctly on your own
5 You have done the procedure correctly so many times that you feel both competent and confident about it
![Page 8: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/8.jpg)
STATISTICAL ANALYSIS
Pearson Correlation’s test
Self evaluation scores X OSCE grade
![Page 9: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/9.jpg)
RESULTSRESULTS
![Page 10: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/10.jpg)
RESULTSRESULTSStation 1
R2 = 0,0336
0102030405060708090
100
0 1 2 3 4 5 6
Self evaluation
OS
CE
gra
de
Station 2
R2 = 0,0271
0102030405060708090
100
0 1 2 3 4 5 6
Self evaluation score
OS
CE
gra
de
Station 3
R2 = 0,0622
0102030405060708090
100
0 1 2 3 4 5 6
Sel- evaluation score
OS
CE
gra
de
Station 4
R2 = 0,0536
0102030405060708090
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
![Page 11: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/11.jpg)
RESULTSRESULTSStation 5
R2 = 0,0568
0
20
40
60
80
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
Station 6
R2 = 0,0341
0
20
40
60
80
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
Station 7
R2 = 0,0238
0
20
40
60
80
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
Station 8
R2 = 0,0258
0
20
40
60
80
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
![Page 12: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/12.jpg)
RESULTSRESULTSStation 9
R2 = 0,0055
0
20
40
60
80
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
Station 10
R2 = 0,0114
0
20
40
60
80
100
0 1 2 3 4 5 6
Self-evaluation score
OS
CE
gra
de
All 10 stations
R2 = 0,0449
0102030405060708090
100
0,0 1,0 2,0 3,0 4,0 5,0
Self-evaluation mean
OS
CE
gra
de
![Page 13: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/13.jpg)
DISCUSSIONDISCUSSION
• The OSCE provides a nearer real clinical situation
• The self-evaluation questionnaire indicates how secure and confident was the candidate about the skill asked in each station
• It would be expected a good correlation between OSCE grade and self evaluation scores
• But no correlation was found
![Page 14: SELF EVALUATION POORLY CORRELATED WITH RESIDENCY ADMISSION GRADES ON OSCE-BASED TEST Júlio César de Oliveira Maria Patrocínio Tenório Nunes Leonardo Borges](https://reader036.vdocuments.us/reader036/viewer/2022062223/551566af55034691418b4cd8/html5/thumbnails/14.jpg)
CONCLUSIONCONCLUSION
Young physicians could not self-evaluate OSCE grades during Residency
Admission Test.