2003iadr gothenburg nikos mattheos nikos mattheos centre for educational research and technology in...

IADR Gothenburg Nikos Mattheos2003

Nikos MattheosNikos MattheosCentre for Educational Research and Technology in Oral Health, Centre for Educational Research and Technology in Oral Health, University of Malmö, SwedenUniversity of Malmö, Sweden

The Interactive ExaminationThe Interactive Examination Assessing students’ self-assessment abilityAssessing students’ self-assessment ability

http://www.nfr.se/index.asp

http://www.netuniversity.se/default.asp?c=3

2003 IADR Gothenburg Nikos Mattheos

Parallel learning goals:Parallel learning goals:

BiomedicalBiomedical knowledgeknowledge Clinical skillsClinical skills

Critical thinkingCritical thinking

Life-long learningLife-long learning

Problem SolvingProblem Solving


Lifelong Learning attitude f or a clinician:

Act

Assess - process - outcome

Identify Needs

Increase Competence

Search Resources


The Interactive Examination:The Interactive Examination: evaluate knowledge, skills and self assessment abilityevaluate knowledge, skills and self assessment ability

•Evaluate process - outcomeEvaluate process - outcome

• Compare to standards - criteriaCompare to standards - criteria

• Identify needsIdentify needs

• Define new learning objectivesDefine new learning objectives


1.1. Self assessment Self assessment (through Internet: 11 ordinal scales 1-6, text)(through Internet: 11 ordinal scales 1-6, text)

The Interactive Examination:The Interactive Examination:


2. Essay question 2. Essay question (40 min) (40 min)

3. Oral examination – discussion 3. Oral examination – discussion (70 min)(70 min)

4. Evaluation 4. Evaluation (10 min)(10 min)




5. Compare own essay with expert’s 5. Compare own essay with expert’s (1 week after)(1 week after)

6. Individual feedback 6. Individual feedback (e-mail)(e-mail) (1 month after)(1 month after)

2. Essay question 2. Essay question (40 min) (40 min)

3. Oral examination – discussion 3. Oral examination – discussion (70 min)(70 min)

4. Evaluation 4. Evaluation (10 min)(10 min)




Material and Method:Material and Method:

One cohort of 2One cohort of 2ndnd year students in Periodontology year students in Periodontology

(2001 n=54)(2001 n=54)

- Essay and oral performance grades (1-6) - Essay and oral performance grades (1-6)

- Self assessment vs instructors judgment - Self assessment vs instructors judgment

- Self assessment and performance / gender - Self assessment and performance / gender

- Qualitative Analysis of comparison documents: - Qualitative Analysis of comparison documents:

(differences, arguments, learning needs) (differences, arguments, learning needs)

- Attitudes towards examination (ordinal scales 1-9, free text) - Attitudes towards examination (ordinal scales 1-9, free text)


Results:Results:

0

40

80

120

160

200

-3 -2 -1 0 1 2 3

n = 506 scores

0,5% 3% 15% 40% 35% 6% 0,5%

• 40 % of the self assessment scores were in agreement with those of the clinical Instructors .

• No gender differences

185

31

• 43 % of the students had significant deviation from instructors ( 34% over – 9% under) (2 tailed Wilcoxon’s signed rank test)

General:General:

Individual:Individual:


Results:Results:

-0,1

0

0,1

0,2

0,3

0,4

0,5

0,6

1 2 3 4 5 6 7 8 9 10 11

Nr. 3 (p=0.0002)“Overall diagnosis and treatment of periodontitis” Nr. 4 (p=0.02)“Competence in clinically differentiating healthy from pathological gingival”

Nr. 5 (p=0.0006)“Measuring of pocket depth”

Nr. 8 (p=0.003)“Competence in evaluating changes in the radiographic image of the periodontium”.


Results:Results:

1. How effective do you think is Interactive Examination for learning?

not effective 1 2 3 4 5 6 7 8 9 very effective median 8 (n=52)

2. How do you think the Interactive examination would affect your learning style from now?

no difference 1 2 3 4 5 6 7 8 9 increase my motivation median 7 (n=52)


Results:Results:• Students underestimating their competence scored better in the oral part of

the exam

• Female students performed significantly better in both oral and written parts of the exam

• Students appreciated the element of discussion, but not the written essay part.


Results: Results: What did the students compare

1. Form and structure:

Length, text diagramme or flow chart,

Use of images, language, style etc

2. Content:Additions, emissions, terms etc

3. Attitude:

Prioritising, elaborating, depth of detail etc


ACT ASSESS

IDENTIFY NEEDS

Student essay

1…. 2…. 6….. 8…..

”Expert” essay

1…. 2…. 3…. 4…. 5….

comparison

differences

arguments

needs

Lifelong Learning attitude f or a clinician:

Act

Assess - process - outcome

Identify Needs

Increase Competence

Search Resources


Conclusions:Conclusions:• 2nd year students in general tended to overestimate their competence

• Students overestimated their competence in diagnostic rather than treatment skills

• The interaction element in examination was highly appreaciated

• Student’s comparison document reflected students’ understanding, prioritising and self-assessment skills through the sequence:

differences arguments needs

2003 IADR Gothenburg Nikos Mattheoshttp://periodont.mah.se/nikos/research

2003iadr gothenburg nikos mattheos nikos mattheos centre for educational research and technology in...

Documents