Changing face of examining

Presentation StructurePresentation Structure

• BackgroundBackground

• Operational support

ki• Emarking

• Application process

IB examiner informationIB examiner information

• May 2010May 2010– 6,500 examiners

1 150 components– 1,150 components

– 100,000 candidates

138 t i– 138 countries

l h f l• Annual growth of approximately 10%

IB examiner information (cont)IB examiner information (cont)

• Majority of examiners are based in a fewMajority of examiners are based in a few countries– UK– UK

– USA

Australia– Australia

• International education – increase the number f i i th t iof examiners in other countries

IB examiner information (cont)IB examiner information (cont)

2008

Examiner rolesExaminer roles

• Different rolesDifferent roles– Senior examiners

• Chief examiners/assessorsChief examiners/assessors

• Examiner responsible for a language

• School based syllabus assessorsy

– Operational examiners

Examiner roles (cont)Examiner roles (cont)

• Operational examinersOperational examiners– Examiner

• Mark candidates work against a markscheme or gmarking notes

– Moderator

• Moderates a teachers marking

– Team leader

P i i l i / d t– Principal examiners/moderators

Recruitment policyRecruitment policy

• InformationInformation– www.ibo.org/examiners

Universal across all DP subjects– Universal across all DP subjects

– Identifies required qualifications and experience

O i t hi i• One years on going teaching experience

• A degree in the subject or a related subject

Examiner contractExaminer contract

• Contract revised for the 2011 sessionContract revised for the 2011 session

• More user friendly

Cl ifi l i h l i d d• Clarifies roles with newly introduced systems

Examining visual artsExamining visual arts

• Ensuring validity of assessmentEnsuring validity of assessment

• An examiner can examine at a school for a maximum of two consecutive yearsmaximum of two consecutive years

• Travelling distance

• Visual arts examiner list

Examiner inductionExaminer induction

• Support newly appointed examinersSupport newly appointed examiners

• Clarifies assessment process

f i f ll i• Reference point for all examiners

Examiner newsletterExaminer newsletter

• Examiner focussedExaminer focussed

• Community of examiners

f l i f i• Useful information

E‐marking – why it’s such an exciting time to join the IB examining team..

The IB have been successfully e‐marking since the November 2009 

session.

Components which will be e‐marked in November 20102010

Biology SL p2 ITGS HL/SL p1gy p

Economics SL p1 & p2

English B HL p1

/ p

Maths SL p1 & p2

Maths studies SL p1 & p2

English B SL p1

Environmental systems & i ti SL 2

Physics SL p2

Theory of knowledge essaysocieties SL p2

– This list of subjects will be extended in the May 2011 session

Ramp up

100%

120%

80%

100%

40%

60%

0%

20%

2010 2011 2012 2013

Ramp up

By 2013 97% of scripts will be e‐markedy p

2010 – 12%

2011 – 36%

2012 – 65%

2013 – 97%

Improving the quality of assessment

• For paper‐based marking, examiner performance is monitored through the use of sample re‐marking and moderation.

• This system, which is the best possible in a paper‐based system suffers from a number ofbased system, suffers from a number of drawbacks. – Examiners cannot learn from this process during the 

marking session

– Examiners do not have the opportunity to improve their standard of marking during the marking session

– Where a complete re‐mark is necessary there is little time to complete the work

Improving the quality of assessment

• From November 2010, the IB will introduce seeding to monitor examiner performance with e‐marked scripts and with this approach these drawbacks are reduced or removed completely.

Summary of the seeding process

Examiners are given more guidance on the application of the g g ppmarkscheme at the start of their marking through practice scripts

E i d ‘ lif ’ k li i b ki 5 Examiners need to ‘qualify’ to mark live scripts by marking 5 scripts correctly , with a second chance of another 5 scripts if necessary 

Once marking, a certain number of scripts (seeds) are randomly added to an examiner’s worklist. The script cannot be identified by the examiner as a seedbe identified by the examiner as a seed 

Summary of the seeding process (2)

Examiners mark these seeds and unless they mark the seed  outside of pre‐agreed tolerances they keep on marking

The first time the examiner marks the seed inaccurately they cannot continue to mark until they have accepted quality feedbackfeedback 

If the examiner marks a second consecutive seed outside of the pre‐agreed tolerance they are stopped from marking until they have received guidance from their team leader

If the examiner marks 3 seeds outside of the pre‐agreed tolerance they are stopped from marking and their scripts willtolerance they are stopped from marking and their scripts will be re‐allocated 

Summary of the seeding process (3)

• The IB and the senior examiners can view the status of each examiner within their marking team during the marking process

Th f d f b i d• Therefore, progress and performance can be monitored more closely

Examiner comments

100%May 10 examination sessionWhich of the follo ing ords/phrases o ld o se to describe the

80%

100%

69%

Which of the following words/phrases would you use to describe the e‐marking system used during the May 10 IB Diploma Programme exam session?

40%

60%

36%

28%

0%

20% 11%

5%11%

Enjoyable Cumbersome Straightforward to use Flexible Complicated OtherAverage response rate to questionnaire:  85% (266 out of 315 examiners)

Examiner comments

• “Like anything else new, it will take some time but I can already tell you that this is easier that marking the scripts themselves.”

• “Quite happy to use this system. Beats having piles of paper all over the house!!!!”all over the house!!!!

• “It is very enjoyable to grade papers with scoris”

• “Actually, I think that this software and its user friendlyActually, I think that this software and its user friendly approach is amazing”

• “I was using a tablet PC with a stylus, so the "stamping" of bl d k ”annotations was very enjoyable and quick.”

Examiner comments

• “Helpdesk at IB was personal and first class.”

• “E‐marking was definitely a nice experience. The experience g y p pof working on scoris assessor was really a very good one. Thanks for providing me an opportunity to learn and improve upon my 'online marking (e marking) skills ”upon my  online marking (e‐marking) skills.

• “I really enjoyed e‐marking and cannot imagine going back to paper.”

• scoris was very easy to do ... the familiarization mode was helpful in getting me ready for scoring ... E‐marking is very, very easy to do and was very enjoyable!very easy to do and was very enjoyable!

• “Far superior to the previous system!”

Examiner support

• For any examiner e‐marking there is a dedicated y gsupport team available 7 days a week during the session.

• Any queries regarding e‐marking should be directed to emarking@ibo.orgg@ g

Why become an examiner?Why become an examiner?

• Professional developmentProfessional development– Deeper knowledge of assessment

• Examiner experienceExaminer experience– Access to materials

– Part of an examining team

• Examiner training

– Candidate answers

Why become an examiner? (cont)Why become an examiner? (cont)

• MoneyMoney

• StatusAdd t t h CV– Adds to a teachers CV

• Input

Why become an examiner? (cont)Why become an examiner? (cont)

Case StudyCase Study• Alison Ryan

• Mathematical Studies team leader

• “I started marking as an assistant examiner and subsequently worked with the IB as the d hi f i f M h i ldeputy chief examiner for Mathematical Studies between 2001 and 2006.  I am currently a team leader for both Mathematical Studies projects and paper 1.”

Why become an examiner? (cont)Why become an examiner? (cont)

Case StudyCase Study• “Since becoming an examiner I have been involved in two 

curriculum reviews, I have run teacher workshops and been involved in school authorisation visits.  Being involved in these activities has allowed me to meet interesting people from around the world who teach a wide variety of subjects ”around the world who teach a wide variety of subjects.

How to applyHow to apply

• IB website – www ibo org/examinersIB website  www.ibo.org/examiners

• OETR – examrecruit@ibo.org

li i f• Application form

• Application process– Policy

– References

– Appointment

QuestionsQuestions