2010 world schools debating championships2010 world schools debating championships chief...
TRANSCRIPT
2010 World Schools Debating Championships
Chief Adjudicators’ Report
Simon Quinn and Chris Erskine
Introduction
1. The 2010 World Schools Debating Championships were held in Doha, Qatar, from 8 to 18 February.
We shared the privilege of being the Chief Adjudicators for the tournament. In this report, we
summarise some of the adjudication innovations that we introduced this year, and note some of the
problems with our approach. We hope that this report may be useful for organisers of future World
Championships, and that it may provoke valuable discussion among the WSDC community.
2. We were assisted as Chief Adjudicators by the members of the Chief Adjudicators’ Panel (‘the CAP’):
Taimur Bandey (Pakistan), Pardip Chopra (Canada), Effie Giannakouri (Greece), Beth James (Wales)
and Eva Spoor (Netherlands). We have been most appreciative of all of their work throughout the
entire process; it was a privilege to work with them and their assistance throughout was invaluable.
3. This report only addresses issues relating to the administration of adjudicators. We were both
members of the Motions Selection Committee this year; we would be very happy to discuss the
Page 1 of 19
selection of motions, but would rather do that in conjunction with members of the Motions Selection
Committee. In this report, we therefore do not discuss motions selection or any other issues arising.
Overall Approach
4. As Chief Adjudicators, we tried throughout to make all major decisions by applying criteria that were
announced in advance. “Transparency” is an over-used word, but has many merits as a guiding
principle for running a tournament such as WSDC. We certainly did not deflect all criticism during
the tournament, and we acknowledge that some decisions were not made as openly as they should
have been. But the major decisions were made in this way, and informal feedback suggests that many
people appreciated understanding how and why particular decisions had been made. We believe that
future Chief Adjudicators should adopt the same overall approach wherever possible.
Sponsored Adjudicators
5. QatarDebate generously funded flights and registration for 30 adjudicators from around the world (in
addition to funding flights and registration for both of us, for members of the CAP and for the Chair
of the WSDC Executive). We worked with the CAP to select the 30 sponsored adjudicators, using
criteria that emphasised the importance both of proven adjudication expertise and of geographical
diversity. The details of this process have been published extensively in previous documents, which
are included as appendices to this report. We believe that the application and selection process was a
success. We hope that, if any future host is willing to provide such sponsorship, that sponsorship
should again be provided both on the basis of proven expertise and on the basis of geographical
diversity.
Adjudicator Briefing
6. We introduced a new approach to the adjudicator briefing this year. We decided that, rather than
providing a comprehensive discussion of the rules and principles of WSDC (much of which would be
Page 2 of 19
extremely familiar to most participants), we would ask every adjudicator to read the rules and
principles of judging before arriving, in anticipation of sitting a ‘revision test’ at the beginning of the
training session. We decided that we would then use the available training time for discussion of
contentious adjudication issues and for the traditional video adjudication. We believed that this
would be a more productive use of time, allowing for more discussion of important and contentious
issues and less discussion of those issues that could be understood by reading the prepared materials.
7. This was less successful than we had hoped. A number of teams raised concerns during the
tournament that some adjudicators did not seem to understand basic concepts, or did not seem to be
weighing competing factors appropriately. In part, this may be a matter of inexperience; many
adjudicators were attending their first World Schools tournament. In part, this may be a matter of
the random assignment of adjudicators (discussed further below), which meant that experienced
coaches may have been more likely than in previous years to have their teams judged by
inexperienced adjudicators.
8. However, in part this may also be a consequence of the way that the briefing was run. Even though a
comprehensive discussion of basic principles is seen by many adjudicators as unnecessary, such
discussion may be particularly useful for less experienced adjudicators.
9. We suggest that future Chief Adjudicators return to the issue of how best to use the time available in
the Adjudicator Briefing. With a limited time available for the briefing, there is a real tension between
the need to revise basic concepts and the need to discuss contentious adjudication issues. We
understand that many people felt this year that we did not emphasise enough the revision of basic
concepts, and we see significant merit in that opinion.
10. Against that, however, is an equally strong view. Adjudicators coming to the championship are
supposed to have read the background material. Merely rehashing what is in the written materials is
not only poor teaching method and a waste of time for those who have already read them, but an
admission that adjudicators have not done what is expected of them and read the materials
Page 3 of 19
thoroughly. The short test was a way of encouraging people to read the materials, and to focus
attention on contentious issues.
11. One possible approach may be to increase the importance of the adjudication test at the start of the
briefing. This year, the test was intended merely to highlight some important issues and to prompt
subsequent discussion (and, hopefully, to have encouraged adjudicators to read all of the relevant
materials in advance). For this reason, we paid almost no attention to the results of the revision test,
and we were quite happy for adjudicators to confer when answering the questions.
12. However, it may be that this revision test – in addition to the existing video adjudication test – could
play a more important role at future tournaments. For example, the test could be conducted ‘under
examination conditions’, and the results used for assessing adjudicators’ expertise. This may have the
added advantage of ensuring that all adjudicators read all of the relevant materials before the
tournament.1
13. Another possible approach was used by the World Universities Debating Championship in Turkey in
January this year, and could be studied for our competition. A test debate was put online on the
championship website, and had to be judged by every adjudicator who wanted to take part in the
championship. Something similar might be used for our championship, noting however that the
Council has now several times voted against accreditation in advance for judges. But using the online
test as a way of identifying judges in need of special training on arrival at the championship might be
an acceptable way to use the briefing time to best advantage.
14. We note one further problem with the adjudicator briefing. At the conclusion of the video debate,
the result from the debate was briefly displayed before the video was stopped. This was an oversight,
as at least one adjudicator noted on his blog. This should not have occurred, and we should have
1 At the WSDC Council Meeting, a concern was raised about the rules regarding adjudicators’ use of ‘specific knowledge’; that is, factual knowledge that a particular adjudicator has that is not generally shared by well-informed members of the public. The rules and principles concerning specific knowledge are set out very clearly in the relevant materials that all adjudicators were asked on several occasions to read before the tournament, and we were disappointed to hear that this had apparently nonetheless been a problem in some adjudications.
Page 4 of 19
taken more care in ensuring that the video was stopped immediately after the final speech of the
debate.
15. One further objection was made: that, because the video debate was from an earlier WSDC
tournament, some adjudicators would anyway have known the result. We acknowledge this, but we
do not see any practical alternative to the use of a previous WSDC debate for the video test. If a
video is used from any other tournament, the style of debate will likely be different (even if only
subtly different) from the WSDC style, and there would still be an objection that some adjudicators
would know the result. A live debate could be conducted for the purposes of adjudicator assessment,
but this would necessarily have a degree of unreality. If such a live debate were to involve debaters
who were to participate in the tournament, this would also be unfair. We therefore suggest that
future Chief Adjudicators continue to use videos from previous WSDC tournaments for the
adjudication test, even though some adjudicators may already know the result.
The Draw
16. We introduced two innovations in running The Draw this year.
17. First, we released publicly a ‘Draw Structure’ document, included as an appendix to this report. This
document, set out in the form of a grid, showed which teams would meet in which rounds. For
example, the grid showed that “Team A1” would be Opposition against “Team A2” in Round Eight.
Importantly, this structure was set out before the draw was done; that is, for example, before Team
A1 was drawn as Ireland and Team A2 drawn as Singapore.
18. The construction of the ‘Draw Structure’ was not a trivial exercise. We developed the structure so
that, as far as possible, every team met one team from each different ‘group’, where groups were
defined on the basis of average performances at the tournament over the past three years. Further,
we sought, as far as possible, to ensure that each team had two debates as Proposition on a prepared
motion, two debates as Proposition on an unprepared motion, two debates as Opposition on a
Page 5 of 19
prepared motion and two debates as Opposition on an unprepared motion. Finally, we were required
to group teams so that the same teams travelled to the same venues on the same day (so that no team
needed to change venues during the day).
19. For this reason, the development of the Draw Structure took substantial time; the final Draw
Structure, for example, was the ninth version that we had tried, after a development process that had
extended over several months. Indeed, the difficulties in the process were highlighted by the fact that
the initial Draw Structure that we published transpired to have a mistake, which was pointed out to
us shortly after the Structure was initially published; this allowed us time to correct the error and
release a revision.
20. We believe that future tournaments should adopt a similar structure, and we suggest that it may be an
efficient use of time to take this year’s Draw Structure as a starting point. We encourage future Chief
Adjudicators to publish the proposed Draw Structure as soon as possible (that is, as soon as possible
once the total number of attending teams is known) to allow for the possibility, as this year, that
members of the WSDC community may identify any errors that have been made.
21. Future Chief Adjudicators may wish to note one anomaly with this year’s Draw Structure: namely,
that teams of a similar ranking tended to meet on unprepared motions, while teams of a substantially
different ranking tended to meet on prepared motions. (For example, a team in group A would have
its four unprepared debates against teams from groups A, B, C and D, and have its four prepared
debates against teams in groups E, F, G and H.) We never intended this outcome, and did not notice
it until it was pointed out to us a few days before the completion of the draw.2 Given the pressing
time constraints, and the difficulties that we had encountered in finding a workable Draw Structure to
that point, we chose not to make further changes. We do not believe that the anomaly caused the
2 Of course, we do not mean to imply by this that the anomaly was purely random, a “complete fluke”. Rather, the anomaly emerged quite unintentionally and unexpectedly from the Draw Structure itself, and we did not recognise this fact until after the Draw Structure had been published and released. We believe that the Draw Structure could be reworked relatively easily to deal with this anomaly if it was thought to be a significant issue.
Page 6 of 19
draw to be any less fair, less random or less balanced. However, future Chief Adjudicators may want
to try to balance the Draw Structure along this dimension as well.
22. The second innovation in the draw process was the running of a transparent and random process to
allocate positions within groups. This was run in different cities around the word, supervised by well-
known members of the WSDC community. In the space of a few days, a different group was drawn
in each city, with the final group being drawn in Doha. None of the draw was released until all of the
teams had been drawn; the draw was then released online in its entirety. The draw videos were then
posted on YouTube.3
23. We strongly encourage future Chief Adjudicators to continue this approach. We believe that this is a
demonstrably open, transparent and random way of assigning teams, and is an important way of
reassuring all teams that they have been treated equally.
24. There were, however, at least two weaknesses with this process this year. First, there was a time lag
of several days between the release of the draw and the release of the videos on YouTube. This was
caused by a delay in editing and uploading the videos; the process of e-mailing and uploading high-
definition videos of the requisite length still currently takes many hours, even on broadband internet
connections. We encourage future Chief Adjudicators to consider editing and uploading the draw
videos to a ‘closed channel’ on YouTube as the videos are e-mailed in; it would then be
straightforward to upload the final video and ‘open’ the channel at the same time that the draw is
released.
25. Second, we believe that there was a perception among some debaters that the draw had been released
selectively several days before it was actually released. This was absolutely not true; the first draft of
the final draw (excluding the team positions drawn in Doha) was only created a few hours before the
Doha draw. The final draft of the draw was sent to relevant e-mail lists within about 30 minutes of
the completion of the Doha draw, and was posted to the WSDC blog several hours later. We are not
3 The videos are still online, available at http://www.youtube.com/wsdc2010.
Page 7 of 19
sure what more we could have done to reassure participants that the draw was being done fairly and
transparently. However, this perhaps only underlines the importance of doing all of the draw groups
within the space of a few days, and releasing the complete results as soon as possible.
ESL and EFL Status
26. We reviewed all applications for ESL/EFL status this year. This included checking issues directly
with a number of teams, as well as using internet sources to confirm the official status of English in
several countries. In all cases in which we disagreed with a team’s application (for example, because a
team had registered as ESL despite internet sources suggesting that English is an official language in
the country), we referred the matter back to the team for further clarification.
27. This process revealed that a relatively large number of teams did not complete their ESL/EFL
registration correctly. We recommend that the ESL/EFL registration form should be redrafted
before the next tournament in order to be clearer and more understandable to applicants who may
not be fully aware of what information is needed.4 This is true both for the registration of ESL/EFL
team status and for the registration of ESL/EFL speaker status. We recommend that future Chief
Adjudicators should again carefully check all ESL/EFL applications against the tournament rules.
The Tab
28. We introduced several innovations regarding The Tab this year. All related to a new suite of
programs that we wrote for the tournament; the programs were coded in Stata 11.0, and produced
PDF output by the LaTeX markup.
29. First, we produced customised mark sheets with the name of each adjudicator and the names of the
debaters involved. We believe that this saved time at the start of each debate, because teams were
4 For example, a revised form could provide a set of ‘yes/no’ questions, with guidance along the lines of “If you answered ‘yes’ to any of the previous questions, your team is not eligible to register as ESL/EFL”, etc.
Page 8 of 19
required only to indicate who would speak in each position, rather than needing to write speaker
names on adjudicators’ mark sheets in full. Further, we believe that this improved the quality of data
entry by minimising the risk of data entry errors in identifying speakers (for example, by a speaker’s
name being misspelled during data entry and that speaker’s mark therefore being incorrectly added to
the tab). We encourage future Chief Adjudicators to continue this approach, and we would be happy
to provide the relevant Stata code on request.5
30. Second, we had every mark from every debate entered into the Main Tab database (that is, every
speaker mark for style, content, strategy, points of information and totals). This was made possible
only by the very diligent work of a large team of local volunteers, and we remain extremely
appreciative of their efforts. We believe that there are several advantages to entering every mark.
First, this allowed for adding errors to be caught and corrected. Second, this reduced the risk of data
entry errors (for example, errors in entering debate results, or total team scores). Additionally, this
will allow for statistical analysis of trends in adjudicating at the tournament.6 We encourage future
Chief Adjudicators to use a system that involves the entry of every mark.
31. Third, we maintained an independent Shadow Tab. This was run by Will Fleming, who attended the
tournament as Chief Adjudicators’ Assistant. This tab recorded results and total team scores only, all
of which were entered in a separate process to the data entry on The Tab. At the conclusion of
Round Eight, we reconciled the Main Tab with the Shadow Tab. This process identified several
further data entry errors in the Main Tab that would not otherwise have been corrected. This year,
with a 16-team break, both the 16th- and 17th-ranked teams had five wins and 16 adjudicators; the
teams were separated on total speaker points, of which there was a margin of just 2.5. We believe
that this underscores the importance of running an independent Shadow Tab, and we encourage
future Chief Adjudicators to continue this innovation.
5 However, we note that Stata is not a cheap package to purchase, and we recognise that future tournaments may wish to consider other packages instead. 6 We have conducted some preliminary analysis of this kind, but much remains to be done. We hope to provide a statistical summary well before WSDC 2011.
Page 9 of 19
32. Though we remain confident that the Tab was correct, there were inefficiencies in the tabbing
process that could be improved at future tournaments. We had hoped to provide updated results
each evening, and this generally did not occur; as at least one adjudicator pointed out on his blog, we
were generally slow to publish results. Further, we had hoped to produce customised PDF files of
mark sheets at the end of each day; these PDFs could then have been printed or e-mailed to
interested coaches. However, the production of these PDFs was delayed substantially by the other
delays in the tabbing process, and we were disappointed that we were not able to give faster feedback
to coaches in this way. We believe that there were several reasons for these delays.
33. First, the data entry process was delayed by the need to chase adjudicators to correct adding errors.
By the end of the preliminary rounds, we had introduced a system of checking adjudicators’ adding at
the venues; this allowed us to find adjudicators immediately to correct any errors, and considerably
accelerated the data entry process. We encourage future Chief Adjudicators to instigate such a system
from Round One.
34. Second, because the Main Tab was run exclusively as an integrated data management system, we were
unable to produce daily summaries of results and margins until we had entered all of the marks for
each debate. Without diminishing the importance of entering every mark for every debate, future
Chief Adjudicators may wish to consider running a simple ‘summary tab’ in parallel to the Main Tab;
indeed, the Shadow Tab could be used for this purpose.
35. Third, future Chief Adjudicators may consider it worthwhile to assign a separate person to run the
Main Tab; we, however, ran the Main Tab ourselves, in parallel to other tasks (including, for example,
the assignment of adjudicators, assignment of shadow adjudicators, production of customised ballots,
etc). Having a separate person for the tab would likely accelerate the data entry process; we were
often required to correct data entry errors in the early hours of the morning, after the assignment of
adjudicators and production of ballots had been done. In our case, this was necessary because we
had coded the software ourselves, but future Chief Adjudicators may find it worthwhile to use a
separate person for this role.
Page 10 of 19
36. We believe that the shadow tab is an essential double check on the Main Tab. It has been used in
one way or another in most championships in the last 10 years. It is not difficult to maintain a
shadow tab; this year, Will operated the shadow tab as an Excel spreadsheet. The first column was
the list of teams. The next three columns were the overall marks, number of judges and win (1 for a
win, 0 for a loss) for that team in Round One. The columns continued in the same way across the
spreadsheet for all subsequent rounds, finishing in a trio of totals columns. Will had access to the
mark sheets when they had been entered in the Main Tab, and was able to enter results in his tab
considerably faster than the Main Tab (because much less information was being captured).
Assignment of Adjudicators: Publication of principles
37. Prior to the tournament, we released a document entitled ‘WSDC 2010: Guidelines for Assigning
Adjudicators’. That document, which is provided as an appendix to this report, set out how we
intended to assign adjudicators for the preliminary rounds, how we intended to assign chairs for the
preliminary rounds and how we intended to assign adjudicators for the finals rounds. We believe that
setting out these policies clearly contributes to the transparency of the tournament, and we encourage
future Chief Adjudicators to publish a similar document before future championships.
Assignment of Adjudicators: the Preliminary Rounds
38. We introduced a new method of assigning adjudicators to debates in the preliminary rounds this year.
In short, we used our data management software to assign adjudicators randomly. We believe that
the random assignment of adjudicators is an important aspect of ensuring the fairness and
transparency of the tournament, because it removes the discretion of the Chief Adjudicators and the
CAP in assigning adjudicators to debates. We see no reason, in principle, why debates between
‘good’ teams should be assigned ‘good’ adjudicators, nor why the Chief Adjudicators and the CAP
should try to predict in advance which debates will prove important for determining the Break.
Page 11 of 19
Further, we believe that the transparency involved in random assignment of adjudicators is inherently
beneficial.
39. We strongly encourage future Chief Adjudicators to maintain the random assignment of adjudicators
in preliminary rounds; indeed, we would be very pleased if this were to become an important guiding
principle for the administration of adjudicators at WSDC.
40. We did not believe that it would be adequate, however, merely to assign adjudicators to debates
randomly without constraints. We decided that the assignment should be done in a way that was
both random and balanced; we believed that it would have been unfortunate, for example, if any
debate were judged by three adjudicators each at their first WSDC tournament. In order to facilitate
a balance of adjudicator experience on each panel, we assessed all adjudicators (other than shadow
adjudicators) as being either ‘Level One’ (highly competent), ‘Level Two’ (very competent) or ‘Level
Three’ (competent); the computer program then randomly assigned one adjudicator of each ‘level’ to
each panel. With the CAP, we made some changes to these rankings each night on the basis of
feedback (and, in some cases, on the basis of convenience; for example, members of the CAP and
other highly competent adjudicators were sometimes listed as Level Two to balance numbers across
the three levels).
41. In addition, we imposed the constraints that (i) no adjudicator could judge a team with which that
adjudicator had a conflict, (ii) no adjudicator could judge with an adjudicator from the same ‘nation
of affiliation’ and (iii) as far as possible, no panel would comprise three men or three women.
42. Additionally, we had hoped to constrain the assignment so that no adjudicator would judge the same
team twice in the preliminary rounds. We were required to drop this constraint when assigning
adjudicators for Rounds Seven and Eight, because the computer was unable to find a solution
Page 12 of 19
otherwise. However, by imposing this constraint until that point, we were at least able to ensure that
no adjudicator judged the same team more than twice in the preliminary rounds.7
43. We believe that this process worked well, and we commend it to future Chief Adjudicators.
Assignment of Adjudicators: Chairs for the Preliminary Rounds
44. We initially intended that the role of chairing each panel (that is, the adjudicator delivering the
adjudication at the end of the debate) would be assigned as a random choice between the Level One
and Level Two adjudicator on the panel. We supported this approach because we believed that it
would allow a large number of adjudicators to have the experience of chairing. We believed that this
would be valuable both for adjudicators and for teams.
45. However, this did not work well. We receive feedback from a number of coaches and adjudicators
that the delivery of adjudications could have been done better; in particular, that many panel chairs
provided only extremely brief discussion of the debate before announcing the result. We therefore
changed the mechanism for assigning chairs after Round Four; for Rounds Five to Eight, the Level
One adjudicator was automatically assigned to be the chair. We remain unsure of how effective this
change was, and we were aware of some feedback that even Level One adjudicators could have
delivered better adjudications.
46. We suggest that future Chief Adjudicators should revisit this issue. We continue to see value in
allowing a large number of adjudicators the opportunity to chair, so that the experience of
synthesising and presenting the opinions of the panel is not limited to Level One adjudicators.
However, we also recognise that many Level Two adjudicators may not have extensive experience at
doing this. Perhaps one approach would be to assign the role of chair randomly in Rounds Five to
Eight (rather than, as this year, in Rounds One to Four); another alternative would be for the Chief
7 Of the 684 adjudicators’ ballots completed in the preliminary rounds, 55 of the ballots (that is, about 8%) were from adjudicators who had judged the same team at an earlier point in the tournament.
Page 13 of 19
Adjudicators and CAP to assign the role as a matter of discretion, after the computer program had
randomly determined the composition of panels.
Assignment of Adjudicators: the Final Rounds
47. We assigned adjudicators to panels for Final Rounds in close consultation with the CAP.
48. In doing so, we applied the following criteria.
(a) No adjudicator judged the team from his or her own nation, nor any other team with which the
adjudicator had a significant association.
(b) In the Octo-Finals, Quarter-Finals and Semi-Finals, no adjudicator judged the same team more
than once. However, as we indicated before the tournament, we decided that an adjudicator
could judge the Grand Final if that person had already judged one of the Grand Final teams in
the Final Rounds. Further, as we also indicated before the tournament, we decided that an
adjudicator could judge a team in the Final Rounds that the adjudicator had judged in the
Preliminary Rounds.
(c) No adjudicator was allowed to judge either Semi-Final if his or her team – or any team with
which that adjudicator had a significant association – was in either Semi-Final.
(d) No Final Round panel had more than one adjudicator from any given nation.
49. With the CAP, we reviewed the composition of all Final Rounds panels to ensure that each panel
comprised adjudicators from diverse backgrounds.
50. We were fortunate this year to have a large pool of very good adjudicators. We therefore decided
that we should have relatively large Final Rounds panels. We believe that, subject to having a large
pool of good adjudicators, having larger panels is an important way of improving the quality of
decisions reached, by reducing the probability that any one adjudicator’s opinion will determine the
Page 14 of 19
result.8 We therefore assigned panels of five adjudicators for the Octo-Finals, seven adjudicators for
the Quarter-Finals, nine adjudicators for the Semi-Finals and a panel of 11 adjudicators for the Grand
Final.
51. We believe that these criteria and this approach worked well. We recommend that future Chief
Adjudicators follow a similar practice.
52. We assigned judges to final rounds by noting that these rounds are structured according to a ‘tennis-
style’ knockout draw in which each team’s potential progress to the final is fixed. One thing stands
out about this fixed draw: each team is always in the same half of the draw, and will not meet any
team from the other half of the draw until the grand final. One of the basic requirements of a judges’
draw is that a judge does not judge the team from his or her own country. We began the judging
draw, therefore, by assigning all judges whose teams were in the final rounds to the opposite half of
the draw from the half in which their team was debating. We then finished the judges draw by
assigning judges whose teams were not in the break rounds.
53. One criticism of previous break round judging draws is that some teams ended up with the same
group of judges on several occasions. To reduce the likelihood of this, we assigned judges to
Quarter-Finals in the same half of the draw as their own team, but in the Quarter-Final in that half of
the draw that did not involve their team. Similarly, for judges whose teams were not in the break
rounds (or had been knocked out in the Octo-Finals), they too were transferred to the other half of
the draw for the quarterfinals. This meant that no adjudicator ever judged the same team in the
Octo- and Quarter-Finals.
8 This can be illustrated by a simple mathematical example. Suppose – for the sake of simplicity – that every adjudicator has a 50% probability of voting Proposition and a 50% probability of voting Opposition; suppose further that adjudicators’ separate decisions are mathematically independent. In that case, the probability that a three-adjudicator panel will deliver a 2-1 split decision (that is, the probability that the result could be changed by a single adjudicator’s vote) is 75%. When the number of adjudicators on the panel increases to five, the probability of a 3-2 split decision is 62.5%. When the number of adjudicators increases to seven, the probability of a 4-3 split is about 55%. When the number increases to nine, the probability of a 5-4 split is about 49%. When the number reaches 11, the probability of a 6-5 split is about 45%. This, of course, is a highly stylised example; in reality, because adjudicators watch the same debate, their decisions are not mathematically independent, so the probability of such close splits in reality is much less. (For example, of the 228 debates in the preliminary rounds, 70 were decided on 2-1 split decisions; a proportion of about 31%, rather than the 75% that is predicted by the stylised example considered here.) However, the simple example nonetheless illustrates the value of larger panels for reducing the likelihood that an individual vote could change the result of the debate.
Page 15 of 19
54. We continued the conventional requirement that no judge could judge either Semi-Final if their team
was one of the semi-finalists. Having adopted the swap-over of judges between Octo- and Quarter-
Finals, it was considerably easier to assign Semi-Final panels of judges which involved little if any use
of judges who had judged the two teams in a particular Semi-Final in the earlier finals rounds.
55. While it would be a nice ideal to ensure that a judge did not judge a team that he or she had judged in
the preliminary rounds, this is not practical. However, the random judges draw outlined earlier
ensured that it was highly unlikely that any judge in the break rounds had judged any of the teams
involved more than once in the preliminary rounds.
Shadow Adjudicators
56. Our management of shadow adjudicators this year was poor. We had intended to provide
comprehensive support for shadow adjudicators, including (i) clear instructions on which venues
such adjudicators should attend, (ii) clear opportunities for mentoring and guidance through
adjudicators experienced in the WSDC style and (iii) clear avenues for those mentors to provide
feedback to us and to the CAP to assist in adjudicator assignment for the following day.
57. We achieved all of these goals, but only in a partial sense. The assignment of shadow adjudicators to
mentors was generally the last thing done each evening (i.e. often in the early hours of the morning),
and the instructions to such adjudicators each day could therefore have been much clearer. In
particular, we should have made it much clearer that shadow adjudicators who did not work with a
senior adjudicator would not be able to be promoted to be full adjudicators, even if those shadow
adjudicators had substantial experience in other competitions. In several cases, adjudicators with
substantial experience in domestic competitions were assigned to be shadow adjudicators (on the
basis that they did not have previous WSDC experience), and were then not able to be promoted to
be full adjudicators because we did not adequately explain to those adjudicators that they were
Page 16 of 19
expected to work as shadows to senior adjudicators, who would then provide feedback on their
performance.
58. This was a disappointing outcome. It is entirely likely that some of the shadow adjudicators who
were never promoted to be full adjudicators would have been extremely competent. However, we
did not believe that we could promote those adjudicators without reports on their performance as
shadow adjudicators, and it was a substantial oversight that we did not provide clearer guidance on
this point from the outset. Moreover, given that the adjudicators affected by this oversight were
generally at their first WSDC tournament, this may have contributed to a sense among some
adjudicators that the adjudication side of the tournament was administered as a clique. This was
never our intention – indeed, most of our innovations this year were designed to ensure transparency
and objectivity wherever possible, and to involve a wide range of people in the process of running the
tournament. We hope that future Chief Adjudicators will prove better at managing shadow
adjudicators than we were this year.
The Break
59. This year, The Break was announced by the Convener (Dr Hayat Maarafi), with the support of a
PowerPoint-style presentation. We believe that this format worked well.
60. However, we regret one problem with The Break. Though all teams and team flags were presented
correctly and in the correct order, one team name was wrongly replaced by another team name on a
slide indicating the assignment of adjudicators for the Octo-Finals. The mistake on the slide was an
innocent one, caused by a tired and careless reading of a three-letter country code. However, some
participants were offended and booed the Chief Adjudicator making the announcement. We
apologised to the team and other members of the delegation concerned, both on the evening of The
Break and again on the following day. The mistake should not have occurred, and we regret the
entire incident.
Page 17 of 19
Gifts for Adjudicators
61. We believe that, on at least one occasion this year, an adjudicator asked a team whether the team had
brought a gift for that adjudicator. We believe that this may have been intended as a flippant remark,
and we recognise that there has been a practice among some teams of giving cheap souvenirs from
their countries to adjudicators and other tournament participants. However, we are concerned that
teams may interpret such comments to suggest that gifts are expected. There is a particular risk that
teams that are relatively new to the tournament may interpret such a remark in this way. We suggest
that future Chief Adjudicators may wish to make clear that adjudicators should never make
comments of this kind, and may also wish to make clear that no team should ever be expected to give
any gift of any kind to any other participant at the tournament. (Indeed, were any team to offer any
adjudicator anything more than a cheap souvenir, we believe that it would be appropriate for the
adjudicator to refuse it. However, we are not aware of this occurring, either at this or at previous
tournaments.)
Thanks
62. We would like to thank many people for their invaluable assistance throughout the entire process.
63. First, we must thank Dr Hayat Maarafi and everyone at QatarDebate for their exceptional
organisation and generosity in hosting the tournament. Similarly, we would like to thank Fiona
Simon and everyone at QMDI for their invaluable assistance throughout the entire process. With 57
teams, WSDC 2010 was comfortably the largest WSDC ever run, and the tournament simply could
not have happened without the outstanding work of Dr Maarafi, QatarDebate and QMDI.
64. Second, we would like to thank everyone involved in the administration of adjudicators throughout
the tournament. In particular, we would like to thank again the members of the Chief Adjudicators’
Panel, and the Chair of the WSDC Executive, Claire Ryan. We would also like to thank again all of
the Qatari volunteers who assisted in the extensive data entry process; it would not have been
possible to run a comprehensive tab without their very generous help.
Page 18 of 19
Page 19 of 19
65. Third, we must thank all of the adjudicators, for their time, their expertise and their patience. The
role of an adjudicator at WSDC can sometimes be as thankless as it is invaluable, and we remain
extremely appreciative of every adjudicator’s efforts.
66. Fourth, we would like to thank Will Fleming for acting as assistant to the Chief Adjudicators. With
the large number of teams this year, and the scale of the administrative tasks involved in running the
judges’ draw, his assistance was invaluable. We believe that it was particularly useful that Will was not
adjudicating, and could therefore work on these tasks during the day; we encourage future Chief
Adjudicators to adopt a similar system.
67. We believe that the World Schools Debating Championships remains an outstanding tournament,
reliant upon the goodwill of many volunteer participants, run for the best interests of a large number
of remarkable young people from around the world. For all of the shortcomings discussed here, we
believe that WSDC 2010 was a great success, and we hope that this sentiment is shared generally by
the WSDC community. We look forward to many successful WSDC tournaments in the years to
come.
Simon Quinn
Oxford, United Kingdom
Chris Erskine
Canberra, Australia
Appendices
Please find attached the following appendices, in order:
1. Call for Applications: Sponsored Adjudicators;
2. Principles for Assessing Applications for Sponsored Adjudication Positions;
3. Sponsored Adjudicators;
4. Draw Structure;
5. Pre-Tournament Rankings;
6. The Draw (explanation);
7. The Draw (teams and positions);
8. WSDC 2010: Guidelines for Assigning Adjudicators.
World Schools Debating Championships Qatar 2010
Call for Applications: Sponsored Adjudicators
QatarDebate is intending to pay for thirty adjudicators to attend the next World Schools Debating Championships, being held in Qatar in February 2010. Sponsored adjudicators will have their tournament registration and return economy airfare paid by QatarDebate; in every other respect, they will participate in the tournament in the same way as all other adjudicators. Applicants will be chosen both on the basis of (i) proven adjudication record at WSDC and (ii) the need for diversity, particularly geographical and cultural diversity. All applicants must be able to be accepted as an eligible judge of the championship; therefore, applications will not be accepted from anybody who has debated at the 2009 WSDC in Athens. The sponsored positions will be allocated by the two Chief Adjudicators (Chris Erskine and Simon Quinn) in consultation with the Chief Adjudicators’ Panel. QatarDebate is now calling for applications for these sponsored positions. If you are interested in applying, please e-mail both [email protected] and [email protected] with a brief summary of your adjudicating experience. Applicants who have not judged at WSDC previously should provide a name and contact information of somebody who has judged at WSDC previously to act as a reference, should it be needed. The deadline for applications is 1 August; the decision will be made and all applicants notified by 1 September. Simon Quinn Chris Erskine 15 June 2009
World Schools Debating Championships Qatar 2010
Principles for Assessing Applications for Sponsored Adjudication Positions
QatarDebate is intending to pay for thirty adjudicators to attend the next World Schools Debating Championships, being held in Qatar on 8 – 19 February 2010. On 16 June, we issued a formal Call for Applications for those positions. That call stated that applicants would be chosen on the basis of adjudication experience and the need for diversity. In this note, we outline in more detail the principles that will be used for implementing those criteria. This note is intended to ensure transparency in the process of selecting sponsored adjudicators. We do not intend this note to change any of the guidance previously issued, and we believe that all applications received so far have provided sufficient information to apply these principles. (Nonetheless, any applicant who wishes to submit further information in light of these guidelines is very welcome to do so.) The aim of the pool of 30 sponsored adjudicators is to provide a core group of competent judges who, between them, are easily capable of judging any debate in the championship. The ‘capability of judging’ requires not only competent and experienced judges of the highest order, but judges drawn from as many nations as is reasonably practicable. The task of selecting the sponsored adjudicators is therefore more complicated than finding the 30 most experienced judges. National diversity is a key requirement. The process will therefore necessarily require subjective weightings to be made. However, we intend to do that in the most transparent manner possible. We therefore intend to follow three steps. 1. The threshold step All sponsored judges must be likely to be assessed, after the judges’ briefing at the beginning of the Championship, as suitable to judge from Round One. The most useful demonstration of this likely suitability will be an applicant’s previous experience judging at the World Schools Debating Championships, especially in the later stages of the Championships. However, other experience will also be useful. Such experience may include (but need not be limited to): (i) judging at the highest level of national schools championships run on rules similar to those of WSDC, (ii) judging at the World University Debating Championships, especially in the later stages, (iii) judging at higher levels of significant regional and international tournaments for schools or universities. In the case of applicants with no previous judging at WSDC, we will seek comments from nominated referees as to their likely ability to judge according to the WSDC requirements.
i
ii
2. The need for experienced judges Judges passing the threshold step will be ranked in order of proven adjudication experience and demonstrated competence (for example, experience in judging finals rounds, being a member of the CAP, etc). The first 10 judges will be selected for sponsorship. In the case of applicants with no previous experience judging at WSDC, we will seek comments from nominated referees including, where necessary, information about the nature of the competitions that applicants have judged, the level in that competition at which they have judged, their competence at judging those competitions, any relevant information about the rules applicable to those competitions in comparison to WSDC, and an opinion about the applicant’s likely competence to judge at WSDC. 3. The need for diversity Judges not yet selected will remain eligible for selection if (i) they have passed the threshold step, and (ii) there is not yet a sponsored judge selected from their country. If an applicant is eligible to be associated with more than one country, the applicant’s own preferred affiliation will generally be taken to be the relevant affiliation unless the applicant’s background shows a significant affiliation with another nation as well as, or instead of, the applicant’s preferred affiliation. If 20 judges or fewer remain eligible for selection, those judges will be selected. In that case, the remaining places will be allocated on the basis of the earlier rank of proven adjudication experience. If more than 20 judges remain eligible, the 20 selected judges will be chosen by the earlier rank of proven adjudication experience. The Chief Adjudicators consider that, for the purposes of the selection process, this step will ensure cultural and other diversity among the sponsored adjudicators. The selection process The selection will be made by the Chief Adjudicators, in ongoing consultation with the Chief Adjudicators’ Panel. In the case that either of the Chief Adjudicators or members of the Chief Adjudicators’ Panel have a friendship with an applicant, that person (i) will offer only limited comment on the applicant, and (ii) will be heavily guided by the opinions of others involved in the process. It will be open to the Chief Adjudicators and members of the Panel to recuse themselves from considering particular applicants if they feel more comfortable doing so. The Chief Adjudicators and Chief Adjudicators’ Panel will not publicise the list of those who applied for sponsored positions, nor release the ranking of proven adjudication experience and competence. Brief reasons will be provided to unsuccessful applicants, but only if requested by that applicant. Those reasons will not include any comparative assessment of the applicant against any other applicants in a way that may reveal the identity of other applicants. Should a sponsored adjudicator be unable to take up the sponsorship, the Chief Adjudicators and Chief Adjudicators’ Panel may replace that person with another judge. This replacement will be selected on the same criteria set out above (for example, if the replaced applicant was chosen in the first 10 judges, a replacement will be more likely to be chosen on the basis of proven experience and competence; if the replaced applicant was chosen on the basis of the need for diversity, a replacement will be more likely to be chosen on this basis). Simon Quinn Chris Erskine 27 July 2009
World Schools Debating Championships Qatar 2010
Sponsored Adjudicators
We have, with the Chief Adjudicators’ Panel, chosen 30 adjudicators to receive sponsored positions at the 2010 World Schools Debating Championships. Listed in English alphabetical order, they are as follows.
Kris Ade (Canada) Chris Bishop (New Zealand) Ria Dharmawan (Indonesia) Branislav Fecko (Slovakia) Peejay Garcia (Philippines) Rob Garlick (South Africa) Diana Gotts (Scotland) Angelika Hoeness (Germany) Miroslav Josic (Croatia) Helen Koliais (Greece) Camilla Korder (Peru) Irena Kotikova (Czech Republic) Derek Lande (Ireland) Aaron Maniam (Singapore) Simona Mazilu (Romania) Irene McGrath (Scotland) Liana Miholic (Slovenia) Justice Modthlabani (Botswana) Rhydian Morgan (Wales) Mehvesh Mumtaz (Pakistan) Debbie Newman (England) Erin O’Brien (Australia) Joshua Park (Korea) Raihanuzzam Raiman (Bangladesh) Dana Schottlaender (Argentina) Ragnar Siils (Estonia) Alfred Snider (USA) Janet Webster (Canada) Asher Weill (Israel) Cameron Wyllie (Scotland)
Simon Quinn Chris Erskine 1 September 2009
[top left]
A1 H7 B7 G1 C1 F7 D7 E1 A2 H2 B2 G2 C2 F2 D2 E2 A3 H3 B3 G3 C3 F3 D3 E3 A4 H4 B4 G4 C4 F4 D4 E4 A5 H5 B5 G5 C5 F5 D5 E5A1 OU PP PU OP OU PP PU OPH7 OP PU PP OU PP OP PUB7 OU PP PU OU PP PU OPG1 PP OU OP PU PP OU OP PUC1 OU PP PU OP PU OP OU PPF7 PP OU OP PU OP PU PP OUD7 OU PP PU OP PU OP OUE1 PP OU OP PU OP PU PP OUA2 PU OU PP PU OPH2 OP PP OU OP PUB2 PU OP OU PP OU PPG2 OP PU PP OU PPC2 PU OP OU PP PU OPF2 OP PU PP OU OP PUD2 PU OP OU PP OU PPE2 OP PU PP OU PP OUA3 OU PP PU OP PU OPH3 OU OP PU OP PUB3 OU PP PU OP OU PPG3 PP OU OP PU PP OUC3 OU PP PU OP PU OPF3 PP OU OP PU OP PUD3 OU PP PU OP OU PPE3 PP OU OP PU PP OUA4 OU PP PU OP OU PP PU OPH4 PP OU OP PP OU OP PUB4 OU PP OU PP PU OP PU OPG4 PP OU PP OU OP PU OP PUC4 PU OP PU OP OU PP OU PPF4 OP PU OP PU PP OU PP OUD4 PU OP OU PP PU OP OU PPE4 OP PU PP OU OP PU PP OU
WSDC 2010: DRAW STRUCTURE
A1 H7 B7 G1 C1 F7 D7 E1 A2 H2 B2 G2 C2 F2 D2 E2 A3 H3 B3 G3 C3 F3 D3 E3 A4 H4 B4 G4 C4 F4 D4 E4 A5 H5 B5 G5 C5 F5 D5 E5A5 PU OP OU PPH5 OP PU PP OUB5 PU OP OU PPG5 OP PU PP OUC5 OU PP PU OPF5 PP OU OP PUD5 OU PP PU OPE5 PP OU OP PUA6 OU PP PU OPH6 OU PP OU OP PUB6 PU OP PU PPG6 OP PU OP PUC6 OU PP PU OPF6 PP OU OP PUD6 PU OP PU PPE6 OP PU OP PUA7 OU PP OU OPH1 OP PP OU PP OUB1 PU OP OU PPG7 OP PU PP OUC7 OU PP OU OPF1 PP OU PP OUD1 PU OP OU PPE7 OP PU PP OUH8 PU PP OP PP PU OU
[bottom left]
[top right]
A6 H6 B6 G6 C6 F6 D6 E6 A7 H1 B1 G7 C7 F1 D1 E7 H8 PP OP PU OU Pr Op Abbreviations:A1 2 2 2 2 4 4 PP: Proposition, prepared motionH7 OU 2 2 2 2 4 4 OP: Opposition, prepared motionB7 OP 2 2 2 2 4 4 PU: Proposition, unprepared motionG1 2 2 2 2 4 4 OU: Opposition, unprepared motionC1 2 2 2 2 4 4 Pr: PropositionF7 2 2 2 2 4 4 Op: OppositionD7 PP 2 2 2 2 4 4E1 2 2 2 2 4 4 Color codes:A2 OU PP OP 2 2 2 2 4 4H2 PU PP OU 2 2 2 2 4 4B2 PU OP 2 2 2 2 4 4G2 OP PU OU 2 2 2 2 4 4C2 OU PP 2 2 2 2 4 4F2 PP OU 2 2 2 2 4 4D2 PU OP 2 2 2 2 4 4E2 OP PU 2 2 2 2 4 4A3 OU PP 2 2 2 2 4 4 Round 0H3 PP PP OU 2 2 2 2 4 4B3 PU OP 2 2 2 2 4 4G3 OP PU 2 2 2 2 4 4C3 OU PP 2 2 2 2 4 4 Reading the draw:F3 PP OU 2 2 2 2 4 4 The code in each cell refers to the positionD3 PU OP 2 2 2 2 4 4 of the left team in debating the top team.E3 OP PU 2 2 2 2 4 4 For example, Team A1 will be OppositionA4 2 2 2 2 4 4 against Team A2 in Round 8, an unprepared H4 PU 2 2 2 2 4 4 round.B4 2 2 2 2 4 4G4 2 2 2 2 4 4C4 2 2 2 2 4 4F4 2 2 2 2 4 4D4 2 2 2 2 4 4E4 2 2 2 2 4 4
Round 7Round 8
Totals
Round 1Round 2Round 3Round 4Round 5Round 6
A6 H6 B6 G6 C6 F6 D6 E6 A7 H1 B1 G7 C7 F1 D1 E7 H8 PP OP PU OU Pr OpA5 OU PP PU OP 2 2 2 2 4 4H5 PP OU OP PU 2 2 2 2 4 4B5 OU PP PU OP 2 2 2 2 4 4G5 OP OU PP PU 2 2 2 2 4 4C5 OU PP PU OP 2 2 2 2 4 4F5 PP OU OP PU 2 2 2 2 4 4D5 OU PP PU OP 2 2 2 2 4 4E5 OP OU PP PU 2 2 2 2 4 4A6 PU OP OU PP 2 2 2 2 4 4H6 OP PU PP 2 2 2 2 4 4B6 OU PP OU OP 2 2 2 2 4 4G6 PP OU PP OU 2 2 2 2 4 4C6 PU OP OU PP 2 2 2 2 4 4F6 OP PU PP OU 2 2 2 2 4 4D6 OU PP OU OP 2 2 2 2 4 4E6 PP OU PP OU 2 2 2 2 4 4A7 PU OP PP PU 2 2 2 2 4 4H1 PU OP PU 2 2 2 2 4 4B1 PU OP OU PP 2 2 2 2 4 4G7 PP PU OU OP 2 2 2 2 4 4C7 PU OP PP PU 2 2 2 2 4 4F1 OP OP PU PU 2 2 2 2 4 4D1 PU OP OU PP 2 2 2 2 4 4E7 PP PU OU OP 2 2 2 2 4 4H8 OP OU 2 2 2 2 4 4 Chris Erskine & Simon Quinn, 30 January 2010.
[bottom right]
WSDC 2010: PRE-TOURNAMENT RANKINGS
Rank Country Wins Judges Wins Judges Wins Judges Wins Judges Group1 New Zealand 8 19 8 23 x x 8.00 21.00 A2 England 8 22 8 21 7 21 7.67 21.33 A3 Australia 8 21 6 18 7 19 7.00 19.33 A4 Ireland 6 16 6 18 8 21 6.67 18.33 A5 Singapore 6 20 6 19 7 19 6.33 19.33 A6 South Africa 7 20 8 22 4 15 6.33 19.00 A7 Canada 7 19 7 21 5 17 6.33 19.00 A8 Pakistan 6 19 7 19 6 19 6.33 19.00 B9 Hong Kong x x 5 15 7 21 6.00 18.00 B10 Slovenia 6 18 6 16 6 16 6.00 16.67 B11 Scotland 5 16 5 15 7 19 5.67 16.67 B12 Greece 6 16 6 18 5 13 5.67 15.67 B13 Korea 6 19 5 15 5 17 5.33 17.00 B14 Israel 3 12 7 19 5 15 5.00 15.33 B15 Argentina 4 11 5 13 5 15 4.67 13.00 C16 Wales 5 16 5 14 3 12 4.33 14.00 C17 Peru 7 20 3 11 3 9 4.33 13.33 C18 Netherlands 5 16 5 14 2 7 4.00 12.33 C19 Qatar 4 11 4 12 x x 4.00 11.50 C20 Philippines 5 14 2 6 5 14 4.00 11.33 C21 United States 3 9 3 9 5 16 3.67 11.33 C22 Sri Lanka 4 12 4 12 3 9 3.67 11.00 D23 Mexico 4 11 3 9 x x 3.50 10.00 D24 Botswana x x 3 10 x x 3.00 10.00 D25 Indonesia 3 12 2 6 4 11 3.00 9.67 D26 Estonia 3 10 2 6 4 10 3.00 8.67 D27 Kuwait 1 6 3 11 4 13 2.67 10.00 D28 Bermuda 4 10 1 7 3 10 2.67 9.00 D29 Lithuania 3 11 1 8 4 8 2.67 9.00 E30 Germany 1 4 3 9 4 11 2.67 8.00 E31 Czech Republic 3 7 3 6 2 9 2.67 7.33 E32 Bangladesh 2 7 4 9 2 5 2.67 7.00 E33 Slovakia 4 9 4 10 0 1 2.67 6.67 E34 India 4 12 2 10 1 3 2.33 8.33 E35 Malaysia x x x x 2 9 2.00 9.00 E (p.t.o…)
2009 2008 2007 Average
36 Montenegro x x 2 7 x x 2.00 7.00 F37 Romania 2 7 0 4 2 6 1.33 5.67 F38 Sweden 1 3 x x x x 1.00 3.00 F39 Thailand 1 3 x x x x 1.00 3.00 F40 Turkey 1 3 x x x x 1.00 3.00 F41 Japan 0 1 0 0 1 3 0.33 1.33 F42 Chile 0 4 x x x x 0.00 4.00 F43 Namibia 0 2 x x x x 0.00 2.00 G44 Mongolia x x 0 0 0 1 0.00 0.50 G45 Bosnia x x x x x x G or H46 Bulgaria x x x x x x G or H47 China x x x x x x G or H48 Croatia x x x x x x G or H49 Cyprus x x x x x x G or H50 Lebanon x x x x x x G or H51 Nigeria x x x x x x G or H52 Oman x x x x x x G or H53 Palestine x x x x x x G or H54 Russia x x x x x x G or H55 Sudan x x x x x x G or H56 UAE x x x x x x G or H57 Uganda x x x x x x G or H
Teams are ranked by: (1) Average wins, (2) average judges, (3) wins in 2009, (4) judges in 2009, (5) wins in 2008, (6) judges in 2008, (7) wins in 2007, (8) judges in 2007, (9) alphabetically.
Chris Erskine & Simon Quinn, 25 January 2010.
i
World Schools Debating Championships 2010
THE DRAW This document explains the structure of the draw for the 2010 World Schools Debating Championships. In past years the teams have been divided into four approximately equal groups, based on their performances (specifically, the number of debates won and number of judges won) at the previous three championships. Each team would debate two opponents from the top-ranked group, two from the second-ranked group, and so on. This year, because the number of teams is so much greater, we have divided the teams into eight groups. This ensures a more even balance: had we stayed with four groups, we would have at least 14 teams in each group, which may span a large range of ability. Instead, we have seven teams in each group, which keeps the pre-tournament results of each group reasonably uniform. Let’s begin by explaining how the draw works where each group is the same size. The closest for us is a draw of 56 teams. With 56 teams there are eight groups of seven teams. This works cleanly for every round except for the round where teams debate against a team from their own group. In that round, six of the seven teams in each group will have an opponent, but the seventh team will not. We have explored a number of options to deal with this. Few of them worked well. For example, if a team in the top-ranked group did not end up debating a team from that group, it would obviously work to their advantage because they would not have an opponent of similar standard. However, when we looked closely at the rankings, we found that there was very little to distinguish the seventh-ranked team in a group from the first-ranked team in the next group. For example, the teams ranked seventh and eighth overall in the rankings both have an average of 6.33 wins and 19 judges from the last 3 years. So, too, does the team ranked sixth, while the team ranked 9th is close behind on 6 wins. Therefore, if we ensure that the team in Pool A without an opponent from that group is the team ranked either sixth or seventh, and that the team in Pool B without an opponent from that group is the team ranked either eighth or ninth, we can arrange for those two teams to meet each other. That ensures that those two teams debate against a team of similar standard, even though the opponent is actually in the adjoining group.
ii
To do this, we must restrict the choice of teams to fill those positions. This is unfortunate, because it slightly reduces the randomness of the draw. However, we believe this is a small price to pay to ensure that the draw for the seeded teams is as fair as we can make it. And, of course, all the other positions are drawn randomly, so it is only three debates out of the entire competition where the choice of teams is restricted. Now we must deal with the fact that we don’t have 56 teams but 57. Because there is an overall odd number, one team somewhere will not debate each round. The 8 teams with byes debate each other in the bye round, or ‘round zero’ as we like to call it. But where do we put the extra team? Again, we explored a lot of options. None of them worked cleanly. For example, putting an extra team into the top group meant that in each round, one team from that group would lack an opponent. All the teams with byes would be from the top group, leading to each of them debating another team from the top group in the bye round. That made the draw for that group very much harder than it should be. Similar problems happened if we put the extra team into any of the groups with seeded teams. With the considerable number of unseeded teams this year, two groups are going to have reasonably random matchings for many of their debates, because they will be against other unranked and unknown teams. If we already have a number of random debates in these groups, it seems less problematic to add one more random debate per round to one of those two groups. For this reason, the extra team has been added to Group H. This means that each team in Group H in turn gets a bye, and this is made up by having round zero between the whole of Group H. This means that each team in Groups H will, randomly, have an extra debate against a team from Group H instead of a debate against one of the other groups. As it stands, the draw is expressed in anonymous terms: “A1”, “A2” and so forth. We will soon assign actual teams to those anonymous labels. Each group will be drawn in a different city around the world. For Groups A, C and E, the draw will begin by randomly selecting one of the sixth- and seventh-ranked teams in their group. This team will be the team that debates against the highest-drawn team from the next group (that is, the teams will be assigned to position “A7”, “C7” and “E7” respectively). For Groups B, D and F, the draw will begin by randomly selecting one of the first- and second-ranked teams in the group; these teams will be assigned to “B1”, “D1” and “F1” respectively. Having done that, they will then draw the remaining teams out of a hat to be assigned to the remaining team codes for their group in the anonymous draw. There will be 42 teams drawn for Groups A to F. There are 44 teams with pre-tournament rankings. Teams ranked 43 (Namibia) and 44 (Mongolia) will therefore be assigned to Group G; the draw for Group G will begin by randomly assigning each of those teams to a position in the Group. The remaining five teams for Group G will be drawn randomly from the 13 unranked teams. This will leave eight teams undrawn; these teams will be randomly assigned within Group H. We believe that this process produces a draw that is as even, balanced and random as possible. In particular, it ensures that the draw for the teams with rankings is even and fair according to those rankings. Any random distortions will happen in Group H, which consists of unranked teams where the debates are already random because of the unknown standard of all the teams in that group.
Chris Erskine and Simon Quinn, 25 January 2010.
GROUP A: Drawn in Dundee, 29 January
A1 IRELAND
A2 SINGAPORE
A3 SOUTH AFRICA
A4 ENGLAND
A5 AUSTRALIA
A6 NEW ZEALAND
A7 CANADA
GROUP B: Drawn in Lima, 28 January
B1 PAKISTAN
B2 SCOTLAND
B3 SLOVENIA
B4 HONG KONG
B5 KOREA
B6 ISRAEL
B7 GREECE
GROUP C: Drawn in Cape Town, 29 January
C1 NETHERLANDS
C2 PERU
C3 QATAR
C4 WALES
C5 PHILIPPINES
C6 ARGENTINA
C7 UNITED STATES
GROUP D: Drawn in Calgary, 30 January
D1 SRI LANKA
D2 ESTONIA
D3 BOTSWANA
D4 INDONESIA
D5 KUWAIT
D6 MEXICO
D7 BERMUDA
GROUP E: Drawn in Singapore, 31 January
E1 GERMANY
E2 SLOVAKIA
E3 BANGLADESH
E4 LITHUANIA
E5 INDIA
E6 CZECH REPUBLIC
E7 MALAYSIA
GROUP F: Drawn in Lahore, 28 January
F1 ROMANIA
F2 MONTENEGRO
F3 TURKEY
F4 CHILE
F5 JAPAN
F6 THAILAND
F7 SWEDEN
GROUP G: Drawn in Athens, 28 January
G1 CHINA
G2 SUDAN
G3 MONGOLIA
G4 OMAN
G5 NAMIBIA
G6 UNITED ARAB EMIRATES
G7 UGANDA
GROUP H: Drawn in Qatar, 1 February
H1 PALESTINE
H2 BULGARIA
H3 LEBANON
H4 BOSNIA AND HERZEGOVINA
H5 NIGERIA
H6 CROATIA
H7 CYPRUS
H8 RUSSIA
Chris Erskine and Simon Quinn, 1 February 2010
WSDC 2010GUIDELINES FOR ASSIGNING ADJUDICATORS
PRELIMINARY ROUNDSAll adjudicators will be assessed by the Chief Adjudicators in consultation with the Chief Adju-dicators’ Panel (‘CAP’). The assessments will be based on the experience of the adjudicators, theadjudication test and the video adjudication at the adjudicators’ briefing.
The first step in assessing adjudicators is to assess whether an adjudicator sufficiently understands:
1. the rules and principles of debate applicable to this championship,
2. what is happening in a debate by reference to those rules and principles, and
3. how to apply that understanding to the task of judging the debate.
Three classes will be formed from the group of adjudicators assessed as able to adjudicate Prelimi-nary Rounds: Level 3 (competent), Level 2 (very competent) and Level 1 (highly competent). Theassessment of all adjudicators (including shadow adjudicators) will be reviewed daily by the ChiefAdjudicators and the CAP; the assessment of any adjudicator may be changed, in either direction.
For each debate, there will be a panel of three adjudicators: one Level 1, one Level 2 and oneLevel 3. This assignment will be done randomly by computer; there will be no attempt to put“good adjudicators” on “close” or “strong” debates. Apart from the fact that defining a “close” or“strong” debate in advance is a difficult task, there is no reason why such a debate is any more orless important to the teams involved than any other debate in the Preliminary Rounds.
The random assignment will be subject to the following constraints.
1. No adjudicator will adjudicate the team from their own nation, nor a team with which theadjudicator has a significant association.
2. No adjudicator will change venues during the day.
i
3. As far as possible, no adjudicator will adjudicate the same team more than once in the pre-liminary rounds; in any event, no adjudicator will adjudicate the same team more than twicein the preliminary rounds.
4. Each panel will have at least one male and one female adjudicator.
5. There will be no more than one adjudicator from the same nation on each panel.
The computer will randomly choose either the Level 1 adjudicator or the Level 2 adjudicator to actas Chair.
The computer program will be demonstrated for anybody who wants to understand the coding.Each evening, the Chief Adjudicators will invite one tournament participant to choose a number tobe used as the ‘seed’ for the computer’s random number generator. (If the computer fails to find asuitable allocation with a given seed, the participant will be asked to choose another seed.) Anyoneis welcome to watch the program run.
FINAL ROUNDSThe assignment of adjudicators to Final Rounds will be done by the Chief Adjudicators in consul-tation with the CAP. The first step will be to select, from the pool of all adjudicators assessed asable to adjudicate the Preliminary Rounds, those adjudicators who are the most competent. Thisselection may include adjudicators who are new to judging at the championship this year. Theassessment of all adjudicators will be made primarily by reference to how they have adjudicated inthe previous debates in this championship.
From this pool, the Chief Adjudicators will select at least enough adjudicators to fill the panels forthe Final Rounds. However, the exact size of the pool will be determined by assessed competence,which means that the pool may be larger than the number of adjudicators required for any individ-ual Final Round. Based on the calibre of the adjudicators attending this championship, there is avery high expectation that the pool selected for the Final Rounds will be from diverse backgrounds.If the pool is not reasonably diverse, the Chief Adjudicators will review the pool to make sure thatother suitable adjudicators have not been inadvertently overlooked.
The panels for the Final Rounds will be:
• Octo-Finals: Five adjudicators;
• Quarter-Finals: Seven adjudicators;
• Semi-Finals: Nine adjudicators;
• Grand Final: Eleven adjudicators.
In assigning adjudicators to Final Round debates, the Chief Adjudicators will apply the followingcriteria (unless, except for the first criterion, it is actually impossible to apply them for a particulardebate).
ii
1. No adjudicator will adjudicate the team from his or her own nation, or any other team withwhich the adjudicator has a significant association.
2. In the Octo-Finals, Quarter-Finals and Semi-Finals, no adjudicator will adjudicate the sameteam more than once. However, an adjudicator may adjudicate the Grand Final if that personhas already adjudicated one of the teams in the Final Rounds. An adjudicator may, in anyof the Final Rounds, adjudicate a team that the person has adjudicated in the PreliminaryRounds.
3. Nobody will not adjudicate either Semi-Final if his or her team — or a team from any othercountry with which that adjudicator has a significant association — is in either Semi-Final.
4. No panel will have more than one adjudicator from any given nation.
The Chief Adjudicators will review any panel to see if it is reasonably diverse. If it is not, the com-position of the panel will be reconsidered to make sure that it is the best panel that can honestly andreasonably be provided for that debate, and that other suitable adjudicators have not inadvertentlybeen overlooked.
Recognising that the Chief Adjudicators and the CAP are in the invidious position of having toassess themselves for assignment to panels, there is no assumption that membership of the CAPentitles a adjudicator to adjudicator any Final Round debate. It is entirely possible that the GrandFinal, for example, might not include either Chief Adjudicator or any members of the CAP.
The Chief Adjudicators and the Chief Adjudicators’ Panel
iii