petrie 2013 exam teaching support

7/29/2019 Petrie 2013 Exam Teaching Support

1/131

EXAM TEACHING SUPPORT:

AN

INVESTIGATION INTO THE

FORMS OF

PROFESSIONAL

SUPPORT

AVAILABLE TO TEACHERS OF ESOLEXAM PREPARATION CLASSES.

DAVID MARTIN ROBERT PETRIE

MAAPPLIED LINGUISTICS AND TESOL

SCHOOL OF LANGUAGES AND AREA STUDIES

PORTSMOUTH UNIVERSITY

JANUARY 2013


2/131

i | P a g e

ABSTRACT:

The professional context of helping teachers prepare learners effectively for

English language exams led to the beliefs that exam class teachers do not have

all the information they need and are not provided with sufficient support. This

study investigates these beliefs by examining the research on exam class

teaching, the characteristics of and approaches to exam class teaching and

forms of support for exam class teaching. Five research questions are

formulated asking whether teachers need support, whether teachers feel as

though they need support, what forms of support teachers use, how teachers

feel about these forms of support and how they would like to see such support

improved in the future.

A descriptive survey was therefore designed with content arising from the

review of exam class teaching literature. The questionnaire was distributed to a

random volunteer sample via the internet and seventy-eight responses were

collated. The survey reveals that teachers do feel as though they need a

degree of support with their exam class teaching but did not uncover any clear

evidence showing teachers did need support with their teaching; though as

some teaching practices identified contradict the evidence from washback

studies, it is likely. It also found that teachers rely predominantly on support

resources that may be found in their staffrooms; namely course books,

supplementary books and the ideas of their colleagues; and that they also rate

these forms of support highly. Finally the survey found that teachers want more

support in the form of practical teaching ideas and materials that they can use

with their exam preparation classes.

This study concludes that the initial beliefs on the nature of exam preparation

class teachers are partially correct and that while there is no evidence to

suggest teachers lack information, there are clear indications that teachers lack

the support they want and that this is something which would be best rectified in

the future.


3/131

ii | P a g e

TABLEOFCONTENTS:

ABSTRACT: ........................................................................................................ I

TABLE OF CONTENTS: .................................................................................... II

LIST OF TABLES AND FIGURES ................................................................... VI

PREFACE AND ACKNOWLEDGEMENTS:.................................................... VII

GLOSSARY OF ACRONYMS: ....................................................................... VIII

CHAPTER ONE: THE RATIONALE AND CONTEXT FOR THE STUDY ......... 1

1.1THE RESEARCHERS PERSONAL CONTEXT ....................................................... 1

1.2A BELIEF ABOUT TEACHERS KNOWLEDGE OF EXAMS ........................................ 2

1.3THE LITERATURE ON TEACHING EXAM PREPARATION CLASSES .......................... 3

1.4A BELIEF ABOUT SUPPORT FOR TEACHERS OF EXAM PREPARATION CLASSES...... 5

1.5CONSTRAINTS OF THE DISSERTATION ............................................................. 6

1.6SUMMARY OF CHAPTER ONE......................................................................... 7

CHAPTER TWO: AN OVERVIEW OF EXAM CLASS TEACHING ................... 8

2.1A DEFINITION OF EXAM CLASS TEACHING......................................................... 8

2.2THE CHARACTERISTICS OF EXAM CLASS TEACHING.......................................... 9

2.3APPROACHES TO EXAM CLASS TEACHING. ..................................................... 11

2.3.1 Constituents of a Methodological Approach ....................................... 11

2.3.2 Overall approaches to exam preparation classes .............................. 13

2.4EFFECTIVENESS OF EXAM PREPARATION CLASSES......................................... 16

2.5FORMS OF SUPPORT FOR EXAM CLASS TEACHING ......................................... 20

2.6SUMMARY OF CHAPTER TWO ...................................................................... 22

CHAPTER THREE: RESEARCH METHODOLOGY ....................................... 24

3.1RESEARCH MATERIALS AND PROCEDURES ................................................... 24

3.2RESEARCHAIMS ........................................................................................ 25

3.3RESEARCH METHODOLOGY......................................................................... 25

3.3.1 Define objectives ................................................................................ 26


4/131

iii | P a g e

3.3.2 Identify target population .................................................................... 26

3.3.3 Literature review ................................................................................ 26

3.3.4 Determine sample .............................................................................. 26

3.3.5 Identify survey instruments ................................................................ 27

3.3.6 Design survey procedures ................................................................. 27

3.3.7 Identify analytical procedures ............................................................. 28

3.3.8 Determine reporting procedures ........................................................ 28

3.4CONDUCTING A PILOT STUDY ...................................................................... 29

3.5RESEARCH DESIGN .................................................................................... 31

3.5.1 Question Types and Technological Affordances ................................ 31

3.5.2 Design of Section 1: Professional Biography .................................... 33

3.5.3 Design of Section 2: Exam Class Teaching ....................................... 33

3.5.3 Design of Section 3: Professional Support ......................................... 34

3.6RESEARCH PARTICIPANTS........................................................................... 36

3.7SUMMARY OF CHAPTER THREE ................................................................... 38

CHAPTER FOUR: PRESENTATION OF RESULTS ....................................... 40

4.1 SURVEY RESULTS PART TWOEXAM CLASS TEACHING .............................. 40

4.1.1 Survey Question 7: Which exams have you prepared learners for? . 40

4.1.2 Survey Question 8: How confident are you in your ability to preparelearners for these exams?........................................................................... 41

4.1.3 Survey Question 9: Which of the following aspects do you include inyour exam preparation classes? ................................................................. 43

4.1.4 Survey Question 10: What proportion of class time do you spend onthose aspects identified in the last question? .............................................. 44

4.1.5 Survey Question 11: Do your current exam preparation courses use abook? .......................................................................................................... 45

4.1.6 Survey Question 12: Do you like teaching exam preparation classes?.................................................................................................................... 45

4.1.7 Survey Question 13: Why? / Why not? ........................................... 46

4.2SURVEY RESULTS PART THREE: PROFESSIONAL SUPPORT........................... 47


5/131

iv | P a g e

4.2.1 Survey Question 14: Which of the following forms of support do youuse now or have you used in the past? ....................................................... 48

4.2.2 Survey Question 15: Given the opportunity, which of the followingforms of support do you prefer to use? ....................................................... 49

4.2.3 Survey Question 16: How frequently do you refer to or use theseforms of support? ........................................................................................ 50

4.2.4 Survey Question 17: How would you rate the forms of support thatyou use or have used in the past? .............................................................. 51

4.2.5 Survey Question 18: Where you have indicated a resource as beingsatisfactory or poor, please could you indicate how you would like to see itimproved? ................................................................................................... 53

4.2.6 Survey Question 19: Is there any additional support you feel you needto improve your exam class teaching? ........................................................ 54

4.2.7 Survey Question 20: If there was one thing that you could changeabout your exam class teaching what would it be? ..................................... 55

4.3CHAPTER FOUR SUMMARY .......................................................................... 56

CHAPTER FIVE: DISCUSSION OF FINDINGS .............................................. 59

5.1RESEARCH QUESTIONA:DO TEACHERS FEEL AS THOUGH THEY NEED SUPPORTIN THEIR EXAM CLASS TEACHING? ...................................................................... 59

5.2RESEARCH QUESTION B: DO TEACHERS NEED SUPPORT IN THEIR EXAM CLASSTEACHING? ...................................................................................................... 61

5.3RESEARCH QUESTION C: WHAT FORMS OF SUPPORT ARE USED BY TEACHERSOF EXAM CLASSES? .......................................................................................... 63

5.4RESEARCH QUESTION D: WHAT ARE THEIR FEELINGS TOWARDS THESE FORMSOF SUPPORT? .................................................................................................. 64

5.5 RESEARCH QUESTION E: WHAT CHANGES TO THIS SUPPORT OR WHATADDITIONAL SUPPORT WOULD TEACHERS OF EXAM PREPARATION CLASSES LIKE TO

SEE IN THE FUTURE?......................................................................................... 66

5.6SUMMARY OF CHAPTER FIVE....................................................................... 68

CHAPTER SIX: CONCLUDING REMARKS ................................................... 70

WORD COUNT ................................................................................................. 72

BIBLIOGRAPHY .............................................................................................. 74

DECLARATION ................................................................................................ 78

APPENDIX 1 COLLATED SCREENSHOTS OF THE ONLINEQUESTIONNAIRE: ........................................................................................... 79


6/131

v | P a g e

APPENDIX 2A RAW DATA FROM THE QUESTIONNAIRE RESPONSES(CLOSED CLASS QUESTIONS) ..................................................................... 86

APPENDIX 2B RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 10............................................................................. 96

APPENDIX 2C

RAW DATA FROM THE

QUESTIONNAIRE RESPONSESTO SURVEY QUESTIONS 12 & 13. ............................................................... 104

APPENDIX 2D RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 18........................................................................... 112

APPENDIX 2E RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 19........................................................................... 115

APPENDIX 2F RAW DATA FROM THE QUESTIONNAIRE RESPONSESTO SURVEY QUESTION 20........................................................................... 118


7/131

vi | P a g e

LISTOFTABLESANDFIGURES

FIGURE 1:THE INTER-RELATIONSHIP OFASPECTS OF EXAM CLASS TEACHING (AFTERSPRATT 2005) _______________________________________________ 18

TABLE 1:A SUMMARY OF WAYS OF CONTINUING PROFESSIONAL DEVELOPMENT,

AFTER JOHNSON (2009) AND WILDEN (2012) _________________________ 20

TABLE 2:FORMS OF EXAM TEACHING SUPPORT __________________________ 22

TABLE 3: COMPARISON OF QUESTION TYPES AVAILABLE IN GOOGLE DOCS TOQUESTION TYPES GIVEN IN NUNAN (1992, P.144) ______________________ 32

TABLE 4:SURVEY PARTICIPANTS EXPERIENCE AND HIGHEST LEVEL OF QUALIFICATION__________________________________________________________ 37

TABLE 5: ELT QUALIFICATIONS IN THE QUALIFICATIONS AND CREDIT FRAMEWORK /NATIONAL QUALIFICATIONS FRAMEWORK FOR ENGLAND AND NORTHERN IRELAND,ADAPTED FROM OFQUAL (2011). _________________________________ 37

FIGURE 2:COUNT OF RESPONSES FOR EXAMS TAUGHT BY RESPONDENTS________ 40

TABLE 6:OTHER EXAM TYPES INCLUDED BY RESPONDENTS TO QUESTION 7 ______ 41

TABLE 7:PERCENTAGE OF RESPONSES INDICATING THE LEVEL OF SUPPORT NEEDED BYEXAM PREPARATION CLASS TEACHERS FOR A RANGE OF ESOL EXAMS._______ 42

FIGURE 3:DEGREE OF SUPPORT REQUIRED BY RESPONDENTS FOR DIFFERENT EXAMS__________________________________________________________ 43

FIGURE 4:NUMBER OF RESPONDENTS INCORPORATING STATED ASPECTS INTO THEIREXAM CLASS TEACHING _________________________________________ 44

FIGURE 5:NUMBER OF RESPONDENTS WHO USE A COURSE BOOK WITH THEIR EXAMPREPARATION CLASSES ________________________________________ 45

FIGURE 6:RESPONSES TO THE QUESTION "DO YOU LIKE TEACHING EXAM PREPARATIONCLASSES?" AND THE PERCENTAGE OF RESPONDENTS WHO GAVE THAT ANSWER 46

FIGURE 7:FORMS OF SUPPORT USED BY SURVEY RESPONDENTS ______________ 48

FIGURE 8:PREFERENCES IN FORMS OF SUPPORT EXPRESSED BY RESPONDENTS___ 49

FIGURE 9:A COMPARISON OF FREQUENCY OF USE FOR FORMS OF SUPPORT ______ 50

FIGURE 10:OVERALL RESPONDENT SATISFACTION WITH FORMS OF SUPPORT _____ 52

FIGURE 11: LEVELS OF SUPPORT REQUIRED FOR ESOL EXAMS CONTRASTED WITHLEVEL OF EXPERIENCE OF ESOL EXAMS_____________________________ 59

FIGURE 12:COMPARISON OF FREQUENCY OF USE WITH COUNT OF TEACHERS WHO USEFORMS OF SUPPORT ___________________________________________ 64


8/131

vii | P a g e

PREFACEANDACKNOWLEDGEMENTS:

I would like to thank John Naysmith and Peter Watkins for their help, their

support and their insightful comments on various versions of this manuscript,

especially for pointing out the obvious when the obvious wasnt so clear to me.

Thanks are also due to the anonymous participants who took part in the survey

and to my friends and colleagues at International House Coimbra and

International House Santa Clara who piloted the study.

Thanks must also go to my wife, Alexis, who not only helped provide a modicum

of sanity in stressful times, but balanced this with writing her own dissertation

and juggling childcare duties. Without your help, none of this would have

happened.

For Rosie and Sam, who would doubtless prefer it if it had dragons in it.


9/131

viii | P a g e

GLOSSARY OF ACRONYMS:

This study uses a number of terms and acronyms, explanations of which aregiven here:

ALTE Association of Language Testers in Europe.

CAE

Certificate in Advanced English. CEFR C1 levelexamination awarded by University of Cambridge ESOLExaminations and also referred to as Cambridge English:Advanced.

CambridgeESOL

The providers and awarding body of a range of ESOL

examinations, including FCE, CAE and CPE, and part of theconsortium that provides and awards the IELTS exam.Cambridge ESOL is the corporate identity adopted by theUniversity of Cambridge ESOL Examinations, which wasalso previously known as UCLES (see below).

CEFR Common European Framework of Reference.

CPD Continuing Professional Development

CPECertificate of Proficiency in English. CEFR C2 levelexamination awarded by University of Cambridge ESOLExaminations and also referred to as Cambridge English:Proficiency

EAP English for Academic Purposes.

ELT English Language Teaching

ESOL English for Speakers of Other Languages

ESP English for Specific Purposes

ETPEnglish Teaching Professional. A professional pedagogicalmagazine for language teachers.

ETSEducational Testing Services, the providers and awardingbody for the TOEFL iBT (see below)

FCEFirst Certificate in English. CEFR B2 level examinationawarded by University of Cambridge ESOL Examinations

and also referred to as Cambridge English: First.

HLTM Humanising Language Teaching Magazine. A professional


10/131

ix | P a g e

pedagogical magazine for language teachers.

IELTS

International English Language Testing System, the namegiven to both the exam and the organisation that providesand awards the exam, the IELTS organisation is apartnership between British Council, IDP: IELTS Australia

and Cambridge ESOL.

LCCIEB

London Chamber of Commerce & Industry ExaminationsBoard. An examinations provider and awarding body. In2002, LCCIEB merged with GOAL plc to create EDI(Education Development International), which waspurchased by Pearson Education Ltd in 2011.

METModern English Teacher. A professional pedagogicalmagazine for language teachers.

TOEFL iBT Test of English as a Foreign Language internet Based Test.

UCLESUniversity of Cambridge Local Examinations Syndicate. SeeCambridge ESOL.

UODLEUniversity of Oxford Delegacy of Local Examinations. Anexaminations provider and awarding body. No longercurrent, it merged with UCLES (see above) in 1995.


11/131

1 | P a g e

CHAPTER ONE: THE RATIONALE AND CONTEXT FOR THE STUDY

This chapter details the professional context and personal experiences that

gave rise to this investigation and describes the rationale that underpins it,

articulating beliefs that guide the exploration of the literature in this area.

1.1THE RESEARCHERS PERSONAL CONTEXT

ESOL Exam preparation classes form the core responsibilities of my current

professional context: helping learners prepare, helping teachers of exam

preparation classes and acting as a school source of ESOL exam related

information and advice.

As a result, I have become interested in the way that teachers prepare their

learners for ESOL exams; in the content and delivery of exam preparation

lessons, the beliefs and attitudes that learners and teachers have towards their

classes, and whether or not what goes on in the classroom can be improved

upon in order to maximize learner exam performance.

Having worked in Asia and Europe, I found that learners in different ESOL

markets often request differing exam types. For example, in China learners

often request IELTS or TOEFL, whereas in Poland and Portugal learners

typically request FCE, CAE and CPE. Higher education institutions usually

require overseas students to demonstrate their ability with English by requesting

specific IELTS or TOEFL scores, for example University of Leicester (n.d.) or

University of Southampton (n.d.), and these requirements influence the exam

choices of the learners. Cambridge ESOL state that their exams are officially


12/131

2 | P a g e

accepted by thousands of universities, employers and governments

(Cambridge ESOL n.d.) and so other stakeholders may also influence these

choices.

Teachers who work in these differing markets often become specialists in

particular exams and lack knowledge of others. This may mean that teachers

are not in a position to offer learners a full overview of all the exam possibilities.

There is also a question of whether language schools drive demand for specific

language exams or whether they react to it, but the tendency to focus on

specific exams means a cyclical process then arises, where lack of knowledge

or information limits choices, which in turn limits the commercial pressures to

provide alternatives, resulting in the specialization of education professionals in

a particular area, reducing the incentive to seek knowledge of alternatives,

resulting in lack of knowledge or information and so on.

1.2A BELIEF ABOUT TEACHERS KNOWLEDGE OF EXAMS

May (1996, p. 159-163) identifies sixteen specific examinations and awarding

bodies, while Burgess & Head (2005, p. 151-153) list fourteen awarding bodies

and a sample range of thirty-five representative exams from those bodies.

europa pages give the four main UK-based awarding bodies as being the

London Chamber of Commerce (LCCIEB), University of Cambridge (UCLES),

Pitman Qualifications, and the University of Oxford (UODLE), providing 47

different exams between them (europa pages, 2011), though as UODLE

merged with UCLES in 1995 (Hawkey, 2009, p. 133) this information may now

be out of date.


13/131

3 | P a g e

English language exams are available for a wide range of ages, abilities and

purposes. With such a wide range, it seems likely that while teachers may have

experience of one exam they will also lack experience of others. This

specialization may not only affect teachers ability to advise on exam choices

effectively, but it may also affect their teaching of exam preparation classes as

they may attempt to transfer techniques and exam preparation strategies from

one exam to another without full knowledge of differences between the target

exams. In his study on the impact of FCE on the EFL classroom, Perrone

(2010), while not setting out to explore this effect directly, does show a degree

of transfer between an FCE preparation class and a general English class. If it

is possible for transfer to exist across categories of teaching, it seems likely that

transfer takes place across exams.

This has then led me to the belief that: Teachers of exam preparation classes

do not always have all the information they need to know in order to be effective

exam class teachers.

1.3THE LITERATURE ON TEACHING EXAM PREPARATION CLASSES

The specialisation of teachers is reflected in the academic literature, with

research that focuses either on specific exams (e.g. Alderson 2009, Hall 2010,

Hawkey 2009) or provides an in-depth focus on aspects of these exams (e.g.

Carrell 2007, Macqueen & Harding 2009, Uysal 2010). Researchers also

approach the area of ESOL exams from different perspectives, for example

investigating validity issues with an exam or considering washback or impact of

an exam on teaching practice (see also section 2.4), or from the perspective of

a specific skill. This specialisation may reflect the sponsorship of researchers


14/131

4 | P a g e

by the exam awarding bodies (Hawkey 2004, Carrell 2007) and the

requirements of the research and development programs of the sponsors. One

outcome of the scope of research relating to language testing and exams is that

information is published in a wide range of sources, some of which are

subscription based academic journals, making it difficult for teachers to access.

More easily accessible to teachers might be the pedagogical journals and

magazines, to which some language schools subscribe and some of which host

their content online. However here there is remarkably little content that deals

specifically with exam classes. Contributors to these journals tend to paint with

a broader brush, for example in The Secret of Reading Prowse (1999)

examines the evidence supporting a reading for pleasure approach.

Gradman and Hanania report that extensive reading was a strongpredictor of TOEFL scores. This is something that teachers preparingstudents for FCE and CPE have always known intuitively, but it is nice to

see it proved through research.

(ibid, p10)

Whether Prowses faith in teacher intuition is justified is debatable (see section

2.4) and although the focus of the article is the value of extensive reading

programs and is aimed at a more general readership, it is a concern to see

something that could have wide implications for exam class teaching relegated

to a single sentence. While some articles do focus on teaching exam classes,

for example Dowlings IELTS Reading without tears (2011), they remain a

minority. There seems to be an assumption that teachers will automatically

apply what they have seen in one aspect of their teaching to another this is

not unreasonable to expect but some research findings (e.g. Wall & Hork

2006) do not bear this out.


15/131

5 | P a g e

1.4A BELIEF ABOUT SUPPORT FOR TEACHERS OF EXAM PREPARATION

CLASSES

The situation of teacher training and development in relation to exam class

teaching is also poor. A review of the online archive of the Teacher Trainer

Journal (n.d.) found nothing that mentioned teaching exam classes. There are

however, currently two books which attempt to influence exam preparation

teaching methodology: May (1996) Exam Classes and Burgess & Head

(2005) how to Teach for Exams. Both provide useful perspectives on

preparing learners for aspects of exams, though the breadth and depth of their

coverage varies.

The counterpoint to this is the wealth of publications available to teachers and

learners from ESOL publishers. These typically focus on one exam in particular

and fall broadly into two categories: the course book and the practice test book.

The former often offers teachers and learners material designed to aid linguistic

development and skills development as part of a broader course, while the latter

generally offers only sample exam tasks and answer feedback, possibly with

some additional task analysis. The inference is that in learning about the

sample tasks given and why the specific answers are correct, the learners will

be able to draw wider conclusions about the target exam in general. There is

evidence to suggest otherwise, as examined in Chapter 2. It could be argued

that if the course book or the practice test book forms the locus of an exam

preparation class, it may also become the primary reference document in terms

of exam information and strategies for approaching exam tasks. This would

mean that any beliefs on the nature of language and on the learning process

held by course book stakeholders form the basis of the exam preparation


16/131

6 | P a g e

course. Depending on what those beliefs are, this is not necessarily a problem.

Nevertheless, it would require the teacher to be able to make an effective

assessment of the strengths and weaknesses of the course book for their

teaching context and concurrently to have the pedagogical tools available to

do so.

The current depiction of support for exam class teaching has therefore led me

to a second guiding belief: The support available to teachers of exam

preparation classes is not sufficient to their needs.

1.5CONSTRAINTS OF THE DISSERTATION

It is not possible within the constraints of this dissertation to investigate these

beliefs with reference to all of the many English language exams available and

the analysis will therefore be confined to discussing three English language

exams: FCE, TOEFL iBT and IELTS.

These have been selected as they are widely considered to be the most popular

with language learners. It is difficult to determine the truth of this as the relevant

awarding bodies seem reluctant to publish candidate numbers. IELTS states

over 1.7 million tests were administered in 2011 (IELTS, n.d.); Cambridge

ESOL state that over 3.25 million people take their tests annually though this

does not state which tests were taken (University of Cambridge ESOL

Examinations, 2011). The Educational Testing Service website states over 27

million people have taken the TOEFL test (2012), though again it does not state

which versions of the test were taken nor does it state over which time period.


17/131

7 | P a g e

1.6SUMMARY OF CHAPTER ONE

This chapter has set out the professional context and experiences that underpin

this investigation and has articulated two key beliefs that will guide the further

exploration of the subject area:

Teachers of exam preparation classes do not always have all the information

they need to know in order to be effective exam class teachers.

The support available to teachers of exam preparation classes is not sufficient

to their needs.

To investigate these beliefs more fully, chapter two of this dissertation will

provide an overview of exam class teaching, the key characteristics and

approaches identified to teaching exam classes. It will further examine the

research on the effectiveness of exam class teaching and discuss forms of

support available to exam class teachers identified from the literature.

Chapter three then re-articulates the beliefs stated above as research questions

that can be investigated with the design of an online questionnaire. It further

describes the research methodology adopted, after Nunans eight step process

(1992, p. 141), describes the conduct of a pilot study and discusses the biases

apparent in the research participants who were obtained as a random volunteer

sample.

Chapter four presents the results of the survey, while chapter five discusses

these results in the context of the five research questions outlined in section 3.2

and considers the implications of these findings and draws conclusions as to

the sufficiency of information and support for teachers of exam preparation

classes and discusses how that support might best be rendered in the future.


18/131

8 | P a g e

CHAPTER TWO: AN OVERVIEW OF EXAM CLASS TEACHING

This chapter provides a definition of exam class teaching, describes its key

characteristics and approaches to exam class teaching identified in the

literature. It further examines the effectiveness of exam class teaching from the

perspective of washback and impact studies that contrast learner performance

in exam classes with alternative instruction modes. Finally it provides a list of

forms of support for exam class teachers derived from an examination of forms

of continuing professional development found in the literature.

2.1A DEFINITION OF EXAM CLASS TEACHING

It is difficult to characterise exam class teaching in an ELT context except in

opposition to the more common term General English, though this term is also

problematic as while it is a term in common currency, it is not one for which a

definition is readily available. Definitions can be found for Business English

(BE) (Ellis and Johnson, 1994), English for Academic Purposes (EAP)

(Hamp-Lyons, 2001), and English for Specific Purposes (ESP) (Dudley-Evans,

2001). There are also many labels and acronyms relating to the nature of

English, for example English as an International Language (EIL) or English as a

Lingua Franca (ELF), and a useful overview of these is provided by Erling

(2005), who also mentionsAhulus (1997) use of General English to describe

a global variety of English as opposed to British and American standard models,

though general English is not used in this way within this study.

Whether it has been clearly defined as so or not, general English has taken on

a meaning of English for everything else. The general English classroom


19/131

9 | P a g e

recognises that learners enter with individual goals and experiences and

abilities with the language (Richards 2001) and tries to take all these into

account, but aims more broadly, to improve the language knowledge and

language skills generally.

Exam class teaching borrows from this but works towards a different purpose.

The activities and teaching techniques that work in general teaching will help

make a successful exams teacher (Burgess and Head, 2005, p. 1) but it is also

necessary to drive students towards the goal of passing their exams (ibid).

Exam class teaching is constrained by the target exam; the syllabus is defined

by it, as are the materials (ibid). The exam class teacher therefore needs to

know as much as possible about the format and content of the exam (ibid, p.

2) and to believe that exam preparation is a worthwhile exercise (ibid, p. 3).

2.2THE CHARACTERISTICS OF EXAM CLASS TEACHINGBurgess and Head (2005) describe a successful exams teacher as having

effective time management skills, a positive view of exams, extensive

knowledge of the target exam, giving pertinent and effective feedback, listening

and responding to learner worries and motivating learners (2005, p. 3). They

also discuss approaches to teaching reading, writing, grammar and vocabulary,

listening and speaking for exams. May (1996) structures his book Exam

Classes in a similar fashion though provides a range of practical activities

rather than discuss the components or characteristics of exam class teaching.

He does state that Exam techniques have to be built on a solid foundation of

language skills. Students cannot handle a difficult exam course without a


20/131

10 | P a g e

thorough grounding in reading, writing, listening and speaking. (1996, p. 5),

which is a useful, but limited viewpoint.

There are similarities between these features of exam classes and aspects

identified by Zabawa (2001) and Wilson (2010) as desirable components of

exam preparation class course books. Though these are not based on

evidence from empirical research and are not designed with the specific aim of

suggesting desirable components of exam classes, it could be argued that if

these criteria are desirable in course books, they are desirable in courses and

consequently in teaching.

The justification and rationale that Zabawa uses for the inclusion of his criteria is

that the FCE textbook must be as similar as possible to the FCE examination

format (2001, p. 162). He extends this to discuss teaching language skills and

states that where sub-skills can be identified as being tested in the exam, these

should be focused on in the text book and by extension the course (2001, p.

163-165).

Wilson describes the exam, candidate needs and problems and general

implications for teachers before arriving at her set of criteria (2010, p. 221),

though she does not describe her information sources or how it was developed.

Wilson draws two main conclusions; firstly that the balance between underlying

skills or language work and exam strategies and practice varies significantly in

exam class teaching and teachers therefore need to consider student needs

during the delivery of a course. Secondly, that many course books assume a

lockstep approach is being taken to teaching, which may not be desirable

(2010, p. 231-232).


21/131

11 | P a g e

From May (1996), Zabawa (2001), Burgess & Head (2005) and Wilson (2010),

the following key characteristics of exam class teaching can be determined:

exam task training / test taking techniques language development (grammar and vocabulary)

listening development

reading development

speaking development

test / exam practice

writing development

As cited from Burgess & Head (2005) previously, many of these aspects bear a

strong resemblance to the content of a general English course, though with an

exam class the focus of the development must be the likely content of the

exam. There is little point in developing the exam class students ability to write

a short story if the primary test task takes the form of a transactional letter.

2.3APPROACHES TO EXAM CLASS TEACHING.

2.3.1CONSTITUENTS OF AMETHODOLOGICALAPPROACH

It is not until the goals, objectives, and content of a language programhave been determined that decisions about methodology can be takenup in detail.

(Richards, 1990, p. 11)

The test is the curriculum.

(unnamed teacher quoted in Wall & Horak, 2006, p. 119)

It is perhaps unfair to suggest that all exam focused classes echo the sentiment

of the teacher quoted above, nevertheless this study has suggested that when

the goals and objectives of an exam course are discussed, those that feature


22/131

12 | P a g e

most prominently are likely to be those featured in the target exam and that as

such, the exam is likely to determine the content also. The relationship

between an exam and the pedagogical events surrounding it is often described

as washback, which is more fully investigated in in section 2.4.

Richards describes methodology as the activities, tasks and learning

experiences selected by the teacher in order to achieve learning, and how these

are used within the teaching /learning process. (1990, p. 11). A methodology

reflects, he suggests, beliefs and attitudes towards language, learning,

participant roles and materials.

To an extent, the beliefs about language are also determined by the exam.

Richards gives, as an example of how to help determine these beliefs, the

analytical questions is reading viewed as a primarily top -down or bottom-

up process ? What does listening comprehension involve and what skills

does it depend on? (1990, p. 12). In the exam class it is the exam designers

whose beliefs and attitudes to language are paramount as these are

incorporated into the test. In this aspect of washback therefore, the belief about

what listening comprehension involves forms the basis of what aspects of

listening comprehension are tested. There is however, the potential for conflict

between the beliefs embedded into the target exams and the beliefs held by

teachers, learners and organisations.

If exam designers hold beliefs about learning, these are not apparent from the

exams. Rather the exams set the end goal or destination and teachers and

learners are free to decide how best to travel. Consequently startlingly different

methods might describe very similar levels of effectiveness. The Callan School,


23/131

13 | P a g e

whose method is based on question, answer, repetition (Callan Method

Organisation, n.d.) claims that by the end of their learning process the student

will have mastered the 5621 most common words of the English language. The

student will now be thoroughly prepared for the Cambridge First Certificate

(F.C.E.) exam (ibid). Chia Suan Chong describes adopting a Dogme approach

with her IELTS class So my one-month IELTS class went the Dogme way and

feedback from my students was overwhelmingly positive. A couple said they

had never learnt so much in a month before. (Chong, 2011).

While it is likely that methodologies or approaches that share beliefs with those

held by exam designers and producers are liable to be more effective in exam

preparation classes than those which do not, it is worth returning to Richards for

a moment to remember that other variables may be at play: Studies of the

effectiveness of specific methods have had a hard time demonstrating that the

method itself, rather than other factors, such as the teachers enthusiasm or the

novelty of the new method, was the crucial variable (1990, p. 36).

The problems in identifying successful teaching practice, with the large number

of variables involved, may account for the relative paucity of articles related to

successful exam class teaching. It is easy to describe teaching practice and

reactions to teaching practice, as with Chong (2011), but it is not so easy to

claim that teaching practice has directly led to successful exam performance.

2.3.2OVERALL APPROACHES TO EXAM PREPARATION CLASSES

Senior provides a very general overview of the constituent parts of exam-driven

teaching (2010): confidence building, skills development, exam strategies and

test practice and provides examples of each. She also outlines some of the

problematic areas, that attempting to teach English largely through tests (ibid)


24/131

14 | P a g e

seldom works well, that teaching to the test in terms of selecting course content

can be limiting and possibly impeding, and that it denies teachers the

opportunity to teach in dynamic and engaging ways (ibid).

Teaching for the test is also an issue that Thompson examines, though his

analysis is kinder as he suggests that when the test and (course) objectives

are in harmony, the effect can be very positive (2001, p. 11). His view, similar

to Zabawa (2001), is that the proper incorporation of the test into the course

curriculum is of prime importance and that tests might lead learners to focus on

areas they might otherwise neglect, for example essay structures. The pitfalls

are those of negative washback, he points out, as including a test either as an

end of course assessment technique or as a course goal can lead learners to

neglect or ignore aspects of their language development that are not included in

the test.

De Hinojosa (2011) takes a more practical and more detailed approach in

Humanising Language Teaching magazine (HLTM) that appears to be based on

her own experience working in a specific context and aimed more at novice

teachers working without the support of colleagues or their institutions. De

Hinojosas suggestions involve making sure classes are relatively homogenous

in their language ability, involving and communicating with parents, giving

homework daily, providing students with exam tips (a broad sample of which are

listed) and testing or assessing the students. De Hinojosas article suffers

slightly from not describing her context, which might provide some

understanding as to why she has made these choices or recommends this

approach. It also ignores alternative contexts in which other stakeholders may

have input on some of the processes she describes, for example the degree of


25/131

15 | P a g e

contact with parents. The overall strategy would appear to be that of managing

expectations of stakeholders and in maximising test performance potential.

Development of learner ability is not discussed and the inference is that by

preparing learners for the exam development of ability will occur as a by-

product, or as positive washback. There is some debate on this point as will be

discussed in section 2.4.

An alternative approach is offered by Case, also in HLTM, who offers

suggestions on how to humanise your exam classes (2002). His key points

seem sensible; personalisation of content to the learners, build group rapport

and encourage peer collaboration, keeping content and activities relevant to the

exam. Case argues that by asking learners to purchase a self-study book and

work individually at home, language work conducted in class is more reactive to

learner needs. This might suggest a belief that prescribing the language

content of a course is inappropriate and that individualised tuition based on

learner needs is more important. Or it could reflect a belief as given in Burgess

and Head (2005, p. 64) that an exam preparation course is not a suitable

vehicle for language development, but should instead consolidate existing

knowledge.

Case, De Hinojosa, Thompson and Senior provide the only attempts at

describing an overall approach to successful exam class practice apparent in

the pedagogical literature, comprising as it does the small number of journals

aimed at helping teachers improve teaching practice rather than the more

detailed investigations of teaching, learning and language that can be found in

the academic literature. In brief, the Thompson and Senior articles provide not

so much an approach to teaching exam classes but more a list of pros and cons


26/131

16 | P a g e

for courses where the test provides the main impetus. These are perhaps

considerations rather than prescriptions. Case similarly makes limited

suggestions for adapting existing procedure rather than a process in itself.

Where De Hinojosa attempts a more detailed description she succeeds in

describing aspects of exam class preparation that seems to lack the

consideration that Thompson and Senior urge.

2.4EFFECTIVENESS OF EXAM PREPARATION CLASSES

Contributions to academic research and understanding of the effects of

language tests and exams upon their wider environments, processes and

participants often do so by examining either impact or washback. Taylor (2005)

defines washback as the way a test affects classroom events in the time period

prior to the test being taken. Washback can be viewed either as positive

(having a beneficial effect on course content and conduct) or negative

(constraining teaching and learning) (ibid). Where washback relates to events

leading up to the test, impact describes the events that come afterwards, or the

consequences of tests (ibid, 154). Impact tends to be applied to a larger

context than the classroom; Taylor (ibid) sees it as the effect of the test on

individual life prospects, schools, educational systems and society as a whole.

While she points out there is some disparity of use of these terms amongst

researchers, this study follows the definitions given above.

Various studies have attempted to examine whether test preparation classes

lead to improvements in test scores: Hayes finds no significant gain (2003: i)

when contrasting IELTS preparation with an English for Academic Purposes

(EAP) course, a result repeated with Green (2007), who found a difference


27/131

17 | P a g e

occurred for course length but not for course type, as might be expected given

the quasi-academic nature of the IELTS exam. Spratt (2005) cites additional

studies by Read and Hayes (2003), Andrews et al (2002), and Cheng (1998) as

reaching similar conclusions. Perrone (2010), in comparing a general English

(GE) course with a FCE course, also found no advantage in test scores for

those students taking the exam preparation class, though as Perrone does not

give pre-course data for either group, it is unclear whether the groups started

from the same intial position or whether one group started from a more

advanced position in terms of test scores, and consequently made less

progress in order to reach the same finishing point as the other group.

Perrones study also examined a total of 35 participants distributed between

four groups, highlighting a common problem in washback and impact research,

whether findings from such small-scale studies can be extrapolated into a larger

context.

Aside from the scope of such studies, other limitations lie in the specific nature

of the principal investigation focus that each study selects. Tsagari (2009)

identifies variables that affect the nature of washback (form, intensity,

specificity, length, intentionality, value) and that are affected by washback

(teachers, students, the subject, resources, materials, classroom conditions,

school management practices, test communication practices, the wider social

context) (ibid, p. 5). Her model of washback as it applies to stakeholders in the

FCE in Greek language schools emphasises that the nature of exam washback

is circuitous and interactive (ibid, p. 8). Studies that investigate single or

particular aspects of washback therefore run the risk of missing the influence

that other aspects may have on the process they seek to examine.


28/131

18 | P a g e

Spratt examines 11 washback studies and categorises the investigation areas

of the studies as follows: curriculum, materials, teaching methods, feelings and

attitudes, learning. (2005, p. 8) and reviews the findings of the s tudies as

broken down into these areas. As mentioned above however, there are

limitations in this approach as choices made in regard to one aspect may well

affect the others. This inter-relationship is diagrammed in Figure 1 below (after

Spratt 2005).

FIGURE 1: THE INTER-RELATIONSHIP OF ASPECTS OF EXAM CLASS TEACHING (AFTER SPRATT 2005)

In Figure 1 exam class teaching is shown as a matrix in which a choice made in

one aspect will also affect all of the other aspects. Here, a choice made to alter

the curriculum may require an adjustment to teaching methods, the use of

different materials, may alter teacher and learner feelings and attitudes towards

the course, and may affect the learning process. Tsagari, for example, noted

that students in her study did not enjoy their classes and felt tired, disinterested

and bored (2009, p. 7); an attempt to alter such a situation does not only


29/131

19 | P a g e

require changes to be made to learners feelings and attitudes. It may require

changes to be made in some or all of the other aspects to achieve the desired

result.

Given this interplay between aspects of exam class teaching, the limitations of

some washback and impact studies become more evident. A contrast between

an EAP class and an IELTS class, (e.g. Read and Hayes 2003), or a FCE class

and a GE class (e.g. Perrone 2010), is useful only if we can be sure that the

teaching methodologies and materials used are discernibly different. Having

measured the different proportions of classroom time spent on developing the

listening, speaking, writing and reading skills, Perrone concludes instruction in

the two courses appeared to differ (2010, p. 227), though he does not seek

to account for how or why this might be and as, at his third measurement point,

the GE class contained 29.1% of exam preparation (2010, p. 198) this might be

useful information to uncover.

This section has shown that there is little evidence to suggest that exam

preparation courses give learners a significant advantage when it comes to

exam performance. The reasons why this might be are unclear, as while

different studies have shed light on how exams influence different aspects of

exam classes, little research has been done on how the different aspects of

exam classes affect exam performance and the inter-relationship between

aspects of exam class teaching tends to obscure the usefulness of such

findings for the exam class teacher. This is an area that would benefit from

further research, though sadly it falls outside the purview of this dissertation.


30/131

20 | P a g e

2.5FORMS OF SUPPORT FOR EXAM CLASS TEACHING

A review of the literature couched in the terms of teacher support reveals that

such support is best viewed through the prism of Continuing Professional

Development or CPD, a term which has become increasingly prevalent over

the course of 2012 (Wilden, 2012) and relates to the process of taking

responsibility for your own development as a teacher (ibid). This highlights the

differing natures of CPD and teacher support as such a view places the onus on

the individual who requires support to find it, rather than on other stakeholders,

e.g. employers, to make it available.

Writers on CPD have identified a number of sources of CPD relevant to ELT

which may also act as sources of support for exam preparation class teachers.

Johnson (2009) and Wilden (2012) suggest ways teachers can professionally

develop and Table 1 below shows a summary of their ideas.

Broadening teaching range (teaching new levels / trying new things)

Conferences (attending / presenting at)

Conversation (informally with colleagues / informally or formally with experts)

Courses (teaching courses / language courses)

Materials Creation

Membership of Professional Associations

Mentoring (being mentored / mentoring another)

Observation (being observed / peer observing)

Online Communities (social media groups and communities) Reading (teaching magazines / books / blogs, individual or group reading)

Research Projects (contributing to / conducting own)

Specialisation of teaching interests

Workshops (attending / giving)

Writing (magazine articles / blogs)

TABLE 1: A SUMMARY OF WAYS OF CONTINUING PROFESSIONAL DEVELOPMENT, AFTER JOHNSON (2009)AND WILDEN (2012)


31/131

21 | P a g e

Not all of these are appropriate in a support context as they rely on the

individual to provide, rather than receive, support; for example mentoring

another or presenting at conferences.

Reading is a common form of both CPD and teacher support and there are a

number of different sources teachers can draw on. This dissertation has

already drawn on teaching blogs (Chong, 2011), professional magazines

(Senior, 2010) and academic journals (Wilson, 2010), as well as professional

development and methodology books (Burgess & Head, 2005; May, 1996). For

the exam class teacher there are also handbooks from exam providers that give

detailed descriptions of the exam and exam components, e.g. UCLES 2012.

There are also a number of online communities and forums that can provide

support, either via social media for the sharing of ideas and resources or via

communities created by publishers and exam providers, though these latter

may have an additional purpose beyond teacher support.

Teacher training and development is another source of support, for example

teaching courses, workshops and conferences as per Table 1 above. Some

organisations also offer in-service teacher training (INSETT) seminars (Waters

& Vilches 2012), and it is also possible to take part in online seminars or

webinars (Webinars, 2013).

The individuals in the teachers context can also provide support and

conversation with colleagues (as per Table 1) and support from senior staff, of

which mentoring and observation can form a part, are both common forms of

individual support.


32/131

22 | P a g e

Finally, it is worth also mentioning course books and supplementary books as

probably nothing influences the content and nature of teaching and learning

more than the books and other teaching material used (Cunningsworth, 1995,

p. v), though whether the course book is viewed as a help or a hindrance is

something that will be investigated during the course of the research component

of this study.

Thus forms of support applicable to exam class teaching can be given as per

Table 2 below:

Reading

Teacher blogs and websitesProfessional magazines and journalsProfessional development and methodology booksExam provider handbooks and materials

OnlineSocial media groups and forumsPublisher websites and forumsExam provider websites and forums

TrainingTeacher development coursesExternally provided seminars, webinars and conferencesIn-house teacher development programme(s) (INSETT)

IndividualsSharing ideas with colleaguesSupport from senior staff

MaterialsCourse booksSupplementary resource books

TABLE 2: FORMS OF EXAM TEACHING SUPPORT

2.6SUMMARY OF CHAPTER TWO

Exam class teaching shares many characteristics with general English teaching

but aims towards the goal of successful exam performance, which influences


33/131

23 | P a g e

choices made in regard to curriculum, materials, methodology and attitudes to

the course.

Methodologies and approaches to exam class teaching may vary considerably,

but successful exam teaching practice remains difficult to identify and there is

little evidence to suggest that exam preparation classes significantly improve

learner exam performance, which suggests there is a need to support exam

class teachers and a number of potential support modes were identified.

The aspects of exam class teaching and forms of support for exam class

teaching that were identified in this chapter will inform the design and content of

the research instrument as described in the next chapter.


34/131

24 | P a g e

CHAPTER THREE: RESEARCH METHODOLOGY

This chapter details the design and administration of a research instrument

created to investigate the degree of support available to teachers of exam

preparation classes. It sets out a rationale for and clarifies the aims of the

research, provides a methodology for the design and administration of the

research instrument, describes the implementation of a pilot study to trial the

research implement, details the design of the research instrument and provides

an overview of the research participants.

3.1RESEARCH MATERIALS AND PROCEDURES

The research materials comprised an online questionnaire in three parts and

containing a total of twenty questions. Each part focused on a separate aspect

and the content of each section is described in section 3.5. In brief however,

part one contained six questions and focused on participants background and

professional experience, part two contained seven questions and focused on

participants experience with and attitude towards teaching exam classes, part

three contained seven questions and focused on participants experience with

and attitude towards the support they have received or would like to receive in

connection with their exam class teaching.

The questionnaire was designed using the form and spread sheet tools in

Google Docs. It was published online and publicised through various websites

and social media networks along with a request for volunteer participants. The

questionnaires were completed by participants in their own time and without

influence or interference from the researcher. The data were automatically


35/131

25 | P a g e

inserted into a spread sheet by the Google Docs programme for later retrieval

and analysis.

3.2RESEARCH AIMS

In Chapter One, the following beliefs about exam preparation classes were

articulated:

Teachers of exam preparation classes do not always have all the information

they need to know in order to be effective exam class teachers.

The support available to teachers of exam preparation classes is not sufficientto their needs.

To more fully investigate these beliefs, it is necessary to rephrase them in light

of the findings of the previous chapters as questions that can be answered with

the design of a suitable research instrument. The aims of the research are

therefore to answer the following questions:

A. Do teachers feel as though they need support in their exam class teaching?

B. Do teachers need support in their exam class teaching?

C. What forms of support are used by teachers of exam classes?

D. What are their feelings towards these forms of support?

E. What changes to this support or what additional support would teachers of exam

preparation classes like to see in the future?

To answer these research questions it is necessary to employ a suitable

research methodology.

3.3RESEARCH METHODOLOGY

This section details the methodology used in carrying out the research following

the eight step procedure described by Nunan (1992, p. 141).


36/131

26 | P a g e

3.3.1DEFINE OBJECTIVES

The objectives of this research have already been stated in the five research

questions given above.

3.3.2IDENTIFY TARGET POPULATION

Oppenheim defines population as those who fall into the category of concern

(1992, p. 38) and as such the target population of this research can be said to

consist of all those who prepare classes for English language exams such as,

but not limited to FCE, IELTS and TOEFL iBT.

3.3.3LITERATURE REVIEW

As more thoroughly detailed in chapter 2, successful exam teaching practice is

difficult to identify and there is some doubt as to the efficacy of exam

preparation classes. Nevertheless, common aspects of exam preparation

classes were identified, as were forms of support commonly available to exam

class teachers. These are incorporated into the questionnaire design as

described below in section 3.5.

3.3.4DETERMINE SAMPLE

Oppenheim defines sample as a smaller group, usually but not always a

representative one, within a population (ibid) and states that in principle every

member of the population should have a statistically equal chance of being

selected. Since this survey was published and distributed online, the sample

may not be completely representative of the population as it precludes those

without internet access. Equally, while the sample might be best described as a

random sample (Nunan, 1992, p. 142), the sample was not selected by the

researcher rather volunteers were asked for and therefore the participants

made the determination as to whether the survey applied to them. The sample


37/131

27 | P a g e

may therefore not reflect the target population. To determine biases in the

sample a number of questions relating to professional qualifications and

experience were included.

3.3.5IDENTIFY SURVEY INSTRUMENTS

In order to answer the research questions, data was collected through the

distribution of a short survey. The terms survey and questionnaire are here

used interchangeably and with Nunans description of purpose attaching to

both, that they seek to obtain a snapshot of conditions, attitudes, and/or events

at a single point in time (1992, p. 140). Oppenheim (1992) differentiates

between descriptive and analytic surveys. An analytic survey attempts to

describe causality and to investigate the relationships between variables,

whereas descriptive surveys tell us how many members of a population have a

certain opinion or characteristic the job of such surveys is essentially fact-

finding and descriptive (ibid, p. 12). The research instrument employed in this

dissertation can therefore be described as a descriptive survey.

3.3.6DESIGN SURVEY PROCEDURES

The questionnaire was created using Google Docs. Google Docs is a suite of

productivity software made available online to subscribers by Google and

includes word processing, presentation, spread sheet and form design tools.

The ability to quickly and easily create a form for the collection of data via a

simple web page with a variety of question types and an unlimited number of

questions led to the selection of Google Docs for the design and dissemination

of the questionnaire.

Once created, the form was published online in two locations; firstly via Google

docs itself and secondly via the researchers professional blog. A hyperlink to


38/131

28 | P a g e

the form was then publicised in and amongst various groups and forums on the

social networking sites Facebook, Twitter and Linked In. An email request

containing a hyperlink to the form was also sent to the researchers professional

colleagues and to schools within the International House World Organisation

network, with the additional request that recipients themselves distribute the

email to colleagues and professional acquaintances.

3.3.7IDENTIFY ANALYTICAL PROCEDURES

One of the features of the Google Docs form and spread sheet tool is that

response data is automatically collated into a spread sheet. Google Docs also

includes a Show summary of responses tool that gives a visual representation

of the survey data, though this is more useful with closed class questions,

where possible responses are selected by the researcher, than with open class

questions, where the respondent is free to make an answer of their own

devising (Nunan, 1992, p. 143).

As mentioned previously, the survey is descriptive, it counts a representative

sample and then makes inferences about the population as a whole

(Oppenheim, 1992, p. 12). As the survey makes use of both closed questions

and open questions, the presentation of the data varies somewhat. The data

from the closed questions is presented with reference to the numerical values

achieved. Nunan suggests that data from open questions is much more difficult

to quantify but can be synthesised to reveal possible patterns (1992, p. 146)

and this approach is adopted for the qualitative data produced by the survey.

3.3.8DETERMINE REPORTING PROCEDURES

The findings of the survey are reported and discussed in the context of the five

research questions listed above. Each question is discussed in the light of the


39/131

29 | P a g e

data generated by the survey and a conclusion is reached. These discussions

are presented in chapter five.

3.4CONDUCTING A PILOT STUDY

The need to conduct a pilot study is stressed by a number of authors. Nunan

states quite plainly that it is imperative to pilot any questionnaire (1992, p.

145), while Oppenheim also emphasises the need to make sure a questionnaire

is fit for purpose.

A pilot study of the questionnaire was therefore conducted. Ten teachers at two

language schools in Coimbra, Portugal were approached and asked to

complete the questionnaire and a further set of feedback questions on the

questionnaire. The teachers were selected as they were known to all have

experience of teaching exam classes and to have a range of professional

experience one participant had two years teaching experience, another had

twenty five. Primarily however, they were selected as a convenience sample as

they are all colleagues of the researcher, could be contacted quickly and easily

persuaded to take part. The downside to using them as a convenience sample

is that their responses were fairly homogenous and represented a fairly similar

attitude to exam class teaching and experience of professional support.

Pilot study participants were emailed a hyperlink to the online questionnaire,

asked to complete it, answer the following questions and email the responses

back to the researcher:

1) How long did it take you to complete?2) Was the wording of the questions clear and easy to follow? If not, which

questions were unclear?

3) Was the supplementary information clear, easy to follow and useful? If not,which questions or sections did this apply to?


40/131

30 | P a g e

4) Were there any options you would have liked included in questions but whichwere not? If so, which question and what was the missing option?

5) Were there any questions you felt should be excluded from the survey? Why?6) Were there any questions you felt should have been asked but weren't? If so,

what are they?

7) Is there anything else you think I should know about the questionnaire?

Feedback to the pilot study was positive as participants did not report problems

with the questions or rubrics, or feel that questions should be added to or

excluded from the questionnaire, though two of the questions were identified as

being overly similar to each other and these were combined into one question

for the final version.

Of more importance was that in considering the responses to the pilot study it

became clear that not all the questions served the purpose of providing

information to answer the five research questions articulated in section 3.2. The

questionnaire was therefore revised and resent to the pilot group for further

consideration.

It may be that in asking the same participants to report on the questionnaire a

second time, the degree of imposition was increased and participants gave the

survey less consideration than on the first occasion, not all respondents

completed the survey the second time and there were four abstentions. The

feedback was also more perfunctory, though similar in content to the first set of

results. In hindsight a secondary trialling of the survey should have enlisted a

separate set of respondents; unfortunately recruitment of a new set of

participants was not possible in the time frame for this stage of the research.

The responses from the second set of feedback provided more relevant data


41/131

31 | P a g e

than previously and so the decision was taken to publish the survey and start

accepting responses.

3.5RESEARCH DESIGN

3.5.1QUESTION TYPES AND TECHNOLOGICALAFFORDANCES

The questions comprised a mixture of closed class questions allowing for

numerical data to identify trends and open class questions to allow the

respondents a chance to offer a personal response from which insights into the

topic may be gathered (Oppenheim, 1992, p. 112-115).

Nunan identifies issues with question types and question wording, in particular

the need to avoid revealing the researchers own attitude to the question by

virtue of using leading language, and specifies that questions should be simple

and unique; that the language used in the question should not be confusing and

no question should ask about more than one thing at a time (1992, p. 143).

Nunan also lists a range of closed question types (ibid, p. 144), which seem

designed for use with a paper-based questionnaire, he does not describe the

form of an open class question perhaps thinking it self-evident, though

Oppenheim states they are usually followed by an amount of space (1992, p.

112). In this study the questionnaire is to be designed using the Google Docs

form tool and the question types available using this tool, the class of question

and the applicability of the options to question types as per Nunan (1992, p.

144) are given in Table 3 below.


42/131

32 | P a g e

Google DocsQuestion Type

DescriptionQuestion

Class

Question Typeapplicabilityafter Nunan

TextShort Answer

(Sentence length)Open not applicable

Paragraph Text Longer Answer(Paragraph Length)

Open not applicable

Multiple ChoiceSelection of one

answer from manyoptions

Closedcategory, quantity

/ frequency

CheckboxesSelection of one ormore answers from

many optionsClosed list

Choose from a listSelection of one

answer from manyoptions

Closedcategory, quantity

/ frequency

Scale

Assign a value inresponse to a singlequestion(values canbe labelled by the

researcher, between3 and 10 values

available)

Closedranking, scale,

quantity /frequency

Grid

Assign values tomultiple options (amaximum of five

values can bedetermined by theresearcher as

column headings.An unlimited number

of options can beadded as rows)

Closed Grid

TABLE 3: COMPARISON OF QUESTION TYPES AVAILABLE IN GOOGLE DOCS TO QUESTION TYPES GIVEN INNUNAN (1992, P. 144)

The questionnaire was divided into three sections. The first section, for reasons

described in section 3.3.4 above, asked about professional biographical data.

The second section asked about exam class teaching and participants attitudes

towards it. The third section asked about experience of support for exam class

teaching and any future support they think important.


43/131

33 | P a g e

3.5.2DESIGN OF SECTION 1: PROFESSIONAL BIOGRAPHY

As mentioned in section 3.3.4 one of the difficulties in using an online survey

tool is in achieving a balanced sample. The questions in section 1 attempt to

spot any biases in the sample. For example question 2 and 3 asked about

professional qualifications and length of teaching experience. If, for example,

all respondents report possessing Masters degrees and over fifteen years

experience, the results of the survey would not adequately reflect the target

population as one might expect more highly qualified and highly experienced

teachers to need or want less support in their exam class teaching.

The biographical questions were given as follows:

1. What is your current teaching environment?

2. What is the highest rated teaching qualification you possess?

3. How many years teaching experience do you have?

4. How many years experience teaching exam preparation classes do you have?

5. How much time do you spend teaching per week?

6. What type of classes do you currently teach?

3.5.3DESIGN OF SECTION2:EXAM CLASS TEACHING

This section of the questionnaire was designed to provide data to answer the

following two research questions:

A. Do teachers feel as though they need support in their exam class teaching?

B. Do teachers need support in their exam class teaching?

The questions that were asked in this section were:

7. Which exams have you prepared classes for?

8. How confident are you in your ability to prepare learners for these exams?


44/131

34 | P a g e

9. Which of the following aspects do you include in your exam preparation

classes?

10.What proportion of class time do you spend on those aspects identified in the

last question?

11.Do your current exam preparation courses use a course book?

12.Do you like teaching exam preparation classes?

13.Why? / Why not?

Questions 10, 12, and 13 were rendered as open class questions with space

provided for personalised answers. Questions 7 and 9 were rendered in the

checklist format, question 11 as a two option choice and question 8 as a grid

where participants assigned a value relating to the level of support they felt they

would need for each exam listed.

Questions 8, 12 and 13 were asked in support of research question A.

Question 8 directly asks respondents to specify the degree of support they

would need in preparing classes for specific exams, while questions 12 and 13

allow respondents to record their attitude to teaching exam classes.

Questions 7, 9, 10, and 11 were asked in support of research question B. They

attempt to check the effectiveness of respondents exam preparation class

teaching by asking respondents to describe habits and procedures in their

teaching.

3.5.3DESIGN OF SECTION 3:PROFESSIONAL SUPPORT

This section of the questionnaire was designed to provide data to answer the

following research questions:

C. What forms of support are used by teachers of exam classes?

D. What are their feelings towards these forms of support?


45/131

35 | P a g e

E. What changes to this support or what additional support would teachers of exam

preparation classes like to see in the future?

The questions that were asked in this section were:

14.Which of the following forms of support for exam class teaching do you use

now or have you used in the past?

15.Given the opportunity, which of the following forms of support for exam class

teaching do you prefer to use?

16.How frequently do you refer to or use these forms of support?

17.How would you rate the forms of support that you use or have used in the past?

18.Where you have indicated a resource mentioned above as being satisfactory orpoor, please could you indicate how you would like to see it improved?

19.Is there any additional support you feel you need to improve your exam class

teaching?

20.If there was one thing you could change about your exam class teaching, what

would it be?

Questions 14 and 15 were rendered as checklist style questions, allowing

respondents to select as many of the options as applicable. Questions 16 and

17 were rendered as grid style questions requiring respondents to assign a

value to the range of exams listed, being frequency of use for question 16 and a

rating from poor to excellent for question 17. Questions 18, 19 and 20 were

rendered as open class questions requiring the respondents to provide their

own answers.

Questions 14 and 16 provide data in answer to research question C as the

responses describe past and present habits of teachers in relation to the

support they use or have access to.


46/131

36 | P a g e

Questions 15 and 17 provide data in answer to research question D as

respondents are asked in question 17 to allocate a rating to the forms of support

described and are also asked in question 15 which forms of support they would

like to use, given the opportunity. This latter question acknowledges that not all

teachers have access to all the forms of support mentioned and may also give

an indication as to the future disposition of exam preparation class support,

providing data in answer to research question E.

However, questions 18, 19, and 20 are designed to provide more significant

data in answer to research question E. These questions were rendered as

open class type questions to attempt to remove researcher bias in pre-selecting

the responses and to allow participants to reflect on their own situation, sharing

these insights with the researcher.

3.6RESEARCH PARTICIPANTSThe survey attracted a total of 78 responses and a brief profile of the

participants follows in this section. No restrictions were placed on who was

eligible to take part, though the initial rubric included the phrase If you teach

exam classes or have taught them in the past, then your input would be

gratefully received. This may have meant that some potential participants

decided they were ineligible to take part.

Additionally, none of the questions in the survey were made mandatory, which

meant that participants could choose to skip some questions and that

consequently, not all of the figures given below add up to 100%.

The majority of participants, 83%, worked in a non-native speaker environment

while 14% stated they worked in an English-speaking country.


47/131

37 | P a g e

A breakdown of participants experience and highest level of qualification is

given below in Table 4.

NotStated

1 - 2years

3 - 5years

6 - 10years

10 - 15years

more than15 years

GrandTotal

Not Stated 2 1 3

Bachelor's degree 1 2 1 7 11

Certificate in ELT 1 6 7 2 3 19

Diploma in ELT 1 5 6 13 25

Doctorate 1 1

Master's degree 2 4 2 9 17

Other 1 2 3

Grand Total 2 1 10 19 14 33 79

TABLE 4: SURVEY PARTICIPANTS EXPERIENCE AND HIGHEST LEVEL OF QUALIFICATION

In hindsight, the question as to what constitutes the highest level of qualification

is open to interpretation. The UK Office of the Qualifications and Examinations

Regulator (Ofqual) publish a comparison chart, within which, the hierarchy of

qualifications rendered in the survey would fall as given in Table 5 below:

Level Type of Qualification

8 Doctoral degrees

7

Masters DegreesPGCE (M) - Post Graduate Certificate in Education (Masterslevel)Diploma in ELT (DELTA Cambridge ESOL)

6Bachelors DegreesPGCE (H) Post Graduate Certificate in Education (Honourslevel)

5 Certificate in ELT (CELTA Cambridge ESOL)

TABLE 5: ELT QUALIFICATIONS IN THE QUALIFICATIONS AND CREDIT FRAMEWORK / NATIONALQUALIFICATIONS FRAMEWORK FOR ENGLAND AND NORTHERN IRELAND, ADAPTED FROM OFQUAL (2011).

As Diplomas in ELT, Masters Degrees and some PGCE certificates all qualify

at level 7, it is difficult to say which of these might be considered highest rated.

Nevertheless, many participants are highly qualified and highly experienced


48/131

38 | P a g e

with 54% possessing either a Diploma in ELT or a Masters degree and with

60% of participants having over ten years of teaching experience.

Furthermore, 56% of respondents reported having five years or more

experience of teaching exam classes but only 64% reported currently teaching

exam classes.

The degree of experience that participants report may well affect their

responses to the survey and as discussed in section 3.3.4 of this chapter, may

not be a representative sample of the target population, though without full

knowledge of the characteristics of the target population this is also difficult to

determine. It may be that exam preparation classes are typically taught by

teachers with more experience and qualifications and that the respondents to

the survey are a representative sample. This data however, obviously has

implications for the discussion of the results that will occur in chapter five.

3.7SUMMARY OF CHAPTER THREE

This chapter has described the design of an online questionnaire to determine

the answer to answer five research questions, as outlined in section 3.2. It has

followed an eight step research methodology procedure as outlined by Nunan

(1992, p. 141).

The objectives of the research have been defined, a target population identified,

and the difficulties associated in determining a sample for the survey, which

might best be described as a random sample of volunteers, discussed.

This chapter has identified the research instrument as being a descriptive

survey (Oppenheim, 1992, p. 12) created in and distributed via the Google Docs


49/131

39 | P a g e

platform. It detailed the types of questions that were available within the Google

Docs platform and how these relate to survey question types identified by

Nunan (1992, p. 144), presenting a brief rationale for the structure of the survey

and the inclusion of specific survey questions, relating these to the stated

research questions. It described the initial piloting of the questionnaire and

subsequent revisions that were made to it. It has outlined how the data is to be

presented and discussed.

Finally, as the sample was identified as being a random volunteer sample, this

chapter presented an overview of the 78 respondents to the survey, in particular

looking at their length of service and degree of professional qualification. It

found that respondents were generally highly experienced and highly qualified,

and discussed whether this might be a representative sample or not before

concluding that the implications of this would need further examination in the

context of discussing the results in chapter five.


50/131

40 | P a g e

CHAPTER FOUR: PRESENTATION OF RESULTS

This chapter presents the results from the second and third parts of the online

questionnaire described in chapter three, which asks about participants

experience of exam class teaching (section 4.1) and experience of support for

exam class teaching (section 4.2)

4.1 SURVEY RESULTS PART TWO EXAM CLASS TEACHING

4.1.1SURVEYQUESTION 7: WHICH EXAMS HAVE YOU PREPARED LEARNERS FOR?

The highest rated answers on this question were FCE with 88% and CAE with

73%. Exams produced by Cambridge ESOL scored much higher than other

exam providers though this may indicate a geographical bias in the survey

sample. Figure 2 below shows the count of responses for each option.

FIGURE 2: COUNT OF RESPONSES FOR EXAMS TAUGHT BY RESPONDENTS

0

10

20

30

40

50

60

70


51/131

41 | P a g e

Respondents

petrie 2013 exam teaching support

Documents