conducting an ‘error gravity’ study for the secondary school teachers in evaluating a writing...

7/30/2019 Conducting an error gravity study for the Secondary School Teachers in evaluating a writing task

1/24

North South University

Department of English

ENG 573: Research Methods in TESOL

SUMMER 2012

Research Paper

Conducting an error gravity study for the Secondary School Teachers in

evaluating a writing task

Submitted to:

MAHJABEEN HUSSAIN (MJn)Lecturer

Department of English

North South University

Submitted by:

SK. SHAFIQUR RAHMAN

ID: 121 1218 055

Date of Submission: 16th

August 2012


2/24

Sk. Shafiqur Rahman | 2

A. Abstract:This study addresses the response of ESL (English as Second Learners) students error

gravity in both grammatical and writing forms to a naturally occurring sample of second

language writing in a secondary school teaching environment. Using a guided grammatical

scale protocol, the secondary school participants are led to participate a one-page essay that

is produced by the students whose first language is Bengali. The faculty evaluates the writing

holistically (on a scale from one to ten) and is invited to identify the most troublesome

errors. Results indicate that the scoring does not affect holistic scores; however, the English

faculties of secondary school do rate the composition differently. While qualitative analysis

demonstrates that, not surprisingly, individual editing styles among faculty are quite variable,

there is a tendency across faculty to edit semantic gaps as opposed to grammatical items. This

indicates a preference by the faculty to clarify content, a finding that supports prior research.

This study also includes the most frequent grammatical errors done by the secondary school

students, which help the English faculties to bear more concentration and the students self-

correction in a better writing skill achievement. The opinion through a bounded system case

study, a term originally coined by Louis Smith, results into a systematic way of giving

importance on particular grammatical parts and structures of meaningful writing skills. The

participants of this study are 5 secondary level students and 3 secondary level teachers. The

materials used in this study are- a case study followed by the researcher while evaluating

the students writing task; a class observation form filled by the researcher that gives a

description of a grammar classroom, teaching skill in learning grammatical items, materials

used by the teacher in a grammar teaching class room, different types of learners and

teachers cooperation with them; 3 questionnaires filled by the secondary level teachers that

give their opinion in importance of grammar in a writing task. Through this procedure, we

hope to reach the result of the study that gives us an identification of teaching grammar

according to its seriousness that improves the secondary school students writing task.This


3/24


study of error gravity for the secondary school teachers will be beneficial for TESOL while

teaching and evaluating writing skill.

B. Motivation for the research:When I was at class eight, in our English language class, our English teacher tasked us to

write a 250-word composition on Climate Change as a home work. I wrote the composition

and gave it to my home tutor for a previous evaluation, and he scored my composition six out

of ten, and showed me some correction of my writing and advised me some grammatical and

textual units.

The next day, I submitted my original uncorrected script of the composition to my course

teacher and he scored my paper eight out of ten. Then I surprised and a question turning

around my brain that why the same script getting different scores. By then, I planed upon an

idea, and I again submitted the same uncorrected paper to the same course teacher of my

school after four months, and that time he scored my paper six out of ten.

Then I understood there are no rules of scoring writing. Scoring writing goes through a

holistic approach. As it is called impressionistic scoring, writing creates different impressions

to different people at different time.

For the tradition of evaluating writing ability in second language learning, I plan an action

research to develop an error gravity frequency table of evaluating a writing task for the

secondary school teachers so that the scoring in a writing task can be done analytically in a

context of different test givers, period and kind.


4/24


C. Purpose of the research: To make an error gravity frequency measurement table through a case study by

doing an action research

To get an idea of the writing ability of the secondary school student To attain a comparison between what is being learnt to the students and to what extent

they are competent in learning grammar through live observation as a part of class

room observation

To what extent CLT is helping secondary school students learning of particulargrammatical items

D. Research questions:1. What is the difference between mistake and error?2. What types of errors are considered most serious by the Secondary School English

Teachers?

3. What is the degree of seriousness in error making among the high school students intheir writing tasks?

4. What are the levels of giving marks of the high school English teachers to theirstudents considering the errors in writing skill?

5. What are the respective expectations of the high school English teachers in makingcorrect and meaningful sentences in a Second Language Learning environment?

6. What is the level of learning grammar among the high school students in a SecondLanguage Learning environment?


5/24


E. Literature Review:'Error' is made unconsciously and corrected by the same person who has made the 'error'.

Error is in the level of performance; however, 'mistake' is because of the lack of knowledge.

Therefore, mistake is in the level of competence. None of the above is intentional. 'Fault' is

completely different from these two. It is because of being careless or doing something

intentionally, in this case the person who had done the fault has to be blamed.

In a well-known experiment, mistakes made by Greek secondary-school children were shown

to Greek teachers of English, British teachers of English and British non-teachers. Members

of each group graded the mistakes on a scale from 1 (least serious) to 5. Interestingly, the

mistakes which the Greek teachers regarded as most serious were often those that troubled

the native speakers least, and vice-versa. Some examples, with the average grading given by

the Greek teachers (GT) and the British non-teachers (BN):

*We agreed to went to the cinema by car. (GT4.6; BN2.2)

*We didnt knew what had happened. (GT4.4; BN1.8)

*Dizzys from the wine we decided to go home. (GT4.2; BN2.1)

*The people are too many so and the cars are too many. (GT3.0; BN4.3)

*The bus was hit in front of. (GT2.6; BN4.3)

*There are many accidents because we have not brought (meaning broad) roads. (GT2.4;

BN4.1)

The native speakers generally gave higher marks to mistakes, which impeded their

understanding: discussing the reasons for their assessments, many mentioned intelligibility.

The non-native teachers seemed more disturbed by infringements of common grammar rules;

in discussion, they referred frequently to basic mistakes. They seemed most upset by the

fact that learners continued to break rules which had been taught at an earlier stage of the


6/24


course, and which they should therefore have mastered. They were effectively teaching

grammar because it was there. (Hughes and Lascaratou 1982)

Grammar can be reassuring and comforting. In the convoluted landscape of a foreign

language, grammar rules shine out like beacons, giving students the feeling that they can

understand and control what is going on. Although this feeling is partly illusory (structural

competence only accounts for a proportion of what is involved in mastery of a language),

anything that adds to learners confidence is valuable. However, the security-blanket aspect

can lead students and their teachers to concentrate on grammar to the detriment of other less

modifiable but equally important aspects of the language. People often regard grammar as a

single interconnected system, all of which has to be learnt if it is to work properly. This is an

illusion. Grammar is not something like a car engine, where a fault in one component such as

the ignition or fuel supply can cause a complete breakdown. It is more realistic to regard

grammar as an accumulation of different elements, some more systematic than others, some

linked together tightly or loosely, some completely independent and detachable. We teach

or should teachselected subsystems, asking for each: 1) How much of this do the students

know already from their mother tongue? (A German speaker, unlike a Japanese learner,

knows the main facts about English article use before his/her first lesson.) 2) How much of

the rest is important? 3) How much of that have we got time for? To try to teach the whole

system is to ignore all three of these questions. Where grammar is given too much priority,

the result is predictable and well known. Course books become little more than grammar

courses. Students do not learn English: they learn grammar, at the expense of other things

that matter as much or more. They know the main rules, can pass tests, and may have the

illusion that they know the language well. However, when it comes to using the language in

practice they discover that they lack vital elements, typically vocabulary and fluency: they

can recite irregular verbs but cannot sustain a conversation. (As J K Jerome put it a century


7/24


ago, few people care to listen to their own irregular verbs recited by young foreigners.) Such

an approach is also psychologically counterproductive, in that it tends to make students

nervous of making mistakes, undermining their confidence and destroying their motivation.

(Michael Swan, 2002)

Many approaches within second language research have presented insights into the intricacies

of native/non-native interactions; one of those has been broadly termed the study of error

gravity. This type of research has aimed to establish hierarchies of error types so that

language teachers might focus on areas of grammar and pronunciation judged by native

speakers to be most disruptive to communication. Measures of comprehensibility,

acceptability, and naturalness are among the constructs used to assess sensitivity to particular

L2 errors.(Rifkin & Roberts, 1995)

Using an error evaluation approach, error gravity hierarchies have been investigated for

second and foreign language classes in English (Johansson, 1978; Khalil, 1985; Sheorey,

1986; Santos, 1987;Tomiyana, 1980);. In most of this research, isolated spoken or written

sentences or contrived prose passages have been used as stimuli. Such studies have provided

useful insights into native speaker perception of errors in L2 writing, but the approach, which

often presents only one error per sentence carrier, may only partially capture a realistic

evaluation of L2 performance. In order to help composition instructors had better prepare

their learners, we argue for the importance of assessing response to L2 error in a more

naturalistic manner. Rather than asking native writers to evaluate L2 errors in terms of

abstract concepts such as acceptability or comprehensibility, a naturalistic performance by the

evaluator (i.e., marking and editing any error they perceive) holds the potential to produce a

more realistic accounting of response to L2 writing. In the current study, we aim to more

approximate actual evaluation of and response to L2 writing by asking university faculty to


8/24


holistically grade and then edit an actual composition written by a student for whom English

is not a native language.

F. Method and Materials:1. Classroom observation: (Appendix C)

Target institution: Rajuk Uttora Model College Target population: Students of class nine Target subjects: students of class nine and their English Language faculty Observer: The Researcher Goals of observation: number of students in an English language classroom;

Evaluation of efficacy of grammar and semantic materials provided by the course

teacher; Practice session that helps the practical use of taught grammar materials

and its effectiveness in the researchers case study materials

Procedure: live observation, notes and logs of the researchers, the researchersparticipation with the student activity, feedback

2. Questionnaire for the English language faculty (Appendix B) Likert scales, Guttman scaling and scaled questions is followed to achieve

leading questions; and highbrow, complex , irritating and negative questions

are kept out of the questionnaire

Questionnaires are associated with class room observation, case study materialsand interviews with individual respondents;

Validity, reliability, lack of redundancy, repetition, independence of items,irrelevant information are maintained;

Little explanation, giving free choice, few verbal links and independentopinions are given importance;


9/24


As a secondary school English language faculty, the teachers academic,professional and non-academic achievements will be considered as ones level of

proficiency

Sex and age are identified in a separate option in the questionnaire

3. Case study: Step 1: 5 students of class nine are given a blank A4 paper; they write an essay

on the give topic mentioned on the top of the paper; the course teacher evaluates

them and identifies the errors with circles using red ball-point pen; and on the

pointed box of the paper the course teacher gives his/ her comments; As students

are reluctant to give their name on the paper, they are identified as participants-

P1, P2, P3, P4 and P5; among them P1 and P2 are female and the rest P3, P4

and P5 are male

Step 2: The researcher counts the particular errors for each paper with countablefrequency and give his opinion on the comment box. (Appendix A: Case

Study)

G. Procedure: Classroom observation

1. Notes and logs of the researchers: Number of the students in the classroom: 42

Materials used by the teacher: A grammar book- A Practical English Grammarby

Thomson and Martin; Several work shits for pair work in classroom participation

Topics discussed by the teacher: Complex sentence structures; Linkers and the use of

linking words and Use of punctuation

Tasks for the students: Teacher gives several work shits to pin point the errors in a

complex sentence, Fill in the blanks with linking words and editing punctuation marks


10/24


Evaluation of the task: Teacher observes the pair work situation and calls for the

correct answers

2. live observation: Classroom environment: Well- lighted large classroom

Response of the students: Some hyperactive students ask questions and teacher helps

them

Cooperation of the teacher with the students: Teacher cooperates with the learners

Teacher-student mutual understanding: well

Handling of weaker students: Teacher knows his weaker students and takes care in

classroom participation willingly

3. Follow-up and paper selection process: Teacher asks students to write anessay on Traffic Congestion and its Possible Solution and tells me to collect

five papers among them, I selected 2 hyperactive learners, 2 shy learners and

one weak learners paper by the suggestion of the teacher for my Case Study

procedure. Among them, two are female and three are male.

Case study:Teacher collects the five papers, identifies the errors with red pen, and scores them. The

following table shows some details of the five participants.

Table- G1

Participants Category Scores (out of

ten)

Sex

P1 Hyperactive 08 Female

P2 Shy 07 Female

P3 weak 05 Male

P4 Shy 08 Male

P5 Hyperactive 07 Male


11/24


Identifying errors: the researchers activity:

Each 0 presents for each error in the frequency column. Based on the total number

of frequency, degree of seriousness is identified.

Table-G2

Grammatical

Errors

Error Frequency Total

number of

frequency

Degree of

seriousness

Subjectverb

agreement

P1- 00

P2- 000

P3- 0000

P4- 0P5- 000 13

o Not seriouso Alrighto Serious

Very serious

Use of tense P1- 0

P2- 0

P3- 000

P4- 0

P5- 00

8

o Not seriouso Alrighto Serious Very serious

Word order P1-

P2- 0

P3- 0

P4-P5- 0

3

o Not serious Alrighto Seriouso Very serious

Spelling mistake P1-0

P2-00

P3-00000

P4-00

P5-0

11


Use of determiner P1-0

P2-0

P3-00

P4-0

P5-0

6


Use of capitalization P1-

P2-

P3-0

P4-

P5-

1


Correct position of

adverb

P1-

P2-0

P3-0

P4-00

P5-00

6



12/24


Punctuation P1-0

P2-0

P3-00

P4-00

P5-00

8


Period (.) P1-P2-

P3-0

P4-

P5-

1


Comma (,) P1-0

P2-0

P3-0

P4-00

P5-00

7


Hyphen (-) P1-

P2-

P3-P4-

P5-

0


Quotation mark ( ) P1-

P2-

P3-

P4-

P5-

0


Direct and indirect

sentences

P1-

P2-P3-

P4-

P5-

0

o Not serious

Alrighto Seriouso Very serious

Passive sentences P1-

P2-0

P3-0

P4-

P5-

2


Incorrect idiomatic

expressions

P1-

P2-

P3-P4-0

P5-

1


Antecedents of

pronouns

P1-

P2-0

P3-0

P4-

P5-0

3

o Not seriouso Alright Seriouso Very serious

Use of appropriate

linkers in making

complex sentences

P1-0

P2-0

P3-00

P4-0P5-0

6



13/24


Redundancy P1-

P2-

P3-

P4-0

P5-0

2


Parallel structure P1-P2-0

P3-0

P4-0

P5-

3


Appropriate

prepositions

P1-0

P2-00

P3-000

P4-0

P5-00

9


Semantics P1-

P2-P3-0

P4-

P5-

1


Questionnaire for the English language faculty:In L2 learning situation writing skill is mostly dependent on grammatical accuracy. Three

secondary school teachers give their opinion on the importance of grammar learning in

building up successful writing skills answering different questions of the questionnaire. This

particular procedure relates researchers case study results with the opinion of the related

faculty members.

Questionnaire query results-

1. Do you think learning grammar according to its importance in a writing task isnecessary in a secondary school L2 learning environment?

o 3 teachers say- yes2. Which type of scoring do you prefer in a writing task evaluation?

o 2 prefer holistic and 1 prefers analytical scoring3. Do you think errors in semantics are more important than grammatical mistakes?

o 2 say they are equally important, one says no


14/24


4. What kind of classroom environment do you prefer to teach grammar in a teacher-student cooperation group task?

a) Large (50 students) b) small (10 students) c) medium (25 students)o Two prefers large classroom and one, medium

5. Do you think scores should be given on the correct use of grammar in a writing task?a) Yes b) No c) there are other facts

like:..

o Two sayyes and one says there are other facts like: total meaningOpinion of the teachers:

1. Only grammatical accuracy comes first in scoring a writing tasko Two say agree and one, disagrees

2. The writing should only convey meaning / follow semantics where grammaticalerrors are not given importance

o Two say disagree and one, strongly disagrees3. Grammar and semantics go side by side

o Two say agree and one, strongly agrees4. Subjectverb agreement is the prime rule of grammar

o Three say strongly agree5. Punctuation problem can be considered at the secondary school level

o Two say neutral and one, agrees6. Must have the knowledge of tense at the secondary school level

o Two say strongly agree and one, agrees7. Must have the knowledge of capitalization at the secondary school level

o Two say agree and one stays neutral8. Spelling mistakes at the secondary school level can be considered

o Two say agree and one, strongly agrees9. Must have the knowledge of word order at the secondary school level

o Two say agree and one stays neutral10. Writing skill reflects the total skill of grammar and meaning

o Three say strongly agree


15/24


H. Analysis:Through Classroom observation procedure, we see grammatical error gets importance in

writing task. In the provided work shit for classroom task, students practice pin pointing error

so that they can find grammatical error in their writing. Among the selected five participants

of the writing test, there is a mixture of different categories of learners- hyperactive, shy and

weak so that I can get different types of error in their writing.

According to the degree of seriousness in Table-G2, we find that most frequent

grammatical errors are- subject-verb agreement, use of tense, spelling mistake, Use of

determiner, correct position of adverb, punctuation (comma), appropriate prepositions. P1

and P4 score highest (8 out of 10) and their frequency of error is quite high in these very

serious area than other areas.

0

2

4

6

8

10

12

14

Most frequent errors

very serious


16/24


Arithmetic mean of the scores is (8+7+5+8+7)/5= 7

Median of the scores is (5,7,7,8,8)=7

Mode of the scores is 7 and 8

Therefore, in this test, 7 is an ideal score.

As, P2 and P5 both score 7. According to Table-G2, Most frequent error of P2 and P5 is in

Subject- verb agreement. That is under very serious in the degree of seriousness.

In the questionnaire, the degree of satisfaction table shows that under opinion no 4. ,Subject-

verb agreement is the prime rule of grammar , 3 teachers of secondary school strongly

agree with it. Therefore, the case study data that we get from the students and the

opinion # 4 of the questionnaire that we get from the teachers meet at a point that

subjectverb agreement is the most frequent error in a writing task.

From the teachers expectation- What are the qualities of a good writing?-

1. Proper form; less grammatical error; meaning of a sentence should be clear2. Less grammatical error3. Less grammatical error; proper form and meaning

From the questionnaire, teachers common expectation is less grammatical meaning.

0

1

2

3

4

5

6

7

8

9

P1 P2 P3 P4 P5

Scores of the participants


17/24


I. Results of the research:1. What types of errors are considered most serious by the Secondary School English

Teachers?

subject-verb agreement, use of tense, spelling mistake, Use of determiner, correctposition of adverb, punctuation (comma), appropriate prepositions

2. What is the degree of seriousness in error making among the high school students in theirwriting tasks?

subject-verb agreement3. What are the levels of giving marks of the high school English teachers to their students

considering the errors in writing skill?

From the questionnaire, Q#2, two teachers prefer with holistic and one analyticalmarking system.

4. What are the respective expectations of the high school English teachers in makingcorrect and meaningful sentences in a Second Language Learning environment?

Less grammatical error5. What is the level of learning grammar among the high school students in a Second

Language Learning environment?

As the most frequent grammatical error is Subject-verb agreement and theteachers expectation Subject-verb agreement is the prime rule, the level of

learning grammar among the high school students is at the beginner level.


18/24


References:

Hughes, A. and C. Lascaratou. (1982). Competing Criteria for Error GravityEnglish Language Teaching Journal, 36/3, 175182.

Johansson, F. (1978).Studies of error gravity: Native reactions to errors produced bySwedish learners of English.Gothenburg Studies in English 44. Gothenburg, Sweden:

Acta Universitatis Gothoburgensis.

Khalil, A. (1985). Communicative error evaluation: Native speakers evaluation andinterpretation of written errors of Arab EFL learners. TESOL Quarterly, 19(2), 335

351.

Rifkin, B., & Roberts, F. D. (1995). Error gravity: A critical review of researchdesign, Language Learning, 45(3), 511537.

Swan, Michael. In Methodology in Language Teaching, ed. Richards and Renandya,CUP 2002, pp.148152)

Sheorey, R. (1986). Error perceptions of native-speaking and non-native-speakingteachers of ESL.ELT Journal, 40(4),306312.

Santos, T. (1988). Professors reactions to the academic writing of nonnative-speaking students. TESOL Quarterly, 22(1),6990.

Tomiyana, M. (1980). Grammatical errors communication breakdown. TESOLQuarterly, 14(1), 7179.


19/24


Appendix A: Case Study

This study is on purpose of a research project named conducting an error gravity study

for the Secondary School Teachers in evaluating a writing task. The researcher is an MA

in TESOL student ofNorth South University and this study is a part of his research work.

The researcher is asking your kind cooperation and giving his words that all the personal

information will be kept secret.

Traffic Congestion and its Possible Solution

Name: Sex: male / female

Mark obtained:

Teachers Comment:Thank you for your nice cooperation


20/24


Identifying errors: the researchers activity

Grammatical

Errors

Error Frequency Total

number of

frequency

Degree of

seriousness

Subjectverb

agreement

o Not seriouso Alrighto Seriouso Very serious

Use of tense o Not seriouso Alrighto Seriouso Very serious

Word order o Not seriouso Alrighto Seriouso Very serious

Spelling mistake o Not seriouso Alrighto Seriouso Very serious

Use of determiner o Not seriouso Alrighto Seriouso Very serious

Use of capitalization o Not seriouso Alrighto Seriouso Very serious

Correct position of

adverb


Punctuation o Not seriouso Alrighto Seriouso Very serious

Period (.) o Not seriouso Alrighto Seriouso Very serious

Comma (,) o Not seriouso Alrighto Seriouso Very serious


21/24


Hyphen (-) o Not seriouso Alrighto Seriouso Very serious

Quotation mark ( ) o Not seriouso Alrighto Seriouso Very serious

Direct and indirect

sentences


Passive sentences o Not seriouso Alrighto Seriouso Very serious

Incorrect idiomatic

expressions


Antecedents of

pronouns


Use of appropriate

linkers in making

complex sentences

o Not seriouso Alrighto

Seriouso Very serious

Redundancy o Not seriouso Alrighto Seriouso Very serious

Parallel structure o Not seriouso Alrighto Seriouso Very serious

Appropriate

prepositions


Semantics o Not seriouso Alrighto Seriouso Very serious


22/24


Appendix B: Questionnaire for the English language faculty

Questionnaire

On

Conducting an error gravity study for the Secondary School Teachers in evaluating a

writing task

Dear Respondent,

I am Sk. Shafiqur Rahman, student of MA in TESOL, North South University

Bangladesh. I would be grateful if you could spare a few minutes to complete this

questionnaire. The researcher is asking your kind cooperation and giving his words that all

the personal information will be kept secret.

(Please answer all the questions correctly and for any query ask to surveyor)

Name:Gender: Male/Female

Faculty position: ..

Academic, professional and non-academic achievements

Please tick the appropriate answer:

6. Do you think learning grammar according to its importance in a writing task isnecessary in a secondary school L2 learning environment?

a) Yes b) No(If Yes then answerthe questions)

7. Which type of scoring do you prefer in a writing task evaluation?a) Analytical b) Holistic c) Other

8. Do you think errors in semantics are important than grammatical mistakes?a) Yes b) No c) Equal

9. What kind of classroom environment do you prefer to teach grammar in a teacher-student cooperation group task?

b) Large (50 students) b) small (10 students) c) medium (25 students)10.Do you think scores should be given on the correct use of grammar in a writing task?

b) Yes b) No c)there are other facts like:..


23/24


Please mark () the appropriate box to indicate your degree of satisfaction.

Opinions Strongly

agree

Agree Neutral Disagree Strongly

disagree

1. Only grammatical accuracycomes first in scoring a writing

task

2. The writing should only conveymeaning / follow semantics

where grammatical errors are not

given importance

3. Grammar and semantics go sideby side

4. Subject verb agreement is theprime rule of grammar5. Punctuation problem can be

considered at the secondary

school level

6. Must have the knowledge oftense at the secondary school

level

7. Must have the knowledge ofcapitalization at the secondary

school level

8. Spelling mistakes at thesecondary school level can be

considered

9. Must have the knowledge ofword order at the secondary

school level

10.Writing skill reflects the totalskill of grammar and meaning

What are the qualities of a good writing? :


24/24

Signature: ........................................ Date: .............

Appendix C: Classroom observation

1. Notes and logs of the researchers: Name of the faculty: Number of the students in the classroom: Materials used by the teacher: Topics discussed by the teacher: Tasks for the students: Evaluation of the task:

2. live observation: Classroom environment: Response of the students: Cooperation of the teacher with the students: Teacher-student mutual understanding: Handling of weaker students:

3. The researchers participation with the student activity:4. Feedback:

conducting an ‘error gravity’ study for the secondary school teachers in evaluating a writing...

Documents