effects of revision training on l2...

25

English Teaching, Vol. 65, No. 1, Spring 2010

Effects of Revision Training on L2 Writing

Yeon Hee Choi

(Ewha Womans University)∗

Choi, Yeon Hee. (2010). Effects of revision training on l2 writing. English

Teaching, 65(1), 25-56.

This study purposes to examine the effects of two types of revision training (teacher

feedback-aided and protocol-aided revision training) on L2 student writers’ revision and

written product and quality, compared with their self-revision and also to analyze L2

writers’ perceptions on revision training through a pre- and post-training questionnaire

survey. The results of the study show positive effects of both types of training; more

positive effects of teacher feedback-aided revision training on the quality and product of

writing, especially in less proficient writers; more significant effects of protocol-based

revision training on revision frequency, content changes, and success ratio of revision,

especially in less-proficient writers; and more local (content preserving) revisions

triggered by teacher feedback, especially in more proficient writers. Though teacher

feedback-aided revision was favored over protocol-aided revision after training, the

participants’ perception on the former’s positive effects and utilization rate decreased,

but their perception on the latter’s positive effects and utilization rate enhanced. The

present study suggests protocol-aided revision as an alternative revision method, and

different revision methods depending on the focus of revision and L2 writing

proficiency.

I. INTRODUCTION

With the increasing focus on writing process, revision has become a main issue in L2

writing class (Ferris, 1995, 1997; Hyland, 1998; Min, 2005, 2006; Sachs & Polio, 2007;

Zamel, 1982, 1983). It has been studied with a variety of topics including differences in L1

and L2 writing (Hall, 1987, 1990; Stevenson, Schoonen, & Glopper, 2006); the effect of

writing tasks/types (Lindgren & Sullivan, 2006b); revision mode (e.g., computer-

∗ This work was supported by the Korea Research Foundation Grant funded by the Korean

Government(KRF-2008-2366-1).

Yeon Hee Choi

26

mediated) (Phinney & Khouri, 1993; van Waes & Schellens, 2003); the effect of revision

on writing quality (Yeonhee Choi, 2006, 2007; Stevenson et al., 2006); and revision

strategy instruction (Sengupta, 2000); and the benefits of peer review (Lundstrom & Baker,

2009). These studies of L2 writing on revision illustrate that generally in L2 writing,

especially in demanding writing tasks, the frequency of revision is higher with a focus on

linguistic forms, and this is salient among learners with low writing proficiency.

Reader awareness is one of the important factors influencing revision. Previous studies on

revision have shown that writers need to be good readers in order to revise their writing from

the reader’s viewpoint (Hayes, 1996). Thus, ‘reading as the reader’ has been considered

important in revision. Research on reader awareness illustrates that proficient writers

examine if their readers can understand their intentions correctly (Hayes, Flower, Schriver,

Stratman, & Carey, 1987) and that skilled writers are more aware of potential readers in

their writing (Hayes et al., 1987; Sommers, 1980). However, whether reader awareness

influences the quality of writing and revision is still inconclusive in L2 writing (Yeonhee

Choi, 2006; Jourderais, 2001; Holliway & McCutchen, 2004; Sachs & Polio, 2007).

Furthermore, how to teach L2 writers to revise effectively has not been explored much,

except in a few studies on revision strategy instruction (Sengupta, 2000) and peer feedback

training (Berg, 1999; Cho, 2005; Liou & Peng, 2009; Min, 2005, 2006; Stanley, 1992).

Sengupta’s (2000) investigation of revision strategy instruction illustrated its positive

effects on L2 secondary learners’ writing performance. Other studies such as Stanley

(1992), Berg (1999), Yunkyoung Cho (2005), Min (2005, 2006), and Liou and Peng (2009)

focused on peer review training and demonstrated that training for peer review was helpful

for revision. However, training for other revision methods such as teacher feedback-aided

or protocol-aided revision has not been explored yet.1

Therefore, the present study purposes to devise a revision training model including

teacher feedback-aided and protocol-aided revision by raising reader awareness. The effect

of two revision training methods (teacher feedback-aided and protocol-aided revision

training) will be compared in terms of quality of writing, written product, revision frequency

and type, and L2 writers’ perceptions on revision training through a pre- and post-training

questionnaire. Effects of revision training will also be compared with self-revision. In

addition, the effects will be examined across L2 writing proficiency levels since it has been

noted that revision methods differ by L2 writing proficiency levels (Yeonhee Choi, 2007;

Zamel, 1983).

The research questions for the study are as follows.

1. What are the effects of teacher feedback-aided and protocol-aided revision training

1 Protocol-aided revision is recommended in L1 writing as a way of increasing reader awareness

(Hayes, 2000). However, it has not been used much to revise writing in L2 as well as in L1.


27

on the quality and product of L2 student writing, compared with self-revision? Does

the influence of revision training vary with L2 writing proficiency?


on the frequency and types of revision, compared with self-revision? Does the

influence of revision training vary with L2 writing proficiency?


on the success and utilization rate of revision, compared with self-revision? Does the

influence of revision training vary with L2 writing proficiency?


on L2 student writers’ perception on revision methods?

II. PREVIOUS STUDIES ON REVISION TRAINING

Revision has drawn a great deal of attention in L1 and L2 writing research; however,

there is not much research on revision training, except for studies on peer feedback

training (e.g., Stanley, 1992; Berg, 1999; Min, 2005, 2006). Since a good writing in

English is defined as a text clear to the reader, reader-oriented writing is essential (Hayes,

1996), especially in revision, as Jones and Myhill (2007) suggest that the process of

revision itself requires more understanding of the reader. Writers with higher writing

proficiency appear to revise more often and have the ability to consider potential readers of

their writing (Hayes et al., 1987; Sommers, 1980). Utilizing feedback provided by the

teacher or the peer is a way to revise writing with a consideration on the reader since they

play the role of the reader when responding to the writing. Although some studies have

concluded that teacher feedback is not always useful (Truscott, 1996), in general, teacher

feedback is recognized most effective when it is provided during the writing process

(Ferris, 1995; Leki, 1990; Zamel, 1985). L2 learners prefer teacher feedback (Huyyun

Choi, 2002; Hedgcock & Lefkowitz, 1994; Lee, 2008; Nelson & Carson, 1998; Yang,

Badger, & Yu, 2006), that is, they appear to prefer specific teacher feedback that includes a

concrete suggestion (Ferris, 1995, 1997, 2003). Writing teachers’ corrective feedback has

gained more interest in recent research (Bitchener, 2008; Bitchener & Knoch, 2009; Ellis,

Sheen, Murakami, & Takashima, 2008). But, they lack clear evidence whether L2 writers

successfully incorporate teacher feedback, that is, whether feedback results in a higher

quality of the revised writing, and ultimately whether it improves students’ writing

proficiency (Conrad & Goldstein, 1999; Ferris, 1997, 2003; Goldstein & Conrad, 1990;

Hyland & Hyland, 2001). Furthermore, training on how to incorporate teacher feedback

effectively has not been specifically explored yet.

Yeon Hee Choi

28

Another way to enhance writers’ awareness of potential readers and help writers become

strategic readers is protocol-aided revision, in which the writer verbalizes whatever comes

into his/her mind while revising. Its positive effects have been revealed in Jourderais

(2001), and Stratman and Hamp-Lyons (1994), such as increased concentration and

learning opportunities, and more accurate revision; however, Sachs and Polio’s (2007)

study in L2 writing illustrates a negative effect on the accuracy of revision. Thus, it is still

uncertain on how think-aloud, that is, protocol-aided revision, can assist the L2 revision

process. In addition to the inconclusive results of the effects of protocol-aided revision,

there is not much research on how to teach L2 writers to become strategic readers in the

writing process.

Therefore, the present study aims at investigating L2 writers’ revision process when they

consider their potential readers, and how revision or quality of the writing changes in

accordance with the writing process. Specifically, the study purposes to devise a revision

training model using teacher feedback-aided and protocol-aided revision by letting the

students become their own readers. The effect of these revision training methods will be

compared with that of self-revision. Ultimately, the study aims at finding an effective

revision method for L2 writers by examining how the two types of revision training affect

the quality of the writing in English after revision training, and how L2 writers evaluate

their revision training.

III. RESEARCH METHOD

1. Participants

Eighteen Korean university students voluntarily participated in the study, including

eight seniors, three juniors, and seven freshmen.2 They were all female, ranging in the age

of 20 to 25. All of the participants were majoring or double-majoring in English Education

(Teaching English as a Foreign Language). Most of them had some experience of learning

composition in English or Korean. All of them stated in the pre-questionnaire that they had

some experience of revising their own writing, but only about a half of them received

2 The study had planned to have two independent groups to compare the impact of revision training

using two revision methods. But, it failed to find EFL writing teachers who were willing to take part in the research. Thus, L2 student writers were recruited. The number of the voluntary participants was not large enough to have two independent groups; thus, the training was administered to one group. The participants might have had more positive attitudes toward English writing or revision, compared to students of a regular English writing course.


29

formal instruction on revision. Based on the pretest score (Writing 1),3 which was scored

by the Test of Written Examination (TWE) scoring criteria (six scales), the participants

were classified into three groups: high-level proficiency (HP) group (five students) (M =

4.08), intermediate-level proficiency (IP) group (nine students) (M = 3.16), and low-level

proficiency (LP) group (three students) (M = 2.17).4 In the analysis of the data, one of the

seniors’ data was excluded because she didn’t complete all the requirements.

2. Materials

The research materials included four writing tasks for the pretest and training, four

sample writings for training, a pre-questionnaire for the participants’ background

information, and two post-questionnaires.

2.1. Writing Tasks

Four argumentative writing tasks were chosen from the prompts of the TWE developed

by ETS, as shown below.5

Writing 1: preference of class type

Writing 2: the issue of athletes’ and entertainers’ high salaries

Writing 3: preference of study type

Writing 4: the most important characteristic in successful life

The TWE prompts were selected since they could be relevant and familiar to the participants.

The first draft of Writing 1 was used as a pretest to divide participants by writing proficiency.

Writing 1 and Writing 3 were the main writing tasks and Writing 2 and Writing 4 were tasks

for in-class revision training sessions. In order to compare revisions in the main tasks, the

prompts of the main writing tasks were similar to each other. The prompts of tasks for

training were also similar to each other. The participants were required to complete each task

within 30 minutes, which is the time limit used in the TWE. However, no time limit was set

for revision. In addition, no word or paragraph limits were set for the writing tasks.

3 One of the participants wrote the essay completely off the given prompt of Writing 1, which was

intended to be used both as a pretest and as one of the main writing tasks. Thus, there were some problems to classify the participants based on the scores of Writing 1 only. As a result, the participants were classified by the combined score of Writing 1 and Writing 2, which was planned to be used for teacher feedback-aided revision training.

4 The number of the low-level proficiency group was very small, which is a main limitation of the research.

5 The topic difficulty was not measured.

Yeon Hee Choi

30

2.2. Sample Writings for In-class Revision Training

For the in-class revision training, four writing prompts were also selected from the TWE,

as shown below.

Sample Writing 1: preference of morning or night person

Sample Writing 2: preference of communication type

Sample Writing 3: the issue of mass media’s attention to public figures and celebrities

Sample Writing 4: the issue of smoking in public places

Sample writings were needed to be used for in-class training, that is, for teacher

demonstrations and group revision work; thus, Korean speakers were recruited to write

sample writings. Their TOEIC scores ranged from 800 to 850. The average length of sample

writings was 234 words, which was shorter than the average length of the pretest of the

participants (311 words). Sample Writing 1 and Sample Writing 2 were used for teacher

feedback-aided training, and thus during the training session teacher feedbacks were

provided in terms of grammar, expression, content, paragraph structure, and overall

organization.6

2.3. Questionnaires

A pre-questionnaire and two post questionnaires were constructed for the study. The pre-

questionnaire was consisted of two sessions. The first part was constructed to obtain

information on the participants’ experience of learning English writing and revision, English

proficiency scores including the TWE score, and the length of residence in English-speaking

countries (five yes-no questions, two choice questions, and 13 open-ended questions). The

second part included questions on their perceptions of teacher feedback-aided and protocol-

aided revision. It contained six 6-point Likert-scale items, two choice questions, and eight open-

ended questions on the participants’ attitudes towards the two types of revision including

their usefulness, foci, difficulties and problems, exploitability, and preference for long-term

effects, which were compared with those after the two training sessions. Two post-questionnaires

were also constructed to examine the participants’ perception of the two revision training

methods. While the questions were identical to those in the second part of the pre-training

questionnaire, they included open-ended questions on suggestions for revision training methods.

6 The teacher feedback in the main writing task and sample writings consists of three parts:

underlining to indicate ungrammatical or awkward expressions, marginal notes stating frequently made errors among underlined errors, and endnotes expressing the general comments on the essay including the comments on the paragraph structure, organization, and the content.


31

3. Data Collection Procedures Including Revision Training

The revision training was conducted by a native writing teacher with a doctoral degree

in TESOL for five weeks, between March 9 and April 11, 2009. The teacher had taught

university-level English composition for five years. All the participants were trained

through two revision training types. Two sessions of revision training were scheduled for

each type of revision training on the second and fourth week of the revision training

experiment (see Figure 1).

FIGURE 1 Revision Training Procedure

Week 1 Pre-questionnaire for background information and participants’ experience and

perception on teacher feedback-aided and protocol-aided revision

↓

Writing 1 (1st draft)

↓

Writing 1 self-revision (W1-SR)

↓


↓

Week 2 In-class revision training 1 for utilizing teacher feedback

- Class 1: teacher’s revision demonstration of Sample Writing 1 and individual

and group revision of Sample Writing 2

- Class 2: individual revision of Writing 2 based on teacher feedback

↓


↓

Writing 3 self-revision (W3-SR) ↓

Week 3 Writing 1 revision utilizing teacher feedback (W1-TF)

Post-questionnaire about teacher feedback-aided revision training ↓

Writing 4 (1st draft) ↓

Writing 4 self-revision (W4-SR) ↓

Week 4 In-class revision training 2 for protocol-aided revision

- Class 3: teacher’s revision demonstration of Sample Writing 3 and group

revision of Sample Writing 4

- Class 4: individual think-aloud revision of Writing 4 ↓

Week 5 Writing 3 revision aided by think-aloud protocol (W3-TA)

Post-questionnaire about protocol-aided revision training

The first training session for teacher feedback-aided revision was held in the second

week and it consisted of two two-hour training classes. The second training session for

protocol-aided revision was held on the fourth week and the number of classes and the

Yeon Hee Choi

32

length of the class were the same as the first training session. In other words, each student

received about 8 hours of in-class revision training. Teacher feedback-aided revision

training preceded protocol-aided revision training since the majority of the participants had

no experience of protocol-aided revision in English writing and it was assumed that

protocol-aided revision might be more demanding.

Before the training, a specific guideline describing the training schedule and the

requirements were distributed to the participants via e-mail. The participants were asked to

submit a pre-questionnaire, the first drafts of Writing 1 and Writing 2, and the self-revised

draft of Writing 1 (W1-SR) by the end of the first week. In the second week the students

had two classes of in-class revision training using teacher feedback.

The first in-class revision training was designed to model how to revise writing based on

teacher feedback. The teacher demonstrated how to revise writing utilizing various types

of teacher feedback such as grammar, vocabulary, expression, content, and organization, with

Sample Writing 1. (See the teacher feedbacks provided with Sample Writing 1 in Figure 2.

They include specific marginal comments on language problems, end notes on content,

organization, and overall problems. All problematic words or phrases are underlined.)

Then the students were asked to revise Sample Writing 2 based on teacher feedback, first

individually and then in a group. After revising Sample Writing 2, the students made

presentations on how they had revised it and submitted their revised drafts. The second in-

class training aimed to give further teacher feedback-aided revision training to the

participants. The students revised their own second writings (Writing 2) in class, which

were marked with teacher feedback. The teacher and a research assistant moved around to

answer questions about how to utilize teacher feedback in their revision.

After the two classes of training, the participants submitted the first and self-revised

draft of Writing 3. In the third week, they submitted the teacher feedback-aided revision of

Writing 1 (W1-TF) with the first post-questionnaire about teacher feedback-aided revision

training, and also the first and self-revised draft of Writing 4.

The second in-class revision training was designed to model how to revise writing doing

think-aloud. The basic structure of the second in-class revision training session (the third

and fourth class in Figure 1) was similar to the first one. During the third class, the teacher

first introduced the concept of think aloud and pointed out the importance of reader

awareness in revision process and then she demonstrated how to revise Sample Writing 3

while she thought aloud, as shown in Figure 3.

After the teacher’s demonstration, the students were asked to revise Sample Writing 4

by taking turns in a group while they thought aloud. Revising while thinking aloud was not

conducted individually due to the fact that most of the computers available were too slow

to be used effectively. In a group, the members thought aloud in turn while revising. Their

think-aloud was all recorded using MP3 players. While the participants were engaged in


33

FIGURE 2 Teacher Feedback on Sample Writing 1 and Teacher’s Revision

Original Draft Teacher Feedback

Teacher Feedback-Aided Revised Draft

I prefer to wake up early in the morning and I prefer to start the day’s work.

Because I believe getting up early in the morning is a more suitable schedule for Human’s life cycle, and it’s also better for our health. However there is a gap between preference and my real life. Logically, I prefer to get up early in the morning and start the day’s work as I mentioned above, but, practically I usually stay my bed just before I can go to work on time. And usually I get up later in the day and stay my bed all day long on weekend. I am doing nothing. I prefer to stay at home and watch movies instead of do work. Therefore there are a lot of efforts to support my preference. For example I usually go to bed before 10p.m to get up early in the morning tomorrow and take some vitamin supplements for increase physical strength. …… So I prefer to wake up early in the morning even if sometimes it is difficult to get up early in the morning and start the day’s work.

need comma after transition missing preposition

verb form word order

I prefer to wake up early in the morning and start the day’s work. I believe getting up early in the morning is a more suitable schedule for human’s life cycle, and it’s also better for our health. However, there is a gap between preference and real life. Logically, I prefer to get up early in the morning and start the day’s work, but actually I usually stay in my bed just before I go to work on time. Nevertheless, I still believe it is important to wake up early in the morning. Waking up early helps me feel

refreshed in the morning. When I get up early in the morning, I can spare time for reading a newspaper and thinking about the day’s work. So in order to get up early in the morning, I try to do my best. For example, I usually go to bed before 10 p.m. to get up early in the morning and take some vitamin supplements for increasing physical strength. And I do not meet my friends during weekdays. If I meet my friends, I try to get home by 9:30. … … work. So I prefer to wake up early in the

morning because it helps me feel refreshed in the morning and prevents me from being lazy on weekends. Sometimes it takes a lot of efforts for me to wake up early, but I think it is worthwhile making efforts to do so.

Comments 1) no clear paragraphing/no indentation

2) no clear structure/organization: introduction (x), body (x), conclusion (x)

3) no clear thesis statement

4) no clear topic sentences/supporting sentences

5) sentences contradict each other

6) sentences do not provide examples to support writer’s choice; rather, they give a description of a

daily routine

the think aloud process, the teacher and a research assistant moved around to remind the

participants to do think aloud when they were silent for more than 15 seconds. However,

they tried not to interfere with the participants’ thinking process. After completing the

revision, the participants listened to their own think-aloud protocols recorded in a group

and discussed to what extent they were aware of readers and to what extent they thought

Yeon Hee Choi

34

aloud about their revision process. In the fourth class the students had opportunities to

individually revise their own writing (Writing 4) while thinking aloud. After the

protocol-aided revision training, the participants submitted their revised draft of Writing

3 (W3-TA) with their think aloud audio files and the second post-questionnaire about

protocol-aided revision training (in the fifth week).

All writing tasks and questionnaires were submitted via e-mail. When the participants

revised Writing 1 and Writing 3, they were asked to revise their first draft without referring

to their self-revision. There was a 2-week interval between self-revision and post-training

revision in order to minimize the influence of self-revision on post-training revision.

FIGURE 3 Teacher’s Demonstration of Protocol-aided Revision

Original draft Think-Aloud Protocols Protocol-aided

Revised draft

Okay, uhm I’m going to demonstrate think aloud for all of

you ……, the purpose of doing this think aloud is to um,

develop a, an awareness of, you know, the fact that

whenever you write anything at all, particularly an

argumentative essay, you have to understand that you have

a reader. Okay, you have a reader. …... So……so you are

going to have a uh, imaginary conversation with your

reader. ……, so you have to imagine that there is

somebody sitting in front of you and that person is…your

reader, and that person is constantly throwing questions

about the essay. What’s wrong with this sentence, how

should I fix it, why did this person write this way. So try to

ask as many questions as possible and you’re going to try

to answer it, and as you have this imaginary conversation

going on, you’re going to revise the essay

simul…simultaneously, okay? ……

I agree that nowadays

television,

newspapers,

magazines, and other

media pay too much

attention to the

personal lives of

public figures and

celebrities. …… in

front of others.

“I agree that nowadays television, newspapers, magazines,

and other media pay too much attention to the personal

…… that they stand naked in front of others.” I think…I

think this introduction for one thing um<1>starts with the

sentence. The first sentence, it has a lot of problems

because the…prompt is …… So I’m going to…take

this…sentence here, which actually belongs at the end of

the paragraph, and it should…be placed at the end.

So…that…is going to function as my thesis statement.

So<2>now I have to delete this sentence…and…the

beginning of the paragraph is now a little bit confusing

…… But as a leading statement it sounds very confusing

so I’m going…I think I need to add something. I need to

add some kind of statement to give a hint what this whole

essay is going to be about.


35

I agree that nowadays

television,

newspapers,

magazines, …… and

celebrities.

{hear typing sound} Do celebrities and public

figures…have a right to maintain their private lives or

should…they…reveal their personal lives to the general

public? So I took the thesis statement I think…and then put

it at the end of paragraph so that it shows the

reader…uh…shows the reader what my position is which

means, which my position I’m trying to argue, that the

media pays too much attention to the personal lives.

But…at the end of intro…at the beginning of the

introduction I start by saying or asking a question uh…to

let the reader know what the essay is going to be about in

general.

Do celebrities and

public figures have a

right to maintain their

private lives, or

should they reveal

their personal lives to

the general public?

I think that is

certainly an act of

invading other’s

privacy.

Um… “I think that is certainly an act of invading other’s

privacy.” This sentence doesn’t seem to connect with the

leading sentence for some reason. So I have to make it a

little bit clearer to connect with the first sentence. The first

thing I’m going to do is change…the statement because I

don’t like the fact that it is in the first person pronoun which

doesn’t sound very academic and professional. So I’m

going to try to connect it to the sentence before. That comes

before.

I think that is

certainly an act of

invading other’s

privacy.

{starts typing} Taking too much interest in the lives of

famous people is certainly an act of invading

other’s…privacy. So I didn’t change the sentence that much

but I think I got rid of…using too much first person

pronouns because it doesn’t sound very professional. I

changed the sentence so that I can avoid using ‘I’ in the

sentence. Uh…and then I just tried to keep the rest of the

sentence the content of the rest of the sentence here.

Taking too much

interest in the lives of

famous people is

certainly an act of

invading other’s

privacy.

Note: think-aloud protocols are italicized, except for the original sentences read and the sentences

typed while verbalizing; six dots indicate parts not specified in the figure.

However, some of the participants did not follow the instruction and they referred to their

self-revision when they did post-training revision. In those cases, the changes already

made in self-revision were not counted as post-training revisions.

4. Data Analysis

The participants’ writing was holistically scored using the TWE CBT/PBT scoring

criteria (6 scales) by the researcher and a native composition teacher who conducted the

in-class revision training. The participants were classified into three L2 writing proficiency

levels based on their pretest scores, as mentioned before.

Before revision coding, the participants’ think-aloud files were transcribed into think-

Yeon Hee Choi

36

loud protocols.7 The transcription was based on the way the participants spoke. This

means that spellings were coded in the way they were articulated.

After counting all the revisions before and after training, revisions were also coded in

terms of their purpose, which was based on Faigley and Witte (1981), and Lindgren and

Sullivan (2006a). First, a distinction was made between revisions that changed the

meaning of the text (content change: CC) and those that did not (content preserving: CP).

The revisions that did not change the meaning of the text were then classified into five

categories: paraphrase, vocabulary, grammar, mechanics (punctuation, capitalization,

paragraph indentation, and spelling correction), and style/tone changes. The category of

style/tone illustrated the revision where the writer tries to make his/her style or tone

neutral.8 The content changes were coded at two levels: 1) micro-level changes (MI)

included addition of new content, deletion of existing content, or substitution of the

sentences; and 2) macro-level changes (MA) were major changes which influenced the

summary of a text.

In addition to coding in terms of revision frequency and types, the degree of success and

the utilization of teacher feedback and think aloud were also coded (see Appendix 1). In

case of teacher feedback, the coding scheme was divided to four categories: ‘utilization

(U),’ ‘no utilization (NU),’ ‘not known (NK),’ and ‘self-revision.’ The term ‘utilization’

refers to a revision triggered by teacher feedback, regardless of the degree of success. The

term ‘no utilization’ refers to a case that the writer does not refer to teacher feedback at all

when revising. The term ‘not known’ refers to a case that the writer already corrected

errors in self-revision prior to teacher feedback. The term ‘self-revision’ is a case that the

writer makes a self-revision though there is no teacher feedback on it. The ratio of

utilization of teacher feedback was calculated by the total number of utilization divided by

the total number of revision.

The utilization of think aloud was coded in terms of three categories: ‘revision while

think aloud (TA-R),’ ‘revision after a pause longer than one second (P-R),’ and ‘revision

7 When the think-aloud files were transcribed, a set of symbols was used: a question mark for a

rising intonation at the end of a phrase; a period for a closing of an idea unit, or an utterance; three dots for a pause shorter than 1 second; the brackets < > for a pause longer than 1 second without any verbalization; the number within < > for the pause length in second; the brackets { } for observation; underlining for the verbalization made while the writer is writing the text; and quotation marks for what the writer was reading aloud, the writing prompt or previously written production.

8 A senior student in the HP group (H4-1) changed the sentence of “If I study in group, I have a chance to contact with other people’s thoughts” to the sentence “It gives more chance to contact with other people’s thoughts.” She deleted the word ‘I’ and rewrote the sentence from the third- person point of view; thus, it sounded more neutral. This kind of revision was coded as a style/tone change. (In the present study, when a participant is specified, the number after the group initial (L, I, and H) refers to her university year (e.g., 4 for senior); and the number after the hyphen is the number of the participant within the group.


37

without any noticeable action (N-R).’ The term ‘revision while think aloud’ refers to a

revision triggered by think aloud, that is, a case that the writer revises his/her writing while

doing think aloud. The term ‘revision after pause longer than one second’ is a case that the

writer makes a revision after a pause longer than one second without any verbalization.

The term ‘revision without any noticeable action’ is a case when the writer revises, that

is, he/ she makes a revision while reading the first draft itself without any verbalization on

his/her thinking process. The ratio of utilization of think-aloud was calculated by the total

number of revision while think aloud divided by the total number of revision.

The degree of success was divided into three categories: ‘successful (S),’ ‘unsuccessful

(US),’ and ‘no difference/improvement (ND/I).’ The term ‘successful’ indicates that

revision enhances the quality of writing in terms of revision purpose (e.g., grammar,

vocabulary, and content) (see Appendix 1). The term ‘unsuccessful’ indicates that revision

fails to improve the writing in terms of its purpose. The term ‘no difference/improvement’

indicates that there is no significant difference or improvement after revision.

The success ratio of revision was measured by the total number of successful revision

divided by the total number of revision, whereas the failure ratio was measured by the total

number of failed revision divided by the total number of revision. The success ratio of

teacher feedback-triggered (U) revision was calculated by the total number of successful

teacher feedback-triggered revision divided by the total number of teacher feedback-

triggered revision. That of revision triggered by think-aloud was measured by the total

number of successful revision while think aloud divided by the total number of revision

while think aloud.

After two research assistants coded revision in a student’s self-revised and post-training

revised drafts with the researcher, one of the research assistants coded the self-revision and

teacher feedback-aided revision of Writing 1, while the other assistant coded the self-

revision and protocol-based revision of Writing 3. After completing the coding, they cross-

examined their coding. When there were any discrepancies about the coding, they

discussed the matters, sometimes with the researcher, and resolved the discrepancies. After

revisions were coded and cross-examined, their frequencies were counted according to

revision types. The frequency of each revision category per 100 words of the original text

besides ratio of each revision category in each revision type was counted to compare pre-

and post-training revisions and also revisions across three L2 writing proficiency groups.9

This method was used because those who write longer would revise more.

9 When revision frequency was counted per 100 words, Hall (1990), New (1999), Stevenson et al.

(2006), and Yeonhee Choi (2007) used the total number of the revised draft. In the present study, however, revision frequency was divided by the total number of the original draft to make a comparison between self-revision and post-training revision.

Yeon Hee Choi

38

Variations across proficiency levels were analyzed in writing quality, written product,

revision frequency and ratio, using one-way ANOVA.10 The ratio of content preserving

(CP) and content change (CC) revision was compared in each revised draft by Wilcoxon

test because the sample size was small and equal variance was not assumed.11 The

participants’ perception on revision training before and after training was also analyzed by

Wilcoxon test.

IV. RESULTS AND DISCUSSION

1. Quality and Product of Pre- and Post-Training Writing

The writing of each participant for the two main writing tasks including their self-

revised (pre-training) drafts (W1-SR and W3-SR) and their post-training revised drafts

(W1-TF and W3-TA) (total three writings per training) was scored holistically using six

scales to see the impact of revision training on the overall quality of writing. As shown in

Figure 4 and Table 1, the scores gradually increased from the first draft to the post-training

revised draft, especially after teacher feedback-aided revision training. Such a positive

effect was also noted in the studies on teacher feedback (Ferris, 1997; Hyland, 1998; Jihye

Lee, 2003; Jeong-Won Lee & Young-Joo Hong, 2001) and peer feedback training (Berg,

1999; Min, 2006; Sengupta, 2000). The score difference was much more noticeable

among W1, W1-SR, and W1-TF (in the teacher feedback-aided revision training) than

among W3, W3-SR, and W3-TA (in the protocol-aided revision training). The score

difference between W1 and W1-TF was larger from that between W3 and W3-TA. These

results suggest a more positive effect of teacher feedback-aided revision training. However,

a caution is required in such interpretation since the effect of the revision training order

(teacher feedback-aided revision training first and then protocol-aided revision training)

may not be excluded.

Moreover, the overall writing quality was compared before and after revision training

across the three L2 writing proficiency groups, as shown in Table 1, to see whether the

impact of revision training varies with L2 writing proficiency. The ANOVA results

show a significant difference in each of the first, self-revised, and post-training drafts in

10 The sample size was too small to run a statistical analysis, which is one main limitation of the

study. The lower-level students were only three students, as mentioned before. Nevertheless, proficiency group variations were analyzed by one-way ANOVA to see the impact of two different types of revision training across L2 writing proficiency groups.

11 No statistical analysis was conducted on writing quality, written product, and revision frequency and ratio between post-training teacher feedback-aided and protocol-aided revision nor between these revisions and self-revision because the repeated treatment effects could not be excluded.


39

FIGURE 4 Pre- and Post-Training Writing Scores

TABLE 1 ANOVA Results of Writing Scores by L2 Writing Proficiency and Post-hoc Analysis

Proficiency Level

Writing Score LP IP HP F

M SD M SD M SD

W1 score 2.17 .76 3.16 .35 4.08 .24 20.925***

A B C

W1-SR score 2.50 .50 3.28 .50 4.20 .27 14.326***

A B C

W1-TF score 2.90 .85 3.49 .43 4.20 .27

7.329**

A B

W3 score 2.83 .76 3.74 .53 4.04 .15 5.603*

A B

W3-SR score 2.90 .85 3.83 .49 4.04 .15 5.269*

A B

W3-TA score 2.90 .85 3.89 .54 4.04 .17 4.953*

A B

Note: A, B, and C indicate grouping in the post-hoc analysis in all the tables henceforth.

* p < .05, ** p < .01, *** p < .001

both training methods. The post-hoc analysis shows that the HP group outperformed the

IP and LP groups in the first draft and self-revised draft before the teacher feedback-

aided revision training, but no significant difference was noted between the HP and IP

groups in all the three drafts before and after protocol-aided revision training as well as

3.2

3.4

3.6

3.8

(writing)

(score)

Original

draft

Self-revised draft

(SR)

Post-training revised

draft (TF/TA)

3.25

3.41

3.59

3.67

3.733.76

(W3)

(W1)

3.2

3.4

3.6

3.8

(writing)

(score)

Original

draft

Self-revised draft

(SR)

Post-training revised

draft (TF/TA)

3.25

3.41

3.59

3.67

3.733.76

(W3)

(W1)

Yeon Hee Choi

40

the post-training draft of teacher feedback-revision training. The LP and IP group’s score

of W3 (the original draft before protocol-aided revision training) was much higher than

that of W1 (the original draft before teacher feedback-aided revision training). The score

difference between the first draft (W1) and the post-training draft (W1-TF) was much

larger in the LP and IP group than the HP group (LP, M = .73; IP, M =.33; and HP, M = .12).

Compared to the LP and IP group, the HP group’s scores for all the six drafts did not

vary much. These results seem to illustrate a more positive effect of teacher feedback-

aided revision training on the LP and IP group, the impact of revision training sequence

(writing and revision practice) and the ceiling effects.

The length of student written products was measured in terms of the total number of

words and sentences, as shown in Table 2. The number of words and sentences tended to

increase from the first draft of W1 to the draft of W3-TA (total number of words from

269.53 to 334.00; that of sentences from 15.00 to 20.18). These results imply that the

participants wrote longer with more sentences over the training period. The revised

drafts (W1-TF, 304.88 words and 16.88 sentences; W3-TA, 334 words and 20.12

sentences) were much longer than the first drafts (W1, 269.53 words and 15 sentences;

W3, 307.12 words and 18.65 sentences), but not than the self-revised drafts (W1-SR,

305.35 words and 16.82 sentences; W3-SR, 327.29 words and 19.59 sentences). Such

text length difference was more noticeable in teacher feedback-aided revision training

than in protocol-aided revision training. Interestingly, the first draft before protocol-

aided revision training (the second training) was slightly longer than post-training

teacher feedback-aided revision (W1-TF, 304.88 words; W3, 307.12 words). This also

suggests the effect of writing and revision practices throughout the two sessions of

revision training.

A significant variation in text length (total number of words and sentences) was also

noted in the results of one way ANOVA among the three proficiency groups in the drafts of

W1, W1-SR, and W1-TF (all of them were significant below the .01 level), because of a

significant difference between the LP and HP groups (see Table 2). Such a difference

was not found in those of W3, W3-SR, and W3-TA. This suggests that the three groups’

text length became similar after the first revision training. The LP group’s post-training

draft was much longer in teacher feedback-aided revision training, compared with the

other groups and also their post-training protocol-aided revision. Interestingly, the IP group

wrote longer than the HP group in W3, W3-SR and W3-TF. These findings imply a

significant benefit of the first revision training (teacher feedback-aided revision training)

for the LP and IP group. Moreover, they suggest writing and revision practices, as noted in

the writing scores. Before the participants submitted W3, they wrote the original, self-

revised, and post-training revised draft of W1, the revised draft of Sample Writing 2 during

the onsite training, and the original and revised draft of W2, a total of six pieces of writing.


41

TABLE 2 ANOVA Results of Writing Products by L2 Writing Proficiency and Post-hoc Analysis

Proficiency Level Length of Written Product LP IP HP F

M SD M SD M SD

Total No. of words in W1 139.00 68.55 272.11 56.91 343.20 78.56 9.137**

A B

Total No. of words in W1-SR 175.33 26.01 305.67 60.49 382.80 64.71 11.926**

A B

Total No. of words in W1-TF 217.33 70.81 298.56 42.38 368.80 61.36 7.768**

A A B

Total No. of words in W3 241.67 112.08 328.11 95.07 308.60 32.49 1.159

Total No. of words in W3-SR 256.00 102.56 358.22 99.87 314.40 23.52 1.677

Total No. of words in W3-TA 247.67 99.03 362.89 99.35 333.80 34.05 2.026

Total No. of sentences in W1 9.33 3.06 14.56 2.19 19.20 4.97 8.377**

A B

Total No. of sentences in W1-SR 11.33 1.15 15.67 3.35 22.20 4.09 10.843**

A B

Total No. of sentences in W1-TF 15.00 3.00 14.78 2.82 21.80 3.96 8.312**

A B

Total No. of sentences in W3 18.00 8.00 19.11 4.68 18.20 2.17 .091

Total No. of sentences in W3-SR 18.67 8.62 20.44 4.16 18.60 2.30 .319

Total No. of sentences in W3-TA 18.33 8.50 20.67 4.36 20.20 3.77 .244

** p < .01

Such writing and revision practices seemed to help the IP group write better before they

started the second revision training.

2. Frequency and Ratio of Pre- and Post-Training Revision

The total number of pre-training (self-revision) and post-training revision (R) in both

teacher feedback- and protocol-aided revision training (see Figure 5) illustrates that all the

participants made more revision after the training regardless of the revision methods,

except for the IP group after teacher feedback-aided training (see Table 3). Revision

frequency per 100 words before and after training displays a similar pattern noted in the

total frequency: a larger number of revision per 100 words after both teacher feedback-

and protocol-aided revision training (see Figure 6). The total number of revision after

protocol-aided revision training was larger than that after teacher feedback-aided revision,

and its difference from that of self-revision was also larger than the difference of revision

Yeon Hee Choi

42

frequency between self-revision and post-training teacher feedback-aided revision.

However, such differences were not noted in revision per 100 words.

FIGURE 5

Pre-and Post-Training Frequency of Total Revisions and Revision per 100 Words

FIGURE 6 Post-Training Revision Frequency per 100 Words by L2 Proficiency

(writing)W1-SR W1-TF W3-SR W3-TA

12.47

16.94

15.12

7.73

4.87

22.29

4.95

7.27

Total R

R per 100

words


12.47

16.94

15.12

7.73

4.87

22.29

4.95

7.27

Total R

R per 100

words


12.47

16.94

15.12

7.73

4.87

22.29

4.95

7.27

Total R

R per 100

words

(R frequency per

100 words)


4.30 4.33

7.48

6.064.80

3.73

16.56

5.22

11.44

4.79

7.00

8.51

LP

HP

IP

(R frequency per

100 words)


4.30 4.33

7.48

6.064.80

3.73

16.56

5.22

11.44

4.79

7.00

8.51

LP

HP

IP


43

TABLE 3 ANOVA Results of Pre-and Post-Training Frequency of Total Revisions and Revision per 100

Words by L2 Writing Proficiency

Proficiency Level Revision Frequency LP IP HP F

M SD M SD M SD

Total No. of R in W1-SR 5.33 3.06 13.89 7.04 14.20 7.79 1.976

Total No. of R in W1-TF 21.67 7.77 11.33 6.69 24.20 13.03 3.710

Total No. of R in W3-SR 18.67 11.59 16.11 11.45 11.20 5.63 .598

Total No. of R in W3-TA 25.67 9.02 19.11 6.49 26.00 9.70 1.551

R per 100 words in W1-SR 4.80 3.94 5.22 2.81 4.30 2.61 .155

R per 100 words in W1-TF 16.56 2.94 4.33 2.81 7.00 3.32 18.941***

A B

R per 100 words in W3-SR 7.48 1.77 4.79 2.35 3.73 2.05 2.805

R per 100 words in W3-TA 11.44 3.03 6.06 2.23 8.51 3.22 4.881*

A B A

* p < .05, *** p < .001

Proficiency level-sensitive variations were noted across the three proficiency levels in

the frequency difference between the total number of self-revision and post-training

revision (see Table 3). The LP group made much more post-training teacher feedback-

aided revision, compared with their self-revision (revision frequency difference of W1-SR

and W1-TF, M = 16.33, SD = 8.14). They also made a great deal of self revision in the

third writing (before protocol-aided revision training). These results illustrate that teacher

feedback-aided revision training led the LP group to make more revision. On the other

hand, a positive effect of protocol-aided revision training was noticeable in the HP group

(revision frequency difference of W3-SR and W3-TA, M = 14.80, SD = 6.97). The IP

group revised less after teacher feedback-aided revision training, though their writing score

of the post-training revised draft (W1-TF) was higher than that of the self-revised draft

(W1-SR).

Revision frequency per 100 words before and after training also displays proficiency-

sensitive variations (see Table 3). Statistical analyses of ANOVA reveal a significant

impact of teacher feedback-aided revision training on the LP group (see Figure 6): their

revision per 100 words was significantly larger than that of the IP and HP group after the

training. Their protocol-aided post-training revision frequency was also distinctively

higher than the IP group (see the results of the post-hoc analysis in Table 3).

Revision was also analyzed in terms of its purpose. It was classified into seven

categories (e.g., paraphrase, grammar, style, and major content change), as explained in

III.4, which was reclassified as content preserving (CP) changes (e.g., grammar and

mechanics) and content changes (CC) (macro- and micro-structure changes). As shown

Yeon Hee Choi

44

in Figure 7 and Table 4, all the participants tended to make CP revision more than CC

revision regardless of revision and training types, except for post-training protocol-aided

revision (W3-TA) due to the IP group’s CC ratio. A high frequency of local revision was

also noted in L2 writers’ online revision in New (1999), Stevenson et al. (2006), and

Yeonhee Choi (2007). CC ratio gradually arose and its average was the highest in

post-training protocol-aided revision, except for the HP group; a noticeable increase

was found in the LP and IP group. A significant difference between the ratio of CP and

CC was found by repeated measures ANOVA because of the significantly higher CP

revision ratio in both W1 self-revision (CP, M = .63; CC, M = .37; F = 7.106, p = .018)

and post-training teacher feedback-aided revision (CP, M = .61; CC, M = .39; F =

11.663, p = .004), but not in W3 self-revision (CP, M = .58; CC, M = .42) and post-

training protocol-aided revision (CP, M = .49; CC, M = .51). A much higher CC ratio of

post-training protocol-aided revision was noted than that of post-training teacher

feedback-aided revision. These results imply that the more L2 writers are trained for

revision, the more they make content changes, that is, the quality change in their

revision, which has been noted in Berg’s (1999) and Min’s (2006) peer review training.

The ratio of grammar changes was much higher in post-training teacher feedback-aided

revision (M = .38) than in post-training protocol-aided revision (M = .17), while that of

micro-level content changes was noticeably higher in post-training protocol-aided

revision (M = .46) than post-training teacher feedback-aided revision (M = .25). The

high ratio of grammar changes might have been led by the nature of teacher feedback.

That is, among the seven categories of teacher feedback given to the participants,

grammar feedback was the most frequent (e.g., grammar, M = 12.65; word choice, M =

4.88; content, M = .82; organization, M = 2.12). This might provide an account for the

fact that in post-training teacher feedback-aided revision, the HP group’s CP revision ratio

FIGURE 7 Pre- and Post-Training CP and CC Revision Ratio

(ratio)

(writing)

0.630.61

0.58

0.49

0.37

0.51

0.42

0.33

CC

CP

(ratio)

(writing)

0.630.61

0.58

0.49

0.37

0.51

0.42

0.33

CC

CP


45

TABLE 4 ANOVA Results of Pre-and Post-Training Revision Ratio and Frequency per 100 Words across

Revision Purposes by L2 Writing Proficiency

Revision Ratio or Revision

per 100 Words

by Purpose Types

Proficiency Level

Total LP IP HP F

M SD M SD M SD M SD

CP ratio in W1-SR .63 .19 .73 .29 .66 .17 .51 .15 1.481

CP ratio in W1-TF .61 .25 .63 .35 .59 .23 .79 .06 2.958

CP ratio in W3-SR .58 .21 .65 .14 .60 .19 .50 .27 .504

CP ratio in W3-TA .49 .15 .54 .06 .44 .18 .53 .13 .814

CC ratio in W1-SR .37 .19 .27 .29 .34 .17 .49 .15 1.481

CC ratio in W1-TF .33 .21 .37 .35 .40 .21 .21 .06 1.403

CC ratio in W3-SR .42 .21 .36 .14 .40 .19 .50 .27 .504

CC ratio in W3-TA .51 .154 .46 .06 .56 .18 .47 .13 .814

CP per 100 words in W1-SR 3.05 1.66 2.95 1.60 3.39 1.70 2.49 1.82 .452

CP per 100 words in W1-TF 4.35 4.23 9.80 7.25 2.26 1.59 4.83 2.83 5.752*

2.96 A B A

CP per 100 words in W3-SR 2.96 1.94 5.10 2.46 2.66 1.47 2.20 1.81 2.869

CP per 100 words in W3-TA 4.00 2.22 6.16 1.80 2.81 1.56 4.84 2.38 4.337*

2.05 A B A

CC per 100 words in W1-SR 1.99 1.66 2.35 3.25 1.92 1.52 1.91 .97 .074

CC per 100 words in W1-TF 2.04 2.04 4.63 4.00 1.61 .91 1.24 .82 4.229*

A B

CC per 100 words in W3-SR 2.05 1.07 2.58 .78 2.13 1.27 1.59 .77 .835

CC per 100 words in W3-TA 3.73 1.34 5.28 1.46 3.24 1.12 3.67 1.14 3.348

A B A

* p < .05

outnumbered their CC revision ratio and also the LP and HP group’s CP revision ratio. On

the other hand, the high ratio of micro-level content changes might be an impact of

repeated revision training or that of protocol-based revision training.

Both CP and CC revision frequency per 100 words was higher after each training than

in self-revision (see Table 4). Compared with CP revision per 100 words in self-revision, a

larger number of CP revisions in post-training revision was noted. CC revision frequency

per 100 words in post-training protocol-aided revision outnumbered that in W3

self-revision and also that in post-training teacher feedback-aided revision.

A comparison of the three proficiency groups illustrates the LP and IP group’s higher

ratio of CP revision over their ratio of CC revision, except for the IP group’s post-training

protocol-aided revision; and also their higher ratio of CP revision in self-revision. A

decline was noted in their ratio of CP revision after each revision training, while an

Yeon Hee Choi

46

increase was found in their ratio of CC revision. This suggests an influence of revision

training on the LP and IP group’s revision purpose, that is, they made more attempts to

make global revisions. On the contrary, the HP group’s ratio of CP revision increased after

each revision training, especially in their post-training teacher feedback-aided revision,

while their ratio of CC revision dropped from .49 to .21 in post-training teacher

feedback-aided revision and slightly increased from .50 to .53 in post-training protocol-

aided revision. These findings are contradictory to those from Zamel’s (1983) pen-and-

paper ESL writing, and van Waes and Schellens’ (2003) and Yeonhee Choi’s (2007)

online FL writing (more proficient L2 writers make more higher-level revisions). This can

be explained by the nature of the teacher feedback they received. As mentioned above,

most of the teacher feedback focused on grammar and word choice including awkward

expressions, which might have triggered CP revisions. Within the three proficiency groups,

the HP group (feedback on grammar, M = 17.80; word choice, M = 8.20) received these

types of teacher feedback more frequently than the other two groups (IP, feedback on

grammar, M = 11.78; IP, word choice, M = 5.00; LP, feedback on grammar, M = 6.67; LP,

word choice, M = 6.00) because their writing was much longer. This might have

influenced their teacher feedback-aided revision.

The ANOVA results of comparing the three groups in the revision frequency per 100

words illustrate group variations, as shown in Table 4. A much larger number of CP

revision was noted in the LP group’s post-training teacher feedback-aided revision,

which might be accounted for by their high frequency of grammar revision (M = 6.04).

After revision training, moreover, the HP and LP group’s revision patterns became more

similar, as noted by the post-hoc analysis. Interestingly, that of the IP group appears

distinctive from the other groups: their CP and CC revision frequency declined after

teacher feedback-aided revision.

3. Success Ratio of Pre- and Post-Training Revision and Utilization Ratio

of Teacher Feedback and Protocol

To see the effect of revision training, the extent to which the participants successfully

revised their first draft after revision training was measured. Compared with the self-

revision, the success ratio of revision after teacher feedback-aided revision training

declined, while the failure ratio increased (see Table 5). It was more noticeable in more

proficient L2 writers (the IP and HP group). On the contrary, it was the opposite after

protocol-aided revision. These results imply a more positive effect of protocol-aided

revision training. But, a caution is required since it is plausible that the declined success

ratio and the increased failure ratio in teacher feedback-aided revision might have resulted

from the fact that a larger number of revisions were triggered by teacher feedback, but the


47

participants did not have enough writing ability to revise successfully.

TABLE 5 ANOVA Results of Success and Failure Ratio of Pre- and Post-Training Revision and

Utilization Ratio of Teacher Feedback and Protocol by L2 Writing Proficiency

Success, Failure, and

Utilization Ratio

Proficiency Level

Total LP IP HP F

M SD M SD M SD M SD

Success ratio in W1-SR .76 .16 .71 .26 .77 .17 .79 .10 .235

Success ratio in W1-TF .71 .28 .81 .17 .66 .38 .73 .06 .299

Success ratio in W3-SR .76 .17 .54 .29 .77 .11 .87 .09 4.966*

A B

Success ratio in W3-TA .81 .12 .73 .13 .82 .13 .83 .10 .840

Failure ratio in W1-SR .20 .17 .29 .26 .19 .17 .17 .13 .499

Failure ratio in W1-TF .23 .23 .19 .17 .23 .30 .24 .09 .036

Failure ratio in W3-SR .11 .09 .17 .08 .12 .08 .05 .07 2.644

Failure ratio in W3-TA .10 .08 .18 .10 .09 .09 .09 .05 1.561

Utilization ratio of teacher

feedback in W1-TF

.51 .21 .54 .29 .44 .21 .60 .17 .935

Successful utilization ratio of

teacher feedback in W1-TF

.65 .33 .84 .16 .63 .38 .58 .33 .556

Utilization ratio of protocol in

W3-TA

.59 .20 .67 .12 .60 .23 .52 .21 .537

Successful utilization ratio of

protocol in W3-TA

.78 .19 .70 .18 .81 .17 .77 .25 .400

* p < .05

In teacher feedback-aided revision, interestingly, the LP revised more successfully than

the IP and HP group, which was different from the questionnaire survey on correct

incorporation of teacher feedback in Lee (2008) (a more positive response by higher-level

Hong Kong secondary school students). They recognized their problems more since they

were pointed out by the teacher; thus, they made an attempt to revise them. These attempts

were relatively successful, as shown by their high successful utilization ratio of teacher

feedback in W1-TF (see Table 5). On the other hand, the higher level students were

able to recognize their problems by themselves and revise them in their self-revision. The

problems which were not self-recognized might have been beyond their ability to revise

successfully. Their successful utilization ratio of teacher feedback was much lower than

the LP group’s. A significant difference in the success ratio of revision in the self-revision

of W3 (W3-SR) was noted between the LP group and the IP and HP group, but not in the

post-training protocol-aided revision of W3 (W3-TA). These results also suggest that the

LP group can take more advantage of revision training than the other two groups. However,

Yeon Hee Choi

48

the IP and HP group’s higher successful utilization ratio of protocol imply that protocol-

aided revision training may be more helpful for more proficient L2 writers.

4. L2 Writer Perception on Revision Training

A pre-questionnaire showed that all the participants had some experience of revising

their own writing, but only nine out of seventeen participants received formal instruction

on revision. Teacher and peer feedback-aided revision were the most frequently

employed revision methods (21 responses out of 34 responses) and they were followed

by protocol-aided revision (eight responses out of 34 responses). The participants

indicated that they would revise their writing mostly in the areas such as organization

and structure. In the responses to the post-questionnaires, the difficulties they pointed

out in teacher feedback-aided revision were that teacher feedbacks were incomprehensible

(11 responses) and unspecific (seven responses), as the illegibility problem of teacher

feedback was noted in Sommers (1982) and Lee (2008). The difficulties they pointed out

in protocol-aided revision were that the participants did not know how to revise (six

responses out of 17 participants) and they did not know appropriate expressions due to

their lack of writing and grammatical competence (10 responses out of 17 participants).

Their lack of writing and grammatical competence was indicated as the main cause of

their failure to revise successfully in both teacher feedback-aided and protocol-aided

revision.

As to the question of effectiveness of each revision method, 13 participants pointed

out that teacher feedback was helpful because they were corrective and they were given

by a teacher, i.e., a writing specialist. Four participants mentioned that protocol-aided

revision was more effective than teacher feedback-aided revision and one of them

pointed out reader awareness as a helpful factor to revise successfully. With regard to

long-term effects on revision, the majority preferred teacher feedback-aided revision (13

participants). Nevertheless, nine participants also preferred protocol-aided revision

because revision could be done at any time by the writers (one participant) and is an

individual work (four participants).

The responses of participants to a pre-questionnaire and two post-questionnaires were

compared to see the participants’ perceptions on revision training and their relationships

with revision and writing quality. The results of Wilcoxon test reveal that after revision

training the participants responded that they revised in their teacher feedback-aided

revision significantly more than in their protocol-aided revision at the .018 level (see Table

6 and Figure 8). After revision training, however, their perception on the positive effect of

teacher feedback-aided revision significantly dropped at the .005 level and their perception

on its utilization in revision also downgraded, whereas their perception on the positive


49

effect of protocol-aided revision significantly enhanced at the .024 level and their perception

on its utilization in revision also significantly increased at the .046 level. The participants’

reader awareness enhanced after revision training, but no significant change was noted.

FIGURE 8 Pre- and Post-training Questionnaire Responses on the Effects and Utilization of Teacher

Feedback- and Protocol-aided Revision and Reader Awareness

TABLE 6 Pre- and Post-training Questionnaire Responses by L2 Writing Proficiency

Questions Pre-/Post-

training

Total

Proficiency Level

LP IP HP

M SD M SD M SD M SD

Positive effects of

teacher feedback

Pre-training 5.35 .60 5.33 .58 5.44 .53 5.20 .83

Post-training 4.00 1.27 4.00 2.00 3.78 1.30 4.40 .89

Positive effects of

think-aloud

Pre-training 3.47 .80 2.67 .58 3.67 .71 3.60 .89

Post-training 3.88 1.32 4.67 .57 3.33 1.32 4.40 1.34

Use of teacher feedback Pre-training 4.94 .56 4.67 .58 5.11 .33 4.80 .84

Post-training 4.47 1.01 3.33 2.08 4.56 .53 5.00 .00

Use of think-aloud Pre-training 3.29 .77 2.67 .58 3.33 .71 3.60 .89

Post-training 3.76 .75 3.33 1.52 3.78 .67 4.00 .00

Reader awareness Pre-training 3.47 .80 3.00 1.00 4.00 .87 3.60 .89

Post-training 3.76 1.20 3.33 1.15 3.78 1.39 4.00 1.00

(question)Positive

effects of

teacher

feedback

5.35

3.29

3.76

3.47

4.474.00 3.88

4.94

(responses)

Positive

effects of

think-aloud

Use of

teacher

feedback

Use of

think-aloud

Reader

awareness

3.71

3.76

(question)Positive

effects of

teacher

feedback

5.35

3.29

3.76

3.47

4.474.00 3.88

4.94

(responses)

Positive

effects of

think-aloud

Use of

teacher

feedback

Use of

think-aloud

Reader

awareness

3.71

3.76

Yeon Hee Choi

50

The comparison of the three proficiency groups’ responses reveals that compared with

those of HP and LP groups, the IP group showed a drastic change in the positive effects

of teacher feedback-aided revision, which dropped from 5.44 to 3.78. On the other hand,

all the responses to the categories of protocol-aided revision became more positive after

training regardless of the groups (except for the IP group’s responses on its

positive effects). A large change was noted in the perceptions of LP group on

protocol-aided revision. Their perception of positive effects of think-aloud on revision

increased from 2.67 to 4.67. A major group variation was noted between the IP group’s

responses and those of the HP and LP group. Except for one category, the use of

think-aloud in revision, the IP group showed negative responses to all the other

categories such as the effects of teacher feedback and that of think-aloud, the use of

teacher feedback, and reader awareness.

V. CONCLUSION

The findings from the study suggest the positive effects of revision training on writing

quality. Teacher feedback-aided revision training displayed more positive effects on the

quality and product of writing than protocol-based revision training. As mentioned above,

this might be accounted for by the sequence of training, since teacher feedback-aided

revision training preceded protocol-aided revision training. Thus, more research is needed

if the former influences revision more effectively than the latter when it is investigated

with two independent groups. As for the scores of the drafts, moreover, the HP group’s

score remained relatively steady, whereas that of the LP and IP group continuously

increased. Thus, the latter groups seem to benefit more from revision training.

In addition, text length differences were more noticeable in teacher feedback-aided

revision training than in protocol-aided revision training. The three groups’ text length

became similar after the first revision training. The LP group’s increase in text length

outnumbered the other groups’ in teacher feedback-aided revision training. These

findings imply a significant benefit of teacher feedback-aided revision training for

lower-level L2 writers.

The comparison of the total number of pre- and post-training revision and revision per

100 words also illustrates the positive effects of revision training regardless of the

revision training types. The total number of revision after protocol-aided revision

training outnumbered that after teacher feedback-aided revision. Compared with the HP

and IP group, the LP group’s noticeable frequency difference between pre- and post-

training revision and the large number of their self revision in the third writing suggest

that teacher feedback-aided revision training led the LP group to make more revision.


51

On the other hand, a positive effect of protocol-aided revision training was noticeable in

the HP group. This implies that effects of different types of revision training vary with

L2 proficiency groups.

A positive effect of protocol-aided revision was noted in the analysis of revision by its

purpose, especially for the lower-level participants. In general, content preserving

revisions outnumbered content changes; however, the ratio of content changes gradually

enhanced and it outnumbered that of content preserving revisions in post-training

protocol-aided revision. The increase was more noticeable in the LP and IP group than

in the HP group. These results imply that the more L2 writers are trained for revision,

the more global revisions they make. Moreover, the HP group’s higher ratio of content

preserving revision in post-training teacher feedback-aided revision than in their self-

revision suggests the impact of the nature of teacher feedback: the high frequency of

feedback on grammar might have triggered local revisions because their writing was

much longer.

The comparison of the success ratio of pre- and post-training revision illustrates a

declined success ratio of post-training teacher feedback-aided revision in the more

proficient L2 writers and an increased success ratio of post-training protocol-aided

revision in the less proficient L2 writers. These results might imply a negative effect of

teacher feedback-aided revision training for higher-level L2 writers but a positive effect

of protocol-aided revision training for lower-level L2 writers. However, a caution is

required in their interpretation because a possibility cannot be excluded that lower-level

writers revise their writing problems explicitly indicated by their teacher which they are

not able to notice by themselves, whereas higher-level writers’ problems pointed out by

the teacher are beyond their competence since they can revise self-recognized problems.

This account seems to be supported by the fact that the higher-level participants’

successful utilization ratio of teacher feedback was much lower than the lower-level

ones.

Although more participants still preferred teacher feedback-aided revision over

protocol-aided revision after training and they utilized teacher feedback more, the results

of the questionnaire survey suggest a more positive effect of protocol-aided revision

training than teacher feedback-aided revision training. In general, the incorporation rate

of teacher feedback after training was lower than that of peer feedback after training in

Yunkyoung Cho (2005) and Min (2006). One of the main reasons seems to be the

incomprehensibility and unspecificity of teacher feedback, as pointed out by the

participants. This implies the importance of the quality or nature of teacher feedback in

teacher feedback-aided revision; thus, the future research on teacher feedback-aided

revision training should take a consideration on what kind of teacher feedback to use

Yeon Hee Choi

52

and how to solve the problems raised by its incomprehensibility or miscommunication

between the teacher and the L2 writer.

The variation in reader awareness before and after revision training was not much

noticeable in the questionnaire survey. The previous L2 writing studies on reader

awareness such as Raimes (1985) and Yeonhee Choi (2006), which provided a prompt

with specified audience, also noted that in L2 writing raising reader awareness was not a

simple process. More studies are thus needed to see whether revision training including

teacher feedback- or protocol-aided revision can enhance reader awareness, especially

for more proficient L2 writers, as noted in Hayes et al. (1987), and suggested in Jones

and Myhill (2007).

Think-aloud, concurrent verbal protocols, is not commonly used to help L2 writers

revise their writing. The findings from the present study suggest that protocol-based

revision can be an alternative to teacher or peer feedback-aided revision. To explore its

effects further, a comparison among different revision types is needed with an

independent larger-sample group for each type. Such a comparison would shed light on

their effects on writing quality and revision and make a more conclusive suggestion on

effective types of revision for different L2 writing proficiency groups.

References

Berg, E. C. (1999). The effects of trained peer response ESL students’ revision types and

writing quality. Journal of Second Language Writing, 8, 215-241.

Bitchener, J. (2008). Evidence in support of written corrective feedback. Journal of Second

Language Writing, 17, 102-118.

Bitchener, J., & Knoch, U. (2009). The relative effectiveness of different types of direct

written corrective feedback. System, 37, 322-329.

Cho, Yunkyoung. (2005). The effectiveness of peer feedback in Korean EFL writing

classrooms. English Language Teaching, 17, 33-59.

Choi, Huyyun. (2002). A study on Korean students’ perceptions on teachers’ feedback in

English writing classes. Unpublished master’s thesis. Sookmyung Women’s

University, Graduate School of TESOL, Seoul.

Choi, Yeonhee. (2006). Effects of L2 writing proficiency and reader awareness on EFL

students’ writing product and process on-line: Computer-based observation. The

Journal of Asia TEFL, 3(2), 37-78.

Choi, Yeonhee. (2007). On-line revision behaviors in EFL writing process. English

Teaching, 62(4), 69-93.


53

Conrad, S. M., & Goldstein, L. M. (1999). ESL student revision after teacher-written

comments: Text, contexts, and individuals. Journal of Second Language Writing, 8,

147-180.

Ellis, R., Sheen, Y., Murakami, M., & Takashima, H. (2008). The effects of focused and

unfocused written corrective feedback in an English as a foreign language context.

System, 36, 353-371.

Faigley, L., & Witte, S. (1981). Analyzing revision. College Composition and

Communication, 32, 400-414.

Ferris, D. R. (1995). Student reactions to teacher response in multiple-draft composition

classrooms. TESOL Quarterly, 29, 33-53.

Ferris, D. R. (1997). The influence of teacher commentary on student revision. TESOL

Quarterly, 31, 315-339.

Ferris, D. R. (2003). Responding to writing. In B. Kroll (Ed.), Exploring second language

writing (pp. 119-140). Cambridge: Cambridge University Press.

Goldstein, L. M., & Conrad, S. M. (1990). Student input and negotiation of meaning in

ESL writing conferences. TESOL Quarterly, 24, 443-460.

Hall, C. (1987). Revision strategies in L1 and L2 writing tasks: A case study. Unpublished

doctoral dissertation. University of New Mexico, Albuquerque, NM.

Hall, C. (1990). Managing the complexity of revising across languages. TESOL Quarterly,

24, 43-60.

Hayes, J. R. (1996). A new framework for understanding cognition and affect in writing.

In C. M. Levy & S. Ransdell (Eds.), The science of writing: Theories, methods,

individual differences and applications (pp. 1-27). Mahwah, NJ: Lawrence

Erbaum.

Hayes, J. R. (2000). A new framework for understanding cognition and affect in writing.

In R. Indrisano & J. R. Squire (Eds.), Perspective on writing: Research, theory, and

practice (pp. 6-44). Newark, DE: International Reading Association.

Hayes, J. R., Flower, L, Schriver, K., Stratman, J., & Carey, L. (1987). Cognitive

processes in revision. In S. Rosenberg (Ed.), Advances in applied psycholinguistics,

Vol 2: Reading, writing and language learning (pp. 176-240). Cambridge, MA:

Cambridge University Press.

Hedgcock, J., & Lefkowitz, N. (1994). Feedback on feedback: Assessing learners'

receptivity to teacher response in L2 composing. Journal of Second Language

Writing, 3, 141-163.

Holliway, D. R., & McCutchen, D. (2004). Audience perspective in young writers'

composition and revising. In L. Allal, L. Chanquoy, & P. Largy (Eds.), Revision:

Cognitive and instructional process (pp. 87-101). Boston, MA: Kluwer Academic

Publishers.

Yeon Hee Choi

54

Hyland, F. (1998). The impact of teacher-written feedback on individual writers. Journal

of Second Language Writing, 7, 255-186.

Hyland, F., & Hyland, K. (2001). Sugaring the pill: Praise and criticism in written

feedback. Journal of Second Language Writing, 10, 185-212.

Jones, S., & Myhill, D. A. (2007). Discourses of difference? Questioning gender

differences in linguistic characteristics of writing. Canadian Journal of Education

30(2), 456-482.

Jourderais, R. (2001). Protocol analysis and SLA. In P. Robinson (Ed.), Cognition and

second language acquisition (pp. 354-375). New York: Cambridge University

Press.

Lee, I. (2008). Student reactions to teacher feedback in two Hong Kong secondary

classrooms. Journal of Second Language Writing, 17, 144-164.

Lee, Jihye. (2003). Hanguk daehaksaeng-euy yeongeo gulssugi sujeong jeonryak

sayong-ey gwanhan saryey yeongu (A case study on the revision strategies of

Korean university students’ English composition). Unpulished master’s thesis.

Ewha Womans University, Seoul.

Lee, Jeong-Won, & Hong, Young-Joo. (2001). Gwajeongjooshim feedback-i yeongeo

ssuginungryeok hyangsang-ey michinun yeonghyang (The effects of process-

based feedback on the development of English writing proficiency). English

Teaching, 56(2), 265-285.

Leki, I. (1990). Coaching from the margins: Issues in written response. In B. Kroll (Ed.),

Second language writing: Research insights for the classroom (pp. 57-68).

Cambridge: Cambridge University Press.

Lindgren, E., & Sullivan, K. P. H. (2006). Analysing online revision. In K. P. H. Sullivan

& E. Lindgren (Eds.), Studies in writing, Vol. 18: Computer keystroke logging and

writing (pp. 157-188). Oxford: Elsevier.

Liou, H.-C., & Peng, Z.-Y. (2009). Training effects on computer-mediated peer review.

System, 37, 514-525.

Lundstrom, K., & Baker, W. (2009). To give is better than to receive: The benefits of peer

review to the reviewer’s own writing. Journal of Second Language Writing, 18,

30-43.

Min, H.-T. (2005). Training students to become successful peer reviewers. System, 33,

293-308.

Min, H.-T. (2006). The effects of trained peer review on EFL students’ revision types and

writing quality. Journal of Second Language Writing, 15, 118-141.

Nelson, G., & Carson, J. (1998). ESL students' perceptions of effectiveness in peer

response groups. Journal of Second Language Writing, 7, 113-131.


55

New, E. (1999). Computer-aided writing in French as a foreign language: A qualitative

and quantitative look at the process of revision. The Modern Language Journal, 83,

80-97.

Phinney, M., & Khouri, S. (1993). Computers, revision, and ESL writers: The role of

experience. Journal of Second Language Writing, 2, 257-277.

Raimes, A. (1985). What unskilled ESL students do as they write: A classroom study of

composing. TESOL Quarterly, 19, 229-258.

Sachs, R., & Polio, C. (2007). Learners’ uses of two types of written feedback on L2

writing revision tasks. Studies in Second Language Acquisition, 29, 67-100.

Sengupta, S. (2000). An investigation into the effects of revision strategy instruction on L2

secondary school learners. System, 28, 97-113.

Sommers, N. (1980). Revision strategies of student writers and experienced adult writers.

College Composition and Communication, 31, 378-388.

Sommers, N. (1982). Responding to student writing. College Composition and

Communication, 33, 148-156.

Stanley, J. (1992). Coaching student writers to be effective peer evaluators. Journal of

Second Language Writing, 1, 217-233.

Stevenson, M., Schoonen, R., & de Glopper, K. (2006). Revising in two languages: A

multi-dimensional comparison of online writing revisions in L1 and FL. Journal of

Second Language Writing, 15, 201-233.

Stratman, J. F., & Hamp-Lyons, L. (1994). Reactivity in concurrent think-aloud protocols:

Issues for research. In P. Smagorinsky (Ed.), Speaking about writing: Reflections

on research methodology (pp. 89-112). Thousand Oaks, CA: Sage.

Truscott, J. (1996). The case against grammar correction in L2 writing classes. Language

Learning, 46, 327-369.

Van Waes, L., & Schellens, P. J. (2003). Writing profiles: The effect of the writing mode

on pausing and revision patterns of experienced writers. Journal of Pragmatics, 33,

829-853.

Yang, M., Badger, R., & Yu, Z. (2006). A comparative study of peer and teacher feedback

in a Chinese EFL writing class. Journal of Second Language Writing, 15, 179-200.

Zamel, V. (1982). Writing: The process of discovering meaning. TESOL Quarterly, 16,

195-209.

Zamel, V. (1983). The composing process of advanced ESL students: Six case studies.

TESOL Quarterly, 17, 165-187.

Zamel, V. (1985). Responding to student writing. TESOL Quarterly, 19, 79-102.

Yeon Hee Choi

56

Appendix 1 Coding Scheme Used to Classify the Degree of Success in Revision

Categories Examples

Successful

(S)

(I9-1) ‘When I study in a group to get bad results,’ to ‘When I got bad results in

group study,’

Unsuccessful

(US)

(I3-4) ‘Thinking in various ways enables creative thinking’ to ‘Thinking in

various ways possible creative thinking’

No Difference/

Improvement

(L2-1) ‘To solve problems is easy in a study-group’ to ‘Solving problems is easy in

a study-group’

(ND/I) (I6-3) ‘…it needs much time to cooperate with others rather than do it alone’ to

‘…it needs less time to deal with problem than with a group’ {Here ‘it’

refers to ‘studying alone.’}

Applicable levels: tertiary

Key words: revision training, teacher feedback, think-aloud, protocol, self-revision, L2 writing

Yeon Hee Choi

Dept. of English Education

Ewha Womans University

11-1 Daehyun-dong, Seodaemun-ku

Seoul, 120-750, Korea

Tel: (02) 3277-2655

Email: [email protected]

Received in December, 2009

Reviewed in January, 2010

Revised version received in February, 2010

effects of revision training on l2...

Documents