xxiv panama tesol annual congress 2010 re-imagining …€¦ · · 2010-11-20re-imagining your...

XXIV Panama TESOL Annual Congress 2010

Re-Imagining Your English Class: Pursuing Professional Excellence

Choosing a good scale to evaluate students’ writing

Presented by

M.A Jorge Luis Espinoza Campos

September 2010

XXIV Panama Tesol Annual Congress 2010M.A Jorge Luis Espinoza

ABSTRACT

Teaching writing can become a very complicated aspect for both EFL

teachers and students. In fact, many EFL students feel that they do not have the

background knowledge (input) to write properly not even in their own language,

since they have been not well assessed previously. On the other hand, teachers

always face difficulties finding the appropriate instruments to evaluate the

learners’ writing which makes their role harder. For this reason, as EFL teachers,

we need to know how to choose and use the writing evaluation scales

appropriately taking into consideration the students’ proficiency level and their

needs to make this process easier.

2


Introduction

Throughout my experience as a language teacher, grading writing is one of

the aspects that requires more attention on the instructor’s part. Previously, I

experienced some particular situations grading students’ written work and that

motivated me to develop this work to choose a good scale to evaluate students

writing and to help other instructors to do it. This work pretends to show EFL

instructors different ways for finding good scales to grade students’ writing using

the appropriate criteria based on the assignment and the course objectives. To

achieve that, this paper includes different questions that guide the way you select

the appropiate writing scales for each of your classes. The work also emphasizes

the way you need to use the evaluation instruments, so that your grading criteria

can be reliable and consistent. In addition, I will include some steps you can

follow to create your own writing evaluation scales in case that you can’t find a

suitable instrument according to your students’ needs. Finally it is important to

know if teachers become aware of the elements previously mentioned, so that

teaching writing can be much easier and rewarding for ourselves and for the

students.

3


Choosing a good scale to evaluate students’ writing

What is a writing scale?

The TLT group defines a writing rubric as an explicit set of criteria used for

assessing the performance and quality of a written work. A good rubric should

include levels of potential achievement, the aspects that are going be graded with a

set of descriptors and a space for the professor to describe the reasons for each

judgment or to make suggestions for the author. Additionally, the scale should

provide the numerical ranks to evaluate each of its descriptors.

According to Merriam Webster’s collegiate dictionary a writing rubric “is

a scoring guide that describes criteria for student performance and differentiates

among different levels of performance within those criteria. Because rubrics set

forth specific criteria, define precise requirements for meeting those criteria, and

often assign numerical scores to each level of performance, they provide teachers

with an effective, objective method for evaluating skills that do not generally lend

themselves to objective assessment methods.” (Merriam Webster Collegiate

Dictionary, 2008). Generally, writing rubrics are commonly used to grade students’

performance on essays, papers, projects, and other assignments.

In summary, as language teachers we may understand a writing scale as a printed

set of scoring guidelines (criteria) for assessing students’ written work

(compositions, essays or papers) and for giving feedback.

4


Why is writing assessment so complicated for teachers and students?

In fact, assessing EFL student’s writing is not easy and sometimes it gets

more complicated of what we as teachers expect. According to H. Douglas Brown

the evaluation of writing is a thorny issue because “If you are a guide and a

facilitator of students’ performance in the ongoing process of developing a piece

of written work, how can you also be the judge? What do you judge?” (Brown,

2001, p. 356). Due to lack of criteria, as EFL teachers we commly attempt to be a

judge and a guide at the same time and that is impossible. However, as instructors

we don’t understand that what Brown mentions is impossible to achieve. For this

reason, EFL teachers tend to use their own evaluation criteria to assess students’

writing without establishing clear objectives and the points are going to be

evaluated. As a result of this, our students will not be able to understand our

evaluation criteria, and writing becomes a frustrating process for many of them.

To avoid this problem, EFL professionals need to be fair and sensitive in their

evaluations.

In Richards and Renandya 2002, Ann Raimes states another reason why

teaching writing turns complicated and not productive. This happens because

teachers do not question what administrators and programs state about writing.

Sometimes, there are programs do not give any importance to writing and because

of that students are never asked to write or they write just occasionally not learn

how to write. Raimes mentions that teachers need to question what the programs

mention if we really want our EFL students to learn to write. Raimes recommends

5


asking ourselves as teachers these questions “Do your students have to pass an

exam that values writing to a formula that rewards above all accuracy of

grammar, spelling and punctuation? Do they even have to compose at all, or just

write sentences, judge grammaticality or pick from multiple-choice responses?”

(Renandya & Richards, 2002, p. 306).

These are two great questions because many professors grade students’

written work just paying attention to grammar, punctuation and spelling. Is that

really helping learns to write? Of course not, we need to teach students to

communicate their ideas based on their experiences and knowledge. To achieve

this, teachers need to teach writing as a process establishing specific goals based

on the students’ needs and their level. By and large, teaching writing implies many

things, but all depends on the course objectives and the goals that we as teachers

have.

Why do teachers need to use a writing scale?

Before starting to choose a writing scale it is important to clarify why it is

important to evaluate students’ written performance using an instrument.

Concretely, we should use writing rubrics because they are one of the best ways to

improve our assessment criteria considering the students’ needs and the course

objectives. Scales allow professors to be fair and explicit because they provide the

aspects that are going to be evaluated and the descriptors so that the students can

easily understand how they are going to be evaluated. Rubrics allow for

6


standardised evaluation according to specified criteria, which makes grading

simpler and more transparent.

Additionally, a scale provides teachers with the exact characteristics of the

performance level that is being evaluated. Another good reason to use a rubric to

assess students writing is that it gives the students clear information about how

well or bad they performed on different aspects. All what students need to do is to

check the descriptors to see their performance explanations. Lastly, another

advantage for students is that rubrics show the areas that need improvement and

their strenghts. This allows students to know what they need to improve in the

future. By and large, the use of rubrics is a great mechanism that enriches and

makes the writing process clearer, more meaningful and less complicated for both

teachers and students.

What aspects do teachers need to take into account before choosing a writing

scale?

There are a lot of aspects that we need to take into account to choose a

rubric or scale to evaluate our students’ writing.

First, teachers need to decide what kind of rubric they need to use to

evaluate student’s writing: holistic or analytic. Using both kinds is highly

recommended.

Holistic Rubric: A Holistic rubric does not list separate levels of performance for

each criterion. It is mostly used to evaluate writing as a whole process without

breaking into separate parts. According to Oakleaf holistic scales “provide a

7

http://en.wikipedia.org/wiki/Standardised_testing


single, overall judgment of quality. They are faster to use, less burdensome for

large-scale assessments, and usually sufficient for evaluating simple artifacts of

learning.” (Oakleaf, 2009, p. 2)

Here is an example of a holistic rubric.

Rubric figure taken from O’Malley & Valdez Pierce, 1996: p 143 just for illustration purposes.

8


When to use a holistic rubric?

Jon Mueller states “holistic rubrics tend to be used when a quick or gross

judgment needs to be made. If the assessment is a minor one, such as a brief

homework assignment, it may be sufficient to apply a holistic judgment (e.g.,

check, check-plus, or no-check) to quickly review student work. But. holistic

rubrics can also be employed for more substantial assignments. On some tasks it

is not easy to evaluate performance on one criterion independently of

performance on a different criterion.” (Mueller, 2010)

Analytic Rubric: “An analytic rubric articulates levels of performance for each

criterion so the teacher can assess student performance on each criterion.”

(Mueller, 2010) Therefore, this rubric provides scores for various skills or parts of

the product or process, then combines the scores to obtain an overall total.

Analytic scales provide more detailed assessment data and they are better for

evaluating complex artifacts of learning. The scoring of this rubric gives more

information to the student and teacher about the specific strengths and weaknesses

of the student's skills than does a holistic rubric. (Taken from the Alabama

Development Modules online document)

This is an example of an analytic rubric.

9


Taken from O’Malley & Valdez Pierce, 1996: p 145 just for illustration purposes

10


When to use an analytic rubric?

Mueller suggests that teachers can use an analytic rubric when they need to

“assess each criterion separately, particularly for assignments that involve a

larger number of criteria. It becomes more and more difficult to assign a level of

performance in a holistic rubric as the number of criteria increases.” (Mueller,

2010)

Here other relevant aspects to consider before choosing a rubric to evaluate the

writing skills of your students:

2. A good rubric addresses the task performance objectives originally stated.

3. A scale has to include at least five general levels of performance like: content,

organization, vocabulary, mechanics (spelling and punctuation, format), grammar

or syntax, language use, conventions, the task among others.

4. Lower performance level labels should be descriptive, not discouraging.

5. The rubric should use a fair range to rate performance.

6. The scale should be suitable to the students’ level and the type of writing task or

assignment requested. For example, if you are evaluating a short story the scale

should be appropriate to evaluate that type of task.

7. The instrument should be applicable to a variety of tasks.

8. the instrument should provide all students with an opportunity to succeed at

some level.

9. Each level of performance should have at least three descriptors.

11


10. The rubric’s levels of performance and the descriptors should be easy to

understand for the students and evaluators.

11. The scale should have a space for the instructor to write specific feedback for

the student.

12. The performance descriptors should be arranged in levels indicating the degree

to which a standard has been met.

13. The rubric should provide consistent results according to the students’

performance, even when it is administered by different language professionals.

14. The scale needs to maintain content consistency over the performance

descriptions for each performance level.

15. The instrument should include criteria that represent significant aspects of the

learning you want to measure.

Here are some examples of scales that you can be considered to evaluate

ESL/EFL students writing since they include some of the aspects mentioned above

and they can be used for different writing tasks.

12


8.1: ESL Composition Profile (Jacobs et al., 1981, p. 30), reprinted with permission from the publisher. (Haswell, 2002)

13


Writing Rating Form by M.A Jorge Luis Espinoza

Value: 40 pts Points Gotten _________ Score:________ Percentage:______

Student’s name:_____________________________________________________________

Content 5 pts

______ Contains all the requested prewriting steps properly done, the composition provides sufficient details, the topic is well narrowed, the content is explicit.

______ Contains the requested prewriting steps done, the composition provides some details, the content is understandable and mostly maintains a focus on the topic

_____ Contains few or no prewriting steps, the composition provides very few details and ideas, the topic isn’t narrowed, the content isn’t organized and the info given is unclear.

Organization 15 pts

______ Contains a good topic sentence that clearly states the main idea with enough supporting ideas, and examples related to the topic and it has a good sequence and a concluding sentence.

______ Ideas are sequenced with a general topic sentence, some supporting ideas and a concluding sentence; text readily comprehensible, requiring no interpretation from the reader.

______ Attempts to develop a topic sentence, contains some supporting details, ideas are not well sequenced, it lacks a concluding sentence.

______ The main idea is not related to the topic, the sequence is confusing, supporting details not related to the topic and it does not have a concluding sentence.

______ Text not comprehensible for the reader it does not provide a sense of beginning and ending (no topic sentence, lack of supporting details, examples and concluding sentence)

Grammar 10 pts

_______ Writes well structured sentences, appropriate use of subject verb agreement and other grammar structures (no use of run-on sentences or fragments).

_______ Writes well structured sentences most of the time with very few limitations in subject verb agreement and other grammar structures.

_______ Presents some problems writing well structured sentences and some limitations in subject verb agreement and in other grammar structures.

_______ Presents constant problems writing well structured sentences and using subject verb agreement and other basic grammar structures.

_______ Writes a lot of run on sentences and fragments, inaccurate use of subject verb agreement and other basic language structures.

14


Vocabulary 5 pts

_______ Appropriate and rich use of vocabulary and transition words, suited for the task perform.

_______ Attempts to use a variety of words, transition words and descriptive language. Some mistakes presented which have some interference in communication.

_______Inadequate and/or inaccurate use of vocabulary and transition words mostly misused or not used, communication is very limited.

Format and Mechanics 5 pts

________Contains a title related to the paragraph; uses indentation, capitalization rules, punctuation marks and spelling correctly.

________ Provides an acceptable title, indents and capitalizes properly. It has some problems with punctuation rules and a few spelling errors.

________ Does not contain a title or the title is not related to the topic, presents constant problems with indentations, spelling, punctuation and capitalization. Comprehension is not carried out or carried out with a lot of difficulty.

COMMENTS

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

______________________________________________________________________________________

_____________________________________________________________________________________.

15


How to use a writing scale?

The first thing that we need to know is that writing scales are used to clarify

expectations and grading methods with learners. Using a writing scale seems easy

and it should not be complicated to any EFL professional. However, sometimes

teachers and students have problems using and understanding the writing rubrics.

There are not specific steps that teachers need to know to be able to use a writing

scale. Based on my experience, there are some good tips that can be followed by

the EFL instructors to make a good use of a writing instrument. It is important to

know what to do before, during and after grading the students work

Before grading

1. It is important to understand the scale and get familiar with it.

2. The instructor needs to share the scale with the students and explain it to them.

3. As practice, students can do peer assessment using the writing scale to help

them to understand the evaluation criteria and to become familiar doing peer

assessment. The instructor should give feedback and clarify the students’

questions regarding the writing scale.

During the grading process

1. Before start grading using the rubric, read the paper for yourself and then look

at the main points of the rubric and focus on them.

2. When grading the instructor needs to evaluate based on the performance

objectives to make the students grades objective and reliable to them.

16


3. To make good use of the writing instrument is important to review how students

write not just what they write.

4. To enhance student’s understanding, the professor needs to write and specify

the strengths and weakness of the student’s work based on the rubric criteria.

After you graded the students work

1. The professor can focus on the rubric aspects that students showed more

problems and point out the students’ strong points.

2. The teacher needs to identify the benchmark papers (papers that have a high

grade according to the scoring rubric) and share these papers with the students so

that learners can see some good writing models that they can emulate.

3. The instructor should go over the students’ questions.

What happens if a teacher cannot find a good scale to evaluate writing?

It is very common that after analyzing different scales you may not be able

to find one that really fulfills all your class and assignment’s goals. In these kinds

of situations, you need to try to develop your own rubric paying attention to the

aspects mentioned before plus your students’ needs, level and the assignment’s

objectives.

Many experts encourage EFL instructors to create their own rubrics since

every assignment and every group of students is different. Although, developing a

rubric is not easy especially at the beginning. To make the scale elaboration

process easier, you can follow some steps and recommendations. For example,

17


O’malley & Valdez Pierce suggested these steps to desing your writing scales

successfully.

1. Determine the focus of your assessment (what is the task that is going to be

graded and what significant knowledge, skills, learning objectives and processes

you are going to evaluate)

2. Review previous students’ work and/or other rubrics to identify any additional

assessment criteria.

3. Define how many performance levels are necessary to describe the knowledge

skills and processes associated with the task.

4. Decide which levels of performance are appropriate to evaluate the task.

5. Choose your assessment criteria.

6. Adopt a scale for describing the range of products/performances and write a

description for each dimension for each point on the scale.

7. Decide the format that you will give to your rubric.

8. Develop a draft rubric.

9. Evaluate the rubric.

10. Have a pilot test, revise the rubric, and try it again.

11. Share the rubric with other teachers and students, so that they can understand it

and give you feedback.

12. Evaluate the end product. Compare students' individual work with the rubric to

determine if the instrument is appropriate for the task.

18


Here is an instrument that you can use to evaluate the writing scales quality. It was designed by SBE Design TeamNorthern Colorado BOCES

19


This is another example of rubric to evaluate other rubrics

20


Conclusion

Evaluating EFL students’ writing is not an easy thing and sometimes as

teachers we are not able to grade students’ written assignments using the right

evaluation criteria. So, it is common that some of our students feel frustrated about

the results of their compositions. We need to know that the evaluation results are

not going to be perfect, but as language teachers we can find different instruments

and ways to grade more accurately considering the students’ needs.

By doing this work I grew as a professional because I could learn useful

ideas to evaluate my students’ written performance. I also found and designed

useful instruments to grade the students’ written work. Additonally, studying the

writing evaluation rubrics gave me a better understanding of the writing process

and how I can help the students to improve their writing skills. In fact, I

encourage all language teachers to take the time to analyze different evaluation

instruments and procedures to make the students’ writing process easier and

successful for yourself and your students.

21


References

Brown, H .D. (2001). Teaching by Principles: An Interactive Approach to Language Pedagogy, Second Edition. New York: Pearson Education

Haswell, R. (2002) Researching Teacher Evaluation of Second Language Writing via Prototype Theory. Retrieved on June 6, 2010 from http://www.writing.ucsb.edu/wrconf08/Pdf_Articles/Haswell-Article.pdf

Mueller, J. (2010). Authentic Assessment Toolbox. Retrieved June, 6, 2010 from http://jonathan.mueller.faculty.noctrl.edu/toolbox/rubrics.htm

Oakleaf, M. (2009). Writing Rubrics Right: Avoiding Common Mistakes in Rubric Assessment. Retrieved on June, 9, 2010 from the World Wide Web http://meganoakleaf.info/writingrubricsright.pdf

O’Malley, J. & Valdez Pierce, L. (1996). Authentic Assessment for English Language Learners: Practical Approaches for Teachers. New York: Addison Wesley Publishing Company

Renandya, W & Richards, J. (2002). Methodology in Language Teaching: An Anthology of Current Practice. United States of America: Cambridge University Press

TLT Group. (2008). Rubrics: Definition, Tools, Examples, References. Retrieved on June, 5, 2010 from http://www.tltgroup.org/resources/flashlight/rubrics.htm

Webster, M. (2008). Merriam Webster Collegiate Dictionary (11th edition).

New York: Merriam Webster

22

http://www.tltgroup.org/resources/flashlight/rubrics.htm

http://www.writing.ucsb.edu/wrconf08/Pdf_Articles/Haswell-Article.pdf

xxiv panama tesol annual congress 2010 re-imagining …€¦ · · 2010-11-20re-imagining your...

Documents