2006 CALL2006 CALL

CALL EvaluationCALL Evaluation

An Exploration of the An Exploration of the Networked Assessment Networked Assessment

in the DNFLLTin the DNFLLT

“ “Assessment is a complex topic, the tAssessment is a complex topic, the technical or political dimensions of wechnical or political dimensions of which used to put teachers off”.hich used to put teachers off”.

---Freeman(2003)---Freeman(2003)

The NDFLLT model requires active The NDFLLT model requires active participation in the assessment of participation in the assessment of teachers and students. teachers and students.

consumers of NA (networked consumers of NA (networked assessment) Systems assessment) Systems

informed evaluatorsinformed evaluators to apply the ready-made network to apply the ready-made network

resources resources to produce new ones in the dynamic to produce new ones in the dynamic

process of ELTprocess of ELT

A sampled surveyA sampled survey of 287 subjects, (270 Ss and 17 Ts) of 287 subjects, (270 Ss and 17 Ts) a questionnairea questionnaire structured interviews,structured interviews, Concerning :Concerning : the popularity of the NA system with the the popularity of the NA system with the

participantsparticipants its effectiveness in evaluating online ELTits effectiveness in evaluating online ELT

““Formative Assessment features Formative Assessment features Computer-based teaching”Computer-based teaching”

1. A Formative Assessment Promoter? 1. A Formative Assessment Promoter? actualizing the notion of formative assessmactualizing the notion of formative assessm

entent Possibilities to Engage Ts in constant and coPossibilities to Engage Ts in constant and co

ntinual assessment of their Ss byntinual assessment of their Ss by * Keeping a record of students' online* Keeping a record of students' online performance;performance; * Keeping files on students' academic achieve* Keeping files on students' academic achieve

ments;ments; **zhangzhang de de shengsheng (1).doc (1).doc **shi shi xiaoxiao kangkang (2).doc (2).doc

* Checking Ss’ fulfillment of the online a* Checking Ss’ fulfillment of the online assignments as to where and for how lonssignments as to where and for how long Ss are accessing our course siteg Ss are accessing our course site

**individual online individual online time.doctime.doc * Getting the statistic data concerning the* Getting the statistic data concerning the

ir performance ir performance Keeping students' learning process undKeeping students' learning process und

er observation, evaluation and supervisier observation, evaluation and supervision. on.

**class a V-L-class a V-L-S.docS.doc

2. A Summative Assessment Reliever2. A Summative Assessment Reliever ? ?

Past practicePast practiceComposing tests:Composing tests: a tiring burdena tiring burdenResorting to marketed Resorting to marketed

papers papers more for EPT than for more for EPT than for

EATEAT poor relevance poor relevance hardly for diagnosis hardly for diagnosis

or consolidationor consolidation misleading Ss into misleading Ss into

despising course despising course book learningbook learning. .

The NA systemThe NA system helping save Ts time helping save Ts time

and labor in and labor in designing and designing and

composing tests composing tests grading the papers grading the papers analyzing the resultsanalyzing the results combined with the combined with the

record of students’ record of students’ other-time other-time performance,performance,

a more objective and a more objective and comprehensive comprehensive assessment of Ss assessment of Ss learning.learning.

3. A mirror for Ss’ self-assessment? 3. A mirror for Ss’ self-assessment? Offering benchmarksOffering benchmarks * for global self-* for global self-

assessment (assessment (in CEP--- Cf. in CEP--- Cf.

CECR Appendix IICECR Appendix II)) for assessing specific for assessing specific

items items * vocabulary exercise,* vocabulary exercise, * cloze work* cloze work * Reading and listening * Reading and listening

comprehension, etccomprehension, etc quick feedbackquick feedback mirroring progress and mirroring progress and

weak points weak points

The NA system:The NA system:a poor assessor a poor assessor in written work like:in written work like: * the short answer * the short answer * Compound dictation* Compound dictation * Translation* Translation * Essay writing* Essay writing In Ss own progress In Ss own progress

concerningconcerning * attitude, * attitude, * strategies, * strategies, * interest, etc.* interest, etc.

4. A spur for greater efforts?4. A spur for greater efforts? Positive answer Positive answer

from 67% from 67% respondents respondents

feedback shown on feedback shown on the Internet made the Internet made them realize the them realize the gap between them gap between them and their peersand their peers

thus driving them thus driving them to greater efforts to to greater efforts to catch up catch up

15% remained 15% remained neutralneutral

18% disagreed18% disagreed Publicizing their Publicizing their

poor performance poor performance on-line: on-line: embarrassingembarrassing

Discouraging them Discouraging them from taking further from taking further risk risk

Problems discussion: Problems discussion: The conveniences and facilitation of the The conveniences and facilitation of the

NA system could not be truly enjoyed NA system could not be truly enjoyed without full accessibilitywithout full accessibility

AccessibilityAccessibility Affected by scant on-line hours: Affected by scant on-line hours: Resulting in limited online assessmentResulting in limited online assessment Adjusting assignments accordingly to Adjusting assignments accordingly to

take advantage of the NA system.take advantage of the NA system. Deciding on the priorityDeciding on the priority

1). Human-computer conversation1). Human-computer conversation---a spur to oral English?---a spur to oral English?

An envisaged spurAn envisaged spur Helping the shy Helping the shy

students overcome students overcome their anxiety in public their anxiety in public speech speech

Improving speaking inImproving speaking in

form and content form and content after full preparationafter full preparation

Facilitating class Facilitating class speaking activitiesspeaking activities

With samples, rich With samples, rich topics and materialstopics and materials

A moonshineA moonshine hardly any private hardly any private

space for the space for the residential Ssresidential Ss

hard for computer to hard for computer to recognize some Ss recognize some Ss pronunciationpronunciation

unconventional unconventional practice practice

not so urgent to the not so urgent to the test-sensitive test-sensitive students students

2). On-line reading assessment? 2). On-line reading assessment?

Functioning well in Functioning well in helping Ts helping Ts grading comprehensiograding comprehensio

n exercises, n exercises, analyzing the resultsanalyzing the results obtaining individual’s obtaining individual’s

score and statisticsscore and statistics Examining Ss academic Examining Ss academic

improvement.improvement. But having to remain idBut having to remain id

le le

86% of students prefer 86% of students prefer printed reading given printed reading given

the limited on-line learning the limited on-line learning hours, hours,

physical discomfort in on-physical discomfort in on-line reading line reading

inconvenience in inconvenience in * note taking, * note taking, * marking, * marking, * margin reflection, etc.* margin reflection, etc. assign printed reading assign printed reading

tasks as off-line worktasks as off-line work Resulting in the idleness of Resulting in the idleness of

the NA systemthe NA system

3). A spur to writing improvement ?3). A spur to writing improvement ? E-mail writingE-mail writing * More for communication * More for communication

than for LL than for LL E-mail exchange E-mail exchange

discontinues ifdiscontinues if * Ts assessing Ss’ e-mail * Ts assessing Ss’ e-mail

in its language elements in its language elements like like *structure, *structure, *appropriateness in *appropriateness in

expression expression * or spelling.* or spelling. Paragraph writing in Paragraph writing in

each unit test ?each unit test ? * 40% choosing to ignore* 40% choosing to ignore * Accounting for little* Accounting for little

* Ts preferring reading and gr* Ts preferring reading and grading Ss’ paper jobading Ss’ paper job

Helping in pre-and post- wrHelping in pre-and post- writing ativitiesiting ativities

* the delivery of writing lectur* the delivery of writing lectures. es.

* the organization of pre-writi* the organization of pre-writing activitiesng activities

* Reviewing on students writt* Reviewing on students written joben job

After all, seeing promotes After all, seeing promotes understanding understanding

4. Facilitation to listening assessment?4. Facilitation to listening assessment? 93% gave a positive 93% gave a positive

response.response. Ts: holding it as aTs: holding it as a great helper great helper * facilitating listening-* facilitating listening-

test composing,test composing, * assignment grading, * assignment grading, *result analyzing, etc.*result analyzing, etc. Ss: embracing it for Ss: embracing it for

their freedom intheir freedom in * pacing,* pacing, * self-assessing * self-assessing * controlling the * controlling the

learning process to learning process to suit his / her own suit his / her own need. need.

Some room for Some room for improvement. improvement.

Its intolerance of Its intolerance of synonyms or similar synonyms or similar expressionsexpressions

Unit 6 Book II++4.doUnit 6 Book II++4.docc

* Listening resource * Listening resource enrichment,enrichment,

* Gradual dictation * Gradual dictation design design

*Content relevance, *Content relevance, etc.etc.

ConclusionConclusion The NA system is gaining its popularity.The NA system is gaining its popularity. The existing problems:The existing problems: 1) hardware construction, 1) hardware construction, * to be solved when schools put in * to be solved when schools put in sufficient e-construction funds sufficient e-construction funds * or when most Ss can afford a PC; * or when most Ss can afford a PC; 2) improvement of the NA-system itself2) improvement of the NA-system itself * It needs the co-efforts from the course-ware and * It needs the co-efforts from the course-ware and

soft-ware developers’;soft-ware developers’; 3) Non-silicon dimension3) Non-silicon dimension * It claims Ts’ caring heart, observant eyes and * It claims Ts’ caring heart, observant eyes and

creative minds as well as Ts’ knowledge of creative minds as well as Ts’ knowledge of assessment theories and techniques.assessment theories and techniques.

The NA system: a necessary The NA system: a necessary supplement and helpersupplement and helper

To give Ss an all-round assessment, To give Ss an all-round assessment, Ts have the non-shifting responsibility Ts have the non-shifting responsibility

to combine:to combine: * the conventional assessment * the conventional assessment approaches approaches * the on-line resources * the on-line resources * alternative devices like portfolio or * alternative devices like portfolio or

performance assessment (Bailey,208-performance assessment (Bailey,208-216)216)