![Page 1: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/1.jpg)
Three strategies for assessment in autonomous language learning
Joan Jamieson, Northern Arizona University, USA
&
Carol A. Chapelle, Iowa State University, USA
![Page 2: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/2.jpg)
Three Strategies
Adaptivity
Feedback
Self-assessment
![Page 3: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/3.jpg)
An Adaptive Strategy
Learner would benefit from more than one form of material
Computer should select appropriate form based on responses to questions
![Page 4: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/4.jpg)
Overview of LEA
Interest and
Ability Finder
Beginning Reading
Beginning Listening
Beginning Writing
Intermediate Reading
Intermediate Listening
Intermediate Writing
Advanced Reading
Advanced Listening
Advanced Writing
Results &
Recommendations
Results &
Recommendations
Results &
Recommendations
![Page 5: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/5.jpg)
The Interest Survey
Select test form
Select recommendations
![Page 6: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/6.jpg)
Items on Interest Survey
Text Use Scores I would like to improve my ability to Used for selecting
business vs. general tests
Understand popular materials in writing
0/1
Understand written technical materials about my business
If 1 then select business reading test
0/1
I live in a place where Used for selecting ESL vs. EFL strategies (on recommendations page/ default = EFL)
Most people speak English If 1 then select ESL strategies
0/1
![Page 7: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/7.jpg)
Example Strategies
Beginning Listening - ESL
Beginning Listening - EFL
Phone an information line for a business or community service. Listen to the recorded message. Write what it is about. Go to a fast-food restaurant or a store. Stand where you can hear a cashier. Write down two questions the cashier asks customers.
Listen to the news on an English-speaking radio station and write down words or numbers you hear. Find an Internet story site. Listen to and read an interesting story. .
![Page 8: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/8.jpg)
A Feedback Strategy
Learner benefits from total scores
Learner might benefit more from part scores
![Page 9: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/9.jpg)
Example Computing Total Score
Vocabulary items
i t e m
answer
correct
value of SCORE
initial value SCORE = 0
1
confident
1
SCORE = 0 + 1
2
cheerful
1
SCORE = 1 + 1
3
exhausted
0
SCORE = 2 + 0
4
in a bad mood
1
SCORE = 2 + 1
Read the dialog. Drag and drop the answers into the blanks. tense relaxed confident in a bad mood exhausted cheerful Amy: Did you meet Sandy, the new receptionist? Isn't she great? Steve: I met her yesterday. She's energetic and hard-working. She also seems very
(1) ___________ that she can do her job well. Amy: I hope so. The last receptionist, Bob, wasn't as (2) _________ as Sandy is. He almost
never smiled. He always seemed to be (3) ____________
and nervous about all the work he had to do.
Steve: You're right. Something always put him (4) _________ and he was mad for the whole day. [Answers: 1. confident 2. cheerful 3. tense 4. in a bad mood]
TOTAL=SCORE/NUMBER OF ITEMS*100
75% = 3 / 4 * 100
![Page 10: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/10.jpg)
Part Scores Reflect Subskills
Tests are often made up of subskills
Each item can be coded according to subskill
Scores for subskills can be computed by including codes
![Page 11: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/11.jpg)
Table of Specifications
Skill # of items % of items Listening
10 25
Vocabulary
4 10
Speaking
6 15
Grammar 10 25
Pronunciation 6 15
Reading 4 10
# of items 40 % of items 100
![Page 12: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/12.jpg)
Tags for LEO Tests
TAG What the TAG means
L listeningLIN listening for informationLID listening for ideasG grammarG1 grammar point 1G2 grammar point 2G3 grammar point 3S speakingV vocabularyR readingP pronunciationP1 pronunciation point 1P2 pronunciation point 2
![Page 13: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/13.jpg)
Tags in Script for Grammar Section
Josh: Hi Kim. How are you doing?
Kim: I'm great! I just finished my science project. <tag> <G, G2> (1)_________ </tag> going to the movies?
Josh: It's <tag> <G, G3> (2)________ </tag> early to go the movies. <tag> <G, G2> (3)________ </tag> we go out to lunch instead? We can have pizza at Pizza Bob's.
Kim: I'm not hungry <tag> <G, G3> (4)________ </tag> to eat pizza. <tag> <G, G2> (5)_______ </tag> go to the Salad Spot instead.
Josh: How would we get there? It's too far <tag> <G, G3> (6) _________ </tag> and I don't have a car.
Kim: We can take the bus!
walking Why don't too Let's not to walk Let's How about enough
![Page 14: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/14.jpg)
Using Tags with System Variables
“score” yields percentage correct
score (tag) yields percentage correct for any items with a given “tag”
score (G2) yields percentage correct of 2nd point of grammar—expressions for suggesting
![Page 15: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/15.jpg)
Combining Tags and System Variables
score (L | G | V | S | P | R)
n/m1= “rawscore(LIN) / tqw(LIN)” n/m2= “rawscore(LID) / tqw(LID)” n/m3= “rawscore(G1) / tqw(G1)” n/m4= “rawscore(G2) / tqw(G2)”
n/m5= “rawscore(G3) / tqw(G3)”
![Page 16: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/16.jpg)
Mock-up of Progress Report Screen
Progress Report LEO 3 Test
Learner’s name:Score: score (L | G | V | S | P | R)
Language area Number correct/number of items
Listening for information n/m1
Listening for ideas n/m2
Grammar (point1*) n/m3
Grammar (point2*) n/m4
Grammar (point3*) n/m5
![Page 17: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/17.jpg)
Screen Shot of Progress Report
![Page 18: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/18.jpg)
Using Tags to Report Scores
TAG What the TAG means Possible scores
L R V
C
listening reading vocabulary comprehension
Total Reading Total Reading Vocabulary Reading Comprehension Listening Total Listening Vocabulary Listening Comprehension Vocabulary—reading and
listening Comprehension—reading and
listening
![Page 19: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/19.jpg)
A Self-Assessment Strategy
Learner may benefit by comparing his/her perspective of performance with score
Computer can collect self-confidence data along with performance data
![Page 20: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/20.jpg)
Example of Self-Confidence Item
Was your answer correct? How sure are you? Click a circle below .
Completely Not sure at allsure
![Page 21: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/21.jpg)
Superimposed Self-Assessment Item
Was your answer correct? How sure are you? Click in a circle for each answer.
1.
2.
3.
Completely Not sure sure at all
![Page 22: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/22.jpg)
Computing Average Confidence (Tarone and Yule, 1989)
Circle clicked 5 4 3 2 1 total average confidence
correct answers 20 5 3 2 0 29 4.52incorrect answers 0 0 4 5 2 11 2.00
(20*5)+(5*4)+(3*3)+(2*2)+(0*1)/29 = 4.52
(4*3)+(5*2)+(2*0)/11 = 2.00
Tarone, E., & Yule, G. (1989). Focus on the language learner. Oxford, UK: Oxford University Press.
![Page 23: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/23.jpg)
Computing Self-Monitoring Index
Derived by subtracting self-confidence rating on incorrect items from self-confidence rating on correct items:
4.52 – 2.00 = 2.52
Index ranges in value from 4 to - 4
Messages could be provided instead of numbers
![Page 24: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/24.jpg)
Self-Assessment Superimposed onto Progress Report
Self-Assessment: You seem to be aware of your own ability. When you gave the correct answer, you were very sure you were correct. When you gave the wrong answer, you were not too sure you were correct.
![Page 25: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/25.jpg)
Implementing Self-Assessment
Tag self-assessment items <SA>
Save value of “rawscore (SA)” separately for correct and incorrect items: IF ANSWER = 1 THEN SAOK = SAOK + rawscore (SA)
IF ANSWER = 0 THEN SANO = SANO + rawscore (SA)
![Page 26: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/26.jpg)
Calculating Average Scores
AVGSAOK = SAOK / # CORRECT ITEMS
AVGSANO = SANO / # INCORRECT ITEMS
MONITORING INDEX = AVGSAOK-AVGSANO
![Page 27: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/27.jpg)
Example of Computing Self-Assessment Scores
Item answer correct value of SCORE
value of SA value of SAOK value of SANO
initial value SCORE =0
initial value SA =0
initial value SAOK =0
initial value SANO =0
1
confident
1
SCORE = 0 + 1
5
SAOK = 0 + 5
2
cheerful
1
SCORE = 1 + 1
4
SAOK = 5 + 4
3
exhausted
0
SCORE = 2 + 0
1
SANO = 0 + 1
4
in a bad mood
1
SCORE = 2 + 1
4
SAOK = 9 + 4
AVGSAOK = SAOK / # CORRECT 4.3= 13 / 3
AVGSANO = SANO / # INCORRECT 1.0 = 1 / 1
TOTAL = SCORE / NUMBER OF ITEMS * 100
75% = 3 / 4 * 100
MONITORING INDEX = 4.3 – 1.0 = 3..3
![Page 28: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/28.jpg)
Three Strategies for Individualizing Assessment
Adapting level, content, and recommendations based on learner’s responses
Additional feedback in the form of diagnostic scores
Self-assessment to heighten learner’s metacognitive awareness
![Page 29: Three strategies for assessment in autonomous language learning](https://reader036.vdocuments.us/reader036/viewer/2022062304/568139fa550346895da1bf66/html5/thumbnails/29.jpg)
Three strategies for assessment in autonomous language learning
Joan Jamieson, Northern Arizona University, USA
&
Carol A. Chapelle, Iowa State University, USA