automated user-centered task selection and input modification rintse van der werf geke hootsen anne...
TRANSCRIPT
Automated user-centered task selection and input modification
Rintse van der Werf
Geke Hootsen
Anne Vermeer
MASLA project
Tilburg University
Outline• Background
• Research
• Discussion and future research
User-centered learning
• Approaches in educational research– Authentic– User initiated– Motivating– Individual needs
MASLA project
• Models of Adaptive Second Language Acquisition
• Combination of Computer Science and Second Language Acquisition
• Goal: building a model for personalized digital language learning web based applications
• How can learning materials automatically be adapted to fit the characteristics and preferences of the language learner?
• Criterion is learning effect.
Requirements for adaptivity
• Annotated learning material– domain model
• Knowledge about learner characteristics– user model
• User model + domain model -> adaptation model (rules)
(Dexter model, 1990; AHAM model (De Bra, 2000))
MASLA Framework
Graphical User Interface
Curriculum
L2 - proficiencies
Learning contents
Learning styles
Learner backgrounds
Task: Vocabulary learning through reading
• Incidental vocabulary learning (side effect of reading for comprehension)
• ZOPD (Vygotsky, 1962); Comprehensible input (Krashen, 1987)
– Assessing learner proficiency
– Assessing text difficulty
based on frequency information from corpora
=> combined in text coverage
Text Coverage
70
75
80
85
90
95
100
1 2 3 4 5 6 7 8 9
learner profficiency (x1000 lemmas)
lem
ma c
overa
ge (
%)
more difficult
easier
Interpreting text coverage
• Hazenberg, 1994; Laufer, 1989; Vermeer, 1998
• Lemma Coverage:
– 85%: Global understanding– 90%: Good understanding– 95%: Almost complete understanding
Effective Instruction
• Comprehensible but challenging
• Lemma coverage 85% - 92%
• Support from input modification– Dictionary/glossary (see Hulstijn et al., 1996; Plass et al.,
1998; Watanabe, 1997)– User initiated “focus on form”
Text Coverage
70
75
80
85
90
95
100
1 2 3 4 5 6 7 8 9
learner profficiency (x1000 lemmas)
lem
ma c
overa
ge (
%)
Top criterion
Bottom criterion
Summary of research background
• Web based tool for automatic adaptive selection of the appropriate text for a specific user.
• Automated analysis of text difficulty.
• User proficiency calculation from score on vocabulary test.
• User gets text that is comprehensible but challenging and has input modification for unknown words to support for understanding the text.
Research questions
• A. Adaptive selection of texts leads to:
• A learning effect for all users• No difference between learners with different proficiency
levels
• B. Using input modification:
• There is a relation between noticing and retention• (There is no difference in this relation for different
proficiency levels)
Method (1)
• Subjects (N=32)
• Reading Texts (16)– 4 clusters
• Input modification
Text coverage for selected texts
60
65
70
75
80
85
90
95
100
1 2 3 5 8
Almost complete comprehension
Global comprehension
Mean text coverage per cluster
60
65
70
75
80
85
90
95
100
1 2 3 5 8
Almost complete comprehension
Global comprehension
Method (1)
• Subjects (N=32)
• Reading Texts (16)– 4 clusters
• Input modification
Method (2)
• Data collection:
– User logging and tracking
– Testing material• Vocabulary proficiency test• Text specific vocabulary tests• Comprehension questions
• Procedure
Learning gains
Learning gains
Learning gains
Learning gains
Procedure
Results (1)
• A mean learning effect occurred for all clusters– 5% learning gains
• No significant difference between groups– both pre and posttest scores– learning gains
Results (2)
• Correlation between noticing and retention– Mean Φ correlation for subjects: .28– Mean Φ correlation for items: .50
• in general, the use of the dictionary was limited – No significant difference between proficiency groups
• In lookup behavior• In correlation
Conclusion
• Automated assessment of texts based on corpora information is a useful indication of text (task?) difficulty.
• Adaptive selection of texts based on vocabulary proficiency works.
• Open, web based learning environment provides flexibility in the curriculum and opportunities for individualized tasks.
Discussion and future work
• Increase learning gains– More adaptivity in text selection
• Increase exposure to target words• Based on observed behavior
• Increase usability of input modification– Individualize annotation
• Based on observed behavior• More focus on form
• Use different corpus for text coverage– Now children’s corpus, future Celex/CGN– Unknown lemmas – Multiword expressions