building the future of autonomous language learning · building the future of autonomous language...
TRANSCRIPT
Building the future of autonomous language learning
Martyn Clark, PhD, Carrie Bonilla, PhD, Ewa Golonka, PhD, Jared Linck, PhD, Catherine Doughty, PhD
LanguageNation project update28 July 2015
University of Maryland CASL
LanguageNation Development Partnership
UMD-CASLJared Linck, PhD (Principal Investigator)Catherine Doughty, PhD (Area Director)C. Bonilla, W. Burns, M. Clark, E. Golonka, C. Green, E. Jones, M. Masters, C.A. Rytting, and M. Tare (Researchers)DoDCarol Van Ess-Dykema, PhD (USG, Chief Scientist)Tara Mechum, PhD (USG, Intern)IBMThanks to Salim Roukos, Jiri Navratil, and Todd Ward at IBM for their assistance in obtaining the data from the pilot participants described in this presentation.
2
LanguageNation Vision
• Anywhere, anytime, any device for professionals• Target users – USG learners • Not “canned” lessons, but truly adaptive learning
o Aptitude Profileo Proficiencyo Performanceo Topic of interesto Mission needs
• Autonomous learning (with a human-in-the-loop coach)
3
INTERNET
Learning Activities
Internal database (corpora)
HLT Tools
Performance Tracking
Pretesting
SLA Principles
Learner model
Learning Planner
Activity Generator
Human curators
HLT Tools
SLA Principles
HLT - Automated Materials Generation
SLA –Guided Learner Management System
Front‐end(visible to user)
Proof-of-concept: activity generation for autonomous learning
7
Context of the Pilot Study Two waves of pilot participants from IBM workforce used
LanguageNation prototype between April 2014 –February 2015.
Participants(n=54) were advanced to native Spanish or English speakers with little Portuguese knowledge reported.
Three CASL researchers with advanced Spanish knowledge also piloted the system.
We analyzed data from those that spent the most time. in the system (>7 hours).
11
Prototype vocabulary activity
• Learners are quizzed on the target words• Options are given if the learner needs a hint• Grammatical information is provided with the right answer
This is Marivi.
é a Marivi .(Hint)
14
The benefit of online learning: Data captureNSECS CTIME UNAME PNAME ATYPE AVAL IVAL FVAL SVAL USER_ID PARAM_IDPARAM_AUSER_PARSVCNAME63535319070 2014‐05‐09‐11.44.30.620566 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319089 2014‐05‐09‐11.44.49.732260 [email protected] NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319089 2014‐05‐09‐11.44.49.733199 [email protected] NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319089 2014‐05‐09‐11.44.49.733766 [email protected] NumConceptHintDiscount concept makeword‐1 3.33E‐01 17313 24 11 435386 makeword63535319089 2014‐05‐09‐11.44.49.734372 [email protected] NumMakeWordContentCompleted content ‐1@106318 1 17313 19735 12 435609 makeword63535319118 2014‐05‐09‐11.45.18.187791 [email protected] NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319118 2014‐05‐09‐11.45.18.188623 [email protected] NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319118 2014‐05‐09‐11.45.18.189230 [email protected] NumConceptHintDiscount concept makeword‐1 0.00E+00 17313 24 11 435386 makeword63535319118 2014‐05‐09‐11.45.18.189773 [email protected] NumMakeWordContentCompleted content ‐1@166879 1 17313 19735 12 435617 makeword63535251702 2014‐05‐08‐17.01.42.690395 [email protected] NumLemmaShown lexicon pessoa 1 17313 15 10 40276963535251702 2014‐05‐08‐17.01.42.738753 [email protected] NumWordPlayed lexicon pessoa 1 17313 7747 10 37972263535319135 2014‐05‐09‐11.45.35.712531 [email protected] NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319135 2014‐05‐09‐11.45.35.713401 [email protected] NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319135 2014‐05‐09‐11.45.35.713983 [email protected] NumConceptHintDiscount concept makeword‐1 0.00E+00 17313 24 11 435386 makeword63535319135 2014‐05‐09‐11.45.35.714565 [email protected] NumMakeWordContentCompleted content ‐1@189171 1 17313 19735 12 435638 makeword63535319141 2014‐05‐09‐11.45.41.892977 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319145 2014‐05‐09‐11.45.45.540560 [email protected] NumConceptChallenged concept makeword‐1 1 17313 22 11 435382 makeword63535319145 2014‐05‐09‐11.45.45.541419 [email protected] NumConceptCorrect concept makeword‐1 1 17313 23 11 435384 makeword63535319145 2014‐05‐09‐11.45.45.541973 [email protected] NumConceptHintDiscount concept makeword‐1 0.00E+00 17313 24 11 435386 makeword63535319145 2014‐05‐09‐11.45.45.545772 [email protected] NumMakeWordContentCompleted content ‐1@282026 1 17313 19735 12 435662 makeword63535319145 2014‐05‐09‐11.45.45.603183 [email protected] NumActivityCompleted content makeword 1 17313 4215 12 435425 lltp63535319146 2014‐05‐09‐11.45.46.791774 [email protected] NumActivityCompleted content makeword 1 17313 4215 12 435425 lltp63535319147 2014‐05‐09‐11.45.47.290347 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319150 2014‐05‐09‐11.45.50.470810 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319152 2014‐05‐09‐11.45.52.455455 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319156 2014‐05‐09‐11.45.56.871358 [email protected] NumActivityCompleted content matching 1 17313 4215 12 381180 lltp63535319157 2014‐05‐09‐11.45.57.356615 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319164 2014‐05‐09‐11.46.04.844827 [email protected] NumFlashCardContentCompleted content 36800 1 17313 29 12 433427 flashcard63535319164 2014‐05‐09‐11.46.04.845586 [email protected] NumLemmaShown lexicon processo 1 17313 15 10 433429 flashcard63535319164 2014‐05‐09‐11.46.04.853555 [email protected] UserEvent event [email protected] user_even 17313 11902 14 37047563535319164 2014‐05‐09‐11.46.04.928987 [email protected] NumWordPlayed lexicon processo 1 17313 7747 10 433431 llaud63535319170 2014‐05‐09‐11.46.10.510097 [email protected] NumFlashCardContentCompleted content 36800 1 17313 29 12 433427 flashcard63535319170 2014‐05‐09‐11.46.10.510986 [email protected] NumLemmaShown lexicon processo 1 17313 15 10 433429 flashcard63535319170 2014‐05‐09‐11.46.10.522356 [email protected] NumLemmaChallenged lexicon processo 1 17313 17 10 433584 flashcard63535319170 2014‐05‐09‐11.46.10.523104 [email protected] NumLemmaCorrect lexicon processo 1 17313 19 10 433586 flashcard63535319170 2014‐05‐09‐11.46.10.523804 [email protected] NumLemmaCorrectHintDiscount lexicon processo 0.00E+00 17313 21 10 433588 flashcard63535319170 2014‐05‐09‐11.46.10.587030 [email protected] NumWordPlayed lexicon processo 1 17313 7747 10 433431 llaud63535319170 2014‐05‐09‐11.46.10.587656 [email protected] NumWordPlayed lexicon processo 1 17313 7747 10 433431 llaud63535319187 2014‐05‐09‐11.46.27.675208 [email protected] NumFlashCardContentCompleted content 36688 1 17313 29 12 421940 flashcard63535319187 2014‐05‐09‐11.46.27.676046 [email protected] NumLemmaShown lexicon passeio 1 17313 15 10 421942 flashcard63535319187 2014‐05‐09‐11.46.27.735006 [email protected] NumWordPlayed lexicon passeio 1 17313 7747 10 421944 llaud63535319192 2014‐05‐09‐11.46.32.318312 [email protected] NumFlashCardContentCompleted content 36688 1 17313 29 12 421940 flashcard
Courtesy of IBM
16
Feedback Given Failure Success Too many attempts
Subsequent success rate
There is no such word 5 14 0 0.737
Try a different word 16 23 1 0.575
Try a word with a different form(). 6 15 1 0.682
Try a word with a different form(gender). 2 8 0 0.8
Try a word with a different form(person). 0 6 0 1.0Try a word with a different form(singular/plural). 0 1 0 1.0
Try a word with a different form. 4 10 1 0.667
Was it a typo? 11 22 1 0.629
Analysis example: Does feedback help?
IntroT:1
√
Pronunciation 1T:1
√
Pronunciation vocabL:42
√
ColorsT:1
√
ArticlesT:1
√
Pronunciation 2T:1
√
seasonsT:1
√
Season quizT:1 √
Article quizL:59 √
familyT:1
√
Body partsT:1
serT:1
√
prepositionsT:1 √
Singular pluralT:1 √
Portuguese Learning Plan (partial)
Courtesy of IBM
‘Any language’
19
ILR Level ILR Function Form (Somali)
0+ Recognize and identify letters Sounds and Orthography
0+ Understand common personal and place names Common names, cities, regions, countries
0+ Understand common words and phrases Common Greetings and Benedictions
0+ Understand common words and phrases Numbers 1-20, dates, months
0+ Understand common words and phrases Numbers 20-100
1 Recognize shared vocabulary Arabic and English borrowings
1 Telling time Telling time
1 Understand descriptions Attributive adjectives
1 Understand descriptions Nouns pluralized by –o
1 Understand descriptions Agreement with definite determiners and demonstratives
1 Understand simple directions Cardinal directions and locatives
1 Understand simple directions Temporal and locative adverbs
1 Understand simple directions Basic adpositions (ku, la, ka, u)
1 Understand simple directions and statements Imperative of weak verbs
1 Understand simple statements Declarative/equative sentences using waa… … …
Directions for future improvements
22
Enhance language contextualization Advancing intelligent feedback on error in open-ended
tasks Develop activities appropriate for learners at higher levels
of proficiency Incorporate more SLA principles in the adaptive learning
system:• Authentic and rich input (e.g., Lafford et al., 2007)• Opportunities to produce the language (e.g., Swain,
1985)• Negotiation for meaning in real-time interaction (e.g.,
Long, 1991)• Feedback on error (e.g., Doughty & Long, 2003)
For more information
ContactJared Linck, PhD (Principal Investigator)Associate Research ScientistPhone: 301-226-8879Email: [email protected]
23