designing for authenticity: a framework and guidelines for conversation simulation courseware
DESCRIPTION
Designing for authenticity: a framework and guidelines for conversation simulation courseware. Prof. dr. Jozef Colpaert Frederik Cornillie LINGUAPOLIS research & development Language Teaching Symposium 2009, Gent. Statement. There’s probably no God . Now stop worrying and enjoy your life. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/1.jpg)
Designing for authenticity: a framework and guidelines for conversation simulation courseware
Prof. dr. Jozef ColpaertFrederik CornillieLINGUAPOLIS research & development
Language Teaching Symposium 2009, Gent
![Page 2: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/2.jpg)
Statement
• There’s probably no God. Now stop worrying and enjoy your life.
![Page 3: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/3.jpg)
Statement
• There’s probably no learning effect in TECHNOLOGY. Now stop worrying and enjoy your research.
![Page 4: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/4.jpg)
Research you say ?
• NSDS: No Significant Difference Syndrome ecological shift: design the entire learning
environment
• SES: Sunday Evening Syndrome– 95 % work, 5 % research – Sunday evening = research moment? how to transform your daily work into research
![Page 5: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/5.jpg)
Research you say ?
• Attitude A:• I am involved in developing and using technology for learning and
teaching, and my perception is that I am rather good at it. I should publish more but I have the feeling that my work is not being valued academically nor recognized as research …
• Attitude B:• I have just developed this tool (which already existed in the early
eighties, but who cares?). Dancing Bearware really, but I present it as coconstruction of knowledge, negotiation of meaning, or empirical validation of theory. It’s fake, but I add Vygotsky and it really works. I get grants and awards, my presentations are accepted at conferences and I even get published. You should try it …
![Page 6: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/6.jpg)
DLL
• Distributed Language Learning– conceptual and methodological framework for
designing learning environments– educational engineering model– based on hypothetical compromise between
pedagogical and personal goals– role of technology and choice of learning/teaching
method are logical consequences
![Page 7: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/7.jpg)
DLL
• Distributed Language Learning– in phase of further theoretical / empirical
validation– contribute to collaborative research
![Page 8: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/8.jpg)
Educational engineering
Analysis
Development
Implementation
Evaluation
conceptualization
specification
prototyping
Design
TheoryTechnology
![Page 9: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/9.jpg)
Architecture
teacher
learner
content
colearner
teacher
learner
content
colearner
Classrm
w
all
![Page 10: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/10.jpg)
Technology & authenticity
Mediated Non-mediated
Information
Interaction
Communication
… … …
IIC model applied to learners:
Mediated = with human or technological intervention
![Page 11: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/11.jpg)
Information
• Non-mediated– use of authentic materials on the Web– examples: WebQuests, www.deredactie.be, …
• Mediated– use of adapted materials (simplified texts, hyperhelp …)– examples: Internet actuel, hyperlinks, http://
podcasts.ox.ac.uk/, http://www.radiolingua.com/, http://mitworld.mit.edu/, …
• Advantages / disadvantages ?
![Page 12: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/12.jpg)
Communication
• Non-mediated– With natives not aware of LL process– Examples: chat, mail, twitter, fora, text, blog, virtual
worlds …
• Mediated– With co-learners, teacher or natives aware of the LL
process– Examples: Tandem learning, http://www.patati.be/
• Advantages, disadvantages ?
![Page 13: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/13.jpg)
Interaction
• Non-mediated– Use of systems not developed for LL purposes– Examples: reservation sites, Second Life, (non-dedicated)
chatterbots, point of sales systems …
• Mediated– Use of systems developed for LL purposes– Examples: language courseware, tests, http://
www.slanguages.net/ …
• Advantages, disadvantages ?
![Page 14: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/14.jpg)
So ?Mediated Non-mediated
Information +: more adapted-: less authenticity, labour-
intensive, logging possibilities
beginner level
+: authenticity, TBL- : errors, quality, appropriateness,
durability … more advanced levels, linguistic
awareness, mature readership, preparation and control
Interaction +: strong didactic functionalities
-: D&P ?? all levels
+: authentic interaction, TBL-: lack of tracking & feedback more advanced levels
Communication +: role of coach, logging possibilities
-: more artificial beginner/intermediate
levels
+: authenticity, TBL-: errors, quality, privacy, no
coaching intermediate to advanced levels
![Page 15: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/15.jpg)
Authenticity
• Different roles of technology tackle authenticity differently
• Different moments/situations/levels require different forms of authenticity
![Page 16: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/16.jpg)
DISCO project
• conversation simulation courseware for language learning
• degrees of authenticity in ...– task type– feedback– content– assessment & progress tracking
![Page 17: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/17.jpg)
DISCO project
• Development and Integrationof Speech technology into Courseware for language learning
• Radboud Universiteit Nijmegen, Polderland Language and Speech Technology, Linguapolis (Universiteit Antwerpen)
• STEVIN research project, Nederlandse Taalunie
![Page 18: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/18.jpg)
DISCO requirements
• to develop and test a prototype of an ASR-based CALL application for training oral proficiency for Dutch as a second language (DL2) at A2 level
• to optimize learning through interaction in realistic communication situations
• to provide intelligent feedback on various aspects of DL2 speaking, viz. pronunciation, morphology and syntax
![Page 19: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/19.jpg)
Degrees of authenticity
• task type• feedback• content• assessment & progress tracking
![Page 20: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/20.jpg)
Degrees of authenticity
• task type• feedback• content• assessment & progress tracking
![Page 21: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/21.jpg)
Task type
• conversation simulation with animated agents• “interactive participatory drama” (Hubbard
2002)• conversation branching (item-based system)• afterwards remediation on specific topics,
coached by pronunciation agent
![Page 22: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/22.jpg)
Conversation environment
![Page 23: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/23.jpg)
Conversation environment
![Page 24: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/24.jpg)
Remediation environment
![Page 25: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/25.jpg)
Degrees of authenticity
• task type• feedback• content• assessment & progress tracking
![Page 26: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/26.jpg)
Feedback in ASR-based CALL
• waveforms and spectograms
• talking heads• feedback based on
pronunciation score• corrective feedback
![Page 27: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/27.jpg)
Feedback in ASR-based CALL
• waveforms and spectograms
• talking heads• feedback based on
pronunciation score• corrective feedback
![Page 28: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/28.jpg)
Feedback in CAPT
Results of 2007 doctoral study, Radboud Universiteit Nijmegen:– The key to successful CAPT is detailed corrective
feedback ...– ... but applied with moderation.
![Page 29: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/29.jpg)
Dutch-CAPT project
![Page 30: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/30.jpg)
Corrective feedback types
feedback type classroom language learning CALL
explicit correction
Neen, ‘gekoopt’ is fout, ‘gekocht’ is juist.
Ik heb een trui gekoopt.
repetition of the error
Gekoopt? Gekoopt?XX SEE HEIFT
recast w(/o) stress
Oh, je hebt een trui gek[ó/o]cht.
Oh, je hebt een trui gekocht.
clarification request
Ik begrijp het niet. Wat bedoel je precies?
Ik begrijp je niet. Kan je herhalen?
metalinguistic clue
Let eens op je voltooid deelwoord.
Het werkwoord is niet juist.XX SEE HEIFT
elicitation Dat is niet juist. Hoe zeggen we dat in het Nederlands?
XX SEE HEIFT
* Ik heb een trui gekoopt.
![Page 31: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/31.jpg)
Ambiguity of recast
• ambiguous function– corrective & mediated (focus on form)
• “Ah, je hebt een trui gekocht.” – communicative & authentic (focus on meaning)
• “Ah, je hebt een trui gekocht. Hoe ziet die eruit?”
• use in classroom teaching– frequently used for treating morphosyntactic errors– less frequently used for treating pronunciation and lexicon
• effect on learning outcome (!!)– higher effect on pronunciation and lexicon– lower effect on morphosyntax
![Page 32: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/32.jpg)
Feedback in DISCO
• two types of corrective feedback, varying with respect to learner preference– explicit correction
• “I want lots of feedback in the conversation,in order to correct my mistakes and achieve perfect pronunciation.”
– recast• “Even when I make mistakes,
I want to go on with the conversation, in order to practice fluency. ”
• non-verbal feedback
![Page 33: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/33.jpg)
Explicit correction
![Page 34: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/34.jpg)
Recast
![Page 35: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/35.jpg)
Degrees of authenticity
• task type• feedback• content• assessment & progress tracking
![Page 36: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/36.jpg)
Content• Seemingly authentic conversations ...
– realistic situations– speech acts
• ... however, very artificial and intricate content authoring– technological constraints: length of sentences, interjections,
disfluencies– phonetic constraints: frequent mistakes at A2 level– grammatical constraints: frequent morphosyntactic mistakes at A2
level– lexical constraints: frequency of vocabulary at A2 level– cultural constraints: Dutch-Flemish bias– didactic constraints: validity of assessment (see further)
=> delicate balance between requirements/constraints and authentic feel of the conversations
![Page 37: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/37.jpg)
Content authoring plan1. studying the cultural context (a.o. Nieuwe Buren videos); 2. devising the narrative plan, with detours and alternative endings;3. writing separate blocks of conversations for each alternative path in the dialogue;4. feeding the conversation trees into the content database;5. double-checking the logic of the narrative in the prototype software;6. checking vocabulary and cultural elements for socio-linguistic consistency (Dutch-Flemish
contexts);7. checking the length of the sentences from the point of view of ASR and NLP requirements;8. checking vocabulary and grammar from a linguistic-didactic point of view (typical A2
vocabulary, possible hyperonyms/hyponyms, ...);9. checking vocabulary and grammar for consistency with Nieuwe Buren vocabulary and
grammar;10. adding grammatical codes to each turn in the branched conversation, so that all linguistic
aspects are more or less equally distributed over the complete conversation tree;11. elaborating prompts for morphology and syntax exercises (which are in fact semantically
identical to the prompts for pronunciation, but differ only in grammatical form); 12. adding contextual information to each pair of turns in the conversation, so that the
graphical artist and photographer know which content to create;13. debugging the complete conversation in the prototype after the insertion of multimedia
content (sounds, graphics).
![Page 38: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/38.jpg)
Content authoring in database
![Page 39: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/39.jpg)
Content authoring in database
![Page 40: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/40.jpg)
Degrees of authenticity
• task type• feedback• content• assessment & progress tracking
![Page 41: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/41.jpg)
Assessment & progress tracking• formative assessment > “How am I doing?”• system keeps scores for each topic on which learner
is tested– phoneme– grammatical topic (e.g. subject-verb concord, verb
position, ...)• scores increase or decrease according to learner
performance
=> topics have to be (more or less) equally distributed in the conversation tree
![Page 42: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/42.jpg)
Topic distribution
![Page 43: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/43.jpg)
Report
![Page 44: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/44.jpg)
Summing up
XX
• kort herhalen wat gezegd geweest is• IIC schema• DISCO als vb van• authenticiteit op 4 vlakken
![Page 45: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/45.jpg)
You’re invited …
• Antwerp CALL 2010 “Beyond motivation”– August 2010– http://www.antwerpcall.be
• Summer School “A Toolbox for Design-Based Research”– Qualitative and quantitative research methods in language
learning and teaching, University of Antwerp, 16 to 20 August 2009
– http://summerschool.linguapolis.be
![Page 46: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/46.jpg)
An authentic thank you
• Credits to– Koen De Maesschalck, illustrations
(www.koendm.be)– STEVIN research funds, Nederlandse Taalunie
• Contact us– [email protected]– [email protected]
![Page 47: Designing for authenticity: a framework and guidelines for conversation simulation courseware](https://reader036.vdocuments.us/reader036/viewer/2022062803/568148ad550346895db5bfa4/html5/thumbnails/47.jpg)
Questions for discussion
• Q1• Q2