integrating icall into synchronous cmc can intelligent computer-generated feedback be provided in...
TRANSCRIPT
Integrating ICALL into synchronous CMC
Can intelligent computer-generated feedback be provided in learner-to-learner interactions?
Markus Dickinson, Soojeong Eom, Natalia Jacobsen, Yunkyoung Kang, Chong Min Lee, Ken Petersen, & Rebecca Sachs
iicall2 conference; December 8, 2007; Waterloo
Intelligent CALL (ICALL)
ICALL provides many potential means of facilitating L2 development Generation of detailed information regarding sources of
learner errors
Precise feedback on learner errors
Can be tailored to learners’ common mistakes, activity goals, proficiency levels, abilities, etc.
Fosters awareness of relevant language forms
Tracking of improvement across exercises, using learner models
(cf. Amaral & Meurers 2006, Heift & Schulze 2007)
Intelligent CALL: Limitations However, there is a tension between
Contextualized language use is increasingly emphasized, and with increasing success (e.g., Amaral et al. 2006; Amaral & Meurers 2006)…
But actual communicative interaction is relatively unexplored (though see Petersen 2006)
In order to manage computational complexity: Exercises are often restricted to the sentence level Activities often do not simulate true communication
The ability of an ICALL system to provide
meaningful, accurate feedback
The flexibility an ICALL system allows for in terms of meaningful, communicative
interaction
Synchronous CMC Synchronous computer-mediated communication (CMC) between
L2 learners can provide a beneficial context for L2 development
Focus on meaningful communication Activities can approximate ‘target tasks’ relevant to real-life
communicative situations, with learners functioning as language users (Ellis 2003)
Possibly optimal psycholinguistic environment for learners to negotiate for meaning and make form-meaning connections (Blake 2000, Doughty & Long 2003)
Less pressing time constraints and less ephemeral language (e.g., through possibility of re-reading) in the written modality Reduced processing demands, which may serve as an equalizer for
learners with lower working memory capacity (Payne & Whitney 2002) Attention to form and monitoring of language production (Chapelle
2003, Pellettieri 2000) Use of more complex language (Warschauer 1996)
Synchronous CMC: Limitations Concerns regarding the quality of learner-learner interactions with
respect to potential benefits for learning
The blind leading the blind? Without feedback from a trusted authority, learners might reinforce
each other’s errors (Kern 1995) and/or simply not have the resources necessary for correcting each other (Blake 2000)
In negotiating for meaning, learners may naturally tend to focus on lexis without attending to morphosyntax (Blake 2000)
Learners might not be concerned with grammatical accuracy (Kern 1995), in which case available planning time is a moot point
Teachers may need to set explicit expectations for grammatically correct language, while simultaneously trying to balance this with a primary focus on meaningful communication (Lee 2001, Pellettieri 2000)
Combining the benefits Can ICALL and synchronous CMC be integrated in
a way that exploits the unique benefits of each while avoiding their limitations?
Intelligent computer-generated feedback in synchronous, task-based, computer-mediated
learner-learner interactions
A balancing act We want learners’ interactions to be as free as
possible, promoting authentic communication
At the same time, we are restricted by limitations of ICALL processing and must constrain the situation somehow (cf. discussion in Amaral et al. 2006)
Guided picture-based information-gap tasks can constrain the vocabulary and domain in ways that: reduce many of the complexities involved in generating
feedback (computationally speaking) still allow for communicative interaction between learners
(interactional authenticity if not real-world authenticity)
Participants English-speaking university students in first-year
Korean classes
Can (or should) beginning learners use CMC? Important for learners to develop the ability to use language
to communicate meaning as opposed to simply ‘displaying’ language or manipulating L2 forms systematically without attending to meaning
However, unconstrained communicative tasks might be stressful (or a long shot) for beginning learners
We hope to design our tasks in such a way that participants will be comfortable communicating meaningfully in the L2 With appropriate guidance Within a constrained and familiar environment
A communicative task we’re considering: Spot the differences Each participant sees one version of a house and must exchange
information in the L2 in order to find similarities and differences between the two versions
Not so authentic in terms of real-world relevance, but ‘interactionally authentic’ – the sort of task often used in interaction research to target specific areas of language and promote negotiation and L2 learning
To increase motivation, set up as a competition between pairs of participants Record (in a provided chart) the activities and locations of all
characters in partner’s house Indicate whether each of these represents a similarity or a
difference between the pictures Compare scores at the end to those of other participant pairs
Spot-the-differences task (picture 1)
Spot-the-differences task (picture 2)
Target of feedback: Korean particles In Korean (as in Japanese, cf. Nagata 1995),
postpositional particles are used to indicate grammatical functions and thematic roles (e.g., who is doing what to whom).
They must be used even in simple sentences. Particles are taught from the beginning of L2 Korean
study, but: The system is quite complex and difficult to master for adult
learners of Korean Particle errors account for a substantial proportion of the
mistakes made by beginning learners (Ko et al. 2004) Errors persist even at advanced levels
Difficulties with Korean particles No one-to-one correspondence between Korean
locative particles and English prepositions
Korean locative particles mark distinctions not made in English
에 (‘e’)
에게 (‘ege’)
께 (‘kke’)
한테 (‘hante’)
English ‘to’
에 (‘e’)
에서 (‘eseo’)
을 / 를 (‘eul/reul’)*
English ‘in’
Location of a static object
Location of a dynamic activity
(*also an object marker)
Location an action moves through
Examples of targetlike and non-targetlike Korean particle use What is in the kitchen?
주방 - 에 뭐 - 가 있어요 ? 주방 - 에 뭐 -* 를 있어요 ?kitchen-LOC what-SUBJ is kitchen-LOC what-OBJ is
Father is grilling meat. 아버지 - 가 고기 - 를 구어요 . 아버지 - 가 고기 -* 가 굽고
있다 .father-SUBJ meat-OBJ grill father-SUBJ meat-SUBJ is grilling
A cat is in the living room. 고양이 - 가 거실 - 에 있어요 . 고양이 - 가 거실 -* 에서 있다 .
cat-SUBJ living room-LOC is cat-SUBJ living room-LOC is
(TL examples are on the left with particles in green; non-TL are on the right with particles in red)
SS
S
D
S D
S D
S
D
S D
S D
D D
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
TASK PICTURE:
Partners have slightly different versions and must communicate to find differences. They can scroll over the picture to enlarge it.
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
WORD & PARTICLE BANKS:
To create a sentence, participants click on
words and particles…
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
…which then appear in the
sentence drafting area.
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D DIf they want help with
Korean particle usage, they can
request feedback on their sentences
before entering them into the conversation.
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
FEEDBACK AREA: Here, participants receive metalinguistic feedback with advice on particle usage.
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
When they are ready, they click SEND to
enter their utterance into the conversation.
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
CHAT WINDOW: They can scroll up and down to review the conversation so far.
[ParticipantA]: 오빠가 어디에 있어요 ?[ParticipantB]: 거실에 있어요 .[ParticipantA]: 오빠가 거실에서 뭐해
요 ? [ParticipantB]: …
CHECK SEND
오빠가 소파를 책을 읽어요 .
Word Bank Particles
에 침대
의자
책
화장실
거실
읽었어요
는 이
…
In your sentence, 소파 is marked with the particle 를 , which
suggests that 소파 is an object. Instead, you
need the particle 에서 attached to 소파 in order to indicate that
소파 is the location of a dynamic activity.
울었어요 소파
에서 가 을
CHARACTER LOCATION ACTIVITY
Mother
Father
Grandma 주방 읽다…
S
S
D
S D
S D
S
S
D
S D
S D
D D
GAME RECORD: When participants find similarities or differences, they drag the relevant words for locations and activities here to record information about their partners’ pictures, then click on ‘S’ or ‘D’ to indicate whether the pictures match in those respects or not.
The importance of instructions Piloting so far (with native speakers of Korean in
face-to-face interactions) has suggested that carefully worded instructions will likely be needed to: Encourage participants to use complete sentences
Emphasize that the characters in the pictures are members of the same family (so that terms such as ‘mother’, ‘brother’, etc., can be used naturally)
Clue participants in to the fact that differences will involve the locations and activities of family members, as opposed to characteristics of furniture (for example)
From activity to processing How can we process CMC as input? Built-in constraints
Intensive feedback is provided on one particular error type Learners’ construction of sentences is guided by…
The nature of the picture-based task (constrains vocabulary) Instructions and the game record chart Word and particle banks, which limit the types of argument
structure by limiting the verbs that can be used, and which moreover…
+ May be necessary for beginning learners who can’t type in Korean
+ May serve as a scaffold for using receptive vocab in conversation
Upshot: Processing can focus just on detecting particle errors in a known domain.
Intensive feedback Some have argued that intensive feedback may be more effective
in certain contexts than wide-ranging incidental feedback on a variety of error types (e.g., Lyster 1998, Nicholas, Lightbown, & Spada 2001) Intensive feedback = focusing consistently on one pre-selected
error type Meta-analyses have not found significant differences, but too few
primary studies to tell for sure; in any case, both produce large effect sizes (e.g., Mackey & Goo 2007, Russell & Spada 2006)
In our study, we will inform learners that they will only be receiving feedback on particles Important for meaning (i.e., communicating who is doing what to
whom) in Korean Will hopefully prevent learners from mistaking non-feedback for
correctness Leaves open the possibility of providing other feedback, if needed
Meaningful communication? A spot-the-differences task provides a possible forum for
learners to use an L2 in purposeful, communicative ways, but…
How much will the participants focus on meaningful communication if it’s clear that the feedback is focusing exclusively on Korean particles?
What else can we do to encourage a balance between focusing on intended meanings and focusing on the forms required to express those meanings accurately?
How free will the participants be to express a creative range of ideas in the context of this task?
What can we do to make the word bank sufficiently rich for their communicative purposes?
How fluid will communication feel if participants have to take the time to search for each word they want to use?
What can we do to make the words in the bank as easy to access as possible?
Future directions Pilot the game with L2 learners
Get a clearer sense of what to expect in learner input Test how the word bank actually plays out
Develop the system Activity, error diagnosis, and feedback modules Processing techniques
Make the tasks more complex, meaningful, and relevant to real-life communicative situations Create realistic rationales for having to find differences (e.g., a
detective and witness discussing a crime scene as it appears just before and after a crime)
Explore other sorts of picture-based information-gap tasks Remove or expand the word bank
This would require additional constraints, however, to keep processing feasible
Develop activities to target more constructions
Opportunity to test techniques A combination of techniques will ultimately be used. Linguistic processing will be kept separate from error
diagnosis and feedback generation.
We can use this setting to experiment with different processing technologies
1. More traditional, anticipation-based regular expressions
2. Basic linguistic abstraction of POS tags and chunks to provide annotation of input (cf. Amaral & Meurers 2006)
3. Experimental ill-formed input checking Use mismatches in 2 different parsers to detect particle errors
(cf. Metcalf & Boyd 2006) One parser captures particle usage patterns from real language One parser captures general argument structure patterns between
words, irrespective of particles
Opportunity to test questions of SLA theory and language pedagogy We will be able to explore learners’ uses of various
types of feedback in CMC, using pre-test / treatment / post-test designs to assess L2 development Intensive vs. non-intensive feedback? Metalinguistic information vs. prompts vs. recasts? (etc.) Content-oriented feedback?
Eventually, we can use such a set-up to investigate ways of optimizing feedback: For different areas of language For learners of different proficiency levels For learners with different aptitude profiles
Using and integrating this system with the Korean language curriculum at Georgetown
Questions or comments?
Please email: Markus Dickinson [email protected] Soojeong Eom [email protected] Natalia Jacobsen [email protected] Yunkyoung Kang [email protected] Chong Min Lee [email protected] Ken Petersen [email protected] Rebecca Sachs [email protected]
References Amaral, L., & Meurers, D. (2006). Where does ICALL fit into foreign
language teaching. CALICO 2006. University of Hawai’i. Amaral, L., Metcalf , V., & Meurers, D. (2006). Language awareness
through re-use of NLP technology. Pre-conference Workshop on NLP in CALL – Computational and Linguistic Challenges. CALICO 2006. University of Hawai’i.
Blake, R. (2000). Computer-mediated communication: A window on L2 Spanish interlanguage. Language Learning & Technology, 4(1), 120-136.
Chapelle, C. (2003). English language learning and technology. Philadelphia: John Benjamins.
Doughty, C.J., & Long, M.H. (2003). Optimal psycholinguistic environments for distance foreign language learning. Language Learning & Technology, 7(3), 50-80.
Ellis, R. (2003). Task-based language learning and teaching. Oxford: Oxford University Press.
Heift, T., & Schulze, M. (2007). Errors and intelligence in computer-assisted language learning: Parsers and pedagogues. Routledge.
References Kern, R. G. (1995). Restructuring classroom interaction with networked
computers: Effects on quantity and characteristics of language production. The Modern Language Journal, 79, 457-473.
Ko, S., Kim, M., Kim, J., Seo, S., Chung, H., & Han, S. (2004). An analysis of Korean learner corpora and errors. Hanguk Publishing Co.
Lee, L. (2001). Online interaction: Negotiation of meaning and strategies used among learners of Spanish. ReCALL Journal, 13(2), 232-244.
Lyster, R. (1998). Form in immersion classroom discourse: In or out of focus? Canadian Journal of Applied Linguistics, 1, 53-82.
Mackey, A., & Goo, J. (2007). Interaction research in SLA: A meta-analysis and research synthesis. In A. Mackey (Ed.), Conversational interaction and second language acquisition: A series of empirical studies (pp. 407-452). Oxford: Oxford University Press.
Metcalf, V., & Boyd, A. (2006, December). Head-lexicalized PCFGs for verb subcategorization error diagnosis in ICALL. Workshop on Interfaces of Intelligent Computer-Assisted Language Learning. The Ohio State University.
References Nagata, N. (1995). An effective application of natural language processing in
second language instruction. CALICO Journal, 13(1), 47-67. Nicholas, H., Lightbown, P. M., & Spada, N. (2001). Recasts as feedback to
language learners. Language Learning, 51, 719-758. Payne, J. S., & Whitney, P. J. (2002). Developing L2 oral proficiency through
synchronous CMC: Output, working memory, and interlanguage development. CALICO Journal, 20(1), 7-32.
Pellettieri, J. (2000). Negotiation in cyberspace: The role of chatting in the development of grammatical competence in the virtual foreign language classroom. In M. Warschauer & R. Kern (eds.), Network-based language teaching: Concepts and practice, 59-86. Cambridge: CUP.
Petersen, K. (2006, December). Measuring L2 development in an ICALL context. Workshop on Interfaces of Intelligent Computer-Assisted Language Learning. The Ohio State University.
Russell, J., & Spada, N. (2006). The effectiveness of corrective feedback for the acquisition of L2 grammar. In J. M. Norris & L. Ortega (Eds.), Synthesizing research on language learning and teaching. Amsterdam: John Benjamins.
Warschauer, M. (1996). Comparing face-to-face and electronic discussion in the second language classroom. CALICO Journal, 13(2), 7-26.