interactive stories and exercises with dynamic feedback for

11
Interactive stories and exercises with dynamic feedback for improving reading comprehension skills in deaf children Ornella Mich * , Emanuele Pianta, Nadia Mana Fondazione Bruno Kessler, via Sommarive 18, 38123 Povo, TN, Italy article info Article history: Received 15 September 2012 Received in revised form 7 January 2013 Accepted 21 January 2013 Keywords: Elementary education Intelligent tutoring systems Interactive learning environments abstract Deaf children have signicant difculties in comprehending written text. This is mainly due to the hearing loss that prevents them from being exposed to oral language when they were an infant. How- ever, it is also due to the type of educational intervention they are faced with, which accustoms them to decoding single words and isolated sentences, rather than entire texts. This paper presents an evolved version of a literacy web tool for deaf children based on stories and comprehension exercises. Two substantial improvements were made with the respect to the rst version of our application. First, the text of the stories is now presented to children in the context of animated web pages. Second, intelligent dynamic feedback is given to the users when resolving the exercises. A preliminary evaluation study with deaf children, as the treatment group, and hearing children, as the control group, assessed the usability and effectiveness of the new system and its graphical interface. Ó 2013 Elsevier Ltd. All rights reserved. 1. Introduction A recent study on 2-days-old hearing infants (Perani et al., 2011) has demonstrated that at such an early stage of brain life the language- related neural substrate is fully active in both hemispheres, thereby providing a strong biological basis for language acquisition. However, functional and structural connectivities are still immature, so that progressive maturation of functional connectivity needs to be established through exposure to language as the brain develops. Pre-lingual deaf children lack this important phase in language development, thus failing to acquire adequate literacy skills (Fabretti, 2000; UNESCO, 1987). As several studies demonstrate, there is a delay in learning to read and write in deaf children compared to hear- ing children (Musselman, 2000; Tomasuolo, 2006; Traxler, 2000). Mean reading ages of deaf students are several years below chronological age (Allen, 1983; Dyer, Szczerbinski, MacSweeney, Green, & Campbell, 2003; Furth, 1966; Harris & Terlektsi, 2011; Paul, 1998). For these reasons, deaf students need explicit instruction in academically relevant skills (reading strategies) that are acquired inci- dentally by hearing students (Borgna, Convertino, Marschark, Morrison, & Rizzolo, 2011). New technologies may effectively support literacy learning (Cedric Wachholz, 2006; Lachs, 2000). This is especially true for children who have special educational needs, as deaf children do (Loeterman, Paul, & Donahue, 2002; Shamir & Shlafer, 2011). Research has demonstrated that multimedia tools based on various combinations of print, pictures, sign language-enhanced video, graphics, and animation is effective in teaching reading to deaf children (Gentry, Chinn, & Moulton, 2005). Cornerstones (Loeterman et al., 2002), a complete system for classroom literacy instruction dedicated to deaf and hard-of-hearing children attending primary school, is one of such tools. Cornerstonesgoal is to work on vocabulary development and decoding skills through the study of written stories enhanced with engaging media components. Another interesting literacy tool is See and See (Nikolaraizi & Vekiri, 2011), developed by the Department of Special Education, University of Thessaly, Greece. It proposes to deaf students narrative texts selected from national school text books, completed with visual aids, such as Greek Sign Language (GSL) videos, pictures and concept maps, and with reading comprehension questions. Highlighter (Cornoldi and Garofalo, 2009), a commercial literacy tool developed by Anastasis (2009) with the collaboration of professor Cornoldi, an expert in psycholinguistics at the Italian University of Padua, is also an effective tool for kids with learning disabilities in reading com- prehension. Highlighter, designed for children attending primary or middle schools, aims to improve those abilities that allow the reader to * Corresponding author. Tel.: þ39 0461 314 582; fax: þ39 0461 314 591. E-mail addresses: [email protected], [email protected] (O. Mich). Contents lists available at SciVerse ScienceDirect Computers & Education journal homepage: www.elsevier.com/locate/compedu 0360-1315/$ see front matter Ó 2013 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.compedu.2013.01.016 Computers & Education 65 (2013) 3444

Upload: doandan

Post on 13-Feb-2017

254 views

Category:

Documents


7 download

TRANSCRIPT

Computers & Education 65 (2013) 34–44

Contents lists available at SciVerse ScienceDirect

Computers & Education

journal homepage: www.elsevier .com/locate/compedu

Interactive stories and exercises with dynamic feedback for improvingreading comprehension skills in deaf children

Ornella Mich*, Emanuele Pianta, Nadia ManaFondazione Bruno Kessler, via Sommarive 18, 38123 Povo, TN, Italy

a r t i c l e i n f o

Article history:Received 15 September 2012Received in revised form7 January 2013Accepted 21 January 2013

Keywords:Elementary educationIntelligent tutoring systemsInteractive learning environments

* Corresponding author. Tel.: þ39 0461 314 582; faE-mail addresses: [email protected], ornella.mich@gma

0360-1315/$ – see front matter � 2013 Elsevier Ltd. Ahttp://dx.doi.org/10.1016/j.compedu.2013.01.016

a b s t r a c t

Deaf children have significant difficulties in comprehending written text. This is mainly due to thehearing loss that prevents them from being exposed to oral language when they were an infant. How-ever, it is also due to the type of educational intervention they are faced with, which accustoms them todecoding single words and isolated sentences, rather than entire texts. This paper presents an evolvedversion of a literacy web tool for deaf children based on stories and comprehension exercises. Twosubstantial improvements were made with the respect to the first version of our application. First, thetext of the stories is now presented to children in the context of animated web pages. Second, intelligentdynamic feedback is given to the users when resolving the exercises. A preliminary evaluation study withdeaf children, as the treatment group, and hearing children, as the control group, assessed the usabilityand effectiveness of the new system and its graphical interface.

� 2013 Elsevier Ltd. All rights reserved.

1. Introduction

A recent study on 2-days-old hearing infants (Perani et al., 2011) has demonstrated that at such an early stage of brain life the language-related neural substrate is fully active in both hemispheres, thereby providing a strong biological basis for language acquisition. However,functional and structural connectivities are still immature, so that progressive maturation of functional connectivity needs to be establishedthrough exposure to language as the brain develops.

Pre-lingual deaf children lack this important phase in language development, thus failing to acquire adequate literacy skills (Fabretti,2000; UNESCO, 1987). As several studies demonstrate, there is a delay in learning to read and write in deaf children compared to hear-ing children (Musselman, 2000; Tomasuolo, 2006; Traxler, 2000). Mean reading ages of deaf students are several years below chronologicalage (Allen, 1983; Dyer, Szczerbinski, MacSweeney, Green, & Campbell, 2003; Furth, 1966; Harris & Terlektsi, 2011; Paul, 1998).

For these reasons, deaf students need explicit instruction in academically relevant skills (reading strategies) that are acquired inci-dentally by hearing students (Borgna, Convertino, Marschark, Morrison, & Rizzolo, 2011).

New technologies may effectively support literacy learning (CedricWachholz, 2006; Lachs, 2000). This is especially true for childrenwhohave special educational needs, as deaf children do (Loeterman, Paul, & Donahue, 2002; Shamir & Shlafer, 2011). Research has demonstratedthat multimedia tools based on various combinations of print, pictures, sign language-enhanced video, graphics, and animation is effectivein teaching reading to deaf children (Gentry, Chinn, & Moulton, 2005). Cornerstones (Loeterman et al., 2002), a complete system forclassroom literacy instruction dedicated to deaf and hard-of-hearing children attending primary school, is one of such tools. Cornerstones’goal is to work on vocabulary development and decoding skills through the study of written stories enhanced with engaging mediacomponents. Another interesting literacy tool is See and See (Nikolaraizi & Vekiri, 2011), developed by the Department of Special Education,University of Thessaly, Greece. It proposes to deaf students narrative texts selected from national school text books, completed with visualaids, such as Greek Sign Language (GSL) videos, pictures and concept maps, and with reading comprehension questions. Highlighter(Cornoldi and Garofalo, 2009), a commercial literacy tool developed by Anastasis (2009) with the collaboration of professor Cornoldi, anexpert in psycholinguistics at the Italian University of Padua, is also an effective tool for kids with learning disabilities in reading com-prehension. Highlighter, designed for children attending primary or middle schools, aims to improve those abilities that allow the reader to

x: þ39 0461 314 591.il.com (O. Mich).

ll rights reserved.

O. Mich et al. / Computers & Education 65 (2013) 34–44 35

detect the most relevant information in a text for understanding the entire text, to memorize it in the working memory and to update itduring the entire reading process.

The goal of this paper is to introduce our literacy application, LODE (LOgic-basedweb tool for DEaf children), which aim is to facilitate thecomprehension of temporal relations in narratives. It proposes:

(a) interactive illustrated stories: we based our tool on stories, because several research studies demonstrated that it is very effective usingstories to teach reading and writing to young students (Calliari & Degasperi, 2007; Smallwood, 2002). We enriched our stories withillustrations, as they help readers improving comprehension of the read text (Brookshire, Scharff, & Moses, 2002; Gambrell & BrooksJawitz, 1993; Waddill, McDaniel, & Einstein, 1988);

(b) a visual dictionary: a good dictionary is essential to help the reader in improving her/his vocabulary (Nagy, Anderson, & Herman, 1987;Pressley, 2000). In our dictionary, we decided to primarily use pictures to illustrate the meaning of words because deaf children arevisual learners (Roccaforte, DeMonte, Groves, Tomasuolo, & Capuano, 2011), i.e. learn most from photos, digital images, flash cards orclassroom posters;

(c) comprehension exercises with intelligent feedback: comprehension exercises allow the reader to monitor his/her reading comprehensionlevel. LODE improves traditional comprehension exercises with an intelligent and appropriate feedback (Murphy, 2007). Feedbackinteracts with students’ motivation and beliefs (El, Tillema, & van Kopper, 2012) and helps students take control of their own learning,favouring the adoption of a self-regulated learning model (Nicol & Macfarlane-Dick, 2006).

In the rest of this paper, Section 2 elaborates, first, on the assumption that using stories to teach reading and writing is effective above allfor young students and then, on the role of illustrations in children books. Section 3 introduces LODE in detail, describing how it wasdesigned and implemented. More specifically, Section 3.2.1 describes LODE’s dictionary and Section 3.3.2 describes LODE’s exercises. Then,Section 4 presents the LODE evaluation, performed involving deaf and hearing children. Finally, the paper concludes discussing the eval-uation results and reporting on directions for future research.

2. Illustrated stories for language learning

This section explains why we chose to base our application on narratives. Our decision is due to the fact that children’s literature offersa complete medium for learning. Indeed, carefully chosen stories allow children to develop their receptive language in an entertaining,meaningful context. Moreover, they provide background knowledge and cultural information, just as emotional, social, and intellectualstimuli for building their own cognition (Calliari & Degasperi, 2007; Mar, Oatley, Djikic, & Mullin, 2011; Smallwood, 2002).

Tales and stories provide a whole imaginary world, created by language and images, that children can enter and enjoy. Stories stimulatestrong emotions of sympathy, anger, fear, anxiety and so forth, opening the door to language learning.

Rich and colourful illustrations are used in most children’s narrative books. They are mainly introduced to transform books intoenchanting and engaging objects, with the aim of attracting young readers to read them.

Engagement is essential in the learning process (Mosenthal, 1999). However, illustrations do not only play a role of embellishment. Theyalso have a critical cognitive function. When appropriately designed, they help children better catch and memorize what they are reading(Brookshire et al., 2002; O’Keefe & Solman, 1987; Waddill et al., 1988).

When may illustrations be considered “well designed for children’s literature”?First of all, illustrations must be text-relevant, i.e. (a) the information conveyed must be central to the text, (b) they must be congruent

with the text content and (c) they must provide a spatial or schematic representation of the interrelations of the text content (Schallert,1980). When illustrations are created in this way, they favour the building of an efficient mental image (Gambrell & Brooks Jawitz, 1993),which is an important cognitive strategy involved in the reading comprehension process (Pressley, 2001). If illustrations are unrelated to thetext, they may produce interferences with the comprehension of the text content (Willows, 1978).

Secondly, illustrations must be physically close to relative text (O’Keefe & Solman, 1987). They should not only illustrate details conveyedin a particular proposition but also illustrate information conveyed by the interrelationship of several propositions in the text: this willincrease recall of fairy tales’ content (Waddill et al., 1988).

Moreover, they should be simple but not minimalist and created with an open mind, taking into account the generally different pro-spective of children with respect to adults in facing problems (Alessio, 2010). Some studies demonstrated that children prefer booksillustrated in a realistic style andwith bright images rather then books illustrated in an abstract style andwith sober images (see for exampleBrookshire et al. (2002) who studied the effect of different types of illustrations on reading comprehension of first and third graders).

A help to build effective illustrations for stories comes from an interesting book by Kress and Leeuwen (2006).

3. Our application: LODE

LODE is aweb application aiming to support deaf children inworking on their reading comprehension skills. In particular, LODEworks ontemporal relations comprehension. The application is based on animated stories and engaging exercises developed as games. The languageused in LODE’s stories is simplified by experts to favour the child’s focussing on the inference aspect and not on other factors involved in thereading comprehension skills (Whitehurst & Longian, 1998). An automatic temporal reasoner is used to create dynamic comprehensionexercises.

We have mainly worked on two LODE prototypes, indicated respectively by LODE-1 and LODE-2 below. The decision to work on twoprototypes is due to the fact that we preferred to separately evaluate the main features of our system. LODE-1, described in 3.2, was used toanalyse the effectiveness of the automatic reasoner applied to the LODE’s exercises. LODE-2, described in Section 3.3, was used to analyse theeffectiveness of story simplification, of the graphical interface and of the exercises’ dynamic feedback.

In the following, we will first give a short presentation of LODE’s design process. Then, we will move on to the description of the twoprototypes.

O. Mich et al. / Computers & Education 65 (2013) 34–4436

3.1. Design process

We designed and realized our prototypes following a user-centred approach. An iterative design process, formed from rapid prototyping,user testing and incorporating user feedback into subsequent versions, was adopted.

The concept of LODE was preliminarily presented to a group of experts (speech therapists, interpreters, psychologists, teachers, andresearchers) with the aim of checking its validity. Experts were involved in two ways: first, through an interview; second, they were invitedto work with an on-line LODE demo, and then their comments were collected by an observer.

Then, a few deaf children evaluated a preliminary mock-up build on the basis of the experts’ suggestions.Afterwards, an improved mock-up was developed taking into account the problems that arose with the first evaluation step. This new

prototype was tested by a group of usability experts to quickly discover the most evident usability problems.The results of these tests allowed the implementation of a complete version of LODE’s graphical interface, with several exercises and

a complete story, connected to a preliminary version of the automated reasoner. This was the LODE-1 prototype, which was subsequentlyimproved through a series of evaluation steps with hearing and deaf children. A complete description of all these design phases can be foundin Mich (2010).

Once reached a reasonably stable version of LODE, weworked on a second prototype, LODE-2, with the aim of studying the effectivenessof a different graphical interface and improved feedback. This prototype was repeatedly evaluated with hearing and deaf children too. Thispaper specifically reports on the final evaluation (see Section 4).

3.2. LODE-1

LODE has a web-based client-server architecture (Fig. 1). The main modules of the server are: (a) the e-stories’ database (DB), which isa simple repository structured as a file system, (b) the ECLiPSe constraint programming system (Apt & Wallace, 2006) (RSN), and (c) theNatural Language Processor (NLP). A switchmodule, the controller (CR), manages the communications between the client and the server, andorganises the flow of data and commands between them. The client, i.e. the GUI, is an AJAX application compatible withmost web browsers,e.g. Firefox-Mozilla, Internet Explorer, Chrome, Safari and Opera. It works as the interface between the user and the real system, i.e. theserver, running on a remote machine. Fig. 2 shows some screen-shots of the LODE-1 graphical interface.

There are two main data flows in LODE (see Fig. 1): an unidirectional data flow and a bidirectional data flow.The unidirectional data flow refers to the path that allows for the creation of the stories’ archive. It starts at the GUI, where the stories are

uploaded. Then, it moves on to the controller that connects the GUI to the NLPmodule. Here the stories are first processed and then stored inthe DB together with the illustrations, the words which are probably unknown to deaf children and the events. The events and the temporalrelations are elaborated by the RSN. LODE’s maintainers or teachers using LODE at school are the managers of this process.

The bidirectional data flow refers to the path that data and commands follow during the use of LODE by the children. This is a completelyautomated process. When the user is reading a story, his/her commands arrive at the controller that activates the DB. The DB gives back datato the controller that gives them to the GUI. When the user is doing an exercise, the controller activates the RSN that returns feedback to theGUI through the controller.

3.2.1. LODE’s dictionaryA dictionary has been added to LODE for two reasons. First, consulting a dictionary has an important role in the language learning

process. In fact, a strategic use of a dictionary can improve the cognitive processes required for vocabulary acquisition: attention to form-meaning connections, rehearsal of words for storage in long term memory and elaboration of associations with other knowledge (Fraser,1998). Moreover, consulting the dictionary facilitates the grapheme-meaning association in beginner readers. Second, it is well knownthat deaf children have problems with vocabulary (see for example Merrills, Underwood, and Wood (1994)); an effective dictionary mayhelp them in comprehending what they are reading.

LODE’s dictionary presents the most unusual words for deaf children, used in the chosen stories. They could be nouns, abstract nouns,adjectives or adverbs of the narrative. In this manner, the level of comprehension of the story is simplified. In LODE-1 single words are

Fig. 1. LODE: a diagram of its architecture.

Fig. 2. LODE-1: some screen-shots.

O. Mich et al. / Computers & Education 65 (2013) 34–44 37

proposed on the screen together with an image explaining their meaning; example sentences based on these words could also be shown(Fig. 2b).

3.3. LODE-2

LODE-2 is an improved version of LODE-1. Themain differences between the two versions concern the graphical presentation of the storypages, which will be described in Section 3.3.1, and the feedback given by the exercises to the users, which will be described in Section 3.3.3.

3.3.1. Representation of story pagesThe story pages in LODE-2 can be classified into two groups: illustrated pages and textual pages. The illustrated pages have an image as

background. When the child first gets to one of these pages, he/she will see only the image (see Fig. 3b). To be able to proceed in reading thestory he/she needs to find an active area of the image and click on it. Active areas are signalled by simple animations. The click will opena text boxwith a textual piece of the story (see Fig. 3c). The user has tomove themouse around the image to find the active area, because theposition of the animated area changes from one page to another. The systemwill visualize the arrow that allows the child to move on to thenext story page only after the child has clicked on the active area and opened the text box. This is an attempt to focus the child on the task ofreading the text as opposed to just browsing through the story images. Textual pages are a small group of pages which include only text (seeFig. 3a). This type of page has been introduced to emphasize the focus of the tool, which is that of letting the user work on her/his readingskills. The number of illustrated pages is however higher than that of text-only pages, to keep the tool entertaining for children.

When the user reaches the end of a story, the system presents him/her with a series of comprehension exercises, aiming to reinforce thereading comprehension skills in the users (see Fig. 4 for three examples of implemented exercises).

LODE’s stories were simplified to have more readable texts with rising level of lexical and syntactic complexity (from level 1 to 3). Thesimplification process was made in two steps: first, the number of subordinate clauses, pronouns and clitics was reduced; then, words thatare not included in the list of words most used in Italian children books (Marconi, Ott, Pesenti, Ratti, & Tavella, 1993) have been eitherparaphrased or substituted with synonyms which do appear in the list. The readability of the simplified texts was measured using theGulpease index (Lucidano & Piemontese, 1988), which is a readability test designed to gauge the understandability of a text. It value goesfrom 0 (corresponding to a less readable text) to 100 (corresponding to a most readable text). In general, a text with a Gulpease index lowerthan 80 is difficult for people who only have the Primary school licence, whereas a text with a Gulpease index lower than 60 is difficult for

Fig. 3. LODE-2: some screen-shots.

O. Mich et al. / Computers & Education 65 (2013) 34–4438

people who have the Secondary school licence and a Gulpease index lower than 40 is difficult for people who have the High school licence.After the modify procedure, we made sure that the new version of stories have a Gulpease index between 80 and 100.

The story text is visualized inside a box shaped as an old parchment. A Sans-Serif typeface was used for the text, because this type of fontis much easier to look at on screen (Lewis & Walker, 1989) and easier to read (Gump, 2001). Research studies also demonstrated that usersprefer Sans Serif fonts for Website text (Shaikh et al., 2006).

Text is visualized with a large font size, because this improves text readability, above all for young readers (DeLamater, 2010; Hughes &Wilkins, 2002).

Some words are highlighted in yellow. These are words that experts selected as difficult or unusual for deaf children (Rinaldi & Caselli,2009). The words highlighted in yellow are linked to a visual dictionary, enhanced with respect to LODE-1 with a video in Italian SignLanguage (see Fig. 3d).

The images used to illustrate the LODE stories always represent the content of the text associated with them. As said in Section 2, onlywhen story’s illustrations are text-relevant do they favour the reading comprehension process. They offer a context for the text she/he willthen read. More specifically, they have been designed to illustrate those main story events which are used to build the final comprehensionexercises. However, special attention was paid not to create overtly rich images which could have overwhelmed the text.

During the moving around the mouse phase, the child activates her/his prior knowledge about the image elements, e.g. by moving themouse over Fig. 3b, she/hemay activate her/his knowledge about the concepts of hen, eagle, tree, stone, etc. and about the relations betweenthem. This course will favour the entire reading comprehension process (Vacca & Vacca, 2003).

LODE images were materially prepared by a team composed by: (a) a university student in Art and Design, (b) an expert in illustrationsfor children’s book, who supported the student in organizing the story board, and (c) a software programmer, who supported the student increating images with the right ratio-aspect and with the most suitable colours for the WEB.

3.3.2. ExercisesAfter reading an entire story, children are invited to resolve some exercises. Exercises are implemented as games which ease and

reinforce learning, as several studies demonstrated (Johnson, Adams, & Haywood, 2011; Munz, Schumm, Wiesebrock, & Allgwer, 2007;Virvou, Katsionis, & Manos, 2005).

Fig. 4. Examples of comprehension exercises.

O. Mich et al. / Computers & Education 65 (2013) 34–44 39

After an introductory warm-up exercise that has the goal of making the user familiar with the metaphor of the left-to-right arrow asmetaphor of temporal ordering, LODE proposes three types of exercises: (a) picture reordering exercises (see Fig. 4a), where the child has tore-build the story through the reordering of some images, (b) word reordering exercises (see Fig. 4c), where the child has to drag and dropa list of word to compose a sentence which describes the temporal relation between two story events, and (c) relation choosing exercises,where the child needs to choose between three LODE temporal relations connecting a pair of events of the story; each event is described inwords and by the related image in the story (see Fig. 4b) or only in words (see Fig. 4d).

The game/exercises are proposed to the child following a predetermined and motivated order. The first exercise has the goal of assessingthe global comprehension of the story and uses only images. The second exercises assess the comprehension of local temporal relations. Theexercises of this second phase use firstly a combination of images and text. Finally, the last exercise is based on pure text, without images.

3.3.3. Exercise feedbackFeedback is an essential part of the educational process. In our system, we have introduced intelligent and rewarding feedback. The

system first tells the user if the exercise is wrong or correct (see Fig. 5a and b). In the case of positive feedback, the message given alwayschanges, i.e. the system casually extracts a positive expression among a list of suitable expressions (e.g. great, wonderful, OK, etc.) aiming toresemble real teacher feedback. In the case of negative feedback, the system invites the child to reread the story’s pages that contain theinformation necessary to correctly resolve the exercise the user did wrong. If the child accepts the invitation, the system moves to exactlythose pages, otherwise he/she can retry the same exercise or move on to the following one.

Fig. 5. Examples of exercises feedback.

O. Mich et al. / Computers & Education 65 (2013) 34–4440

In addition to the described feedback, we introduced another feature to improve not only the educational effectiveness of our system butalso its appeal to children. This new feature consists of a medal panel where all the exercises’ names are listed. When the child finishes anexercise, an icon appears near the exercise name in the list: a green check-mark when the exercise was correctly answered and a red crossotherwise. The child can check the state of his/her work session by opening the medal panel with a click on a medal-shaped icon positionedon the right top corner of the exercise page. At the end of the work session, when the child completes the exercises series, the medal panel isautomatically visualized (see Fig. 6). When an exercise was correctly done at first attempt, a gold medal is visualized close to the exercisename, whereas in the case of awrong answer the system invites the child to re-try the exercise. The user only has to click on theword redo tobe moved to the exercise page. The objective of the medal panel is to stimulate children to not randomly answer the multiple-choicesexercises.

3.3.4. The entire working processThis section presents thework-flow of a complete working sessionwith LODE. To explain all the features of our application, it is sufficient

to consider aworking sessionwith a story only composed of two pages, an illustrated page and a textual page, and two exercises. The sessionis summarized in Fig. 7. The standard path represents the sequential work flow, whereas the other continuous lines represent parallelactions and the dashed path represents the path back to the start of the session. The non-active paths are represented in light-grey.

The working session starts with the menu of the stories. After choosing which story to read, the child is presented with the first storypage. Being an illustrated page, he/she has to look for the animation moving the cursor of the mouse on the visualized image. When he/shefinds the animation, he/she has to click to it. This action visualizes both the parchment with the text and the arrow that the child has to clickto move on to the next page. The child has the possibility of only reading the text and then moving on to the second page, or he/she can alsoopen the dictionary to read the explanation of an unknown word he/she has found in the text. In our example, the second page is onlytextual. After reading this page the child moves on to a training exercise, which helps him/her practising with the structure and interface ofthe exercises. The following step consists of resolving the first exercise and the third one. When the child is on an exercise, he/she can openthe medal panel and have a look at his/her results. The child can also click on the help icon and go back to the part of story related to thatexercise.

At each step of the work flow, the reader can go back to the main menu (see the dashed path) and choose another story.

4. LODE-2: the evaluation

To check if our simplified and animated stories are effectively more comprehensible to deaf children, we performed a test involvingeighteen Italian deaf children. The target group was divided into two subgroups: ten primary school children, aged 8–11 (in the followingnamed yD, to mean ‘Younger Deaf’) and eight middle school children, aged 12–14 (in the following named oD, to mean ‘Older Deaf’). Finally,a group of twelve Italian hearing children, eight aged 8–11 (named yH, i.e. ‘Younger Hearing’) and four aged 11–14 (named oH, i.e. ‘OlderHearing’) participated as a control group.

4.1. Objectives

Our evaluation was aiming at testing three hypotheses, according to the literature:

H1. Hearing children perform better than deaf children in comprehension exercises (Paul, 1998).

H2. Both hearing and deaf children perform better in comprehension exercises when reading simplified stories (Coleman, 1962).

Fig. 6. The feedback medals panel.

Fig. 7. The flow chart.

O. Mich et al. / Computers & Education 65 (2013) 34–44 41

H3. Both deaf and hearing children perform better in comprehension exercises when reading a simplified and illustrated story (Carney &Levin, 2002).

4.2. Method

In our experiment we used two sets of stories: one for younger and one for older children. Each set was composed by three stories: a)a story in its original version, visualized without images; both stories for younger and for older children had a Gulpease readability index(G.i.) of 55, corresponding to a mean readability level (Lucidano & Piemontese, 1988); b) a simplified story (in (Mich & Vettori, 2011) anexplanation of the method used to simplify our stories) visualized without images and presenting a G.i. 64 (the story for younger children)and G.i. 58 (the story for older children); and c) a simplified story illustrated with drawings and including definitions, with G.i. 72 (the storyfor younger children) and G.i. 60 (the story for older children).

Ideally the test should be conducted on the same story presented in three different versions but that was impossible because, unfor-tunately, we had to test all the children in one unique session. Therefore, we used three different stories, with similar characteristics(readability index, lexicon and illustrations’ style), each presented in one of the forms described above.

Each child was asked to read each story in the age-related set (presented in the same order to all the participants, a-b-c) and answer theeleven questions foreseen for the comprehension exercises (Cornoldi & Colpo, 2010).

4.3. Results

The evaluation was basically centred on the comparison of the mean scores achieved by the target groups (deaf children) with thoseachieved by the control groups (hearing children).

Fig. 8 shows the percentage of correct answers given by the two groups. Not surprisingly, the control groups performed better than thedeaf groups. However, while in the control groups the older children (oH, age 12–14) globally performed better than the younger ones (yH,age 8–11) with respect to each version of story, in the target group the older children (oD) performed worse than the younger ones (yD).Furthermore, we found a significant difference among the responses in the yD group but not in the oD group. In both cases there is

Fig. 8. Comprehension exercises: the percentage of correct answers.

Fig. 9. Comprehension exercises results: regression analysis.

O. Mich et al. / Computers & Education 65 (2013) 34–4442

a significant improvement of the correct answer mean from the original story (yD ¼ 0.79; oD ¼ 0.57) to the simplified illustrated version(yD¼ 0.88; oD¼ 0.70). Nevertheless, in the oD group the difference between themean of correct answers given on the original story and theone given on the simplified story is almost inexistent (0.57 versus 0.58). Moreover, in the yD group the mean of correct answers to thesimplified story without images is lower (0.68) than the one achieved in the exercises for the other two story versions.

Themain results are confirmed by a regression analysis (see Fig. 9) which underlines that there is a significant difference among the threestories and that the third one always appears to be the most comprehensible for deaf children. From the same analyses it is evident that thetwo hearing groups always performed better than the deaf ones, but the difference in terms of correct answer mean is larger among theolder children (0.295 versus 0.084).

4.4. Discussion

With respect to the hypothesis (see Section 4.1), the results show that:

1)Accordingly to H1, hearing children perform better than deaf children in comprehension exercises, especially the oldest ones. On thecontrary, the primary school deaf children performed better than the middle school ones on each of three story versions.2)Concerning H2, we cannot draw any firm conclusions about the simplification impact. Clearly enough, lexical and syntactic sim-plification alone increases the readability index but that is not sufficient to guarantee the comprehension of a text by a deaf readership. Inthis case, it seems that the story’s structure itself (i.e. event sequence, length etc.) plays a crucial role. That story’s structure has to becarefully designed in order to avoid confusion and boredom in the deaf reader, especially in the absence of drawings which help con-textualize the events and information read.3)Accordingly to H3, the simplified and illustrated stories, having a higher readability index, make the comprehension exercise easier andboth children groups (deaf and hearing) got the best results. We can therefore state that visual aids and text simplification do helpchildren to understand a story more easily.

5. Conclusion and future work

With our research, we tried to develop an application that could help deaf children improve their reading comprehension skills in an easyway, while also having fun.

With respect to other similar systems, see for example Cornerstones or See and See cited in Section 1 which present stories with staticpages, we proposed an interactive way of reading. We also introduced a more active and intelligent method of feedback respect to othersimilar applications. For example, Highlighter, also cited in Section 1, gives a feedback consisting of a message simply telling the student ifhe/she is going onwell or if he/she is doing errors. Moreover, we based our system on a modular client-server architecture, which is easy tomaintain and to improve. Finally, we implemented it using a programming language, HTML 5.0, that allows for the implementation of lightweight code, and that will be the standard for future animated web applications.

A preliminary evaluation has been conducted involving 18 deaf children, grouped in primary school and middle school children.The results globally confirm the starting hypotheses (see Section 4.1). In particular, simplified stories, illustrated with drawings and

extended with definitions, turned out to be more effective for the reading comprehension of deaf children.In the near future, the effectiveness of the proposed feedback systemwill be more deeply evaluated by a larger group of deaf and hearing

children. The children performancewill be possibly tested on three versions of the same story in order tomake the results more comparable.Moreover, in those experiments, more attentionwill be paid to the simplification task of the story, focussing not only on lexical and syntactic

O. Mich et al. / Computers & Education 65 (2013) 34–44 43

aspects but also on structure ones. Finally, we are also considering the possibility of implementing an editor to allow teachers and childrento add new stories and their related exercises.

Acknowledgements

Wewould like to acknowledge all the peole and institutions who contributed to this project, in particular: Chiara Vettori, for the help andinspiration she extended; Fondazione Caritro, for funding this research; Istituto dei Sordi of Torino, its director Enrico Dolza and all theeducators, for hosting our test and collaborating during its execution; ABC Onlus for providing the sign language-enhanced videos; MicheleMarchesoni, for his active participation in the project; Giulia Agnoletto, for her beautiful illustrations; Nella Valentini for her inspiringadvice; Charles Callaway, for proofreading this paper. Finally, our special thanks go to all the children who participated in our test and totheir parents.

References

Alessio, J. (February 2010). Why are children’s books good for designers?. On Design Shack. http://designshack.net/articles/inspiration/why-are-childrens-books-good-for-designers.

Allen, J. (1983). Maintaining knowledge about temporal intervals. Communications of the ACM, 26, 832–843.Anastasis. (2009). Anastasis cooperative society. Website. http://www.anastasis.it/.Apt, K., & Wallace, M. (2006). Constraint logic programming using ECLiPSe. Cambridge University Press.Borgna, G., Convertino, C., Marschark, M., Morrison, C., & Rizzolo, K. (2011). Enhancing deaf students’ learning from sign language and text: metacognition, modality, and the

effectiveness of content scaffolding. Journal of Deaf Studies and Deaf Education, 16(1), 79–100.Brookshire, J., Scharff, L., & Moses, L. (2002). The influence of illustrations on children’s book preferences and comprehension. Reading Psychology, 23(4), 323–339.Calliari, P., & Degasperi, M. (Eds.), (2007). I Bambini Pensano con le Storie. Provincia Autonoma di Trento: IPRASE del Trentino.Carney, R., & Levin, J. (2002). Pictorial illustrations still improve students learning from text. Educational Psychology Review, 14(1), 5–26.Cedric Wachholz, E. E. M. (Ed.), (2006). Using ICT to develop literacy. Bangkok: UNESCO.Coleman, E. (April 1962). Improving comprehensibility by shortening sentences. Journal of Applied Psychology, 46(2), 131–134.Cornoldi, C., & Colpo, G. (2010). Prove di Lettura MT. Giunti OS.Cornoldi, C., & Garofalo. (May 2009). Highlighter. Anastasis and Asphi.DeLamater, W. (April 2010). How larger font size impacts reading and the implications for educational use of digital text readers. http://www.ereadia.com/research/How_Larger_

Font_Size_Affects_Reading.pdf.Dyer, A., Szczerbinski, M., MacSweeney, M., Green, L., & Campbell, R. (2003). Predictors of reading delay in deaf adolescents: the relative contributions of rapid automatized

naming speed and phonological awareness and decoding. Journal of Deaf Studies and Deaf Education, 8(3).El, R., Tillema, H., & van Kopper, S. (2012). Effects of formative feedback on intrinsic motivation: examining ethnic differences. Learning and Individual Differences, 22(4), 449–

454.Fabretti, D. (2000). L’Italiano Scritto dai Sordi: un’Indagine sulle Abilità di Scrittura dei Sordi Adulti Segnanti Nativi. Rassegna di Psicologia, xviii(1), 73–93.Fraser, C. (1998). The role of consulting a dictionary in reading and vocabulary learning. Canadian Journal of Applied Linguistics, 2(1–2), 73–89.Furth, H. (1966). A comparison of reading test norms of deaf and hearing children. American Annals of the Deaf, 111, 461–462.Gambrell, L., & Brooks Jawitz, P. (1993). Mental imagery, text illustrations, and children’s story comprehension and recall. Reading Research Quarterly, 28(3), 264–276.Gentry, M., Chinn, K., & Moulton, R. (2005). Effectiveness of multimedia reading materials when used with childrenwho are deaf. American Annals of the Deaf, 149(5), 394–403.Gump, J. (2001). The readability of typefaces and the subsequent mood or emotion created in the reader. Journal of Education for Business, 76(5), 270–273.Harris, M., & Terlektsi, E. (2011). Reading and spelling abilities of deaf adolescents with cochlear implants and hearing aids. Journal of Deaf Studies and Deaf Education, 16(1),

24–34.Hughes, L., & Wilkins, A. (2002). Reading at a distance: implications for the design of text in children’s big books. British Journal of Educational Psychology, 72(2), 213–226.Johnson, L., Adams, S., & Haywood, K. (2011). 2011 Horizon report: K-12 edition. Tech. rep.. Austin, Texas: The New Media ConsortiumKress, G., & van Leeuwen, T. (2006). Reading images: The grammar of visual design (2nd ed.)..Lachs, V. (2000). Making multimedia in the classroom: A teachers guide. London: Routledge.Lewis, C., & Walker, P. (1989). Typographic influences on reading. British Journal of Psychology, 80, 241–257.Loeterman, M., Paul, P., & Donahue, S. (February 2002). Reading and deaf children. http://www.readingonline.org/articles/art_index.asp?HREF¼loeterman/.Lucidano, P., & Piemontese, M. (1988). GULPEASE: una Formula per la Predizione della Difficolt dei Testi in Lingua Italiana. Scuola e Citt 3(31).Marconi, L., Ott, M., Pesenti, E., Ratti, D., & Tavella, M. (1993). Lessico Elementare. Dati statistici sull’italiano letto e scritto dai bambini delle elementari. Bologna: Zanichelli.Mar, R., Oatley, K., Djikic, M., & Mullin, J. (2011). Emotion and narrative fiction: interactive influences before, during, and after reading. Cognition & Emotion, 25(5),

818–833.Merrills, J., Underwood, G., & Wood, D. (1994). The word recognition skills of profoundly, prelingually deaf children. British Journal of Developmental Psychology, 12, 365–384.Mich, O. Usability Methods and Deaf Children. The Case of the LODE e-Tool. Ph.D. Thesis, Free University of Bozen-Bolzano, Faculty of Computer Science, 2010.Mich, O., Vettori, C. E-Stories for Educating Deaf Children in Literacy. The DAMA Procedure. Tech. rep., FBK, January 2011.Mosenthal, P. B. (1999). Understanding engagement: historical and political contexts. In J. T. G, & D. E. A (Eds.), Engaged reading: Processes, practices, and policy implications

(pp. 1–16). New York: Teachers College Press.Munz, U., Schumm, P., Wiesebrock, A., & Allgwer, F. (2007). Motivation and learning progress through educational games. IEEE Transactions on Industrial Electronics, 54(6).Murphy, P. (October 2007). Reading comprehension exercises online: the effects of feedback, proficiency and interaction. Language Learning & Technology, 11(3), 107–129.Musselman, C. (Winter 2000). How do children who can’t hear learn to read an alphabetic script? A review of the literature on reading and deafness. Journal of Deaf Studies

and Deaf Education, 5(1), 9–31.Nagy, W., Anderson, R., & Herman, P. (1987). Learning word meanings from context during normal reading. American Educational Research Journal, 24(2), 237–270.Nicol, D., & Macfarlane-Dick, D. (April 2006). Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. Studies in Higher

Education, 31(2), 199–218.Nikolaraizi, M., & Vekiri, I. (2011). The design of a software to enhance the reading comprehension skills of deaf students: an integration of multiple theoretical perspectives.

Education and Information Technologies1–19, URL. http://dx.doi.org/10.1007/s10639-011-9152-1.O’Keefe, E., & Solman, R. (1987). The influence of illustrations on children’s comprehension of written stories. Journal of Literacy Research, 19(4), 353–377.Paul, P. (1998). Literacy and deafness: The development of reading, writing, and literate thought. Allyn & Bacon.Perani, D., Saccuman, M., Scifo, P., Awander, A., Spada, D., Baldoli, C., et al. (2011). Neural language networks at birth. Proceedings of the National Academy of Sciences, 108(36).Pressley, M. (2001). Comprehension instruction: what makes sense now, what might make sense soon. Reading Online, 5(2).Pressley, M. (2000). Comprehension instruction: what makes sense now, what might make sense soon. In Kamil., Mosenthal., Pearson., & Barr. (Eds.), Handbook of reading

research. International Reading Association, Ch. 30.Rinaldi, P., & Caselli, M. (2009). Lexical and grammatical abilities in deaf Italian preschoolers: the role of duration of formal language experience. Journal of Deaf Studies and

Deaf Education, 14(1), 63–75.Roccaforte, M., DeMonte, M., Groves, K., Tomasuolo, E., & Capuano, D. (2011). Strategies for Italian deaf learners. In S. Editore (Ed.), Proceedings of the 4th international

conference on ICT for language learning, ISBN 978-88-7647-677-8.Schallert, D. (1980). The role of illustrations in reading comprehension. In R. J. Spiro (Ed.), Theoretical issues in reading comprehension (pp. 503–524). Hillsdale, NJ: Erlbaum.Shaikh, A., Chaparro, B., & Fox, D. (February 2006). Perception of fonts: Perceived personality traits and uses. Usability News.Shamir, A., & Shlafer, I. (2011). E-books effectiveness in promoting phonological awareness and concept about print: a comparison between children at risk for learning

disabilities and typically developing kindergarteners. Computers & Education, 57(3), 1989–1997.

O. Mich et al. / Computers & Education 65 (2013) 34–4444

Smallwood, B. (November 2002). Thematic literature and curriculum for English language learners in early childhood education. ERIC Digest.Tomasuolo, E. (2006). La Valutazione delle Abilità Linguistiche in Bambini e Ragazzi Sordi. Ph.D thesis.Traxler, C. (2000). The Stanford achievement test, 9th edition: national norming and performance standards for deaf and hard-of-hearing students. Journal of Deaf Studies and

Deaf Education, 5, 337–348.UNESCO. (1987). Education of deaf children and young peopleIn Guides for special needs education, Denmark.Vacca, R., & Vacca, J. (2003). Content area reading. New York: HarperCollins.Virvou, M., Katsionis, G., & Manos, K. (2005). Combining software games with education: evaluation of its educational effectiveness. Educational Technology & Society, 8(2),

54–65.Waddill, P., McDaniel, M., & Einstein, G. (Dec 1988). Illustrations as adjuncts to prose: a text-appropriate processing approach. Journal of Educational Psychology, 80(4),

457–464.Whitehurst, G., & Longian, C. (1998). Child development and emergent literacy. Child Development, 69, 848–872.Willows, d. (1978). A picture is not always worth a thousand words: pictures as distractors in reading. Journal of Educational Psychology, 70(2), 255–262.