an augmented reality gamebook for children with autism spectrum disorder (asd)
TRANSCRIPT
![Page 1: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/1.jpg)
An Augmented Reality GameBook for Children with Au8sm Spectrum Disorder (ASD) Jorge Brandão University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
Pedro Cunha University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
José Vasconcelos University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
Vítor Carvalho University of Minho/Algoritmi R&D Centre, Guimarães, Portugal IPCA-‐EST, Barcelos, Portugal
Filomena Soares University of Minho/Algoritmi R&D Centre, Guimarães, Portugal
InternaKonal conference focused on e-‐learning in the workplace June 10-‐12, 2015, at Columbia University in New York.
![Page 2: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/2.jpg)
• WHAT IS AUTISM
• ABSTRACT • ABOUT AUTISM
• INTRODUTION • THE GAMEBOOK • CHARACTERS • SCENARIOS • IMPLEMENTATION
• CONCLUSIONS • GAME DEMONSTRATION
SUMMARY
![Page 3: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/3.jpg)
• Technologies, like augmented reality (AR), have the ability to catch children’s imaginaKon and to promote their aYenKon. • Children with ASD usually have difficulty to recognize facial expressions and to understand associated emoKons. • We propose to design and develop an innovaKve GameBook to assist children with ASD to recognize and acquire emoKons by engaging their aYenKon and moKvaKon. • The GameBook will contain a story that describe some scenarios and real world situaKons which will conduct the children to become involved on ficKonal contents associated with emoKons.
ABSTRACT
![Page 4: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/4.jpg)
ABSTRACT Play with AR Avatar
Play Quiz Games
Play Memory Games
The child will have to interact with these scenarios, by playing with one 3D AR avatar with different
facial expressions
In this GameBook the child will have to idenKfy the correct
emoKonal face to the situaKon in a Quiz Game.
This GameBook the child will have to idenKfy the correct emoKonal face to the situaKon in a Quiz
Game.
x x
![Page 5: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/5.jpg)
AUTISM SPECTRUM DISORDER IN USA
Source: Center for disease Control and PrevenKon -‐ hYp://www.cdc.gov/ncbddd/auKsm/data.html
“The AuKsm Society esKmates that the United States is facing almost $90 billion annually in costs for au8sm. (This figure includes research, insurance costs and non-‐covered expenses, Medicaid waivers for auKsm, educaKonal spending, h o u s i n g , t r a n s p o r t a K o n , employment, related therapeuKc services and caregiver costs.)” hYp://www.auKsm-‐ society.org/about-‐auKsm.
![Page 6: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/6.jpg)
hWp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
HIGH FUNCTIONING
LOW FUNCTIONING
![Page 7: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/7.jpg)
1 out of every 54 boys in the United States has an au8sm spectrum disorder
hYp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
![Page 8: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/8.jpg)
1 out of every 252 girls in the United States has an au8sm spectrum disorder
hYp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
![Page 9: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/9.jpg)
hYp://thautcast.com/drupal5/category/tags/centers-‐disease-‐control
![Page 10: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/10.jpg)
INTRODUTION • Technology is ubiquitous and can be a safe and good way to moKvate and engage children in interacKve learning acKviKes in order to promote their cogniKve and social skills
• It is being used increasingly on a variety of pedagogical contexts, both as assisKve technologies and as tools for helping us to understand user’s moKvaKon
• It has the potenKal to provide individually tailored intervenKons that are suitable for a wide variety of abiliKes • Technological intervenKons allow its use at different speeds and locaKons, and never lose paKence with the frequent repeKKon that many people with ASD has
![Page 11: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/11.jpg)
INTRODUCTION
• ASD children struggle with significant relaKonships and behavioural challenges that in most cases have serious implicaKons for inclusion social adulthood
• One of the most promising applicaKons in the intervenKon process of ASD children came from developing tools for promoKng children’s social and communicaKon skills
• This offers the opportunity to reduce some of the crippling personnel costs associated with auKsm care
![Page 12: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/12.jpg)
INTRODUTION
• Children with ASD usually have difficulty to recognize facial expressions and to understand associated emoKons, to imitate or use emoKonal expressions, to understand and control their own emoKons, or to interpret emoKons or empathy with others
• Several studies show that the majority of people with ASD exhibit a natural affinity with technology and a posiKve ahtude towards computer-‐based training
![Page 13: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/13.jpg)
INTRODUTION • Technologies, like augmented reality (AR), have the ability to catch the children’s imagina8on and to promote their aWen8on, because they can experiment ar8ficial, safe and fascina8ng environments.
• AR is an opKmal interface technology and a helpful tool to support auKsKc children capabiliKes. With AR it is possible to create more aYracKve and interacKve interfaces that can be manipulated by hand, without using convenKonal peripherals such as the keyboard and the mouse. • This characterisKc offers greater interacKon between the child and the object, raising their interest and their curiosity, providing greater adherence to the task/acKvity.
![Page 14: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/14.jpg)
THE GAMEBOOK Given the potenKal benefits of AR we aim to design a tool based on AR in order to improve capabiliKes of ASD children Our goal is to create an innovaKve serious game based on a book style with interacKon inputs to assist children with social interacKon and communicaKon skills. We aim to promote children with ASD recognizing and acquiring emoKons by engaging their aYenKon and moKvaKon, increasing their competence on this handicap.
![Page 15: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/15.jpg)
THE GAMEBOOK
The GameBook presents the story of Tobias’s adventure during a visit to a zoo park. The player can read the story and interact with it. The story will describe scenarios and Tobias’ interacKons with animals and real world situaKons which will conduct the children to become involved on ficKonal contents associated with emoKons.
![Page 16: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/16.jpg)
THE GAMEBOOK
We believe that the GameBook will put the ability of the child to interact with the story and to promote his/her imagina8on and engaging.
hYp://www.tobiasadventures.in/thezoo
The GameBook can be played on any mobile technology, such as a tablet, smartphone or
laptop, with either an external web camera or an inbuilt camera.
The GameBook is available on:
![Page 17: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/17.jpg)
GAMEBOOK CHARACTERS • The character Tobias was designed to capKvate the
children aYenKon and to empathize with them. • On his design was pretended to promote a face with
characterisKcs that allows the players to feel some aYracKon with him.
• The expressions are designed by special details in the
avatar, essenKally, with his lips and eyes. • It was given a cartoonish look to Tobias, which was
deemed by the group members as potenKally more familiar and more fun to the children than a photo-‐realisKc alternaKve.
![Page 18: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/18.jpg)
GAMEBOOK SCENARIOS The game includes a total of 5 different scenarios, designed to involve the child
on real life situa8ons. The scenarios, all in the zoo, will promote the child’s contact with different environments and
will create some emoKonal reacKon on Tobias.
ANIVERSARY GO TO THE ZOO WITH ELEPHANT WITH LION WITH BIRD
![Page 19: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/19.jpg)
IMPLEMENTATION OF THE GAMEBOOK To create this serious game, the following technologies were used:
The 3D flip book uses the HTML5/CSS3 and JavaScript/Jquery web languages. To develop the AR feature it was selected the ArUco® (Augmented Reality library from the University of Cordoba). This opKon was held on the easiness of use, reliability and cross-‐plakorm compaKbility.
![Page 20: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/20.jpg)
CONCLUSIONS AND FUTURE WORK • The aim of this project was to make available a cost effec8ve tool implemented in a plagorm
usually pleasant to ASD children. • In order to test the GameBook, an exploratory study in a classroom context with ASD children
will be performed. • In the near future, a system that will allow monitoring the child’s results and these outcomes
will be available on the website of the game. • Furthermore, new stories, as for example, Tobias adventures in the Sea or in the Space, with
new characters, scenarios and situa8ons can be easily included. • It is the authors’ belief that the GameBook will increase ASD children engagement in the
learning ac8vi8es promo8ng their memory, cogni8ve and social skills and allowing them to reinforce facial expression recogni8on.
![Page 21: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/21.jpg)
GAME DEMONSTRATION
![Page 22: An Augmented Reality GameBook for Children with Autism Spectrum Disorder (ASD)](https://reader031.vdocuments.us/reader031/viewer/2022031919/55c9647ebb61eb82578b46bf/html5/thumbnails/22.jpg)
QUESTIONS?
Jorge Brandão University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected]
Pedro Cunha University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected] @gmail.com
José Vasconcelos University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected]
Vítor Carvalho University of Minho/Algoritmi R&D Centre, Guimarães, Portugal IPCA-‐EST, Barcelos, Portugal E-‐mail: [email protected]
Filomena Soares University of Minho/Algoritmi R&D Centre, Guimarães, Portugal E-‐mail: [email protected]