Wearable Assistive Technologies

for AutismBy Paraskevi Fasouli

and Taraneh Aminosharieh Najafi

Contents

• What is Autism?

• What is Assistive Technology?

• Wearable Technologies

• Social Robot

• Solar Smart Cap

• Wearable IoT

• AR Smart Glasses

• Super Power glass

• Virtual Reality

• Proximity awareness

• Detection of Stereotypical Motor Movements

• Conclusion

What is Autism?

• Autism spectrum disorder (ASD) is a broad term used to describe a group of neurodevelopmental disorders.

• These disorders are characterized by problems with communication and social interaction. People with ASD often demonstrate restricted, repetitive, and stereotyped interests or patterns of behavior.

• According to the Centers for Disease Control and Prevention (CDC), autism does occur more often in boys than in girls, with a 4 to 1 male-to-female ratio.

• The CDC estimated in 2014 that nearly 1 in 59 children have been identified with ASD.

• There are indications that instances of ASD are on the rise. Some attribute this increase to environmental factors.

Skills of Concern

• Communication Skills• Talking, Reading, Writing

• Learning new vocabulary

• Social Skills• Social understanding, awareness

• Emotion identification, recognition

• Functional Life Skills• Mobility, Planning, Organizing

• Daily tasks e.g. cooking

• Educational tasks

Assistive technology makes things possible!

Assistive Technology

• Any customized item, piece of equipment, software or product that is used to increase, maintain, or improve the functional capabilities of people with disabilities

• Wearable AT enhances and supports daily life in a practical way due to the capability to wear or carry around the equipment

Assistive Technology Categories

Low tech AT Mid tech AT High Tech AT

Tools that do not needbatteries or electricity and require little-to-no training to operate

Tools that require a bit oftraining, they may bebattery-operated and add a level of specialization

Electrically-powereddevices (and software) that require the most training andthey are highly specialized and customizable based on individual needs

Wearable Technologies

• Social Robot

• Solar Smart Cap

• Wearable IoT

• AR Smart Glasses

• Super Power glass

• Virtual Reality

• Proximity awareness

• Detection of Stereotypical Motor Movements

Social Robot

• Communicates with autistic children by following social behavior

• Helps them in improving their eye contact during communication

• Laser detection system for analyzing expressions

• Approaches the child and displays images and play sounds in order to change the mood of the child

• Touch screen display for interaction -Child can give input to the robot

Solar Smart Cap

• Rechargeable DC battery with 2 charging modes: normal or solar cell

• Sensors, for children vitals data collection

• GSM module, for delivering children health condition to the smartphone of the parents or caregivers

• GPS technology, for locating and sending the position of the children

• Emergency button, for alerting parents if the child feels uncomfortable

Wearable IoT

-Measuring pair-wise human interaction between individuals with a nametag-sized sensor badge and subsequent wireless upload of data to an access point

-Multi-modal sensing including:• ultrasound sonar• Face-to-face time - the amount of

time two individuals face each other• Proximity - the physical distance

between the two participants• accelerometer• Activity level - the physical

movement of participants

AR Smart Glasses

• Data Gathering module: sensors send data to the next module through MQTT

• Data integration module: Smart glasses and Alexa work as subscribers and Data Gathering module works as publisher

• Therapist can model a task in a high level of abstraction through web platform

• Improves daily living activities skills

Emotion recognition by Superpowerglass

• Components:

• Google glass

• Android phone

• Activities:

• Capture the smile game: the child is challenged to provoke prompted emotions in an adult.

• Guess the emotion game: challenges the child to guess the emotion.

• Unstructured Activity: at home glasses can be used as emotional aid during social activities (e.g. dinner)

Emotion recognition by Superpowerglass

• Feedback Choices:

• Visual Emotion and Color Cues: three possible heads-up interfaces

• Auditory cues: emitted via bone-conducting speaker on the glass

• combined visual and auditory cues

• Computational heavy tasks such as frame processing, video encoding and data storage are handled on the android phone due to limited processing power on glass CPU and battery life.

Virtual Reality Social Cognition Training

• Use of VR to enhance social skills, social cognition and social functioning in young adults with ASD

• Focused on:

• Real-time emotion: recognizing other’s feelings and tone of voice recognition

• Theory of mind: recognizing and responding to other’s thoughts and desires

• conversational skills: initiating, maintaining and closing

ProCom prototype Mobile and wearable system for proximity

awareness

• system includes:• a wearable sensor module and

mobile application connected via Bluetooth.

• hardware includes a 3D printed box with two infrared (IR) sensors and a servomotor on its top.

• The internal components include an Arduino Uno, a Bluetooth module, and a 9V battery.

• Function:• IR sensors sweep between −30°

and 30°, collecting distance values.

• mobile interface:• an aerial view with the user

depicted at the bottom

• A traffic light represent distance zones

Detection of Stereotypical Motor

Movements in Autism using a Smartwatch-

based System

• system:

• Motorola Moto 360 with a 3-axis accelerometer and a 3-axis gyroscope.

• Android tablet with Bluetooth

• model:• Person-independent model to

recognize four target behaviors: Hand flapping, Head banging, repetitive dropping, others

• Accuracy 92.6%

• Limitation: simulated data from adults imitating children with ASD is used to train the system.

Wearable Assistive Technologies

Type Technology Targeted Social Challenge

Smart Glasses Super Power Glass [1] Emotion Recognition

Smart Glasses Smart Glasses [2] Emotion Recognition

Humanoid Robot Social Robot [3] Eye Contact -Communication

Smart Glasses Little Helper [4] Communication

Smart Wear Wearable Sensors [5] Health Monitoring

Smart Wear Smart Cap [6] Health Monitoring –Location Tracking

Smart Watch Filip2 [7] Location Tracking

Smart Watch [8] Movement Detection

Smart Wear ProCom [9] Proximity Detection

Smart Wear Accelerometers [10] Stereotypical behaviors

Smart Wear Sensor IoT Badge [11] Pair Communication

Smart Glasses / VR VR Training [12] Social Training

Smart Wear E4 Wristband [13] Health/Stress Monitoring

Smart Glasses AR Smart Glasses [14] Task Achievement

Conclusion

• WATs are an important advance in improving autistic people’s opportunity to integrate in society and enhancing their understanding and awareness in different social scenarios.

• Although WAT is a hot topic, most of the technologies presented are in research and prototype phase, and therapy sessions. It is a challenge to integrate them to ASD people’s daily life. Further research is needed to further develop the area.

• WAT can be seen as a useful addition in the work with ASD children and adults.

References

1. P. Washington et al., “SuperpowerGlass: A Wearable Aid for the At-Home Therapy of Children with Autism,” Proc. ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 3, 2017

2. Z. Ye et al., “Detecting Eye Contact Using Wearable Eye-Tracking Glasses,” Proc. 2012 ACM Conf. Ubiquitous Computing (UbiComp 12), pp. 699–704, 2012.

3. Christiane M. Goulart, Javier Castillo,Carlos T.Valadão,Eliete Caldeira,“Mobile Robotics: A Tool for Interaction With Children With Autism”,IEEE, pp.1555-1559,June 2014.

4. Qingguo Xu, Sen-ching Samson Cheung, Neelkamal Soares,” Littlehelper: An Augmented Reality Glass Application To Assist Individuals With Autism In Job Interview”, Proc. 2015 APSIPA Annual Conf., p. 1276-1279, December 2015.

5. André G. Ferreira, Duarte Fernandes,Sérgio Branco,João L. Monteiro,“A Smart Wearable System for Sudden Infant Death Syndrome Monitoring”, IEEE.pp.1920-1925, Mar 2016.

6. Iftekhar Uddin Ahmed, Nazia Hassan, Humayun Rashid, “Solar Powered Smart Wearable Health Monitoring and Tracking Device Based on GPS and GSM Technology for Children with Autism”, Proc. 4th International Conference on Advances in Electrical Engineering, p.111-116, September 2017.

7. S. Parez, FiLIP 2, “A Smartwatch For Kids, Arrives Today At AT&T For $100”. Retrieved Dec 2019, from TechCrunch: http://techcrunch.com/2014/11/07/filip-2-a-smartwatch-for-kids-arrives-today-at-att-for-100/, 2014.

8. Sarker, Hillol et al, “Detection of Stereotypical Motor Movements in Autism using a Smartwatch-based System”, AMIA Annual Symposium proceedings AMIA Symposium,vol. 2018 p.952-960, Dec 2018.

References

9. L.E. Boyd, X. Jiang, and G.R. Hayes, “ProCom: Designing and Evaluating a Mobile and Wearable System to Support Proximity Awareness for People with Autism,” Proc. 2017 CHI Conf. Human Factors in Computing Systems (CHI 17), 2017, pp. 2865–2877.

10. T. Westeyn et al., “Recognizing Mimicked Autistic Self-Stimulatory Behaviors Using HMMs,” Proc. 9th IEEE Int’l Symp. Wearable Computers (ISWC 05), 2005, pp. 164–169

11. Yan Shi, Saptarshi Das, Sarah Douglas, Subir Biswas, “An Experimental Wearable IoT for Data-driven Management of Autism”, 9th International COMSNETS, p.468-471, 2017.

12. N.Didehbani, T.Allen, M.Kandalaft, D.Krawczyk, S.Chapman, “Virtual Reality Social Cognition Training for children with high functioning autism”, Computers in Human Behavior v.62, p.703-711, September 2016

13. “Empatica Inc,” E4 Wristband. Retrieved Dec 2019, from empatica: https://www.empatica.com/e4-wristband, 2016.

14. Eduardo Machado, Ivan Carrillo, David Saldana, Feng Chen, Liming Chen, “An Assistive Augmented Reality-based Smartglasses Solution for Individuals with Autism Spectrum Disorder”, IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Confon Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress, p.245-250 , 2019.

15. E. M. Benssassi, J. Gomez, L. E. Boyd, G. R. Hayes and J. Ye, "Wearable Assistive Technologies for Autism: Opportunities and Challenges," in IEEE Pervasive Computing, vol. 17, no. 2, pp. 11-21, Apr.-Jun. 2018.