Slide 1

INTERACT : M OTION S ENSOR D RIVEN G ESTURE R ECOGNITION C LOUD S ERVICE School of Electronic & Computer Engineering Technical University of Crete, Greece Dr. Stelios Sotiriadis K. Stravoskoufos, A. Preventis, E. Petrakis

Slide 2

Data! 640K ought to be enough for anybody (1981). Google processes 20 PB = 2.19902326 10 13 kilobytes a day

Slide 3

Cloud Computing

Slide 4

The Internet of Things (IoT)

Slide 5

Slide 6

Cloud in Europe: FIWARE Open cloud infrastructure for cost-effective creation and delivery of FI applications and services Develop Cloud Services called Generic Enablers. EU FP7 project. Offers open specification services featuring powerful but user-friendly APIs based on RESTFul standards.

Slide 7

Generic Enablers (GEs) Speeds up the creation of FI applications. Cloud-based modular components which offer reusable and commonly shared functions. Serve multiple areas of use across various Sectors e.g. health! The fundamental building blocks of FIWARE.

Slide 8

FI-STAR FP7 FI-STAR Experimentation Sites and Use Case Scenarios. Servicing around 6 Million people throughout Europe. Work in parallel to apply and validate the Core FI- STAR Platform and its subset of GEs. www.fi-star.eu

Slide 9

XIFI FP7 Crete Node

Slide 10

XIFI FIWARE Node Crete CPU : 208 Cores RAM: 512 TB HDD: 11.7 TB IaaS services FIWARE Instances Blueprint instances SaaS APIs Feel free to contact me for requesting information!

Slide 11

Interact! Motion Sensors Cloud Driven Gesture Recognition Cloud open source software based on FIWARE Generic Enablers. It operates FIWARE cloud platform and offer their functionality through RESTful APIs.

Slide 12

Interact Architecture

Slide 13

IoT Connectivity & Protocol Adapter Connecting sensor with the FI application components. Uses the Protocol Adapter module to adapt to the specific connectivity protocol that the sensor is using (e.g., Bluetooth).

Slide 14

Sensor Data Collector Collects the sensor data Converts data into the desirable format (e.g., JSON, XML etc.) and schema so it can be processed in the back-end without causing interoperability problems.

Slide 15

Connectivity Service Establishes a connection between the front-end and the back-end Transfers the sensor data received from the Sensor Data Collector to the Back-End

Slide 16

Application Logic Application specific module. Encapsulates the business logic of the FI- application. Handles and processes sensor data by using the Complex Event processing, Cloud storage and Publish subscribe modules

Slide 17

Complex Event Processing Used for decision-making thorough the analysis of complex conditional events. Processes custom event patterns Based on user defined conditions, decides the flow of the data.

Slide 18

Cloud Storage Responsible for storing or retrieving sensor data using a database. Functionality should be offered as a RESTful service. Developers should be able to create or retrieve gestures from the database

Slide 19

Publish Subscribe Responsible for publishing the results of the sensor data processing. Publishes context to context subscribers. Example:

Slide 20

Identity Management Used for user authentication and access authorization. Users and Developers authorized to access applications and services

Slide 21

Motion Sensors Sensors that can track the human body or parts of it (skeletal tracking). Asus Xtion Pro MS Kinect Leap Motion

Slide 22

Interact FI application, based on the FIWARE core platform Example functionality: Gesture Recognition. GESTURE MOTION SENSORINTERACTRESULT Victory"

Slide 23

Interact Functionalities Recognize Gestures: Static, Motion, Drawing Store Gestures: Developers can use interact to store gestures & create their own gesture pools.

Slide 24

Interact Functionalities Subscribe to gesture pools: Developers can subscribe to other developers gesture pools in order to use them in their projects. Create gesture pools: Public gesture pool - Can be accessed by everyone - Only super administrators can add new gestures Custom gesture pools - Created by developers

Slide 25

Interact Functionalities Train the System: Interact features a machine learning mechanism that allows users to train the system to recognize gestures. Implements Naive Bayes Classification Accuracy 75%

Slide 26

Interact Architecture

Slide 27

Use Case 1 Gesture to voice: Using Google text to speech API to transform the recognized gestures into voice in order to enable communication with people with speaking disabilities (e.g., deaf-mute).

Slide 28

Use Case 2

Slide 29

Use Case 2 Demo

Slide 30

Use Case 2 Architecture

Slide 31

Conclusions & Future Work Highlights new means of communication for people with special needs. Interact is online! Ask for more details. Try different machine learning algorithms to increase gesture recognition accuracy.

Slide 32

THANK YOU! Questions ?