GESTURE TECHNOLOGY

INTRODUCTION Gesture recognition is a language technology with

the goal of interpreting human gestures via mathematical algorithms. Gestures can originate from any bodily motion or state but commonly originate from the face or hand.

Gesture recognition enables humans to interface with the machine (HMI) and interact naturally without any mechanical devices. Using the concept of gesture recognition, it is possible to point a finger at the computer screen so that the cursor will move accordingly.

EXAMPLE Military air marshals use hand and

body gestures to direct flight operations aboard aircraft carriers.

GESTURE TYPES In computer interfaces, two types of gestures are

distinguished: We consider online gestures, which can also be regarded as direct manipulations like scaling and rotating. In contrast, offline gestures are usually processed after the interaction is finished; e.g. a circle is drawn to activate a context menu.

Offline gestures: Those gestures that are processed after the user interaction with the object. An example is the gesture to activate a menu.

Online gestures: Direct manipulation gestures. They are used to scale or rotate a tangible object.

A BASIC WORKING OF THE GESTURE RECOGNITION SYSTEM

USES Sign language recognition Facial Expressions Face Recognition Electrical Field Sensing Vision Based Technologies Device Gesture Technologies

SIGN LANGUAGE RECOGNITION

Effective Sign language recognition would grant the deaf expanded tools for communicating with both other people and machine.Certain types of gesture recognition software can transcribe the symbols represented through sign language into text.

FACIAL EXPRESSIONS Facial gesture recognition creates

an effective non- contact interface between users and their machines.

The goal of facial gesture recognition is to make machine effectively understand human emotions of the countless physical difference between individual.

Facial gesture recognition can be used for purposes related to personal security & law enforcement

FACE RECOGNITIONThe Face Recognition Technology (FERET) program database is a large database of facial images,divided into development and ... Maximum likelihood detection of faces and hands. International Workshop on Automatic Face and Gesture Recognition

ELECTRICAL FIELD SENSING

Proximity of a human body or body part can be measured by sensing electric fields .These measurements can be used to measure the distance of a human hand or other body part from an object; this facilitates a vast range of applications for a wide range of industries.

VISION BASED TECHNOLOGIES There are two approaches to vision

based gesture recognition: Model based techniques:They try to create a three dimensional model of the users hand and use this for recognition. Image based methods:Image-based techniques detect a gesture by capturing pictures of a user’s motions during the course of a gesture.

DEVICE GESTURE TECHNOLOGIES Device-based techniques use a glove, stylus, or other

position tracker, whose movements send signals that the system uses to identify the gesture.

The glove is equipped with a variety of sensors to provide information about hand position, orientation, and flex of fingers.

INPUT DEVICES Wired gloves Depth-aware cameras Stereo cameras Controller-based gestures Single camera

WIRED GLOVES A wired glove (also called a "data

glove“ is an input device for human–computer interaction worn like a glove.

DEPTH-AWARE CAMERAS

Depth sensors’ ability to capture large amounts of 3D information will completely change the way that digital photography is done. Users would be able to seamlessly remove objects, isolate single objects, and even remove and replace the entire background of a picture.

Navigation software like Google Maps would be more accurate than ever by combining GPS data with the ability to accurately sense the user’s immediate surroundings; they could offer the ability to navigate inside of buildings. Devices could even be developed that allow the visually impaired to navigate the world around them, and the hearing impaired could have sign language translated in real time.

STEREO CAMERAS A stereo camera is a

type of camera with two or more lenses with a separate image sensor or film frame for each lens. This allows the camera to simulate human binocular vision, and therefore gives it the ability to capture three-dimensional images, a process known as stereo photography.

CONTROLLER-BASED GESTURES

These controllers act as an extension of the body so that when gestures are performed, some of their motion can be conveniently captured by software. 

SINGLE CAMERA The single-camera setup, or single-

camera mode of production, also known as Portable Single Camera, is a method of filmmaking and video production.

ALGORITHMS 3D model-based algorithms Appearance-based models

3D MODEL-BASED ALGORITHMS

The 3D model approach can use volumetric or skeletal models, or even a combination of the two. Volumetric approaches have been heavily used in computer animation industry and for computer vision purposes. The models are generally created of complicated 3D surfaces, like NURBS or polygon meshes.

The drawback of this method is that is very computational intensive, and systems for live analysis are still to be developed.

SKELETAL-BASED ALGORITHMS Instead of using intensive processing of the 3D

models and dealing with a lot of parameters, one can just use a simplified version of joint angle parameters along with segment lengths. This is known as a skeletal representation of the body, where a virtual skeleton of the person is computed and parts of the body are mapped to certain segments.

APPEARANCE-BASED MODELS These models don’t use a spatial representation

of the body anymore, because they derive the parameters directly from the images or videos using a template database. Some are based on the deformable 2D templates of the human parts of the body, particularly hands. Deformable templates are sets of points on the outline of an object, used as interpolation nodes for the object’s outline approximation.

CHALLENGES There are many challenges associated with the

accuracy and usefulness of gesture recognition software. For image-based gesture recognition there are limitations on the equipment used and image noise. Images or video may not be under consistent lighting, or in the same location. Items in the background or distinct features of the users may make recognition more difficult.

FLUTTER APP Flutter allows you to play & pause Spotify, iTunes,

Quicktime, and VLC (latest version) using gestures via your built-in webcam. - Play/Pause your music & videos using simple hand gestures - Works via your webcam -- no additional hardware necessary- Works best from 1-6 feet away from webcam- Works even when Spotify & iTunes is playing music in the background Coming Soon- Gestures for Skip & Like Songs- Mute- Volume Control

Since the official debut in late May on the Mac App Store, Flutter has been quite popular worldwide. Within two weeks of their debut in the App Store, Flutter became part of the top 5 downloads in over 30 countries among all free entertainment apps. Users in over 90 countries have downloaded it. At one point the app was #2 in the United States, along with being the #1 free entertainment app in 14 countries, as well as the overall #1 free app in Korea, Italy, Sweden, and Taiwan. https://flutterapp.com

FUTURE Currently, gesture recognition is mostly

commonly used in gaming applications but the potential for future applications is immense. What if we could use gesture recognition to manage multimedia in our home environment? Use a gesture to turn our TV or heating system on and off? What if we could use gesture recognition to teach, do remote surgeries or conduct facial recognition in airport security lines?