data acquisition system for sign language

8/12/2019 Data Acquisition System for Sign Language

1/16

Process Instrumentation

Semester Project

Data acquisition system for Sign Language

Submitted By

Asad Ali KhanMuhammad

Haseeb Fayyaz AbbasiYasir Mehmood

Jibran LatifFahad Ahmed

Degree Program: MS Systems Engineering

Course Instructor: Dr. Kamran Ullah Khan

Department of Electrical Engineering

Pakistan Institute of Engineering &Applied Sciences Islamabad


2/16

Contents

1.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 System Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1.2.1 Flex Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21.2.2 Accelerometer (ADXL335) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.2.3 USB (Universal Serial Bus) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.2.4 Programming of USB in MikroC . . . . . . . . . . . . . . . . . . . . . . . . . 7

1.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1


3/16

Abstract

Sign Language is a useful tool to ease the communication between the deaf, mute communityand even the normal people who understand the language. Yet there is a communication barrierbetween these communities with the normal people as most of them do not learn or know the sign

language. Therefore there is a need to develop an electronic device that can translate the signlanguage into text in order to make the communication take place. With this project, the requireddata to perform sign language can be acquired which will be very helpful later on to recognizethe performed signs which will be converted into text so that normal people can understand theirexpression.

The main objective of this project is to develop a system that can acquire the data of differentsigns made by the different people so that the signs can be recognized efficiently. The data is sentto the computer via USB port where it is stored in a database. If a computer is not availablesomewhere, LCD can also be used to read the data.


4/16

1.1 Introduction

A sign language is a language which, instead of acoustically conveyed sound patterns, uses manualcommunication and body language to convey meaning. This can involve simultaneously combininghand shapes, orientation and movement of the hands, arms or body, and facial expressions to ex-press a speakers thoughts.

Wherever communities of deaf people exist, sign languages develop. While they utilize spacefor grammar in a way that spoken languages do not, sign languages exhibit the same linguisticproperties and use the same language faculty as do spoken languages. Hundreds of sign languagesare in use around the world and are at the cores of local deaf cultures. Sign language is the languageused by deaf and mute people. It is a combination of shapes and movements of different parts ofthe body. These parts include face and hands. The area of performance of the movements may befrom well above the head to the belt level. Signs are used in a sign language to communicate wordsand sentences to audience. Sign language is the most frequently used tool when the transmissionof audio is almost impossible or forbidden, or when the action of writing and typing is difficult,but the possibility of vision exists. Moreover, sign language is the most natural and expressive way

for the hearing impaired. Since sign language is gesticulated fluently and interactively like otherspoken languages, a sign language recognizer must be able to recognize continuous sign vocabulariesin real-time.

American Sign Language (ASL) is a complex visual-spatial language that is used by the Deafcommunity in the United States and English-speaking parts of Canada. It is a linguistically com-plete, natural language. It is the native language of many Deaf men and women, as well as somehearing children born into Deaf families. ASL shares no grammatical similarities to English andshould not be considered in any way to be a broken, mimed, or gestural form of English.

Figure 1.1: American Sign Language

1


5/16

1.2 System Architecture

In this project, a data glove is implemented to capture the movement or gesture of a users fin-gers as it has been used by previous researchers for sign language recognition. When the userperforms sign languages with the data gloves, all the sensors (Flex Sensors and Accelerometer)will provide measurements of hand gestures in analogue signals. The analogue signals are then fedto the PIC microcontroller for analogue to digital signal conversion. At the same time, the PICalso collects all gesture data and sends it to computer via USB port where it is stored in a database.

The basic components used in this system are given below:1. Flex Sensors (4.5, 2.2)2. Accelerometer (ADXL335)3. PIC Microcontroller (18f4550)4. USB cable5. Operational Amplifiers (LM741)6. An ordinary glove

Figure 1.2: Basic system architecture

1.2.1 Flex Sensor

Flex sensors are sensors that change their resistance depending on the amount of bend on thesensor. They convert the change in bend to electrical resistance - the more the bend, the more theresistance value. They are usually in the form of a thin strip from 1 inch - 5 inch long that vary inresistance from approximately 10 to 50 k.

The resistance of the flex sensor changes when the metal pads are on the outside of the bend(text on inside of bend). The membrane construction is both resilient and somewhat durable, andcan be used within a temperature range of -35C to +80C for an operational life rating of over 1

million movements if the sensor is secured properly.

Flex sensors are used in gaming gloves, auto controls, fitness products, measuring devices, as-sistive technology, musical instruments, joysticks, and more. They are often used in gloves to sensefinger movement. The Flex point bend sensor was first developed for automotive airbags and nowis also used in car horns, toys that detect different degrees of pressure or bending, robots, machinecontrol, medical devices and assistive technology.

2


6/16

Figure 1.3: Flex sensor

Flex sensors work as variable analog voltage dividers. Inside the flex sensor are carbon resistiveelements within a thin flexible substrate. More carbon means less resistance. When the substrateis bent the sensor produces a resistance output relative to the bend radius.

Figure 1.4: How flex sensor works?

Mechanical Specifications: (4.5)- Life Cycle: >1 million- Height: 0.43mm (0.017)- Temperature Range: -35C to +80C- Part length: 112.24 mm

- Active length: 95.25 mmElectrical Specifications: (4.5)- Flat Resistance: 10k- Resistance Tolerance: 30%- Bend Resistance Range: 15k to 50k- Power Rating : 0.5W continuous, 1W PeakMechanical Specifications: (2.2)- Life Cycle: >1 million- Height: 0.43mm (0.017)- Temperature Range: -35C to +80C- Part length: 73.66 mm- Active length: 55.37 mmElectrical Specifications: (2.2)- Flat Resistance: 25k- Resistance Tolerance: 30%- Bend Resistance Range: 45k to 125k- Power Rating : 0.5W continuous, 1W Peak

In our system we have used both the sensors. The 4.5 sensors are used on index, middle andring finger while 2.2 sensors are used on thumb and little finger. We have used the flex sensors

3


7/16

as voltage dividers in the manner shown in the figure.The impedance buffer is used to prevent thecircuit from loading due to very high impedance of sensors. Suggested op amps are the LM741 orLM324. These flex sensors can also be tested using the simplest circuit without the op amp. R2used is of 33k resistance for 2.2 sensors and of 17k for 4.5 sensors.

Figure 1.5: Basic flex sensor circuit

1.2.2 Accelerometer (ADXL335)

An accelerometer measures acceleration of anything that its mounted on. Single axis accelerom-eters measure acceleration in only one direction. Dual-axis accelerometers measure accelerationin two directions perpendicular to each other. Three-axis accelerometers measure acceleration inthree directions.

Figure 1.6: ADXL335

Accelerometers are very handy for measuring the orientation of an object relative to the earth,because gravity causes all ob jects to accelerate towards the earth. A two-axis accelerometer canbe used to measure how level an object is. With a three-axis accelerometer, you can measure anobjects acceleration in every direction.

Accelerometers are real workhorses in the sensor world because they can sense such a widerange of motion. Theyre used in the latest Apple PowerBooks (and other laptops) to detect when

4


8/16

the computers suddenly moved or tipped, so the hard drive can be locked up during movement.Theyre used in cameras, to control image stabilization functions. Theyre used in pedometers,gait meters, and other exercise and physical therapy devices. Theyre used in gaming controls togenerate tilt data. Theyre used in automobiles, to control airbag release when theres a suddenstop. There are countless other applications for them.

Figure 1.7: Functional block diagram

The ADXL335 is a small, thin, low power, complete 3-axis accelerometer with signal condi-tioned voltage outputs. The product measures acceleration with a minimum full-scale range of 3

g. It can measure the static acceleration of gravity in tilt-sensing applications, as well as dynamicacceleration resulting from motion, shock, or vibration. The ADXL335 is available in a small, lowprofile, 4 mm x 4 mm x 1.45 mm, 16-lead, plastic lead frame chip scale package.

The ADXL335 operates on 2.2-3.6 VDC, and uses very little current (500uA). It has threeanalog outputs, one for each axis. Acceleration on each axis generates a voltage from 0 to approxi-mately 3.3V. When theres no acceleration on a given axis, the output for that axis is about 1.65V.With acceleration in a positive direction along the axis, the output voltage for that axis rises. Withnegative acceleration along the axis, the voltage goes down. In other words:- At rest the voltage is in the middle- At full forward acceleration, the voltage is at its highest

- At full backward acceleration, the voltage is at its lowest.

The device consists of two surface micro machined capacitive sensing cells (g-cell) and a signalconditioning ASIC contained in a single integrated circuit package. The g-cell is a mechanicalstructure formed from semiconductor materials (polysilicon) using semiconductor processes (mask-ing and etching). It can be modeled as a set of beams attached to a movable central mass that movebetween fixed beams. The movable beams can be deflected from their rest position by subjectingthe system to acceleration.

5


9/16

As the beams attached to the central mass move, the distance from them to the fixed beamson one side will increase by the same amount that the distance to the fixed beams on the otherside decreases. The change in distance is a measure of acceleration. The g-cell beams form twoback-to-back capacitors. As the center beam moves with acceleration, the distance between thebeams changes and each capacitors value will change.

Figure 1.8: G-Cell in accelerometer

Specifications:- Measurement range: 3.6g- Non linearity: 0.3% of full scale- Operating voltage range: 1.8V - 3.6V- Supply current: 350uA- Operating temperature range: -40C to +85C

We have used the following schematic to connect the accelerometer to the microcontroller. TheRC filter with 1.0 k resistor and 0.1 F capacitor on the outputs of the accelerometer is used tominimize noise.

Figure 1.9: Interfacing accelerometer to microcontroller

6


10/16

1.2.3 USB (Universal Serial Bus)

The USB is a high-speed serial interface that can also provide power to devices connected to it. AUSB bus supports up to 127 devices connected through a four-wire serial cable of up to three or

even five meters in length. Many USB devices an be connected to the same bus with hubs, whichcan have 4, 8, or even 16 ports. A device can be plugged into a hub which is plugged into anotherhub, and so on.

The maximum power available to an external device is limited to about 100mA at 5.0V.USB isa four- wire interface implemented using a four-core shielded cable.

Figure 1.10: Pin configuration

In USB communication unique vendor ID and product ID should be provided at the start ofcommunication. For commercial purposes these should be purchased. As we are not using thisproduct for commercial purposes, we are using 0x5600 and 0x2323 as product ID and vendor ID

respectively. When the device is attached to computer, these are transferred to computer andauthenticated by it and a message of Device is ready to use is displayed.

In order to communicate between microcontroller and computer, we are using mcHID librarywhich provides different functions to interact with USB device. Similarly on microcontroller sidefunction libraries provided by MikroC Pro for USB communication are used. They basically providedata in accordance with protocol of USB 2.0. With the help of these libraries we are able to usehigh level programming languages for writing our program.

The computer end software is generated using Visual Studio 2012 using Visual Basic.Net. Thedata read by microcontroller (obtained from user for recording gestures) is displayed in text boxesand is sent to data base by clicking appropriate button. Data Base is generated using SQL ServerCompact Edition 4.0. There are two tables in it. One contains user information while the second

table contains data obtained by microcontroller from user. Relation between two tables is generated.The user information table serves as primary table while that of data is foreign key table.

1.2.4 Programming of USB in MikroC

The mikroC language supports a number of functions for USB HID-type communications. Eachproject based on the USB library should include a descriptor source file which contains vendor IDand name, product ID and name, report length and other relevant information. To create a de-

7


11/16

Figure 1.11: Block diagram showing USB programming algorithm

scriptor source file we can use mikroCs integrated USB HID terminal tool (Tools -> HID Terminal).

The mikroC language supports the following USB bus library functions when a PIC microcon-

troller with built-in USB is used (e.g. PIC18F4550), and port pins RC4 and RC5 are connected tothe D+ and D- pins of the USB connector respectively.Hid Enable: This function enables USB communication and requires two arguments: the read-buffer address and the write-buffer address. It must be called before any other functions of theUSB library and it returns no data.Hid Read: This function receives data from the USB bus and stores it in the receivebuffer. It hasno arguments but returns the number of characters received.Hid Write: This function sends data from the write-buffer to the USB bus. The name of the buffer(the same buffer used in the initialization) and the length of the data to be sent must be specifiedas arguments to the function. The function does not return any data.Hid Disable: This function disables the USB data transfer. It has no arguments and returns no

data.

1.3 Conclusion

This project was meant to be a prototype to check the feasibility of acquiring sign language datausing sensor gloves. The completion of this prototype suggests that sensor gloves can be used foracquiring sign language data which can be later used in pattern recognition of language. Moresensors can be employed to acquire more data and recognize full sign language.

8


12/16

Figure 1.12: Flow showing USB communication

One problem that was faced in the project was that some of the alphabets involved dynamicgestures. Their data was not acquired using this glove. So these were left out from the domain ofthe project. Also, some gestures require use of both hands. This requires two sensor gloves.

9


13/16

Figure 1.13: Simulation in Proteus

Figure 1.14: Graphical User Interface

10


14/16

Figure 1.15: Database

11


15/16

Figure 1.16: Schematic for flex sensors

12


16/16

Figure 1.17: Main schematic

13

data acquisition system for sign language

Documents