mouse simulation using coloured tape
DESCRIPTION
Mouse simulation using coloured tapeTRANSCRIPT
![Page 1: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/1.jpg)
MOUSE SIMULATION USING TWO COLOURED TAPES
Presented ByNeena R Krishna
S7 CSEBTECH,SNGIST
![Page 2: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/2.jpg)
Contents
• Introduction• Realted Works• Proposed System• Hand Recognition and Colour Tape Detection• Mouse Cursor Movements• Click Events• Conclusion
![Page 3: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/3.jpg)
Introduction• Novel approach for Human Computer
Interaction (HCI) .• control the cursor movement using a real-
time camera .• Current method
- involve changing mouse parts - adding more buttons -changing the position of the tracking ball.
![Page 4: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/4.jpg)
Contd..
• Proposed Method-use a camera and computer vision technology,
-image segmentation and gesture recognition
-to control mouse tasks 1)left and right clicking
2) double-clicking and scrolling
• Perform everything as current mouse devices can.
![Page 5: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/5.jpg)
Contd..• The software will be developed in JAVA language.• Use two colour tapes on finger to perform
actions.- One of the tapes will be used for controlling cursor
movement while the relative distance between -the two coloured tapes will be used for click events of the
mouse. -the system will provide a new experience for users in interacting with the computer.
![Page 6: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/6.jpg)
Contd..
• Modern Graphical user interface (GUI)- current standard interface on personal computers is well-defined- it provides an efficient interface for a user to use various applications on a computer
• GUIs combined with mice and track pads - reducing the richness - variety of human communication down to a single point.
![Page 7: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/7.jpg)
Related Work• Index finger for cursor movement .• Angle between index finger and thumb for
clicking events.• Finger tip tracking to control the motion of the
mouse.• A click of the mouse button was implemented
by defining a screen. • Such that a click occurred when a user’s hand
passed over the region.
![Page 8: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/8.jpg)
Contd..
• Another approach- only the finger-tips to control the mouse cursor and click .
- based on image density.- required the user to hold the mouse cursor on the desired spot for a short period of time.
• Next approach– used the motion of the thumb to mark a clicking
event thumb.
![Page 9: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/9.jpg)
Contd..
• A study of the existing systems for on-screen choice selection reveals that people are still limited to the use of devices
• Such as mouse, touchpad, joystick, trackball and touch screen.
• All these devices need contact of hand with them.
![Page 10: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/10.jpg)
Proposed System -System Overview
![Page 11: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/11.jpg)
Hand Recognition and Colour Tape Detection
1. System is to separate the potential hand pixels from the non-hand pixels.
-done by background subtraction scheme.
2. System startup, a pair of background images is captured
-represent the static workspace from camera view
3. After background subtraction, the process of skin segmentation is done .
![Page 12: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/12.jpg)
Contd..
4.Histogram-based skin classifier assigns each of the RGB pixels in the training set
- to either a 3D skin histogram or non-skin histogram.
- the probability is computed that a given RGB color belongs to the skin or non-skin classes .- skin segmentation process outputs an image which is ready for detection of color tapes in the finger.
- algorithm based on HSV color space is used .-HSV stands for Hue,Saturation,Value.
![Page 13: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/13.jpg)
Contd..
5. convert the RGB pixels into the HSV color plane- so that it is less affected to variations in shades of similar color. -H runs on 0 to 360 degree-S value is degree of strength or purity and from 0 to 1
- s=1 purest colour.
-V for brightness and from 0 to 1 -0 is black
![Page 14: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/14.jpg)
Contd..
• resulting binary image is then run through a convolution phase to reduce the noise introduced.
![Page 15: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/15.jpg)
Mouse Cursor Movement
![Page 16: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/16.jpg)
Contd..
• Using the index finger with yellow colour tape as a cursor controller to control mouse cursor movement
• Two different approaches for moving the mouse cursor:1. position mapping the index finger on a camera
screen to a desktop screen position.2. weighted speed cursor control
- the difference of the finger of the current image and the previous image is found.
![Page 17: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/17.jpg)
Contd..
-And the distance between the two is computed
- the mouse cursor is moves fast if the gap between the two finger images is far. - if the gap is close then the cursor moves slow.
• In proposed system First method is used.-More accurate than second method.
![Page 18: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/18.jpg)
Click Events
• The click events for the mouse are mapped with different hand gestures.
• processing the distance between the two coloured tapes in the fingers.
1. Left Click- the system records the distance between the yellow and red tapes in the index finger and the thumb respectively.
![Page 19: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/19.jpg)
Contd..
-the index and thumb must be apart as much as possible so as to get maximum distance
- This distance is regarded as the threshold distance for the event
- as the thumb moves towards the index finger, the distance between yellow and red tapes is decreases .- when the thumb is close to the index finger the system records the reduced distance between them.
![Page 20: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/20.jpg)
Contd..
-When the distance between the tapes is reduced or less ,then consider the event as the left click event of the mouse cursor.
• Thus mathematically, D’ < D
-where D is the distance between the yellow and red tapes in the index finger and the thumb
• Suppose the distance between the tapes at any time is d then for left click event
d ≤ D’
![Page 21: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/21.jpg)
Contd..
-where D’ is the reduced distance-where d is distance between the tapes at any time.
![Page 22: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/22.jpg)
Contd..
Initialisation of threshold distance (D)
![Page 23: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/23.jpg)
Contd..
Reduced distance (D’)
![Page 24: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/24.jpg)
Contd..
2.Right Click- The right click event of the cursor is simulated using the concept of waiting time.- If the yellow tape on the index finger is waiting for 7 seconds(say) in front of the camera pointing at the same location
- then the event is recognised as the right click event of the mouse cursor
![Page 25: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/25.jpg)
Contd..
-Mathematically,D’ < d ≤ D Waiting time = 7 sec.
- the distance between the red and yellow tapes should be between D and D’ respectively
![Page 26: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/26.jpg)
Contd..
![Page 27: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/27.jpg)
Contd..
3. Double Click
-Similar to right click.-The only difference is that the finger gesture used .- If both the colour tapes are waiting for the time 7 seconds and the distance between the colour tapes is D’ then the event is recognised as double click event of the mouse cursor.
![Page 28: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/28.jpg)
Contd..
![Page 29: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/29.jpg)
Contd..
-Mathematically
d ≤ D’ Waiting time = 7 sec.
![Page 30: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/30.jpg)
Conclusion
• The proposed system will completely revolutionize the way people would use the computer system.
• Presently, the webcam, microphone and mouse are an integral part of the computer system.
• Product which uses only webcam would completely eliminate the mouse.
![Page 31: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/31.jpg)
Contd..
• Also this would lead to a new era of Human Computer Interaction (HCI) where no physical contact with the device is required.
![Page 32: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/32.jpg)
QUERIES….????
![Page 33: Mouse simulation using coloured tape](https://reader033.vdocuments.us/reader033/viewer/2022050906/553baf6b4a79596c7d8b4715/html5/thumbnails/33.jpg)