Section 1

Touch Your Vision

MULTI-TOUCH INTERACTIVE

SCREEN

Faculty of computer and information science Ain Shams University

Supervisors:Professor Dr. Mohammed RoushdyFaculty of computer and information sciences

Dr. Haythem El-MessiryFaculty of computer and information sciences

T.A. Ahmad SalahFaculty of computer and information sciences

Sponsors:

Teamwork: Abu-Bakr Taha Abdel Khalek

Hadeel Mahmoud Mohammed

Hager Abdel Motaal Mohammed

Mahmoud Fayez El-Khateeb

Yasmeen Abdel Naby Aly

Agenda:1. Introduction.2. Interactive Screen Vs Other systems.3. Market Research &Customer needs.4. Physical Environment.5. System Framework.6. System Modules.7. Applications.8. Limitations.9. Future Work.10. References.

Overview:Projector and two cams system HCI (Human Computer Interaction) system.Interact with hand gestures(shapes).Extension of interactive wall ‘09

Introduction: Problem Definition: Human interacts

normally with another human by using motions, It might be annoying and impractical to use hardware equipments to interact with someone/something.

Introduction: Motivation:

1. “Interactive Wall 2009”.

2. multi touch

technology

3. large size of touch screen with appropriate cost.

4. flexibility.

future without annoying input devices, and to be proud of being a part of accomplishing such a dream.

develop the Interactive Screen System to be able to handle more features and gestures and to get over its limitation which will make the user get satisfied with its usability and flexibility of use.

Time Plan:

Milestones: . Segmentation modules. May-2010 • Multi-hand tracking. April-2010• Automatic hand detection. April-2010 • Z-Depth Module. April-2010• Dynamic gesture Module. May-2010

Physical Environment:

• Simple components constructs a new environment of interactive screens that overcomes limitations of other systems.

•Environment limitations

Traditional Environment

Camera

Mirror

Z-Depth Camera

Interactive Screen

Projector

Proposed Physical Environment:

Physical Environment:

other alternative solutions VS proposed solution

Interactive screen Vs other Systems

Microsoft surface.

Diamond Touch.

Touch Screens.

• Cost• No need To touch the screen.• Gesture Recognition.• Dynamic Gesture Recognition.• Bare Hands.• No sensors, pure image

processing.

• In 1991, First Smart White Board.

• Over 1.6 million smart whiteboards have been installed throughout the world.

• Surveys indicates that interactive whiteboards benefit student engagement, learner motivation and knowledge retention.

Market Research

Framework

System Controlling

Input

Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

Experimental Results

Input apply a geometric calibration using the four calibration points acquired

by the configuration module. Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

Experimental ResultsSimple back ground

Complex back ground

It’s main task is to generate a binary image from the captured

image represents the foreground

Input

Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

Experimental Results

responsible for detecting the hand position automatically at any position,

with certain gesture (Open Hand)

Input

Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

Experimental Results

responsible for keep track with the user hand, know the actual position of the hand all the time

Input

Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

Experimental Results

the main task is to decide whether the user did touch the

screen or not

Input

Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

Experimental Results

Responsible for recognize the shape of the hand.

•Static.•Dynamic.

Input

Calibration

Segmentation

Hand Detection

Multi hand Tracking

Touch Detection

Gesture Recognition

Event

Interface

1 Main Finger

2 Main Fingers

Open Hand Closed Hand 2 Fingers0

50

100

150

Smart whiteboard application

• In class rooms.• In meeting rooms.

Application

Limitations

• Non Skin color for the user top clothes.

• user must wear Long Sleeves.

•Enter the system with certain gesture.

1. Multi-user system.

2. Body Tracking.

3. Detecting any shape of hand.

Future work

[1]. Mennat-Allah Mostafa Mohammad,Nada Sherif Abd El Galeel,Rana Mohammad Ali Roshdy,Sarah Ismail Ibrahim, Multi Touch Interactive Surface, Faculty of Computer and Information Sciences, Ain shams University, Cairo, Egypt, 2009.

[2]. Kai Briechle, Uwe D. Hanebeck, Template Matching using Fast Normalized Cross Correlation, Institute of Automatic Control Engineering,Technische Universität München, 80290 München, Germany.,2001.

[3]. Rafeal C. Gonzalez, Richard E. Woods, DIGITAL IMAGE PROCESSING, Third edition, Pearson, 2008.

[4]. Gray Bradski, Adrian kaebler, Learning Open CV, O'Reilly Media, 2008.

[5]. Alan M. McIvor, Background Subtraction Techniques In Proc. of Image and Vision Computing, Auckland, New Zealand, 2000.

[6]. Francesca Gasparini, Raimondo Schettini , Skin Segmentation using multiple thresholding, Milano Italy, 2007.

References

[7]. Hideki Koike, Masataka Toyoura, Kenji Oka and Yoichi Sato, 3-D Interaction with Wall-Sized Display, IEEE computer society,2008.

[8]. Mahdi Mirzabaki , A New Method for Depth Detection Using Interpolation Functions using single camera, INTERNATIONAL ARCHIVES OF PHOTOGRAMMETRY REMOTE SENSING AND SPATIAL INFORMATION SCIENCES, 2004, VOL 35; PART 3, pages 724-726.

[9]. Patrick Horain and Mayank Bomb, 3D Model Based Gesture Acquisition Using a Single Camera,proceedings of sixth IEEE on applications of computer vision ,2002.

[10] Z. Cˇernekova´, N. Nikolaidis and I. Pitas, SINGLE CAMERA POINTING GESTURE RECOGNITION USING SPATIALFEATURES AND SUPPORT VECTOR MACHINES,EUSIPCO,Pozan,2007.

References

Thank You

AnyQuestions