utilizing kinect control for a more immersive interaction with 3d environment
DESCRIPTION
Utilizing Kinect Control for a More Immersive Interaction with 3D Environment. Implemented by Saed Haj Ali, Kinda Tarboush and Marah Halawah and Supervised by me, Dr. Noor Shaker and Dr. Ammar Joukhadar.TRANSCRIPT
![Page 1: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/1.jpg)
Utilizing Kinect Control for a More
Immersive Interaction with 3D
Environments
Supervised byDr. Noor Shaker Dr. Ammar Joukhadar Eng. Mohammad Shaker
Designed, Implemented and Tested by
Mohammed Saed Haj Ali Marah Halawa Kinda Tarboush
F.I.T.E of Damascus, Syria – AI Department 2014
![Page 2: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/2.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 3: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/3.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 4: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/4.jpg)
“Why using a
stylus when God
gave us ten
fingers!”
Steve Jobs when introducing
the first iPhone, 2010
![Page 5: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/5.jpg)
![Page 6: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/6.jpg)
Good design looks great, yes,
![Page 7: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/7.jpg)
Good design looks great, yes,
but
![Page 8: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/8.jpg)
smell great and sound great?
Good design looks great, yes,
but
why shouldn't it also feel great,
![Page 9: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/9.jpg)
smell great and sound great?
Good design looks great, yes,
but
why shouldn't it also feel great,
![Page 10: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/10.jpg)
Jinsop Lee
“Design for all 5 senses”
The Concept of
![Page 11: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/11.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 12: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/12.jpg)
The player mostly has only one method to interact with the game (most games has the same playing peripherals) like
joystick or mouse
![Page 13: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/13.jpg)
Most games don’t let the player to create their own content\ worlds and share it with others.
![Page 14: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/14.jpg)
For most games there’s no direct interaction between the game and the player (only one way from the player to the game)
![Page 15: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/15.jpg)
For most games there’s no direct interaction between the game and the player (only one way from the player to the game)
![Page 16: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/16.jpg)
A Game For You to Use
Your Senses
![Page 17: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/17.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 18: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/18.jpg)
Sketching a Level
Sketch Your Level
![Page 19: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/19.jpg)
Using Kinect
![Page 20: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/20.jpg)
Interaction with the environment
Hand Gesture
![Page 21: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/21.jpg)
Interaction with the environment
Voice Command
![Page 22: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/22.jpg)
Our Game
Put here a picture or a short video of the game. Just so that people know what are you talking about
![Page 23: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/23.jpg)
The Immersion Process
Player
Sketch
Generating
basic Level
Canvas
Allow Edit and
Generation of
Content
![Page 24: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/24.jpg)
The Immersion Process
Player
Sketch
Generating
basic Level
Canvas
Allow Edit and
Generation of
Content
![Page 25: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/25.jpg)
Hand Gesture
Interaction with
Dynamic
Hand Gesture
Interaction with
Static
Hand Gesture
![Page 26: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/26.jpg)
Dynamic Hand Gesture
• Animated gestures
![Page 27: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/27.jpg)
Static Hand Gesture
• Comparison with reference images
• Using machine learning
• Rule based system
![Page 28: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/28.jpg)
Select
Rotate
Delete
Move
T he ge s tu re s
that we will use
![Page 29: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/29.jpg)
Hand gesture in our application
Dynamic
Hand Gesture
Static
Hand Gesture
![Page 30: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/30.jpg)
Hand gesture in our application
Dynamic Hand Gesture
Discrete Markov
Processes
Static Hand Gesture
Rule based
system
![Page 31: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/31.jpg)
Hand gesture in our application
![Page 32: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/32.jpg)
Interaction via Voice Commands
![Page 33: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/33.jpg)
Voice Command
• Audio to text conversion
![Page 34: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/34.jpg)
Voice Command
• Speech Recognition Hidden Markov Model
![Page 35: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/35.jpg)
Voice Command
• Words Meaning Conceptual Grammar
![Page 36: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/36.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 37: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/37.jpg)
Our ApplicationGenerate 3D environment and edit it using most of your senses!
![Page 38: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/38.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 39: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/39.jpg)
Future Perspective
• Enhance the social aspect of the game
• Multiple player playing at the same time at the same level
• Assess the game immersions metrics
• Allow for more immersive and enhanced gestures
• Analyze the applicability of the approach on other hardware (like Oculus Rift)
• Analyze the applicability of the approach on other game genres
![Page 40: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/40.jpg)
Content
• Motivation
• The Problem with Today’s Games
• Modeling
• Sketching a Level
• Generating the Basic Level Structure
• Allow Edit and Creation on Content through in-game Interactions
• Hand Gesture
• Voice Command
• Our Application
• Future Perspectives
• Demo
![Page 41: Utilizing Kinect Control for a More Immersive Interaction with 3D Environment](https://reader033.vdocuments.us/reader033/viewer/2022060109/5553b535b4c905d4448b4cda/html5/thumbnails/41.jpg)
Thanx for listening!