Implementation of Gesture Recognition in the Immersive Visualization
Environment
By Danny Catacora
Under the guidance of:Judith Terrill andTerence Griffin
National Institute of Standards and Technology
To add a set of gestures that were intuitive to the users, and have actions that could be easily reconfigured to be applied to different visualizations
Problem Statement:
Courtesy of capital-benefits-group.com.au
Outline:
• Introduction to the IVE in the RAVE
• Importance of needed gestures in the IVE
• Procedure consisted of designing a gesture, record data, and coding it
• Created similar Wii gestures that are easily defined and recognized
• Future work includes more complicated gestures
Introduction:
IVE?• Immersive Visualization
Environment
• Produces visualizations from large amounts of computed data
• A visualization technology that allows human interaction
Courtesy of NIST
Introduction:
Courtesy of NIST
RAVE?• Reconfigurable
Automatic Virtual Environment
• Takes up the size of a room
• Consists of a wand, headset, and 3 screens
Importance of Problem:
• Flexible and intuitive navigation methods, but lacks pointer gestures
• Newer interactions needed
• Applicable and basic gestures should be investigated
Courtesy of NIST
Research & Design:
Game Technology:• The Wii :
o Uses a remote controller to recognize gestures• Xbox Kinect :
o Removes the controller, recognizes human movement and actions
Improving 3D Gesture Recognition with Spatially Convenient Input Devices
Procedure:
Steps for each gesture: • Gesture definition:
o Specify what happens when the gesture is performedo Define what the gesture program will recognize/look for
• Data collection:
o Figure out how each variable will respond to the gestureo Determine correlations between gestures and changes in
position, speed, and acceleration
Procedure:
Steps for each gesture: • Program implementation of gesture:
o Write code that will recognize gesture accuratelyo Use algorithms and data analysis to write code in C++
• Gesture testing:
o Test the written code for the gesture in both environmentso See if calibration for the gesture works
Data Interpretation:
• Had program record the movement of the wand
• Wand data could be analyzed for specific changes in the x, y, z, yaw, pitch, and roll values.
• Code was written to catch the patterns of gestures.
Results:
• Resulting algorithms and code that recognizes gestures
• Further
complex gestures later defined from these basic gestures
Results:
• Displaying gesture recognition in the RAVE• Display: from terminal, to pop-ups, to 3d arrow
Conclusion
• In the end, able to successfully identify a total 15 gestures
• Actions completely user definable
• Gestures will help current interaction within the IVE • Set basis for more complex gestures in the future
Future Research:
• Eventually remove the wires, glasses, or the remote
• Set basis for more complex gestures in the future
• Establish a greater user-friendly interaction between man and machine in the RAVE
Acknowledgments:
• Judith Terrill - N.I.S.T • Terence Griffen - N.I.S.T • John Hagedorn - N.I.S.T • Steven Satterfield - N.I.S.T • Elizabeth Duval - Montgomery Blair High
School
Thank You.Questions?