Francois RiouxFrank Rudzicz

Mike Wozniewski

HCI Project PresentationMcGill University, April 6 2004

:: Overview

Proposed to improve visualization of 3D design.Application is *immersive*.

Allows user to place him/herself inside the 3D world they are designing.

Uses a *two handed* gesture-based interface.Allows for real-time rendering of display.

no need to switch between design-mode and render-mode.

:: Usage

The system could benefit many disciplines: Computer Assisted Design (CAD). 3D game design. Architecture, engineering, 3D layout. Etc.

:: Usage

In our prototype, we focus on “Interior Design”. User can place models of furniture, decorations, &

appliances in a room. User can modify these models:

• Rotate & Translate• Apply Textures/Colors• Etc.

User can save/restore various configurations. User can invite his/her client to navigate around

room to gauge a sense of the space before construction begins.

:: Design Decisions

Unique Design Decisions:

o Bimanual (Two Handed) Interactiono Toolglass interface widgets

:: Bimanuality

Must consider the properties of bimanual (two-handed) interaction as design constraint: There exists structure to bimanual manipulations:

asymmetry & division of labor (Guiard 1987). Non-preferred vs. preferred hand. Preferred hand is typically organized relative to a

dynamic frame of reference provided by the non-preferred hand.

Benefits:• Cognitive load lessened (Leganchuck 1998).• Performance increase (Buxton & Meyers 1986).• Provides additional kinesthetic feedback.

:: Toolglasses

Toolglass metaphor (Bier et al. 1993). Semi-transparent menu. Positioned over a target using the non-preferred

hand. Preferred hand clicks “thru” the menu to apply an

operation to the target.

:: Toolglasses

Toolglasses were chosen and designed as pie menus with handles. Provided the affordance that one could “grab”

them. Provided the affordance of crosshair-like targeting

of objects behind the toolglass.

ToolglassProposed byBier et al.

Our Concept

:: Design Decisions

Other design decisions: Toolglass rack – instead of one toolglass, have

many and separate similar tasks among them. 3DS models – allow users to import the

common .3ds (3D Studio Max) file format. High-level modelling – users perform high-level

tasks such as placing 3D models, applying textures/colors etc. (As opposed to constructing models and scenes from low-level primatives).

:: Initial Prototype

Coded in OpenGL. Allowed modifications (coloring, scaling, rotation, translation) of

GLUT primitives (sphere, cube, torus, teapot).

:: Prototype Evaluation

Suggested changes: Add visual feedback to show which model is currently

targeted. Add yes/no confirmation for irreversible actions (such as the

reset command). Add undo/redo. Implement collision detection so models and user cannot

pass through the walls of the world. Instead of translating models with toolglasses, allow user to

simply ‘grab’ the model and move it. Differentiate ‘system’ toolglasses from ‘model’ toolglasses.

:: Alpha System

Many of the above changes were implemented.

:: Alpha System

Different toolglass handles:

Included support for importing .3DS (3D Studio Max) models, although texture support was buggy and disabled for the release:

Modification toolglasses System toolglasses

:: Alpha System

Also: Added XML config files to configure toolglasses and options

without need for recompiling. Adds flexibility to the design. Added collision detection with walls.

Added undo/redo:Added file saving: Added sub-toolglasses:

:: Beta System

Deployed in the SRE immersive environment, with gesture tracking.

:: Beta System

Added support for gesture tracking: Toolglasses allowed keeping the gesture set at a minimum

Pointing:Holding arm partially

extended moves virtual cursors.

Selection:Fully extending

non-preferred hand selects a toolglass.

Selection:Fully extending preferred hand

invokes a wedge.

Deselection:Arms at side of

body drops toolglass.

Also, Navigation gesture: offsetting ones body from the center of the environment moves the camera in that direction.

:: Beta System

Improved highlighting of targeted models:

Ability to apply different textures and colors to

sub-components of models:

:: Beta System

Clarity & visibility improvement

of toolglasses in rack:

Confirmation of actionsBefore irreversible actions:

:: Future Work

Add support for placing pictograms/textures on toolglass wedges instead of text.

Improve tracking of gestures: Employ a robust hand tracking algorithm rather

than using ‘furthest point from center’ algorithm. Use statistical tracking methods (eg. Condensation

algorithm) to make tracking less susceptible to noise.

Experiments to test benefits of the interface and interaction techniques.

:: HCI Lessons

Visibility of system state & feedback was very important for users. ie, knowing which model they were targeting, which toolglass wedge was

selected, etc.

User feedback early in the process saved lots of time in the long term. Eg: An initial user was concerned about being able to click on a wedge. We now

use angular targeting, which was simpler to code and thus saved time.

A major challenge to gesture-based interfaces is tracking / recognizing the gestures. Currently, we feel our design is robust yet poor gesture tracking makes it almost

unusable.

It is important to prototype the physical aspects of a system. It is quite physically demanding to keep one’s arms outstretched for a long time.

Our prototype used a mouse and keyboard, so no users never tried performing our gesture set for an extended period of time.