Virtual MuseumRamón Oliva, Miguel Pasenau & Eugeni Casadesús

2012, January 24th

Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course

Master in Computing ( LSI – UPC)

2012, January 24th / 2

Contents• Goal

• Implementation

• Results

• Conclusions

2012, January 24th / 3

Contents• Goal

• Implementation

• Results

• Conclusions

2012, January 24th / 4

Goal• Virtual Museum: 3 rooms

• User can move around:– Collision detection

• Objects exposed, selectable for inspection

2012, January 24th / 5

Environment & tools• Cave or Power-Wall:– Limited space– User gestures to move around scene Kinect

• VRJuggler:– Targets Cave, Power-Wall and desktop computer

as well• Kinect interface:– Natural user gestures– NITE + OpenNI + FAAST

• Bullet: collision detection

2012, January 24th / 6

Used tools• gMeshSim ( PFC-EI)• GiD 10.1.9b• VRJuggler 3.0.0• Bullet 2.78• openNI 1.5.2.23• Nite 1.5.2.21• FAAST 0.10.0

2012, January 24th / 7

Contents• Goal

• Implementation

• Results

• Conclusions

2012, January 24th / 8

Museum floor plan

13.964 triangles

2012, January 24th / 9

Objects

1.024 triangles

28.688 triangles

69.451 triangles

111.658 triangles

54.931 triangles

51.925 triangles

51.132 triangles 97.425 triangles

99.877 triangles

2012, January 24th / 10

User movements• Start & finish navigation: left arm up• Forward, backward, left, right:– Segway: lean forward / backward: moves fw / bw – Rotate torso left / right: rotates left / right– Left arm up / down: starts / ends movement

• Selection– Using wand– Stretching right arm to ‘grasp’ object in front

• Inspection:– Zoom in / out

• Help– Raising right hand

2012, January 24th / 11

VRJuggler approach• User stays fixed, scene moves

• Init():– Initialize devices– Build scene:• read ply models: museum walls, pedestals & objects

– Create bullet physics:• Museum, objects and pedestals containers

2012, January 24th / 12

VRJuggler approach• preFrame():– Update physics– Gets and process kinect and other events data

• Draw():– Navigation mode: draw museum and objects– Inspection mode: draw selected object zoomed

2012, January 24th / 13

Actual state• Museum, pedestals and objects are rendered• Collision detection is enabled• User movements are enabled:– Forward and backward, left and right turning– Grasp object for inspection

• Help mode• Inspection mode:– Scale object up and down– Automatic rotation

• Kinect integration:– Gestures recognition and application responds

2012, January 24th / 14

Actual state• Museum, pedestals and objects are rendered• Collision detection is enabled• User movements are enabled:– Forward and backward, left and right turning– Grasp object for inspection

• Help mode• Inspection mode:– Scale object up and down– Automatic rotation

• Kinect integration:– Gestures recognition and application responds

2012, January 24th / 15

VRJuggler issues• No x64 alone build & install• Only mixed x32 and x64 Linux build and install• Using x32 build:

– https://github.com/rpavlik/vrjuggler-windows-binaries

• Units in feet and inches, not meters• User centred, scene moving • No restrictions or definition of movements• No event filtering possible• No seamless platform transition

2012, January 24th / 16

Bullet + VRJuggler issues• User centred, scene moving • Translations whole scene is translated the hard way• Rotations whole scene is rotated the hard way• Collisions:– User: forward ray– Scene: bbox and cylinders moving with scene– Object selection = collision wand ray - object

2012, January 24th / 17

Kinect issues• Unstable FAAST 0.10• Using .jconf: lot of noise• Gesture recognition using FAAST:– Send keys / events to window with focus– Depends on sensor position / orientation

• Some delay

2012, January 24th / 18

Contents• Goal

• Implementation

• Results

• Conclusions

2012, January 24th / 19

Results: navigation mode

2012, January 24th / 20

Results: inspection mode

2012, January 24th / 21

Results: help

2012, January 24th / 22

Results• Source code at:

http://code.google.com/p/museum-navigator/

2012, January 24th / 23

Contents• Goal

• Implementation

• Results

• Conclusions

2012, January 24th / 24

Conclusions• Developing a RV application:– Hard to do– Still a juggling art

• VRJuggler:– Hard to build and install: documentation so-so– Only x32 is known to work

• Bullet: easy to install and to start with• Kinect: difficult to incorporate to VRJuggler:– Gesture analysis and interpretation– Complicated device and proxies structure– “Easy” using FAAST patch

2012, January 24th / 25

Future work• Solve problems with power-wall + kinect:– Inverse camera parameters– Inverse movements

• Use transformations instead of modifying coordinates

• Avoid noise depending on sensor position and orientation

• Interactive museum floor plan editor

2012, January 24th / 26

References• Bullet documentation• VRJuggler documentation• FAAST documentation• http://www.keyboardmods.com/2010/12/howto-kinect-openninite-skelet

on.html• https://github.com/avin2/SensorKinect• http://www.cs.unc.edu/Research/vrpn/• http://projects.ict.usc.edu/mxr/faast/

Virtual Museum

Virtual Reality, Immersive Interaction, Usability and Presence (RVA) course

Master in Computing ( LSI – UPC)

Ramón Oliva, Miguel Pasenau & Eugeni Casadesús2012, January 24th