1997 npsnet - large-scale virtual environment technology ... · npsnet research group overview •...

Calhoun: The NPS Institutional Archive

Faculty and Researcher Publications Faculty and Researcher Publications

1997

NPSNET - Large-Scale Virtual

Environment Technology Testbed

(presentation to University of North

Carolina, Chapel Hill)

Zyda, Michael

10 October 1997: "NPSNET - Large-Scale Virtual Environment Technology Testbed," Brown University.

http://hdl.handle.net/10945/41475

NPSNET - Large Scale Virtual NPSNET - Large Scale Virtual Environment Technology TestbedEnvironment Technology Testbed

Michael Zyda, Don Brutzman, Rudy Darken, Robert McGhee, John Falby,

Eric Bachmann, Kent Watsen,& Ben Kavanagh

Naval Postgraduate [email protected]

Michael Zyda, Don Brutzman, Rudy Darken, Michael Zyda, Don Brutzman, Rudy Darken, Robert McGhee, John Falby, Robert McGhee, John Falby,

Eric Bachmann, Kent Watsen,Eric Bachmann, Kent Watsen,& Ben Kavanagh& Ben Kavanagh

Naval Postgraduate SchoolNaval Postgraduate [email protected]@siggraph.org

Talk OutlineNPSNET Research Group Overview• Techno - software technology for the networked VE.

• Interact - interaction technology for the networked VE.

• Apps - VE applications.

Educational Infrastructure - MOVES CurriculumHow I spent my summer vacation ...

NPSNET Research Group OverviewNPSNET Research Group Overview•• Techno - software technology for the networked VE.Techno - software technology for the networked VE.

•• Interact - interaction technology for the networked VE.Interact - interaction technology for the networked VE.

•• Apps - VE applications.Apps - VE applications.

Educational Infrastructure - MOVES Educational Infrastructure - MOVES CurriculumCurriculumHow I spent my summer vacation ...How I spent my summer vacation ...

NPSNET Research GroupThe focus of the NPSNET Research Group is on the complete breadth of human-computer interaction and software technology for implementing large scale virtual environments (LSVEs).

In addition, we apply that research in constructing VEs useful for the DoD.

The focus of the NPSNET The focus of the NPSNET Research Group is on the Research Group is on the complete breadth of human-complete breadth of human-computer interaction and computer interaction and software technology for software technology for implementing large scale implementing large scale virtual environments virtual environments (LSVEs).(LSVEs).

In addition, we apply that In addition, we apply that research in constructing VEs research in constructing VEs useful for the DoD.useful for the DoD.

NPSNET Research Group

AppsApps InteractInteract TechnoTechno

NPSNETNPSNET

LSVE NetworkLSVE Network Software Arch. Software Arch.Web-Based Web-Based Interoperability InteroperabilityBamboo -Bamboo - VE Toolkit VE ToolkitWorld-BuildingWorld-Building

NPSNET-VNPSNET-VNPSNET-IVNPSNET-IVAmphibious VEAmphibious VE

Inertial Motion Inertial Motion Tracking TrackingLocomotion Devices Locomotion Devices Human ModelingHuman Modeling in the VE in the VESpatial SoundSpatial SoundWayfindingWayfinding

LSVE Network Software Architecture

LSVE - large scale virtual environment - • a networked virtual environment with greater than

1,000 players in it.– When we say this large a number of players, we

mean both live/interactive and autonomous/computer-generated characters.

LSVE - large scale virtual environment - LSVE - large scale virtual environment - •• a networked virtual environment with greater than a networked virtual environment with greater than

1,000 players in it.1,000 players in it.

–– When we say this large a number of players, we When we say this large a number of players, we mean both live/interactive and autonomous/mean both live/interactive and autonomous/computer-generated characters.computer-generated characters.

LSVE Network Software Architecture

Network Software Architecture (NSA) - • the combination of the network protocol used for the

LSVE and the software architecture that supports that protocol within the confines of available bandwidth and processor cycles. – There are many protocol-only and software

architecture-only solutions. There are few full NSA solutions.

Network Software Architecture (NSA) - Network Software Architecture (NSA) - •• the combination of the network protocol used for the the combination of the network protocol used for the

LSVE and the software architecture that supports LSVE and the software architecture that supports that protocol within the confines of available that protocol within the confines of available bandwidth and processor cycles. bandwidth and processor cycles.

–– There are many protocol-only and software There are many protocol-only and software architecture-only solutions. There are few full NSA architecture-only solutions. There are few full NSA solutions.solutions.

NSA for the VE Research Directions

Web-Based VE Interoperability• What are the issues?

– What types of information need to be transferred between web-based networked VEs?

– How do we transfer that information on the Internet (protocol)?

– What is the NSA for this?

Web-Based VE InteroperabilityWeb-Based VE Interoperability•• What are the issues?What are the issues?

–– What types of information need to be transferred What types of information need to be transferred between web-based networked VEs?between web-based networked VEs?

–– How do we transfer that information on the How do we transfer that information on the Internet (protocol)?Internet (protocol)?

–– What is the NSA for this?What is the NSA for this?

Web-Based VE Interoperability

What types of information need to be transferred between networked VEs?• State changes/entity interactions (peer-to-peer).• Heavy-Weight Objects (http client/server requests).• Network pointers (URLs).• Real-time streams (Mbone audio/video).

What types of information need to be What types of information need to be transferred between networked VEs?transferred between networked VEs?•• State changes/entity interactions (peer-to-peer).State changes/entity interactions (peer-to-peer).

•• Heavy-Weight Objects (http client/server requests).Heavy-Weight Objects (http client/server requests).

•• Network pointers (URLs).Network pointers (URLs).

•• Real-time streams (Mbone audio/video).Real-time streams (Mbone audio/video).


How do we transfer that information (protocol)?• Projects

– Dial-a-protocol project (peer-to-peer).– virtual reality transfer protocol - vrtp (the

applications-layer protocol used to wrap together dissimilar application layer protocols).

How do we transfer that information How do we transfer that information (protocol)?(protocol)?•• ProjectsProjects

–– Dial-a-protocol project (peer-to-peer).Dial-a-protocol project (peer-to-peer).

–– virtual reality transfer protocol - vrtp (the virtual reality transfer protocol - vrtp (the applications-layer protocol used to wrap together applications-layer protocol used to wrap together dissimilar application layer protocols).dissimilar application layer protocols).


Dial-a-protocol project• Development of tools/methods for the rapid

generation of peer-to-peer VE applications layer network protocols.– How do we formally specify the state change and

entity interaction information?– How do we embed semantics in syntax in

general? How do we do this on-the-fly?

Dial-a-protocol projectDial-a-protocol project•• Development of tools/methods for the rapid Development of tools/methods for the rapid

generation of peer-to-peer VE applications layer generation of peer-to-peer VE applications layer network protocols.network protocols.

–– How do we formally specify the state change and How do we formally specify the state change and entity interaction information?entity interaction information?

–– How do we embed semantics in syntax How do we embed semantics in syntax in in general? How do we do this on-the-flygeneral? How do we do this on-the-fly??


Dial-a-protocol project• How do we formally specify the state change and

entity interaction information?– Formal BNF specification of DIS.– PDU specification editor and code generator for

DIS readers/writers.– HTML combo-form PDU specification editor.

Dial-a-protocol projectDial-a-protocol project•• How do we formally specify the state change and How do we formally specify the state change and

entity interaction information?entity interaction information?

–– Formal BNF specification of DIS.Formal BNF specification of DIS.

–– PDU specification editor and code generator for PDU specification editor and code generator for DIS readers/writers.DIS readers/writers.

–– HTML combo-form PDU specification editor.HTML combo-form PDU specification editor.


Dial-a-protocol project• How do we formally specify the state change and

entity interaction information?– DIS-Java-VRML

Δ There is a working group working on how we provide DIS-like interoperability for the web. This is a fast-track effort with early success (1500 PDUs per second).

Dial-a-protocol projectDial-a-protocol project•• How do we formally specify the state change and How do we formally specify the state change and

entity interaction information?entity interaction information?

–– DIS-Java-VRMLDIS-Java-VRML

ΔΔ There is a working group working on how we There is a working group working on how we provide DIS-like interoperability for the web. provide DIS-like interoperability for the web. This is a fast-track effort with early success This is a fast-track effort with early success (1500 PDUs per second).(1500 PDUs per second).


Dial-a-protocol project• How do we embed semantics in syntax in general?

How do we do this on-the-fly?– What we are looking at very much has the flavor

of Internet agents.Δ Behavior is encoded in a packet.

Δ Syntax is (identifier type, number of bits).

Δ Semantics is Java object & methods.

Dial-a-protocol projectDial-a-protocol project•• How do we embed semantics in syntax How do we embed semantics in syntax in general? in general?

How do we do this on-the-flyHow do we do this on-the-fly??

–– What we are looking at very much has the flavor What we are looking at very much has the flavor of Internet agents.of Internet agents.ΔΔ Behavior is encoded in a packet.Behavior is encoded in a packet.

ΔΔ Syntax is (identifier type, number of bits).Syntax is (identifier type, number of bits).

ΔΔ Semantics is Java object & methods.Semantics is Java object & methods.


VRMLVRMLJava3DJava3DOpenGLOpenGL

BambooBambooJavaJavaC++C++

InternetInternet

RenderingRendering

Computation &Computation &Network AccessNetwork Access

Shared Data/InteractionShared Data/Interaction

General Entity ModelGeneral Entity Model

Objects areObjects areanything youanything youwant.want.


How do we transfer that information (protocol)?

– The dial-a-protocol project looks at a very small piece of the peer-to-peer, light-weight entity interaction problem in networking VEs.

– If we go back and look at our list of types of information to be transferred between VEs, we see there are other types of data in our VE ...

How do we transfer that information How do we transfer that information (protocol)?(protocol)?

–– The dial-a-protocol project looks at a very small The dial-a-protocol project looks at a very small piece of the peer-to-peer, light-weight entity piece of the peer-to-peer, light-weight entity interaction problem in networking VEs.interaction problem in networking VEs.

–– If we go back and look at our list of types of If we go back and look at our list of types of information to be transferred between VEs, we information to be transferred between VEs, we see there are other types of data in our VE ...see there are other types of data in our VE ...


virtual reality transfer protocol - vrtp • the application-layer protocol used to wrap together

dissimilar application layer protocols.• vrtp will support:

– Light-weight entity interactions (peer-to-peer).

– Network pointers (URLs).

– Heavy-weight objects (http client/server request).

– Real-time streams (Mbone audio/video).

virtual reality transfer protocol - vrtp virtual reality transfer protocol - vrtp •• the application-layer protocol used to wrap together the application-layer protocol used to wrap together

dissimilar application layer protocols.dissimilar application layer protocols.

•• vrtp will support:vrtp will support:–– Light-weight entity interactions (peer-to-peer).Light-weight entity interactions (peer-to-peer).

–– Network pointers (URLs).Network pointers (URLs).

–– Heavy-weight objects (http client/server request).Heavy-weight objects (http client/server request).

–– Real-time streams (Mbone audio/video).Real-time streams (Mbone audio/video).

vrtp & http

HTML

VRML 2.0

http

vrtp

vrtp - a full spectrum application layer protocol

client server

peer peer

httpweb browsermulti-user worlds

audiovideo

DIS behaviors

“reliable” multicast

group-cached http servers (NCSA)


The primary problem we are trying to solve is how to optimize available bandwidth and available processor cycles for our LSVE.• We don’t want to bury the CPU in processing packets at

the applications layer of the operating system.

• We don’t want to flood the network with unnecessary packets.

The primary problem we are trying to The primary problem we are trying to solve is how to optimize available solve is how to optimize available bandwidth and available processor cycles bandwidth and available processor cycles for our LSVE.for our LSVE.•• We don’t want to bury the CPU in processing packets at We don’t want to bury the CPU in processing packets at

the applications layer of the operating system.the applications layer of the operating system.

•• We don’t want to flood the network with unnecessary We don’t want to flood the network with unnecessary packets.packets.

Area of Interest ManagementWe have done a lot of software experimentation and systems testing to get a good understanding of the software architecture issues.• We are now focusing on a layer of software we call

the Area of Interest Manager (AOIM).

We have done a lot of software We have done a lot of software experimentation and systems testing to experimentation and systems testing to get a good understanding of the software get a good understanding of the software architecture issues.architecture issues.•• We are now focusing on a layer of software we call We are now focusing on a layer of software we call

the Area of Interest Manager (AOIM). the Area of Interest Manager (AOIM).

Area of Interest Management

VE SoftwareVE Software

AOIMAOIM

State Changes/Entity InteractionsState Changes/Entity Interactions

Packets to Appropriate Multicast GroupsPackets to Appropriate Multicast Groups

InternetInternet

Area of Interest ManagementState Changes/Entity Interactions• instead of being broadcast to everyone, are

assigned to particular multicast groups (groups are interested parties, spatial partitioning, functional partitioning, temporal partitioning, ...).

State Changes/Entity InteractionsState Changes/Entity Interactions•• instead of being broadcast to everyone, are instead of being broadcast to everyone, are

assigned to particular multicast groups (groups are assigned to particular multicast groups (groups are interested parties, spatial partitioning, functional interested parties, spatial partitioning, functional partitioning, temporal partitioning, ...).partitioning, temporal partitioning, ...).

Area of Interest ManagementPackets to Appropriate Multicast Groups• Multicast groups are subscribed to by appropriate

parties.• Packets not part of subscribed multicast groups are

killed off at the network interface unit rather than at the applications layer in the CPU!

• So your VE only gets packets from groups formally subscribed to.

Packets to Appropriate Multicast GroupsPackets to Appropriate Multicast Groups•• Multicast groups are subscribed to by appropriate Multicast groups are subscribed to by appropriate

parties.parties.

•• Packets not part of subscribed multicast groups are Packets not part of subscribed multicast groups are killed off at the network interface unit rather than at killed off at the network interface unit rather than at the applications layer in the CPU!the applications layer in the CPU!

•• So your VE only gets packets from groups formally So your VE only gets packets from groups formally subscribed to.subscribed to.

Area of Interest ManagementResearch Issues in AOIMs• How do we program in a general way an AOIM?• AOIMs are application dependent. How do we

design such systems for dynamic AOIM replacement?

• In a distributed fashion, how do we dynamically assign information to a particular set of multicast groups?

Research Issues in AOIMsResearch Issues in AOIMs•• How do we program How do we program in a general way in a general way an AOIM?an AOIM?

•• AOIMs are application dependent. How do we AOIMs are application dependent. How do we design such systems for dynamic AOIM design such systems for dynamic AOIM replacement?replacement?

•• In a distributed fashion, how do we dynamically In a distributed fashion, how do we dynamically assign information to a particular set of multicast assign information to a particular set of multicast groups?groups?

DoD Standards for Interoperability

The NPSNET Research Group has always been on the leading edge in the development and utilization of DoD standards for interoperability.• NPSNET-IV is SIMNET and DIS compliant.• We are currently evaluating HLA for its utility in

interactive LSVEs.

The NPSNET Research Group has always The NPSNET Research Group has always been on the leading edge in the been on the leading edge in the development and utilization of DoD development and utilization of DoD standards for interoperability.standards for interoperability.•• NPSNET-IV is SIMNET and DIS compliant.NPSNET-IV is SIMNET and DIS compliant.

•• We are currently evaluating HLA for its utility in We are currently evaluating HLA for its utility in interactive LSVEs.interactive LSVEs.

Bamboo - An Extensible Framework for Networked VEs

Motivation: • A general-purpose, cross-platform, high-level toolkit

providing a framework facilitating VE application research and development.

Motivation: Motivation: •• A general-purpose, cross-platform, high-level toolkit A general-purpose, cross-platform, high-level toolkit

providing a framework facilitating VE application providing a framework facilitating VE application research and development.research and development.

What is Bamboo?• Core mechanisms common to networked VEs.

– an API toolkit for serious programmers.• Dynamically extensible runtime environment.

– an application which puts it all together.• Based on OpenGL++.

– which implies OpenGL, C++, STL.

•• Core mechanisms common to networked VEs.Core mechanisms common to networked VEs.

–– an API toolkit for serious programmers.an API toolkit for serious programmers.

•• Dynamically extensible runtime environment.Dynamically extensible runtime environment.–– an application which puts it all together.an application which puts it all together.

•• Based on OpenGL++.Based on OpenGL++.

–– which implies OpenGL, C++, STL.which implies OpenGL, C++, STL.

What is OpenGL++?Multi-platform (SGI, Windows NT, BeOS, Linux, etc.)Visualization toolkit (Performer + Inventor)• Mechanisms:

– RTTI, linear math routines, scene graph, construction and manipulation, draw/probe/sound actions (traversals), engines (spline, morph), fields, routes, and some extensibility.

Multi-platform (SGI, Windows NT, BeOS, Multi-platform (SGI, Windows NT, BeOS, Linux, etc.)Linux, etc.)Visualization toolkit (Performer + Inventor)Visualization toolkit (Performer + Inventor)•• Mechanisms:Mechanisms:

–– RTTI, linear math routines, scene graph, RTTI, linear math routines, scene graph, construction and manipulation, draw/probe/sound construction and manipulation, draw/probe/sound actions (traversals), engines (spline, morph), actions (traversals), engines (spline, morph), fields, routes, and some extensibility.fields, routes, and some extensibility.

How to extend OpenGL++?Bamboo sits along-side rather than on top of OpenGL++ ...Based on Plug-ins

Bamboo sits along-side rather than on top Bamboo sits along-side rather than on top of OpenGL++ ...of OpenGL++ ...Based on Plug-insBased on Plug-ins

Bamboo Runtime Environ.Bamboo Runtime Environ.

OpenGL++/Bamboo APIOpenGL++/Bamboo API

OSOS OpenGLOpenGL Misc.Misc.

SGI IRIX/Windows NT SGI IRIX/Windows NT

Bamboo’s MechanismsExtensibility

Networking

Security

Process Management

Inter-Process Communication

Remote Procedure Calls

Callbacks

Event Handling

ExtensibilityExtensibility

NetworkingNetworking

SecuritySecurity

Process ManagementProcess Management

Inter-Process CommunicationInter-Process Communication

Remote Procedure CallsRemote Procedure Calls

CallbacksCallbacks

Event HandlingEvent Handling

Device Manager

Database Handling

Statistics Manager

Graphical User Interface

Undo/Redo

Physically-Based Modeling

Friendly OpenGL++ Abstractions

Device ManagerDevice Manager

Database HandlingDatabase Handling

Statistics ManagerStatistics Manager

Graphical User InterfaceGraphical User Interface

Undo/RedoUndo/Redo

Physically-Based ModelingPhysically-Based Modeling

Friendly OpenGL++ Friendly OpenGL++ AbstractionsAbstractions

VE World Building

InteractInertial Motion Tracking of Humans in a Networked VE Locomotion DevicesHuman Modeling in the VESpatial SoundWayfinding

Inertial Motion Tracking of Humans in a Inertial Motion Tracking of Humans in a Networked VE Networked VE Locomotion DevicesLocomotion DevicesHuman Modeling in the VEHuman Modeling in the VESpatial SoundSpatial SoundWayfindingWayfinding

Inertial Motion Tracking of Humans in a Networked VE

Current technologies are unable to provide a natural and intuitive interface for inserting a human into a large scale networked virtual environment.

Current technologies are unable to Current technologies are unable to provide a natural and intuitive interface provide a natural and intuitive interface for inserting a human into a large scale for inserting a human into a large scale networked virtual environment. networked virtual environment.


Advances in the design of micromachined accelerometers and angular rate sensors and 3D spatial positioning using RF, present the possibility of tracking human body motion inertially in a manner similar to that of an inertial navigation system (INS).

Advances in the design of micromachined Advances in the design of micromachined accelerometers and angular rate sensors accelerometers and angular rate sensors and 3D spatial positioning using RF, and 3D spatial positioning using RF, present the possibility of tracking human present the possibility of tracking human body motion inertially in a manner similar body motion inertially in a manner similar to that of an inertial navigation system to that of an inertial navigation system (INS). (INS).


Current human motion capture systems suffer from numerous limitations:• User encumbrance• Restricted Range• Susceptibility to Interference• Latency• Shadowing

Current human motion capture systems Current human motion capture systems suffer from numerous limitations:suffer from numerous limitations:•• User encumbranceUser encumbrance

•• Restricted RangeRestricted Range

•• Susceptibility to InterferenceSusceptibility to Interference

•• LatencyLatency•• ShadowingShadowing


The hybrid inertial tracking system is essentially sourceless and does not suffer from these limitations.Inertial body tracking is based upon same algorithms as inertial navigation of missiles, aircraft and ships.Physics resemble those of the human vestibular system.

The hybrid inertial tracking system is essentially The hybrid inertial tracking system is essentially sourceless and does not suffer from these sourceless and does not suffer from these limitations.limitations.

Inertial body tracking is based upon same Inertial body tracking is based upon same algorithms as inertial navigation of missiles, algorithms as inertial navigation of missiles, aircraft and ships.aircraft and ships.

Physics resemble those of the human vestibular Physics resemble those of the human vestibular system.system.


Proposed tracking system advantages:• High data rates without perceptible lag.• Tolerant to environmentally induced errors.• Will produce consistent registration between physical

and virtual worlds.• Allow multiple users in a large working volume

simultaneously.

Proposed tracking system advantages:Proposed tracking system advantages:•• High data rates without perceptible lag.High data rates without perceptible lag.

•• Tolerant to environmentally induced errors.Tolerant to environmentally induced errors.

•• Will produce consistent registration between physical Will produce consistent registration between physical and virtual worlds.and virtual worlds.

•• Allow multiple users in a large working volume Allow multiple users in a large working volume simultaneously.simultaneously.


Basic Concept:• Instrument the Human body with 15 micromachined

inertial sensors.• Each sensor contains:

– Three-axis accelerometer.– Three-axis rate sensor.– Three-axis magnetometer.

Basic Concept:Basic Concept:•• Instrument the Human body with 15 micromachined Instrument the Human body with 15 micromachined

inertial sensors.inertial sensors.

•• Each sensor contains:Each sensor contains:

–– Three-axis accelerometer.Three-axis accelerometer.–– Three-axis rate sensor.Three-axis rate sensor.

–– Three-axis magnetometer.Three-axis magnetometer.


Complementary filtering would correct for sensor errors and drift.The sensors would provide an orientation vector for each body segment.RF positioning of a single point on the body would accurately place the articulated human within the VE.

Complementary filtering would correct for Complementary filtering would correct for sensor errors and drift.sensor errors and drift.The sensors would provide an orientation The sensors would provide an orientation vector for each body segment.vector for each body segment.RF positioning of a single point on the RF positioning of a single point on the body would accurately place the body would accurately place the articulated human within the VE.articulated human within the VE.


Human Body Tracking Sensor ConfigurationHuman Body Tracking Human Body Tracking Sensor ConfigurationSensor Configuration

Locomotion DevicesMost of our work in locomotion devices has been to integrate other people’s locomotion devices into the NPSNET-IV software platform.Two of our most successful demos ...

Most of our work in locomotion devices Most of our work in locomotion devices has been to integrate other people’s has been to integrate other people’s locomotion devices into the NPSNET-IV locomotion devices into the NPSNET-IV software platform.software platform.Two of our most successful demos ...Two of our most successful demos ...

AUSA 95 TreadportJoint project with Sarcos Engineering, UPENN & NPS.• NPS role was to

integrate treadport into NPSNET-IV.

Joint project with Joint project with Sarcos Engineering, Sarcos Engineering, UPENN & NPS.UPENN & NPS.•• NPS role was to NPS role was to

integrate treadport into integrate treadport into NPSNET-IV.NPSNET-IV.

AUSA 96 Omni Directional Treadmill (ODT)

Human Modeling in the VEWe have explored commercial solutions for animating articulated humans in the VE.• UPENN Jack-ML• BDI DI-Guy• All of these solutions are too slow once you get

above about 5 to 10 humans in view.

We have explored commercial solutions We have explored commercial solutions for animating articulated humans in the for animating articulated humans in the VE.VE.•• UPENN Jack-MLUPENN Jack-ML

•• BDI DI-GuyBDI DI-Guy

•• All of these solutions are too slow once you get All of these solutions are too slow once you get above about 5 to 10 humans in view.above about 5 to 10 humans in view.

Human Modeling in the VELow-cost, articulated humans.• We want to develop

articulated human support software capable of displaying 100 to 150 humans in the NPSNET-V VE.

Low-cost, articulated Low-cost, articulated humans.humans.•• We want to develop We want to develop

articulated human support articulated human support software capable of software capable of displaying 100 to 150 displaying 100 to 150 humans in the NPSNET-V humans in the NPSNET-V VE.VE.

Spatial Sound for the VEVirtual Environment Auditory & Visual Intersensory Modality Issues• Measuring performance & immersion during

wayfinding and target acquisition tasks in a VE.– Are manipulating both auditory & visual display

fidelities.– Want to find out the effects of both modalities on

performance & immersion.

Virtual Environment Auditory & Visual Virtual Environment Auditory & Visual Intersensory Modality IssuesIntersensory Modality Issues•• Measuring performance & immersion during Measuring performance & immersion during

wayfinding and target acquisition tasks in a VE.wayfinding and target acquisition tasks in a VE.

–– Are manipulating both auditory & visual display Are manipulating both auditory & visual display fidelities.fidelities.

–– Want to find out the effects of both modalities on Want to find out the effects of both modalities on performance & immersion.performance & immersion.

Wayfinding in the VETraining Spatial Knowledge Acquisition Using Virtual Environments • The goal of the work is to show that VEs can be

used to acquire spatial knowledge of a specific real space (training transfer).

Training Spatial Knowledge Acquisition Training Spatial Knowledge Acquisition Using Virtual Environments Using Virtual Environments •• The goal of the work is to show that VEs can be The goal of the work is to show that VEs can be

used to acquire spatial knowledge of a specific real used to acquire spatial knowledge of a specific real space (training transfer). space (training transfer).

Wayfinding in the VE• Future work will involve the investigation of

environmental features and perceptual stimuli and their role in navigation and wayfinding so that we may understand how to degrade the fidelity of a VE without substantially degrading navigation performance.

• We will then extend these concepts to training general navigation skills, including map usage (perspective transformation) and landmarking abilities.

•• Future work will involve the investigation of Future work will involve the investigation of environmental features and perceptual stimuli and their environmental features and perceptual stimuli and their role in navigation and wayfinding so that we may role in navigation and wayfinding so that we may understand how to degrade the fidelity of a VE without understand how to degrade the fidelity of a VE without substantially degrading navigation performance. substantially degrading navigation performance.

•• We will then extend these concepts to training general We will then extend these concepts to training general navigation skills, including map usage (perspective navigation skills, including map usage (perspective transformation) and landmarking abilities.transformation) and landmarking abilities.

Applications of VE Technology

NPSNET-IV CompletionNPSNET-VAn Amphibious VE

NPSNET-IV CompletionNPSNET-IV CompletionNPSNET-VNPSNET-VAn Amphibious VEAn Amphibious VE

NPSNET-IV CompletionNPSNET-IV Capabilities• Building walkthroughs.

• Articulated humans - mounting/dismounting capability.

• Networking - play across the multicast backbone of Internet.

NPSNET-IV NPSNET-IV CapabilitiesCapabilities•• Building walkthroughs.Building walkthroughs.

•• Articulated humans - Articulated humans - mounting/dismounting mounting/dismounting capability.capability.

•• Networking - play across the Networking - play across the multicast backbone of Internet.multicast backbone of Internet.

• Terrain database integration, terrain paging (70km x 70km).

• Any vehicle capability - air, ground, articulated human.

• Testbed for VE NSA issues.

• Interoperability - SIMNET/DIS

– Constructive model integration - Janus World Modeler

– ModSAF

•• Terrain database integration, Terrain database integration, terrain paging (70km x 70km).terrain paging (70km x 70km).

•• Any vehicle capability - air, Any vehicle capability - air, ground, articulated human.ground, articulated human.

•• Testbed for VE NSA issues.Testbed for VE NSA issues.

•• Interoperability - SIMNET/DISInteroperability - SIMNET/DIS

–– Constructive model Constructive model integration - Janus World integration - Janus World ModelerModeler

–– ModSAFModSAF

NPSNET-ODT Press Conference & AUSA 96

NPSNET-IV

NPSNET-VFeatures & Plans• Redesigning the network software architecture of

NPSNET.• Plug & play VE where additional modules of

capabilities (networking, human models, terrain models, walkthroughs, mounting/dismounting of humans, general interaction ...) can be developed with a uniform, published API.

Features & PlansFeatures & Plans•• Redesigning the network software architecture of Redesigning the network software architecture of

NPSNET.NPSNET.

•• Plug & play VE where additional modules of Plug & play VE where additional modules of capabilities (networking, human models, terrain capabilities (networking, human models, terrain models, walkthroughs, mounting/dismounting of models, walkthroughs, mounting/dismounting of humans, general interaction ...) can be developed humans, general interaction ...) can be developed with a uniform, published API.with a uniform, published API.

NPSNET-VTechnologies for NPSNET-V• Bamboo - Next Generation Toolkit

– OpenGL++– C++/Java– vrtp & HLA

Technologies for NPSNET-VTechnologies for NPSNET-V•• Bamboo - Next Generation ToolkitBamboo - Next Generation Toolkit

–– OpenGL++OpenGL++

–– C++/JavaC++/Java–– vrtp & HLAvrtp & HLA

An Amphibious VE

Educational InfrastructureMS & PhD Program in Computer Science, with specialization in Computer Graphics & Visual Simulation

MS & PhD Program in Computer Science, MS & PhD Program in Computer Science, with specialization in Computer Graphics with specialization in Computer Graphics & Visual Simulation& Visual Simulation

Educational Infrastructure - MOVES Curriculum

Modeling, Virtual Environments & Simulation Curriculum• The MOVES Curriculum provides the MS and Ph.D.

student both fundamental & specialized courses in applied computer simulation technology & the application of quantitative analyses to human-computer interaction in simulation technology.

Modeling, Virtual Environments & Modeling, Virtual Environments & Simulation CurriculumSimulation Curriculum•• The MOVES Curriculum provides the MS and Ph.D. The MOVES Curriculum provides the MS and Ph.D.

student both fundamental & specialized courses in student both fundamental & specialized courses in applied computer simulation technology & the applied computer simulation technology & the application of quantitative analyses to human-application of quantitative analyses to human-computer interaction in simulation technology. computer interaction in simulation technology.

Educational Infrastructure - MOVES Curriculum• There are two tracks that support the curriculum's

research efforts, the Visual Simulation Track and the Human-Computer Interaction Track.– The Visual Simulation track is focused on

developing the technolgy for VEs.– The Human-Computer Interaction track is focused

on evaluating human performance with the developed technology.

•• There are two tracks that support the curriculum's There are two tracks that support the curriculum's research efforts, the Visual Simulation Track and the research efforts, the Visual Simulation Track and the Human-Computer Interaction Track.Human-Computer Interaction Track.

–– The Visual Simulation track is focused on The Visual Simulation track is focused on developing the technolgy for VEs.developing the technolgy for VEs.

–– The Human-Computer Interaction track is focused The Human-Computer Interaction track is focused on evaluating human performance with the on evaluating human performance with the developed technology.developed technology.

NRC CSTB - Modeling & Simulation Linking Entertainment & Defense

Modeling & Simulation: Linking Entertainment & DefenseISBN 0-309-05842-2

National Academy Press

http://www.nap.edu/readingroom

Modeling & Simulation: Modeling & Simulation: Linking Entertainment & Linking Entertainment & DefenseDefense

ISBN 0-309-05842-2ISBN 0-309-05842-2

National Academy PressNational Academy Press

http://www.nap.edu/http://www.nap.edu/readingroomreadingroom

Mobile Agents - Explanations & Examples

Mobile Agents - Explanations & ExamplesWilliam R. Cockayne &

Michael Zyda

ISBN 1-884777-36-8

Available from Manning Press

http://www.manning.com

Mobile Agents - Mobile Agents - Explanations & ExamplesExplanations & Examples

William R. Cockayne & William R. Cockayne & Michael ZydaMichael Zyda

ISBN 1-884777-36-8ISBN 1-884777-36-8

Available from Manning Available from Manning PressPress

http://www.manning.comhttp://www.manning.com

Networked Virtual Environments

Networked Virtual EnvironmentsSandeep Singhal &

Michael Zyda• ACM Press

Networked Virtual Networked Virtual EnvironmentsEnvironmentsSandeep Singhal & Sandeep Singhal &

Michael ZydaMichael Zyda•• ACM PressACM Press

NPSNET Research Group Sponsors

DARPAONRAdvanced Networks & ServicesNPS Internal Funds

DARPADARPAONRONRAdvanced Networks Advanced Networks & Services& ServicesNPS Internal FundsNPS Internal Funds

NPSNET Web SitesNPSNET Pagehttp://www.npsnet.nps.navy.mil/npsnet

Interact Pagehttp://www.npsnet.nps.navy.mil/interact

Michael Zydahttp://www.cs.nps.navy.mil/people/faculty/zyda

NPSNET PageNPSNET Pagehttp://www.npsnet.nps.navy.mil/npsnethttp://www.npsnet.nps.navy.mil/npsnet

Interact PageInteract Pagehttp://www.npsnet.nps.navy.mil/interacthttp://www.npsnet.nps.navy.mil/interact

Michael ZydaMichael Zydahttp://www.cs.nps.navy.mil/people/faculty/zydahttp://www.cs.nps.navy.mil/people/faculty/zyda

1997 npsnet - large-scale virtual environment technology ... · npsnet research group overview •...

Documents