supporting complexity in distributed virtual reality systems

School of Computer Science and Information TechnologyUniversity of Nottingham

Jubilee CampusNOTTINGHAM NG8 1BB, UK

Computer Science Technical Report No. NOTTCS-TR-1996-6

Supporting complexity in distributed virtual reality systems

Chris Greenhalgh

First released: September 25, 1996

© Copyright 1996 Chris Greenhalgh

In an attempt to ensure good-quality printouts of our technical reports, from the supplied PDF files, we

process to PDF using Acrobat Distiller. We encourage our authors to use outline fonts coupled with

embedding of the used subset of all fonts (in either Truetype or Type 1 formats) except for the standard

Acrobat typeface families of Times, Helvetica (Arial), Courier and Symbol. In the case of papers prepared

using TEX or LATEX we endeavour to use subsetted Type 1 fonts, supplied by Y&Y Inc., for the

Computer Modern, Lucida Bright and Mathtime families, rather than the public-domain Computer Modern

bitmapped fonts. Note that the Y&Y font subsets are embedded under a site license issued by Y&Y Inc.

For further details of site licensing and purchase of these fonts visit http://www.yandy.com

Techhical Report NOTTCS-TR-96-6 1

Supporting complexity in distributed virtualreality systems

Chris Greenhalgh

Technical Report NOTTCS-TR-96-6Department of Computer Science

The University of NottinghamNottingham, NG7 2RD, UK

September 25, 1996

Future applications of virtual reality will require increasingly complicated vir-tual worlds to yield the richness and functionality demanded by increasingnumbers of users. This document introduces some qualitative measures ofcomplexity which may be used to asses virtual reality systems. A number ofcurrent virtual reality systems are considered with regard to the facilities andcomplexity which they support. Some key methods are identified which areused by these systems to provide support for larger and more complicatedworlds. Finally, these observations and assessments are brought together in alist of recommendations for creating virtual reality systems which support verylarge and complicated virtual worlds.

1. Introduction

It is easy to imagine a virtual world with thousands, even millions, of simultaneous users.Within such an virtual world people might be conducting business, “visiting” friends, shop-ping, or being entertained. However, while the idea is easily grasped, the realisation is some-what more elusive. Before such a world, or many such worlds, will become reality, a numberof problems must be overcome. The user’s interface must be acceptable, in terms of cost,comfort, ease of use and capability. The users must want to enter the virtual world - there hasto be something there that they want. Communications infrastructure must be in place to allowusers access to a common virtual world. And the technology for supporting the creation andanimation of virtual worlds must be able to cope with sufficient complexity.

This document focuses on the last of these considerations: supporting complexity in virtualreality systems. Other workers in the field are developing the user interface, for example, sup-ported by the entertainment industry. The content of virtual worlds will depend on the capabil-ity of these worlds, and the flexibility of a virtual reality approach gives many potentialapplications. Communications infrastructure is improving all the time, as telecommunications


companies move to fibre optic links, and cable companies bring copper or fibre optic cableinto many homes. Virtual worlds will be just one of the services or applications to use thisinfrastructure, though perhaps the most significant.

Dimensions of complexity in distributed virtual reality include:

• number of users;

• number of different applications or uses;

• number of worlds;

• interconnectedness of worlds;

• sizes of worlds;

• number of objects in a world;

• object complexity, e.g. level of detail;

• richness of object behaviour;

• richness of user interaction with the world;

• richness of interaction between users;

• spatial scope of activity within worlds;

• extent to which users can extend the world or create new worlds;

• level of user-specific customisation;

• support for interoperation with other systems (including other virtual reality systems); and

• utilising the full capabilities of user equipment (from the very powerful to the very lim-ited).

The limits on achievable complexity, against which the system competes, are:

• technological and physical limits;

• implementation time and man-power;

• financial cost;

• memory;

• computation; and

• communication.

This document considers each area of complexity and considers the support provided for eachby a number of current systems. In many case, the support for complexity will be consideredfrom the perspective of scalability - the way in which approaches and techniques are affectedby their application to increasingly complicated situations. Virtual reality systems alreadyexist which support to some extent all the things above. But no current systems (in their cur-


rent incarnations, at least) will scale well in all of these dimensions even to the levels forwhich we can anticipate an immediate use. This document then outlines the limiting factors inachieving complexity and the approaches taken by these systems to support increasing com-plexity. This is drawn together into a list of recommendations for virtual reality systems sup-porting increasing complexity in distributed virtual worlds. A number of areas in need offurther work are identified.

2. Dimensions of Complexity

This section considers each dimension of complexity in turn. The range and significance ofeach is described, with illustrations from current systems.

2.1 Number of Users

The meaning of this dimension is fairly self-evident; it is the number of users who may simul-taneously participate in a single world. More users indicates increasing complexity, and thereis (notionally) no limit on the possible number of users.

Current systems may be divided into three classes with reference to the number of simultane-ous users which they support: single user; small-scale multi-user; and large-scale multi-user.

Single user systems include VUE [2] and to some extent MRToolkit [8], which began life withan emphasis on single user systems, though the peer package provides some support for mul-tiple users. The facilities for distributed computation in single user systems are generallyaimed at utilising parallel computation to improve the user’s interface, for example coordinat-ing multiple renderers and other servers.

Multi-user systems include DIVE [6], dVS [7], AVIARY [15] and MASSIVE [9]. These sys-tems all support multiple users by treating users largely within the same distributed objectframework as for other objects in the virtual world. These systems, at least in their currentimplementations, will support a small number of users (3 to 5, say) reasonably well, butwould be poorly suited to supporting a large number of users (e.g. 100 or more).

The third class of system comprises SIMNET and NPSNET [14]. These systems have beendesigned specifically to support large numbers (100s or 1000s) of users. However, by theirnature (military field exercise simulation) the users are fairly dispersed in the virtual worldand tightly constrained in terms of what they can do and how quickly they can do it.

2.2 Number of Applications

This dimension indicates the extent to which the virtual world can support different activitiessimultaneously. This may be in the context of one user performing multiple tasks or usingmultiple tools, or in the context of multiple users who may each be engaged in different activ-


ities and using different tools within a shared space. More applications corresponds to greatercomplexity, and there is (again, notionally) no limit to the number of applications.

Dynamic and distributed virtual reality systems such as DIVE, MASSIVE, AVIARY andVEOS [5] allow the organic addition of new applications and tools to a world. This is by vir-tue of their dynamic nature and an object-based programming metaphor which is shared by allinteracting processes. See [10] for a more extended discussion of dynamism and configurabil-ity in distributed virtual reality systems.

Other systems are more geared to running a single application to completion (e.g. dVS, VUE,RB2 [4]). This generally reflects a tighter binding between the interface components (e.g. ren-dering) and the application components.

2.3 Number of Worlds

This dimension indicates the number of worlds which may exist simultaneously. Larger num-bers indicate greater complexity, and the is no upper limit. This dimension must be qualifiedto some extent in the context of the next factor, interconnectedness. Clearly, it is possible torun even a single-user, singe-world system independently at many sites, without affecting sys-tem complexity. So we must restrict the number of worlds to be the number of simultaneouslyexisting worlds to which a user can gain access from the same terminal equipment.

dVS is essentially a single world system; to enter a new world the user shuts down the systemand restarts with new configuration information. MRtoolkit, without the peer package, is alsoa single world system, as is VUE.

AVIARY, DIVE, MASSIVE all support multiple worlds and DIVE and MASSIVE provideportals between worlds (see next section).

The complexity of multiple worlds derive from the problems of naming and binding to a largenumber of worlds. For example, DIVE and MASSIVE use simple text names to identifyworlds, which could lead to naming conflicts and difficulties in locating relevant worlds.

2.4 Interconnectedness of Worlds

This is related to the above, in that a system must support multiple worlds before interconnec-tion becomes an issue. Richer interconnections between greater numbers of worlds reflectgreater complexity.

Portals in DIVE and MASSIVE are examples of world interconnections. The basic unit ofinterconnection in both cases is a one-way portal which specifies a destination world and posi-tion. In both cases the choice of destination world is limited: in DIVE it must share the sameISIS support (port number and host set); and in MASSIVE it must have been created with thesame master collision manager.


Other forms of interconnection could include two-way channels, choices of destinations, user-initiated jumps and context sensitive “suggested” jumps (left by other users or generated bythe system).

2.5 Size of World

This describes the spatial extent of a world. Larger worlds are considered more complicated.It must be noted that size has no intrinsic meaning: the choice of units is arbitrary. Rather it isratios that are significant. For example, the ratio of the smallest feature size to the largestworld size, the ratio of the smallest object size to the largest object, the ratio of the user’sembodied size to these other sizes, and the ratio of the user’s largest embodied size to theirsmallest embodied size where this can be varied.

Limits of this nature are generally the result of the limited mathematical precision used inimplementing the world. Current systems seem to cope well with “normal” sizes. I.e. whereusers have fixed size and perceptive capabilities similar to those in the everyday world, givinga lower limit on object and feature size, and where worlds are finite (typically a number ofmiles equivalent across), giving an upper limit on world size. However current systems wouldhave trouble with unconstrained worlds, where for example, a user could shrink themselves(and their ability to perceive) to the size of an atom or expand to the size of a galaxy. In gen-eral, however, the number of objects and their complexity is likely to present rather more of aproblem than is increasing the mathematical precision of operations in such worlds.

It should be borne in mind that where systems include a mapping to “real” sizes (metres) thatsuch a mapping should be able to cover the spectrum from atoms to galaxies, as virtual realityapplications may be addressed to both of these extremes.

2.6 Number of Objects

This is the number of objects simultaneously present within a world. Larger numbers ofobjects correspond to greater complexity. There is no upper limit on the number of objects.

All systems allow multiple objects: it would seem incredibly limiting to allow only a singleobject. This is a key factor in assessing complexity, and dealing with this is likely to havesome of the most profound effects on system design.

2.7 Object Complexity

This expresses the flexibility and capability of individual objects. More flexible and capableobjects correspond to greater complexity. In some cases a more complicated object may beequivalent to a number of less complicated ones, while in other cases a limit in object com-plexity may be a limit in the capability of the system.


Object complexity may include graphical, audio or textual complexity (limits on objectappearance) or behavioural complexity, considered in the next section.

2.8 Object Behaviour

This is the aspect of object complexity which deals with the dynamics of objects. Objectbehaviour might be expressed by, for example, code in a procedural programming language,by rules, or by a neural network. The richer the possible behaviour (in terms of available oper-ations, size of behaviour specification and range of object interaction) the greater the com-plexity.

Current systems take a number of approaches. Many systems define objects using a normalprogramming language (e.g. VEOS uses LISP and AVIARY uses C) or allow objects to bemanipulated by a program (e.g. DIVE, dVS and MASSIVE). DIVE also has finite statemachines to describe light-weight object behaviour while dVS has a simple response to eventmodel for dVISE objects.

2.9 Interacting with the World

This dimension reflects the richness of interaction with a virtual world. Greater complexitycorresponds to more modes of interaction, a greater range of operations, greater flexibility inthose operations on more efficient interaction.

Some systems provide quite limited interaction. The first version of MASSIVE addressesinteraction between users but not interaction with the world: the world can only be perceived,not changed. Other systems, such as dVS and DIVE, allow selection and movement ofobjects. However, many of the systems, while they can support interaction, but do not providea general high-level framework.

2.10 Interacting with Users

This is the richness of interaction available between users. More modes of interaction (e.g.different media), more detailed rendering and more flexible and expressive controls corre-spond to greater complexity.

Clearly, multi-user capability is a pre-requisite for user interaction. MASSIVE allows users tointeract over a combination of graphics, audio and text channels, as controlled by the spatialmodel of interaction [3]. Other systems such as DIVE and dVS only enable users to see eachother embodied in the virtual world.


2.11 Spatial Scope

This factor is related to the number of objects in a world, and is particularly important in thecontext of the spatial model of interaction [3]. Often an object or user need not be aware ofallof the objects in a world; it may be sufficient for it to be aware of those that are “close” to it(e.g. within some limiting distance). The greatest complexity is where there is no spatial scope- all of the objects in the world have mutual awareness. Larger scopes correspond to greatercomplexity.

MASSIVE’s communication model is based fundamentally on spatial scopes to enable inter-action. AVIARY uses a spatial scope (the caching volume) to limit the objects considered bythe renderer. The WAVES architecture includes message filtering which might be used toimplement spatial scoping. Most other systems (e.g. DIVE and dVS) have no scoping facility.

2.12 World Building

This dimension measures the flexibility and ease with which worlds may be created and mod-ified. Aspects which increase complexity include “immersive” modification, i.e. modifying aworld while experiencing it, and simultaneous modification by multiple users.

dVS offers limited immersive editing. The author is not aware of any systems which currentlysupport immersive editing of general behaviour. Multi-user editing is also limited: to one userper file of light-weight objects (e.g. DIVE, dVS) or per heavy-weight object (e.g. DIVE, dVS,AVIARY, MASSIVE). The world builder also requires privileged access to the system.

2.13 Customisation

This expresses the extent to which and flexibility with which each user can uniquely custom-ise their interaction with and perception of the virtual world.

Some systems, such as dVS, DIVE and MASSIVE, allow users to specify their embodimentin the virtual world (their appearance to other users). MASSIVE allows users so chose anycombination of three interfaces: graphics, audio and text. Most systems allow some customi-sation to reflect the interface equipment available, e.g. desktop or immersive, type of tracker.

Otherwise there is little scope for, and therefore little support for, individual customisation.See [10] for a longer discussion on subjectivity, objectivity and customisation in virtualworlds.


2.14 Interoperation with Other Systems

This measures the systems ability to interoperate with other systems, be they other virtualreality systems or other more “mundane” systems (e.g. databases, spreadsheets, word proces-sors).

The author is not aware of any support for interoperation between the virtual reality systemsdiscussed in this document. Specific applications may be created to provide a virtual front-endto traditional applications, for example Nottingham University have a DIVE applicationwhich provides a limited browsing interface to the UNIX file system.

2.15 Utilising Terminal Equipment

This measures the systems ability to make effective use of whatever equipment the user has.To make more effective use of a greater range of equipment corresponds to greater complex-ity.

Some of this support is quite low level: programs naturally run faster on faster processors, andthe rendering code may support a range of graphics accelerators. Often support must be sup-plied explicitly by the system’s creators, e.g. support for immersive verses desktop use (e.g. inDIVE). At a more VR-specific level support may be provided for varying levels of detail orfor completely different interface types (as in MASSIVE, which supports both graphical andtextual interfaces).

3. Limitations on Complexity

The previous section outlined some of the aspects of distributed virtual reality which may beused to measure the complexity of a world or system. This section identifies factors whichlimit the achievable complexity in a real system. The goal of the virtual reality system is touse the resources represented by these limitations efficiently, in order to provide “sufficient”complexity in the above areas. Exactly what is considered “sufficient” may vary in differentapplication domains, but some approaches may be expected to be better than others in most orall domains. One goal of this document is to identify such superior approaches and, wherenone is clear, to highlight the need for further work.

The sub-sections which follow list and briefly describe the key limitations. Section 4 consid-ers the approaches taken by a number of virtual reality systems to address these limitations.

The first three limitations fall largely outside the influence of virtual reality system implemen-tors. They are presented to retain the “big picture” within which this implementation anddesign takes place. The second three limitations are much more within the influence of thesystem implementors, and will be the focus of subsequent consideration and analysis in theassessment of current systems. The aim of the system’s designers and implementors is to


achieve the most capable system given these limitations. Often this will require a trade-off tobe made between different aspects of complexity, or between different limitations, to find thebest match to current and anticipated facilities.

3.1 Technological and Physical Limits

There will always be technological and physical limits on what can be achieved by a system,and these will limit the total achievable complexity. Some of these limits are shifting continu-ally, e.g. screen size and resolution, processor power, communications bandwidth. These tech-nological limitations are expected to continue moving for at least a number of orders ofmagnitude in performance. However, other limits are less likely to move, most notably thespeed of light which limits communication latency.

These limitations should be borne in mind when designing architectures and protocols, bothwith regard to making systems function effectively on contemporary technology and to avoidhindering the use of anticipated technologies.

3.2 Implementation Time and Man-power

This limitation, corresponding to the resource represented by peoples’ work, and impactsprincipally in two areas: the implementation and maintenance of the virtual reality system;and the creation and evolution of worlds realised using that system.

It is clearly up to each developer to decide how effort should be allocated to the differentaspects of system and world design. But it is suggested that more efficient use can be made ofthis resource within a simple (but not simplistic), logical, consistent and flexible system. Also,the system designers must consider the effort that subsequent users must expend in creatingand maintaining worlds. In the long term, many more years of effort may be invested in creat-ing worlds than in the system itself, and additional effort spent in system design and imple-mentation may ease the burden of subsequent users.

3.3 Financial Cost

There is more scope for compromise within this limitation. The greatest compromise isbetween more expensive systems which are likely to be taken up only in small numbers, andcheaper systems which are likely to be taken up in larger numbers. There are also a number ofcritical boundaries in the cost dimension, for example to gain acceptance in the game market,the home appliance market, or the general or specialist technical market.

The continual shifting of technological limits (“the state of the art”) also has a profoundimpact on cost: almost everything is becoming cheaper. So once again, system designers mustconsider not only what can be bought today, but what will be available in two years, or tenyears time.


3.4 Memory

Memory is the resource consumed within the computer in order to maintain state information.This includes all types of storage, from cache, through main memory, to disk storage and off-line tape or removable disc storage.

State information required to maintain a virtual world includes:

• object type data, e.g. class definitions, behaviour code, initial values, geometry and audiodata;

• object instance data, for “live” and “passivated” (off-line) objects;

• current messages, including audio and video packets;

• world state information;

• system configuration data;

• system program executable images;

• transient state used in program execution (e.g. stack space);

• operating system state; and

• operating system executable image.

The system designer can influence the use of memory (and computation and communicationresources) in many ways. Sufficient performance and flexibility must be achieved at minimalcost. Trade-offs can also be made between different storage demands, and between memoryuse and the use of other resources (e.g. see the techniques in section 4).

3.5 Computation

The computation resource measures the capability of the systems or systems to execute code,and is measured roughly using measures such as MIPS (Millions of Instructions Per Second)and MFLOPS (Millions of FLOating point operations Per Second).

Computational resources will be consumed by:

• realising object behaviour;

• marshalling, sending and receiving messages;

• handing peripheral devices, e.g. mouse and tracker input;

• rendering output, e.g. 3D object manipulation and graphical rendering, audio output andother forms of feedback;

• maintaining the system context;

• system management (reporting and control); and


• maintaining machine operation and other services (the operating system).

The same kinds of decisions must be made with regard to computation as for memory, above.

3.6 Communication

The communications resource measures the ability to transfer information between the dis-tributed elements of the system. Key parameters in assessing communications include band-width, latency, jitter, error rate, data loss rate. Together these are termed quality of service(QoS).

Communications resources are consumed by:

• messages moving between objects;

• audio and video streams;

• system management and reporting messages; and

• system house-keeping messages.

Again, the same kinds of decisions must be made about communications as for computationand memory.

4. Achieving Scalability

This section draws out some of the specific ways in which current systems, as outlined in theprevious sections, aim to achieve scalability and support complexity.

4.1 Distribution

A key technique shared by all of these systems is that of distributing computation over anumber of processors. This allows greater performance through the parallel execution of code,and allows more peripherals to be used, e.g. multiple displays. Distribution is also required tosupport geographically dispersed users. The obvious cost of distribution is in communica-tions. LANs are much slower than computer buses, and WANs are generally slower still. Also,the communications bandwidth doesn’t necessarily increase as the number of processorsincreases, whereas the computational resources, and hence the need to communicate, doesincrease.

Distribution may also make more memory available (additional memory being associatedwith the additional processors). However, in afully replicated system, this will make little orno contribution to scalability.


4.2 Object migration and load balancing

An additional technique for making the most of computational resources is that of load bal-ancing, supported by object migration. Here, objects may be moved from more heavily loadedprocessors to less heavily loaded ones, so as to make the best use of the available processingpower. WAVES supports load balancing and AVIARY supports object migration but does notyet implement load balancing.

The cost of supporting load balancing and object migration is an additional overhead (compu-tation and memory) in message transfer and routing.

4.3 Requiring explicit interest declarations

Requiring all objects and processes to explicitly declare interest in other objects and eventssupports two techniques in different types of application. Firstly, in a distributed database,replication and distribution of database items may be minimised; this avoids the need to fullyreplicate the database (as in DIVE). For example, dVS uses explicit registration of interest tosupport as-required replication of database items and events. Secondly, in a message-passingor event-driven system explicit declaration of interest can be used to limit distribution of mes-sages and events to only those objects which will use them. WAVES’ message filtering is anexample of this, and AVIARY often operates in a similar manner.

These techniques should result in reduced communications overheads, lower memory require-ments and subsequently reduced computation at the receiving nodes or objects. However, thesending objects and intermediate system components will consume some additional memoryand processing to support registration of interest and filtering of messages. Coordinating theelements of the system in order to distribute the correct data items will also impose a commu-nications overhead. Generally, however, there should be an overall saving, especially in largersystems.

One additional complication involves the use of hardware multicasts, and is considered insection 4.9.

4.4 Spatial localisation of interaction

This technique is based on the observation that objects and events that are close in the (virtual)world demand greater interest and attention, and have greater effects, than others which arefar off. A critical region is defined (for each object or by some other means) beyond whichobjects and events are deemed to be insignificant. Object interaction and propagation ofevents is then constrained to occur only within the critical region. This may be considered tobe an extreme form of distancing (section 4.6). One system which makes extensive use of spa-tial localisation is MASSIVE; the aura concept of the spatial model is used to limit objectinteractions based on spatial proximity.


The benefits and hazards of this approach are the same as for explicit interest, and for distanc-ing. I.e. an individual object faces reduced communications and lower memory and computa-tion requirements because it is aware and can interact with only a subset of the objects in theworld. However, there is a system overhead, including communications, memory and compu-tation, to support the operation of spatial localisation and to distribute message appropriately.

4.5 Adaptive spatial subdivision

This method is related to spatial localisation, above, but applies more to system componentswhich require some sort of world overview, e.g. for collision detection. As the number ofobjects in a world increases so the load (in terms of communication, computation and mem-ory) on such a process increases. Adaptive spatial subdivision is the technique whereby thetotal space is divided up into subspaces, which are generally non-overlapping and adjacent.Each subspace may be handled by a separate process (on a separate processor), or the divisionmay be purely internal.

In most situations this technique depends for its effectiveness on object interactions beinggenerally local. I.e. most interactions should fall within a subspace, since interactionsbetweensubspaces result in additional communications to coordinate between the processes handlingthose subspaces. The benefits of this technique are principally an increase in computation (byparallel handling of subspaces), but also each process should be able to localise communica-tions and limit memory requirements. The overhead, as noted above, is in coordinating theparallel subspaces, especially where objects in different subspaces interact.

4.6 Distancing

Distancing is the general technique of exploiting distance-related perceptual limitations. I.e.making use of the fact that distant objects can be seen (or heard) in less detail, and are gener-ally less significant for interaction than are close objects. The basis of this method is to exploitknowledge of interaction limitations (e.g. rendered size) to provide onlysufficient rather thancomplete information to distant viewers. The best known applications of this are visual, butthe same concepts can be applied in any context where some equivalent of “level of detail” ismeaningful. Superscape Virtual Reality Toolkit, a simple PC-based virtual reality system, sup-ports distancing of graphical objects.

Spatial localisation is an extreme form of distancing, which considers the distance at whichobjects become insignificant (theoretically, indiscernible). As with spatial localisation andexplicit interest, distancing aims to reduce communications, computation and memoryrequirements by systematically limiting object interaction and inter-visibility. Again, there is asystem overhead to support distancing which includes elements of communications, computa-tion and memory use.


4.7 Location extrapolation

This technique involves the use of historical information about object position and movementto predict future position and movement. This can be used to compensate for different framerates in cooperating systems (e.g. smoothing the apparent movement of a remote object exe-cuting on a slow processor) - this is not strictly an aid to supporting complexity. A simpleextension to this is to require the use of location extrapolation for objects, and to drop positionupdate messages which would be predicted anyway (within some working tolerance).

This technique can reduce the number of position messages (i.e. reduce communicationsrequirements) but at the expense of additional computation: the extrapolation calculations anderror compensation. This is a fundamental technique in NPSNET [14] and SIMNET to reducethe numbers of messages on the network.

4.8 Behaviour modelling

This technique is a generalisation of location extrapolation, above, in that it applies remoteprediction to behaviour in general, in order to reduce communications and compensate fornetwork latency. Depending on the way in which behaviour is predicted the additional compu-tational load may be much greater then for location extrapolation. This is a key technique forsupporting scale in the WAVES architecture.

4.9 Hardware supported multicast

This last technique exploits the fact that messages or database updates must often be propa-gated identically to a number of destinations, possibly all destinations in the world. Mostshared medium networks, e.g. ethernet, support some kind of hardware multicast or limitedbroadcast. By using this hardware multicast, the communications load is reduced - one mes-sage on the wire is sufficient for many destinations - and the sender’s computational load isreduced, since it sends the message only once. There is likely to be some overhead to supportthis facility, e.g. administering multicast addresses and handling message routing and for-warding.

A difficult trade-off in the use of hardware multicast is in the scale at which it is used. Withunicasts, each message must be sent once to each destination, but every destination receivesonly messages intended for it. Whereas with multicasts, the physical multicast destinationaddress may be forced to be a superset of the logical destination set, so some nodes or objectswill receive spurious messages, which they must process and reject. Where hardware multi-cast addresses are plentiful and highly dynamic then logical and physical destination sets maybe closely matched, but where addresses are limited or configuration is slow then physicaldestinations may need to be significantly larger than logical destinations (i.e. a single physicaldestination address may have to serve multiple logical destination sets). It should be noted


that physical address limits may exist within the transmission technology, the network inter-face and within the operating system.

Reliability must also be considered in the use of multicasts. Given an unreliable multicastprimitive, a confirmation will typically be required from every destination, which is a signifi-cant communication and computation overhead (the same as would apply for unicast sends).

NPSNET uses multicast messages to reduce network load. All objects in the world are mutu-ally aware and so a single multicast group can serve the whole world. No confirmation isrequired and the unreliable multicast is assumed to be “sufficiently reliable”, given the sys-tem’s use of location extrapolation which can mask the loss of messages.

5. Current Systems

This section introduces a number of current virtual reality systems, and outlines their basicdesign. Section 6 gives a basic assessment of the complexity they support and identifies thetechniques they use to aid scalability.

5.1 DIVE

The DIVE system [1, 6] is based around a fully replicated database of objects. Objects have astandard form which comprises a simple geometry plus an optional event-driven finite statemachine for behaviour. More complex behaviour is achieved by manipulating the objectsusing specifically written C code. There can be many worlds, each identified by a simple textname.

When a program joins a world it receives a complete copy of the current world state (all of theobjects in the world) via TCP/IP. Changes to the world are propagated by update messages,which are reliably multicast to all processes in the world using the ISIS toolkit. There is also afixed set of events which includes collision, input and interaction events, plus a limitednumber of simple user events. These events are all broadcast to the entire world, and eachprocess interprets them independently.

Programs can join and leave worlds independently, and can move between worlds supportedby the same ISIS process by means of portals.

5.2 dVS

dVS [7] is based on a partially replicated database. There is a single world (the API appears tosupport the possibility of multiple worlds, but it is not implemented in v2.0), and each processwhich joins that world may express interest in individual database items, or in all the items ofa given type in the database. The principal communication is via database operations: creat-ing, deleting and updating database items.


The database is maintained by one agent per machine, and the agents communicate to distrib-ute database items as required. The processes cooperating in the virtual world (actors) includethe renderer, I/O handler and a light-weight object context called dVISE. All objects (those indVISE and application-specific objects) are realised using a standard set of types in the data-base. dVISE objects can have simple event-response behaviours.

5.3 MR Toolkit

In the MR Toolkit [8] communication is based on copying simple data structures betweenprocesses. All connections are explicit, and transfers are asynchronous and uncoordinated.The majority of the MRToolkit is aimed at fixed configurations of cooperating processes withmutual knowledge of each others existence. Generally a single master process creates all ofthe other processes. The peer package provides some low-level communications facilitiesbetween such groups of processors.

There is no object model, or notion of a database or world. These things would have to beimplemented independently using the communications facilities provided by MRToolkit.

5.4 AVIARY

AVIARY [15, 16] is based on a general-purpose distributed object system with message pass-ing for communication. Messages may be sent to single objects, lists of objects, all objects ina world or all objects in the system. In many situations, explicit interest must be expressed byone object before another will send it messages. Objects may be light-weight, executingwithin the context of an object server process, or heavy-weight, having their own heavy-weight UNIX process. Light weight object types must currently be compiled into the objectservers before execution. Object instances may be dynamically created and destroyed and canmove between object servers.

AVIARY supports multiple worlds, identified by a simple name. Objects can move betweenany worlds on the same distributed object system. Light weight objects are implemented usingC in an object-oriented style, which includes inheritance. Heavy weight objects are also writ-ten in C.

AVIARY includes a collision detector (with adaptive space subdivision) for each world. Thisis used by the renderer to limit the objects considered for rendering. The renderer has a cachevolume, somewhat larger than the view cone. The renderer expresses interest in collisionsbetween objects and this caching volume, and uses these collision events to identify objects tobe rendered and to request state updates on those objects.


5.5 VEOS

VEOS [5] is based on a collection of node processes, each of which is a LISP interpreter(based on Xlisp) with the addition of a number of primitive operations implemented in C tosupport communications and other key functions. Entities (objects) in VEOS are lisp pro-grams which communicate using reliable message passing. This message passing is used toimplement a shared database called a tuple space. Each entity has a private and a public tuple-space, plus an imported tuple-space in which it can access items from other entities’ publicspaces.

There may be multiple worlds, each managed by an entity. Interaction is principally achievedby putting items in the database and by getting items from the database using pattern-match-ing criteria.

5.6 WAVES

The WAVES architecture [13] is another architecture based on message passing objects. Thesystem comprises a message manager and a number of hosts which provide execution con-texts for objects (called “objoids”). Message passing can theoretically be limited by each hostspecifying a message profile, which characterises the messages it wishes to receive. This maybe according to the objoid’s location, or semantic constraints (e.g. particular kinds of objects).However this functionality is not present in the first implementation, HYDRA [12].

As well as the objoids being executed on a host, there is a cache of remotely located objoidswhich are of interest to the host. These “clones” model the behaviour of their master object,with the aim of reducing the number of coordinating messages required and enabling predic-tions of behaviour to compensate for network latency.

5.7 NPSNET

NPSNET [14], now in its fourth implementation, is designed for military simulation withmedium to large numbers of users (100s or 1000s), using standard high-end graphics worksta-tions. The world comprises a largely unchanging terrain over which vehicles may move. Eachof these vehicles multicasts its behaviour and appearance to all other participants in the world,using the DIS protocol [11]. Position extrapolation methods such as dead reckoning are usedto calculate the expected position of other vehicles and to minimise the number of positionupdates to be sent (by dropping position updates which fall within some error bound of thatpredicted by the extrapolation method in use).

The constrained behaviour available in NPSNET, together with the use of position extrapola-tion and multicast messages allow the system to potentially support hundreds of users on cur-rent ethernet-based networks.


5.8 VUE

VUE [2] is a single user client-server system. A number of servers provide the main systemfunctionality, such as rendering and application-specific functions. A single client coordinatesall of the servers. Communication is by event broadcast, and the client implements an applica-tion specific set of event-transforming rules in order to coordinate the servers in a single appli-cation. There is no explicit model of objects or worlds.

5.9 MASSIVE

MASSIVE [9] is a prototype testbed for the spatial model of interaction [3]. It is based onpoint to point communication via connected interfaces. Interfaces include actions (RPCs),streams and attributes. An object is characterised by an aura interface and a peer interface.The aura interface is connected to a specific aura manager (depending on the object’s worldand medium) and detects collisions between object auras. Upon collision, the objects begin tocommunicate via their peer interfaces, which may be used to exchange medium-specific infor-mation about appearance or sound, or to pass messages. This peer-to-peer communication iscontrolled by mutual awareness levels, which are calculated using focus and nimbus, twoother components of the spatial model.

Objects may be hand-coded in C, or passive objects (e.g. scenery) can be maintained by anobject server which reads object descriptions from a file. There can be many worlds, eachidentified by a simple name. Objects can move between worlds associated with a single mas-ter aura manager.

6. Summary of current systems

The following table (in two parts) summarises the supported complexity of the systems intro-duced in section 5.

TABLE 1. Comparison of supported complexity in VR Systems

System UsersAppli-cations Worlds

Inter-connection

Objectnumber

Objectcomplexity

Objectbehaviour

DIVE several many many portals many visual only,fixed

process (C) /FSM

dVS several several one N/A many gooda process (C) /event rules

MRToolkit

several one N/Ab N/A N/A(many)

N/A process (C)

AVIARY several many many potentially many good process /class (C)

VEOS several many many possible many good Lisp code


A number of points are apparent from the table:

• support forlarge numbers of users is still lacking - even NPSNET expects to support onlya few hundred users with current technology;

• userinteraction with the world requires further development;

• interaction between users has only limited support in almost all systems; and

• world building facilities, support forcustomisation, andequipment utilisation all requirefurther development.

a. but appearance is specified only by a file name.

b. MR Toolkit provides only low level support with which worlds and objects might be realised.

a. facilities may be provided on a per-application basis by the application programmer.

WAVES several many many possible many good class (C)

NPSNET many one one N/A many simple, fixed process

VUE one one one N/A N/A N/A N/A

MASSIVE several many many portals many good process (C) /passive

TABLE 2. Comparison of supported complexity in VR Systems (cont.)

SystemWorldinteraction

Userinteraction Spatial scope

Worldbuilding

Custom-isation

Equipmentutilisation

DIVE fair limited global only fair good fair

dVS fair limited global only good fair fair

MRToolkit

N/A - - N/A - fair

AVIARY fair, exten-sible

limited in visualiser fair fair fair

VEOS -a - - fair good fair

WAVES - - yes (messagefiltering)

fair fair fair

NPSNET limited fair global only N/A fair fair

VUE - N/A none N/A N/A fair

MASSIVE limited good yes (aura) fair good good

TABLE 1. Comparison of supported complexity in VR Systems

System UsersAppli-cations Worlds

Inter-connection

Objectnumber

Objectcomplexity

Objectbehaviour


The following table (in two parts) summaries the techniques used by these systems to supportincreasing complexity.

Clearly, no system makes use of all of these techniques. Especially under-used techniques aredistancing and hardware multicast.

a. application specific - not part of system as supplied.

a. Any of these systems could be evolved to make use of multicast com-munications. The table indicates current implementation state, to the bestof the author’s knowledge.

TABLE 3. Comparison of scalability techniques used in current VR Systems

System DistributionLoadbalancing

explicitinterest

Spatiallocalisation

Spatialsubdivision

DIVE yes no no no no

dVS yes no yes no no

MRToolkit

yes -a - - -

AVIARY yes potentially yes yes yes

VEOS yes potentially yes - -

WAVES yes yes yes yes -

NPSNET yes no no no no

VUE yes no no no no

MASSIVE yes no yes yes -

TABLE 4. Comparison of scalability techniques used in current VR Systems (cont.).

System DistancingLocationextrapolation

Behaviourmodelling

Hardwaremulticasta

DIVE no no yes no

dVS no no no no

MRToolkit

- - - no

AVIARY - - - no

VEOS - no - no

WAVES - yes yes no

NPSNET no yes no yes

VUE no no no no

MASSIVE - no no no


7. Conclusions and Future Work

What functionality can be considered essential in a general purpose, complex virtual realitysystem? The discussion of the aspects of complexity outlined in section 2 suggests:

• multi-user support;

• multiple application support;

• multiple worlds;

• interconnected worlds;

• support for large numbers of objects;

• support for complicated objects;

• support for rich object behaviour;

• rich interaction between users and the world;

• rich forms of interaction between users;

• world building facilities, including immersive manipulation;

• facilities for customisation; and

• optimised use of terminal equipment.

Some of these points need simply to be kept in mind when designing the system, to ensurethat they are not excluded. Others, most notably those related to the number and complexity ofobjects in the world, have much greater impact on the distributed architecture of the system.Drawing on the preceding discussion and the features found in some current virtual realitysystems, what recommendations can be made about future virtual reality systems to supportgreater complexity?

First, distribution is likely to remain critical to achieving adequate and cost effective perform-ance in very complicated worlds. However powerful single processors become, it is easy toimagine adding objects to a world until the processor is unable to cope satisfactorily. Moreflexible distributed systems, such as those supporting object migration and load balancing, arelikely to yield greater gains in more varied situations. One important effect of this is that com-munications will remain a critical issue. Many of the other techniques covered in this docu-ment aim to limit communications between these distributed components to acceptable andsupportable levels.

Second, any broadcast style approach will be severely limited in its scalability. This includessystems where all data items or events propagate to all objects in a world. To avoid this situa-tion demands some form of on-demand forwarding and replication. This must be supported byrequiring explicit statements of interest to be registered for interactions. This interest may beexpressed as spatial scoping, as more abstract constraints, or as fully explicit object identifica-


tions. Spatially scoped systems, in particular, show significant promise in controlling com-plexity, and deserve further research and testing.

Third, distancing must be applied at a number of levels to support worlds with very largenumbers of objects. Distancing must be applied at the single object level, so that, for example,the graphical detail may be reduced on distant objects. Beyond this, an additional level of dis-tancing should be applied to treat distant groups of objects as a whole. These two approaches,taken together, should permit an effective and efficient distribution of resources between thepotentially many objects with which some level interaction is possible.

Fourth, some form of behaviour modelling and prediction should be available within the sys-tem. This may not be applicable in all situations, but the facility should be available, becausein many situations the benefits could be considerable.

Fifth, the system should support hardware multicasts where appropriate. The benefits fromthis are chiefly for message senders, but in many situations and especially real-time streams(e.g. voice, video) these benefits are considerable, and indeed critical for supporting largernumbers of users.

A system which draws together all of these techniques should provide a platform for support-ing very large and complicated virtual worlds. To make these worlds rich and rewarding to userequires development in other areas which will have a lesser impact on system resources, butwill still require significant development resources.

Areas which require special focus in future work are:

• managing explicit interest requests and data;

• utilising spatial localisation;

• effective use of distancing; and

• supporting the use of hardware multicast in communications.

It has yet to be established in what situations and to what extent each of these will be of bene-fit. Efficient realisations must be developed for each, characterised and assessed, and mergedwith general-purpose distributed computing facilities.

References

1. Anderson, M., Carlsson, C., Hagsand, O., and Stahl, O., DIVE - The Distributed Inter-active Virtual Environment, Technical Reference Manual, July 1993, part of the DIVEdistribution, available from Swedish Institute of Computer Science, Box 1263, 164 28Kista, Sweden, tel. +46 8 752 1500, fax. +26 8 751 7230.


2. Appino, A.P., Lewis, J.B., Koved, L., Ling, D.T., Rabenhurst, D.A., and Codella, C.F.(1992) An Architecture for Virtual Worlds,Presence, 1(1), Winter 1992, pp 1-17.

3. Benford, S.D., and Fahlén, L.E., A Spatial Model of Interaction in Large Virtual Envi-ronments, Proc. Third European Conference on CSCW (ECSCW’93), Milano, Italy,1993, Kluwer.

4. Blau, B., Hughes, C.E., Moshell, J.M. and Lisle, C. (1992) Networked virtual environ-ments,Computer Graphics 1992 Symposium on Interactive 3D Graphics, 157.

5. Bricken, W., and Coco, G. (1993) The VEOS Project, technical report from the HumanInterface Technology Laboratory, University of Washington FJ-15, Seattle 98195, ftp-able from ftp.u.washington.edu as “public/VirtualReality/HITL/papers/tech-reports/Veos_Project.ps”.

6. Carlsson, C., and Hagsand, O., DIVE - A Platform for Multi-User Virtual Environ-ments,Computer & Graphics, Vol. 17, No. 6, 1993, pp. 663-669.

7. dVS product documentation (VL Technical Reference, version 2.0.3, 1993,VCToolKit Technical Reference, version 2.0.3, 1993, dVISE User Guide, version2.0.3, 1993) from DIVISION limited, 19 Apex Court, Woodlands, Almondsbury,BRISTOL, BS12 4JT.

8. Green, M., and White, L., Minimal Reality Toolkit Version 1.3 Programmer’s Manual,February 1994, Department of Computer Science, University of Alberta, Edmonton,Alberta.

9. Greenhalgh, C.M., An Experimental Implementation of the Spatial Model, Comicworking paper COMIC-NOTT-4-15, available from Department of Computer Science,University of Nottingham, University Boulevard, Nottingham.

10. Greenhalgh, C.M., Approaches to Distributing Virtual Reality Systems, internal work-ing paper, available from Department of Computer Science, University of Notting-ham, University Boulevard, Nottingham.

11. Institute of Electrical and Electronic Engineers, International Standard, ANSI/IEEEStd 1278-1993,Standard for Information Technology, Protocols for Distributed Inter-active Simulation, March 1993.

12. Kazman, R., HIDRA: An Architecture for Highly Dynamic Physically Based Multi-Agent Simulations, International Journal of Computer Simulation, 1993.

13. Kazman, R., Making WAVES: On the Design of Architectures for Low-end Distrib-uted Virtual Environments, IEEE Virtual Reality International Symposium(VRAIS’93), September 18-22, 1993, Seattle, Washington.


14. Macedonia, M. R., Zyda, M. J., Pratt, D. R., Barham, P. T., and Zeswitz, S., NPSNET:a network software architecture for large scale virtual environments, inProc. ForthInternational Conference on Artificial Reality and Tele-Existence (ICAT’94), July 14-15, 1994, Nikkei Hall, Tokyo.

15. Snowdon, D.N., and West, A.J., The AVIARY VR System: A Prototype Implementa-tion, 6th ERCIM Workshop, June 1994, Stockholm.

16. West, A.J., Howard, T.L.J., Hubbold, R.J., Murta, A.D., Snowdon, D.N., and Butler,D.A., AVIARY - A Generic Virtual Reality Interface for Real Applications, in VirtualReality Systems, ed. E.A. Earnshaw, M.A. Gigante and H.Jones, Academic Press Ltd.London, 1993.

supporting complexity in distributed virtual reality systems

Documents