Proceedingsof the2002Winter SimulationConferenceE. Yücesan,

�C.-H. Chen,J. L. Snowdon,andJ. M. Charnes,eds.

VRML CLIENTS LINKED THROUGH CONCURRENT CHAT

LeeA. BelforeII SudheerBattula

Departmentof ElectricalandComputerEngineering YnotLearn,Inc.VirginiaModelingAnalysisandSimulationCenter VirginiaBeach,Virginia,U.S.A.

Old DominionUniversityNorfolk, Virginia,U.S.A.

ABSTRACT

Internetbasedvirtual reality offers the opportunityto ren-der contentin threedimensions. In addition, the Internetprovides a mediumto supportcollaborative activities. Inthiswork, wedescribehow collaborativecapabilitiesarein-tegratedinto the Interactive Land useVRML Application(ILUVA). ILUVA isaVRML basedapplicationthatsupportshighly interactivefunctionality, liveupdates,andthedynamicgenerationof VRML content. The collaborative functionshave beenaddedin the context of an Internetchatsessionin which multiple usersmay participatefrom the Internet.In additionto theusualfunctionssupportedby chatapplica-tions, userinformationsharingis supported.The union orintersectionof sessionsfromdifferentusersmaybeproducedandreviewedin theworld.

1 INTRODUCTION

Internettechnologyprovidesboththefabricandfunctionforcollaborative enterprise.Providing theability to effectivelycommunicatefrom any location on the Internet is an im-portantconsiderationgiven the impact the Internethasonmany people’s personaland businesslives. Several tech-nologiesprovide usefulbasesfor exploration. Our VRMLapplicationshave evolved,first throughdevelopmentof theclientpart(Belfore2001)andthenhavebeenextendedto in-cludeclientserverapplications(BelforeandChitithoti 2000,Belfore and Chitithoti 2001). The client applicationsarestand-aloneapplicationsthatsupport liveupdates,or theabil-ity to addcontentto theworld without reload. A varietyofmodelscanbeaddedto aclientsessionincluding,for exam-ple,buildings,roadways,andareasof interest.Mostmodelsincludetheability tobemovedandeditedtosuittheparticularcircumstances.Collectively, the sessionrepresentsan areaof interestthatincludesimprovementsandmodificationsthatarevisualized.TheapplicationintegratesGIS andplanninginformationappropriatefor thethemeof theapplication.Toexpandthe capabilities,server processinghasbeendevel-opedto increasetheflexibility andcapabilities(Belforeand

Chitithoti 2000,BelforeandChitithoti 2001,Belfore2002).The server processingprovides the opportunityto enforceaccesscontrol,enableselectionof thecontentto beviewed,and to generatecontentdynamically. Indeed,in (Belfore2002),GISVRML contentis generatedby translatingnativeGIScontent,in theform of ESRIShapefiles(ESRI1998),ondemand.

To supportinformationsharing,a mechanismmustbein placeto allow usersto determinewhatinformationcanbesharedandhow that informationis shared.As describedinprior works (BelforeandChitithoti 2000,Belfore andChi-tithoti 2001,Belfore2002),auserhasseveralusageoptionsincludingstartinga new session,revising anold session,orincludingasessionmadeavailableby anotheruser. Sessionsfrom otherusersarereadonly to protecttheintegrity of theoriginal user’s session. The applicationsupportsmultiplesimultaneouslogins from a singleuserwho is restrictedtoreviewing sessionsnot actively beingreviewed. Interactionamongusersislimitedsinceinformationcanbesharedonly atlogin. Theresultsdescribedin thispaperextendprior effortsby enablingthecreationandmanagementof virtual worldsthatallow interactivecollaborationthroughachatsession.Inadditionto the expectedcapabilitiesassociatedwith a chatapplication,usersmayshareinformationfrom sessionseachhascreated.Furthermore,simpleoperationsfor combininginformation from differentusersessionshave beenimple-mentedthroughintersectionandunionoperations.Theop-erationsgive participantstheability to determinecommon-ality betweentheirrespectiveeffortsthroughtheintersectionoperationandcombinetheir effortsusingunionoperations.

Severalotherresearchershavemadecontributionsin thedevelopmentof VRML basedcollaborative virtual worlds(Testani,Wagner, andWehden1999,Brutzman,Zyda,Wat-son,andMacedonia1997,TheWeb3DConsortium,Incor-porated2000,Kirner, Kirner, Kawamoto,Cantao,Pinto,andWazlawick 2001, Damer, Marcelo, and Revi 1999). Thework most similar to ours includesa concurrentchat thatlinks clients in an educationalapplication(Kirner, Kirner,Kawamoto,Cantao,Pinto,andWazlawick 2001).A chatthatlinksclientsandtheirworldstoachatserverenablescommu-

BelforeandBattula

nicationandmodificationof theworlds. Onecharacteristicthatdistinguishesourapplicationis themannerin whichob-ject propertiesarechanged.In (Kirner, Kirner, Kawamoto,Cantao,Pinto, andWazlawick 2001),a dialog appearsforediting object locationandappearancewhereasin our ap-plication,a usermodifiesobjectsthroughcandraganddropactionson objectswithin theworld. TheNerve Gardenar-chitecture(Damer, Marcelo,andRevi 1999)describeshowalgorithmsmimicking life processesareusedto createandplantartificialplants.NerveGardenisacollaborativelabora-toryallowinguserstocreateandgerminateplantsin anappletwindow with subsequentinsertioninto a scenegraphcom-mon to all users. The vrtp (Brutzman,Zyda, Watson,andMacedonia1997)describesa generalapproachfor linkingVRML basedworldsandseveralexampledeployments(TheWeb3DConsortium,Incorporated2000). (Testani,Wagner,andWehden1999)describesa virtual trainingenvironmentallowing participantsat distantlocationsto collaborate.In-teractionamongusersis accomplishedusing a multi-userdomain(MUD). A key contribution of this work is the de-scriptionof a softwarearchitecture.

This paperis organizedinto five sectionsincluding anintroduction,anoverview of theILUVA architecture,a pre-sentationof thechatarchitecture,anexamplesession,andaconclusion.

2 OVERVIEW OF THE ILUVA ARCHITECTURE

The applicationarchitectureis assembledto link four pri-marycomponents(BelforeandChitithoti 2000,BelforeandChitithoti 2001,Belfore2001,Belfore2002).Thefirst com-ponentis thedomaindatathat is usedto setthecontext forthe world. The secondcomponentis the server that satis-fies userrequests,convertsbetweendataformats,andlogssessions.Thethird componentis theclient applicationthatrenderstheworld. The fourth component,thechatcompo-nent is describedin the next section. The interrelationshipamongthefirst threecomponentsis illustratedin Figure1. Atstartup,theuserrequeststheURL for theworld. In response,theserverprovidesbothstaticinformationanddynamiccon-tentthatis assembledto createtherequestedworld. Duringa session,the userhasthe ability to make live updates,i.e.modify theworld,saveapplicationinformationontheserver,andrestorea prior session.

2.1 Server Architecture

A key challengein creatingvisualizationsis beingable toprovide timely, up to dateinformation to clientsautomati-cally. In order to achieve this flexibility , the server needsto be able to determinewhat information is available andthenpresenttheappropriatechoicesto theuseruponloggingin. Furthermore,the maintenanceproceduresfor updatingandaddinginformationshouldbe simpleandreliable. Ta-

VRML Plug-in

ApacheWebServer

ContentURLURLContent

zones

urban

Display

RecordDocumentDirectory

Servlets

Host Machine

Client Machine

DatabaseReset

Figure1: Top Level Architecture

ble1 showsahigh level sequenceof theactivities thatoccurin a session.The client andserver have specificfunctionsthat occur in eachstep. In addition,the administrative as-pectsfor eachactivity aredescribedin thethird column.Theserver architectureis built upontheApachewebserver andApacheJServservletengine.Apacheprovidesall of thenec-essarycapabilitiestoservefilesontheInternet.ApacheJServenablesthecodingof servlets,light weightJavamethods,foraccesscontrol,sessioncontrol,generationof dynamiccon-tent, and logging useractivity. The server architectureisdesignedto makeupdateof currentanddevelopmentof newdeploymentsaseasyaspossible. The currentwork buildson server functionality describedin (Belfore andChitithoti2001,Belfore2002)andaddssignificantcapabilitiesfor inte-gratingdynamiccontent.Usersmaylogandafterloggingin,

Table1: User, Client,andAdministrativeAspects

Client Server AdministrativeLogin Login page, user

authenticationUserinformation

Configure Provide choices,take responses

Datarepositories

Generate Content, assem-bly, generation,conversion

Methods to sup-port generationofworlds

Explore Monitor, log, con-tent

Associateloggingwith user

theuserhastheopportunityto selectinformationto view andotheraspectsrelatedto theconfigurationof theworld. Fromthis configurationinformation,therequestedinformationisformattedandpresentedto theuserfor verification. At thispoint,theserverassemblestheinformationandmakeswhat-

BelforeandBattula

everconversionsarenecessaryto includetheinformationintheworld. Oncetheclient hasrenderedtheworld, theuserexplorestheworld. At varioustimes,informationis passedbackto theserver to providearecordof certainevents,suchastheadditionor modificationof a feature.

2.2 Client Architecture

Thedetailsof theclientarchitecturearedescribedelsewhere(Belfore andChitithoti 2000,Belfore andChitithoti 2001,Belfore2001). Theclient is written entirely in VRML andsalientaspectsof the architecturearedescribedhere. Thepurposeof theclient is to provide a platformuponwhich torenderthe world andallow the userthe ability to navigatethroughandinteractwith theworld. In additionto managingthe direct interactionswith the user, the client architecturemustalsocommunicatewith theserver. Theclientarchitec-tureis shown in Figure2. GISinformationcanbeintegrated

Server

Server

points

command

grant

object type

grant

requ

est

request

number of points required

parameters

ResourceManager

ManagerCollision

Object

Work ZoneMain

Arbiter

ManagerSimulation

collision data

bounding box

Initializationhandshake

Menu

Figure2: Client Architecture(BelforeandChitithoti 2001)

in several ways. In the basevisualization,GIS providesabasicframeof referencefor theapplication,describingter-rain,imageryandotherdesiredfeatures.In addition,theGIScanbeacomponentin theapplicationthatausercanchangethe appearanceof or otherwisemanipulate(Belfore 2002).To control the differentcapabilitiesandfeatures,a userin-terfaceis includedto helpguidetheuserthroughtheworld.Theuserinterfaceis implementedasa head’s up displayintheworld thatalwaysfollowstheuser. Theuserinterfacecanincludesuchfeaturesasnavigationaidesto assisttheuserinidentifying importantfeaturesor locations.A live updateisaprocesswherebynew contentis integratedinto anexistingworld. Dependingon thecomplexity of theupdatedcontentandits linkageto theworld, theprocessof performinga liveupdatecanbe complicated. In ILUVA, addedcontentcanbemovedandmanipulatedrequiringa separatemodule,theresourcemanager, to coordinate.Furthermore,caremustbetakenin definingthearchitectureof thecontentto beadded.Figure3 givesanoverview of theobjectarchitecturethat isdeployed in ILUVA (Belfore 2001). Browserapplicationstypically have limited capabilitiesto protectthe userfrommaliciousor inadvertentmodificationof dataon the clientmachine.Thus,in orderto captureinformationfrom a usersession,communicationwith the server is necessary. The

ModelControl

Updates to Server

Any update

Monitor

fields

type

dimensions

location

orientation

parking

etc.

CollisionManager

Bounding Box

Collisioncollision

LocalArbiter

selectedRequest

GrantHighlight

Object Activated

CommandConfiguration and state

Interaction Layer

User interactionSensors

Touch and Planeselect, position, orientation

Edit Layer

Edit Controls

BehaviorLayer

Figure 3: Object Architecture Supporting Live Updates(Belfore2001)

client applicationcommunicateswith theserver by request-ingURLsfor servlets.Theseservletsmaydirecttheservertolog theinformationor mayalsobearequestfor VRML con-tent thatcanbesubsequentlybeaddedto theworld. Thesecommunicationissuesaredescribedin detailin (BelforeandChitithoti 2001).

3 THE CHAT ARCHITECTURE

Building onprior work, ILUVA chatis aclient serverarchi-tecturethatimplementsthechatthatrunsconcurrentwith thesession.The technologyfor constructingchatapplicationsis well defined(Harold2000). In this work, usershave twomethodsfor communicating.Previously, ILUVA providedinformationsharingonly throughthesharingof sessionsina non-interactive fashionasdescribedin (Belfore andChi-tithoti 2001).A chatapplicationhasbeenintegratedinto theILUVA to extendits multiusercapabilities.Thechatappli-cation,implementedin Java,providesamethodallowing im-mediatecommunicationamongclientsthroughachatserverapplication.Thechatsupportstheexportingandimportingof sessionsbetweenusers.Thelatterpartof this sectionex-plains the chatclient, chatserver, importing a sessionandexportinga session.Thechatapplicationarchitectureis il-lustratedin Figure4. In orderto assuresessionownersretaincontrolovertheirsession,sharingof sessionsisaccomplishedonly with their consent.Theusersnegotiatesharingby ex-changingchat messages.Export of a sessionoccursafterthe provider exportsthe requestedsessionthroughthe chatclient. Uponnotificationof theexport, thefirst userreloadsthescenethatincludesthecombinationof thecurrentuser’ssessionandtheexportedsession.

3.1 Chat Protocol

Thechatprotocoloperatesin a client-servermodewith oneserver and several clients. Sockets are openedbetweenasingleserverandeachclientaseachuserlogsin andusersare

BelforeandBattula

Chat Server

Interlock

VRML Frame

URLs

Client

Content

Apache

Servlets

Server

Database

Chat Client

Figure4: ChatApplicationArchitecture.

added.Thechatserver is capableof acceptingconnectionsfrom chatclientsandrelayingmessagesamongsttheclients.Thechatclient takesinput from theuserandcommunicatesthis informationto theserver. In addition,theclientdisplaysany updatesfrom theserver. This formsthegroundwork forthechatapplication.Thechatprotocolissimpleandsupportsa handfulof basiccommandssummarizedin Table2.

Table2: ChatCommandSummary

Command DescriptionStartUserList informsthechatclientthatit isabout

to receive is thelist of usersAdd addsa userto theuserlist box in all

thechatclientRemove removesauserfrom theuserlist box

of all thechatclientsEndof UserList no more user to be updatedin the

userslist box of the chatclientsofall users

exportunion export operationwith union the oftwo sessionshasto beperformed

export intersection export operation with intersectionthe of two sessionshas to be per-formed

3.2 Chat Client

Thechatclientthatis embeddedinto ILUVA is aJavaapplet(Harold 2000). Thechatclient appearsin a browserframeandhastheappearanceshown in Figure5. Informationre-lating to individual usersis storedas browser parameters,previously setduring the login processby servlets,that thechatclientusestonegotiateasocketconnectionwith thechatserver. Thechatclient interfaceincludesthreebuttons,tworadiobuttons,alist, andtwo text boxes.In onetext box,mes-sagesfrom otherusersaredisplayed,servingasthebulletin

boardfor thechatserver. In typical “chat” fashion,theuserrespondsto the messagesin the bulletin boardby enteringa messageto send.Thetwo remainingbuttonsareusedforexportingasessionto a userandimportinga sessionfrom auser. Thelist boxdisplaysall theuserswhohavebeenloggedon this virtual environment. If a sessionis to be exported,theuserhasto selectauser(to whomtheuserlikesto exportasession)from thelist. A mouseclick on theuserin thelistsetstheenvironmentfor exportingasessionto him/her. Theexportbuttonhasto bepressedin orderto transfera sessionto theselecteduser. Throughthechatbulletinboardtheuserscannegotiatetheexchangeof information. Oncetheexportis confirmed,the recipientselectsthe reloadbutton to viewall theobjectsthathavebeenexportedfor theselectedoper-ation. The radio buttonsfor union andintersectioncontroltheexportfunction. Everyapplethasathread(Harold2000)thatreadsto andwritesfrom thechatserver. Thecommandsthatareexchangedarelistedin Table2. In addition,thechatclient handlesfunctionssuchas addinga userto the userlist uponloggingin, deletingtheuserwhenloggedout,andmanagingthe client chatdisplay. User informationfor thechatserverisprovidedthroughservletcallsontheserverthatmaintainandupdatelist of concurrentchatclients.

Figure5: ChatClient Appearance

3.3 The Chat Server

The chat server operateson the samemachinethat servesInternetcontentandservletmethods. This is necessaryinourapplicationbecausebothaccessa commondatabaseonthis machine. The chatserver will responduponreceivingany of theclient commandssummarizedin Table2 andper-form therequiredprocessing.Whena new userlogsin, theuseris connectedto thechatserver by issuinganaddcom-mand.Sinceseveralusersmaybeloggedin simultaneously,multi-threading(Harold2000)is necessary. Requestsfromtheclient areprocessedandthechatserverprovidesseveralresultsincludingalist of chatparticipants,availableexportedsessions,andsessionoperations.Thechatservermustinter-actwith the samedatabaseastheservletsdescribedabove,so an interlock mechanismis implementedto ensurethatthedatabaseis not corrupted.Wheneithertheexport-unionor export-intersectioncommandsareissued,theserver willcombinethesessionsin therequestedfashion.Currentlytheunionandintersectionoperationsaredeterminedby identify-ing coincidentobjectsin thesceneaslistedin therespectivesessionfiles for theusers.Theunionoperationenablescol-

BelforeandBattula

laboratorsto combinetheir work into a singlesession.Theintersectionenablesuserstoexaminedifferencesbetweenthesessions,useful if both arecomparingmodificationsto thesamearea.

4 SESSION

In this section,anexampleof aninterchangebetweenusersis presented.The ILUVA applicationsupportssharingofinformationbetweenusersin two methods.Theinformationsharingmechanismdiffers in whetheror not the direct andimmediateinteractionwith anotheruseris necessary. In thefirst case,whenuserslog in, sessionsexportedby otherusersareavailabletheir review (BelforeandChitithoti 2001). Bythis method,sessionsareexportedin a read-onlyfashiontoprotectthe integrity original user’s session. In this work,a second,moreinteractive sharingmethodis presentedthatreliessupportedby a chatapplication. Using the chat, theuserscan merge their sessionsusing a union operationoridentify the commoncomponentsthroughthe intersectionoperation.Figure6 shows theprocessandresultof a unionoperation. As notedearlier, the significanceof the unionoperationis in the ability to combinework betweenusers.In Figure6.a, a userrequestsa sessionfrom anotheruser.Figure6.bshowsthestateandchatclientof theseconduser.This userselectstheoperation,eitherunionor intersection,andexportsit to thefirst user. After theexport, theseconduser informs the first that the export hasoccurred. Withthe confirmation,the first userreloadsthe sceneasshownin Figure6.c. In addition,Figure7 shows the intersectionsbetweentheinitial sessionandthemodifiedsession.

5 SUMMARY

In this paper, we have presentedextensionsto the ILUVAarchitecturethatsupportaconcurrentcollaborativechatses-sion. Thechatsessionsupportsimmediatecollaborationandsharingof informationamongusersthatarecurrentlyloggedin. Unionandintersectionoperationsaresupportedto showmodificationsand cumulative efforts respectively. Futurework will focus on developing a wider rangeof chat op-erations,closerintegrationof the chatserver with the webserver, andalsoto supportconcurrentVRML worldswheremodificationsbyoneclientareimmediately reflectedin otherclients’worlds.

REFERENCES

Belfore, L. A. 2001, September. An architecturefor con-structing large VRML worlds. In Transactionsof theSocietyfor ComputerSimulation, 24–40.

Belfore, L. A. 2002,April 14-18.An architecturesupportlive updatesanddynamiccontentin VRML basedvir-

tual worlds. In Military, Government,and AerospaceSymposium, 138–143.SanDiego,California.

Belfore,L. A., andS.Chitithoti. 2000,December. An inter-active landuseVRML applicationILUVA with servletassist.In 2000Winter SimulationConferenceProceed-ings, 1823–1830.Orlando,Florida.

Belfore, L. A., andS. Chitithoti. 2001,April 22-26.Mul-tiuserextensionsto the interactive landuseVRML ap-plication ILUVA. In Thirty-Fourth Annual SimulationSymposium, 159–166.Seattle,Washington.

Brutzman,D., M. Zyda, K. Watson,and M. Macedonia.1997,June18-20.virtual realitytransportprotocol (vrtp)designrationale.In Workshopson EnablingTechnolo-gies: Infrastructurefor CollaborativeEnterprises(WETICE): Sharinga DistributedVirtual Reality. CambridgeMassachusetts:MassachusettsInstituteof Technology.

Damer, B., K. Marcelo,andF. Revi. 1999,January. Nervegarden:A virtual terrariumin cyberspace.In Proceed-ingsof the1999Virtual Worlds andSimulationConfer-ence(VWSIM’99), 131–135.SanFrancisco,CA.

ESRI1998,July.ESRIShapefiletechnical description.ESRI.Harold,E. R. 2000.Javanetworkprogramming. O’Reilly &

Associates,Inc.Kirner, T. G., C. Kirner, A. L. S. Kawamoto, J. Cantao,

A. Pinto,andR. S. Wazlawick. 2001,February. Devel-opmentof acollaborativevirtualenvironmentfor educa-tional applications.In WEB3D2001, 61–68.Paderbon,Germany.

Testani, S., E. Wagner, and K. Wehden.1999, January.CIMBLE: TheCADETTinteractivemulti-userbusinesslearningenvironment.In Proceedingsof the 1999Vir-tual Worlds and SimulationConference(VWSIM’99),105–109.SanFrancisco,CA.

TheWeb3DConsortium,Incorporated2000.Thedistributedinteractive simulation DIS-Java-VRML workinggroup.http://www.web3d.org/WorkingGroups/vrtp/dis-java_vrml.

AUTHOR BIOGRAPHIES

LEE A. BELFORE II is an AssistantProfessorof Elec-trical andComputerEngineeringat Old Dominion Univer-sity. He received the B.S. degreein electricalengineeringfrom Virginia Techin 1982,theM.S.E.degreein electricalengineeringand computersciencefrom PrincetonUniver-sity in 1983, and the Ph.D. degreein electricalengineer-ing from the University of Virginia in 1990. From 1982to 1985,he wasa Memberof TechnicalStaff at AT&T In-formationSystems.From1987to 1988,hewasa researchscientistin theDepartmentof ElectricalEngineering,Centerfor SemicustomIntegratedSystemsat theUniversityof Vir-ginia. From 1990to 1997,he waswith the DepartmentofElectricalandComputerEngineeringat MarquetteUniver-sity, Mil waukee,Wisconsin.Since1997,hehasbeenwith the

BelforeandBattula

(a)User1 Request (b) User2 ExportOperation

(c) ResultingUnion

Figure6: ExampleChat: UnionOperation

Departmentof ElectricalandComputerEngineeringat OldDominionUniversity, Norfolk, Virginia. His researchinter-estsincludeneuralnetworks,datacompression,andInternetbasedvirtual reality applications. He is a Seniormemberof theIEEE anda memberof theComputerSocietyandtheSystems,ManandCyberneticsSocieties.His e-mailaddressis <lbelfore@odu.edu>.

SUDHEER BATTULA is currently employed withYnotLearn,Inc., Virginia Beach,Virginia. In addition,Mr.Battulais currentlyworkingtowardsanM.S.Degreein elec-trical engineeringfrom Old DominionUniversity. Mr. Bat-tula received his B.E. in electronicsand communicationsfrom OsmaniaUniversity, Hyderabad,India.

BelforeandBattula

(a)User1 (b) User2

(c) Intersection

Figure7: IntersectionOperation