emil m. petriu, thom e. whalen, rami abielmona, and alan ...petriu/mag04-...

Monitoring environment parameters is acomplex task of great importance in manyareas, such as the natural living environ-ment; homeland security; industrial or lab-

oratory hazardous environments (biologically,radioactively, or chemically contaminated); polluted/toxicnatural environments; water treatment plants; nuclear sta-tions; war zones; or remote, difficult-to-reach environ-ments, such as the deep space or underwater [1]–[4].

This article will discuss a new generation of intelligent,autonomous, wireless robotic sensor agents (RSAs) forcomplex environment monitoring. Figure 1 shows thearchitecture of an RSA system under development in ourlaboratory at the University of Ottawa [5]. Monitoring isdone by continuously collecting sensory data from sta-tionary and mobile RSAs deployed in the field.

Monitoring uncertain natural environments is a taskthat is significantly dependent on each application. Thecomplexity of this task is a function of the nature of theparameters and of the environment to be monitored. Thetype, number, and bandwidth of sensors, the difficultiesencountered while deploying the sensors in the field, andthe reliability of the sensory data and of the data commu-nication network are all requirements that have to be con-sidered when designing the intelligent sensor agents andthe distributed sensor network architecture.

The partial and heterogeneous sensor views of theenvironment are fused into a coherent virtualized realityenvironment (VRE) model of the explored environment[6]. Based on information about the real/physical worldobjects and phenomena as captured by a variety of sen-sors, VREs have more “real content” than the ordinary

Emil M. Petriu, Thom E. Whalen,

Rami Abielmona, and Alan Stewart

©1996 MASTER SERIES

46 IEEE Instrumentation & Measurement Magazine September 20041094-6969/04/$20.00©2004IEEE

September 2004 IEEE Instrumentation & Measurement Magazine 47

virtual reali ty environments,which are entirely based on com-puter simulations.

The VRE model of the exploredenvironment allows human teleop-erators to experience the feeling ofbeing immersed in that naturalenvironment, while keeping themout of harm’s way. To find efficientsolutions to the complex percep-tion tasks, the humans will have tocombine their intrinsic reactivebehavior with higher-order worldmodel representations of theimmersive VRE systems. A numberof human-computer interface(HCI) modalities allows the humanoperators and the VREs to be con-nected as transparently as possible.

VREs provide a cost-effectivesolution for the off-line trainingof personnel and virtual proto-typing environment for no-penal-ty, what-if experiments.

Autonomous RSAsMonitoring the environment, inpractical terms, is a game withlimited resources. There is a lim-ited number of RSAs, which havelimited operational parameters,communicating via a l imitedquality of service wireless com-munication network.

RSAs are not functionally andoperationally identical. A variety ofintelligent, autonomous RSAs (Figures 2–7) are currently beingdeveloped to cover all four perception phases of the environ-ment parameters: far away, near to, touching, and manipulation.

RSAs should be capable of selective environment per-ception focusing on parameters that are important for the

specific task and avoid wasting resources on processingirrelevant data. Different sensor planning strategies areused for the placement of the fixed and mobile sensor

Fig. 1. Distributed wireless network of mobile and stationary intelligent robotic sensor agents deployed in thenatural environment.

Fig. 2. Two-wheel robotic platform for an experimental mobile RSA equippedwith wireless camera and IR sensors.

Fig. 3. Six-wheel robotic platform for an experimental mobile RSA equippedwith video camera and sonar and infrared range sensors.

Wireless Communication

Mobile RoboticSensor Agent



StationarySensor Agent



Virtualized Reality

48 IEEE Instrumentation & Measurement Magazine September 2004

agents in such a way as to get optimum performance dur-ing specific sensing tasks and for the real-time selection ofsensing operations to minimize the observed systementropy [7], [8].

System ArchitectureA multisensor data-fusion mechanism is used for theintegration of heterogeneous sensory data into compositemodels of three-dimensional object shape, surface andmaterial properties, heat transfer, and radiation (electro-magnetic, thermal, radioactive, optical, etc.) characteris-tics. The multisensor fusion framework [9] deals in aconsistent way with a diversity of measurement data pro-duced by RSAs. Such a multisensor fusion system has to

◗ organize data collection and signal processing fromdifferent types of sensors

◗ produce local and global world models using themultisensor information about the environment

◗ integrate the information from the different sensorsinto a continuously updated model of the system.

In many applications, there is a considerable commu-nication delay and randomness between the informationcollected from the different field deployed sensor agents.This may affect the fidelity and consistency of the inte-

gral world model of the monitored environment. We areusing the “time clutch” and “position clutch” conceptsproposed in Conway et al. [10] to maintain the coherencebetween the information acquired from different sensoragents measuring parameters of the same object or envi-ronment region.

To avoid fatal errors and reduce the effect of the com-munication delay, we are using a distributed virtual envi-ronment allowing for the maintenance of a shared worldmodel of the physical environment that is explored [11].

Intelligent Autonomous Agent BehaviorHuman-to-human communication and cooperationrequire a common language and an underlying system ofshared knowledge and common values. To achieve a sim-

Fig. 4. Four-leg RSA platform powered by a solar panel.

Fig. 5. Frog-leaping RSA powered by a solar panel equipped with two lightdetector sensors and a touch probe.

Fig. 6. RSA platform on tracks with an onboard robotic arm for hazardousmaterial manipulation.

Fig. 7. Two-wheel hexagonal-shaped modular RSA robotic platform equippedwith onboard manipulator arm.


i lar degree of machine-to-machine RSA interaction andcooperation, an RSA social frame-work should be available for themanagement of heterogeneousfunctions and knowledge for alarge diversity of RSAs.

To provide the high degree ofautonomy required by their func-tions, RSAs have onboard sens-ing, computing, and wirelesscommunication capabilities aswell as the ability to autonomous-ly perform exploratory motions,as illustrated in Figure 8.

Learning allows autonomousrobots to acquire knowledge byinteracting with the environmentand, subsequently, adapt theirbehavior. Behavior-learning meth-ods are used to solve complexcontrol problems that autonomousrobots would encounter in anunfamiliar real-world environ-ment. Neural networks, fuzzylogic, reinforcement-learningmethods, and evolutionary-learn-ing methods can be used to implement basic behavioralfunctions [12], [13].

The intelligent, autonomous RSAs use a combination ofintrinsic reactive behaviors with higher-order world modelrepresentations of the environment [14]. A forward model-learning method is used to recover explicit structural repre-sentations of the dynamic environments from the sensordata and robot navigational experience.

We are investigating the use of Brooks’ reactive behaviorparadigm [12] as an alternative to the traditional function-ori-ented sensing-planning-acting control strategies. The behaviorparadigm is based on a task-wise decomposition of the con-trol functions in special-purpose, simple task-achieving mod-ules. We are verifying claims that neither strategic planningnor carefully calibrated sensors are necessary to producerobust intelligent reactive behavior in autonomous roboticagents [12], [13].

Social Behavior of RSAsAll RSAs are, by definition, instinctive information-seek-ing agents. When the RSA deployment costs are pro-hibitive, these sensor agents would benefit from havingsurvival behavior/instinct, cooperation skills, and adap-tation and learning abilities [15]. Like any sentient humanbeing, the intelligent RSAs (bots) [16] each have theirown artificial personality.

Cooperating agents should be able to work togetherwith other RSAs toward the overall goal, which is to max-imize the information acquired from the environment. For

instance, two cooperating RSAs could join their forces tocarry a bigger object (Figure 9). Or, an RSA could illumi-nate the subject for another RSA to take images.

To make possible the implementation of these characteris-tics, RSAs have modular, reconfigurable structures with com-ponents that are accessible and easy to assemble/disassemble.We are currently evaluating the performance of an experi-mental RSA platform with a hexagonal shape (Figure 7).

As shown in Figure 10, these RSAs are able to tem-porarily couple, forming new structures better adapted tosolving specific problems. For instance, two or moreRSAs can couple to make a bridge over a trench, or onecould be helped by another to get over an obstacle.

Fig. 9. Two RSAs collaborating to carryout a given task.

Fig. 8. Onboard control system for an autonomous wireless robotic sensing agent.

50 IEEE Instrumentation & Measurement Magazine September 2004

An evolutionary mechanismwill allow an RSA to pass thelearning experience and behav-ioral genes i t acquired whileoperating in the field over toother descendents.

To allow for bot species sur-vival [17], RSAs should be able tocannibalize/recycle other agentsthat are operationally dead, whichotherwise will be abandoned in thefield. Providing RSAs with such abehavior would allow for theupgrading of the operational capa-bility of surviving agents.

An intriguing and controversial question is the value ofextending the recycle_the_dead (i.e., the useful) cannibal-ism of RSAs to a more aggressive big_fish_eats_smaller_fish survival of the fittest behavior. Obviously, such anaggressive survival behavior should not allow for suicidalactions. An agent should not prey on equally strong ormore powerful RSAs. Although such behavior may lead tomore individually efficient RSAs through an evolutionaryprocess, it may affect the global mission by decimating thepopulation of agents deployed in the environment. It isquite possible that fewer, more operationally efficientRSAs, from an information-gathering standpoint, will do aworse global job than a greater number of agents that maybe less efficient but nevertheless still operational.

To facilitate recycling, each RSA has a status-advertis-ing mechanism telling other agents about its job-relatedfunctional qualification and health level.

Networking RSAsEach RSA is supported at the hardware level by a real-time resource management system using a real-timeoperating system. However, another higher-levelresource management framework is needed to addressthe communication needs of the agents. This frameworkshould work at a level that is higher than the classicalnetwork protocols and even distributed computingframeworks such as CORBA, which mainly provide dis-tribution transparency.

Heterogeneous RSAs cannot realistically be expectedto talk exactly the same language. However, they willshare domain-specif ic knowledge, which may beexpressed by each of them in a different dialect .Accordingly, the communication management frame-work should define a domain-specific semantic for com-mon knowledge and functions. This framework isexpected to act as a universal translator between speak-ers of different dialects.

To provide a flexible, extensible, open framework allow-ing for interoperability, methods should be developed toallow different agents to exchange the grammars describingtheir own dialects and to learn to understand each other.

This way, the agents would beable to advertise their own func-tions, search and discoverproviders of required services, andexpress their needs in a collabora-tive environment.

Extensible Markup Language(XML) [18] could provide high-level protocols for exchanginginformation between differentinformation appliances. XMLprovides syntax for building aformal document type definition(DTD) model, which describesrelationships between elements

and attributes of each class of documents. As a DTDgives a standard format for information related to a spe-cific domain, it could be used to simplify the exchange ofinformation between different sources that refer to thesame domain regardless of the internal format of eachsource.

AcknowledgmentsThis work was funded in part by the Natural Sciences andEngineering Research Council of Canada, theCommunications Research Centre Canada (CRC), and theCommunications and Information Technology Ontario(CITO). The authors gratefully acknowledge the assistanceof the Larus Technologies Corporation of Ottawa.

This article is based on materials previously publishedby the authors in [5] and [19].

Fig. 10. Two RSAs are temporarily joining to better perform their mission.The lead RSA has a video camera mounted on the hand of its manipulator arm.

The intelligent,

autonomous RSAs use

a combination of

intrinsic reactive

behaviors with

higher-order world model

representations of

the environment.


References[1] G. Giralt, R. Chatila, and R. Alami, “The remote-operated

autonomous-robot concept: From deep sea to outer space

applications,” in Proc. Int. Planetary Rovers Symp. , Laberge,

France, 1992.

[2] D.M. Wilson, S. Hoyt, J. Janata, K. Booksh, and L. Obando,

“Chemical sensors for portable, handheld field instruments,”

IEEE Sensors J., vol. 1, no. 4, pp. 256–274, 2001.

[3] A.M. Zoubir, I.J. Chant, C.L. Brown, B. Barkat, and C.

Abeynayake, “Signal processing techniques for landmine

detection using impulse ground penetrating radar,” IEEE

Sensors J., vol. 2, no. 1, pp. 41–51, 2002.

[4] R. Merritt, “Sensor nets top R&D list for Homeland Security

agency,” EE Times, 2003 [Online]. Available:

http://www.eetonline.com/article/showArticle.jhtml?articl

eId=18310193

[5] E.M. Petriu, G.G. Patry, T.E. Whalen, A.H. Al-Dhaher, and V.Z.

Groza, “Intelligent robotic sensor agents for environment mon-

itoring,” in Proc. VIMS’02, IEEE Int. Symp. Virtual Intelligent

Measurement Syst., Girdwood, AK, May 2002, pp. 14–19.

[6] T. Kanade, Virtualized Reality, Robotics Institute, Carnegie

Mellon University, Pittsburgh, PA [Online]. Available:

http://www.cs.cmu.edu/~virtualized-reality/

[7] G. Roth and M.D. Levine, “Minimal subset random sampling

for pose determination and refinement,” in Advances in

Machine Vision, C. Archibald and E. Petriu, Eds. Singapore:

World Scientific, 1992, pp. 1–21.

[8] K.A. Tarabanis, P.K. Allen, and R.Y. Tsai, “A survey of sen-

sor planning in computer vision,” IEEE Trans. Robot.

Automat.,

vol. 11, no. 1, pp. 86–104, 1995.

[9] R. Joshi and A.C. Sanderson, Multisensor Fusion: A Minimal

Representation Framework. Singapore: World Scientific, 1999.

[10] L. Conway, R.A. Voltz, and M.W. Walker, “Teleautonomous

systems: Projecting and coordinating intelligent action at a

distance,” IEEE Trans. Robot. Automat., vol. 6, no. 2, pp.

146–158, 1990.

[11] F.C.A. Groen, N. Vlassis, M. Boasson, H.J.W. Spoelder, and

H.E. Bal, “A framework for maintaining a shared world

model in dynamic environments between differentiated

embedded systems and allowing interaction with human

supervisors,” in Proc. 2nd PROGRESS Workshop Embedded

Syst., Veldhoven, The Netherlands, 2001, pp. 49–53.

[12] R. Brooks, “A robust layered control system for a mobile

robot,” IEEE J. Robot. Automat., vol. 2, no. 1, pp. 14–23, 1986.

[13] R.C. Arkin, Behavior-Based Robotics. Cambridge, MA: MIT

Press, 1998.

[14] J. Tani, “Model-based learning for mobile robot navigation

from the dynamical systems perspective,” IEEE Trans. Syst.,

Man, Cybern. B, vol. 26, no. 3, pp. 421–436, 1996.

[15] L.E. Navarro-Serment, R. Grabowski, C.J.J. Paredis, and P.K.

Khosla, “Heterogeneous teams of modular robots for map-

ping and exploration,” Inst. Complex Engineered Syst.,

Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. ICES Apr.

11, 1999.

[16] A. Leonard, Bots—The Origin of New Species. San Francisco,

CA: Hardwired, 1997.

[17] R. Dawkins, The Selfish Gene—New Edition. Oxford, U.K.:

Oxford Univ. Press, 1989.

[18] World Wide Web Consortium (W3C), Extensible Markup

Language (XML) [Online]. Available:

http://www.w3.org/TR/REC-xml

[19] E.M. Petriu, T.E. Whalen, and R. Abielmona, “Autonomous

robotic sensor agents,” in Proc. ROSE’03, 1st IEEE Int.

Workshop on Robotic Sensing 2003, Orebro, Sweden, 2003.

Emil M. Petriu ([email protected]) is a professor in theSchool of Information Technology and Engineering at theUniversity of Ottawa, Canada. His research interestsinclude intelligent sensors, robot sensing and perception,and interactive virtual environments. He is chair of TC-15,Virtual Systems, and cochair of TC-28, Instrumentation andMeasurement for Robotics and Automation, of the IEEEInstrumentation and Measurement Society. He is a fellowof the IEEE.

Thom E. Whalen completed his doctorate in experimentalpsychology at Dalhousie University in 1979. Since then, hehas been conducting research in human-computer interac-tions at the CRC, a research institute of the Government ofCanada. He is an adjunct professor in the PsychologyDepartment of Carleton University and in the Departmentof Management Science at St. Mary’s University.

Rami Abielmona is a doctorate candidate in computer andelectrical engineering at the School of InformationTechnology and Engineering, University of Ottawa. Hereceived the B.A.Sc. and the M.A.Sc. degrees in computerengineering, from the University of Ottawa. He received the“Student Research Excellence Scholarship” by CITO in 2003.He is cochair of the local IEEE Computational IntelligenceSociety chapter.

Alan Stewart is a computer and electronic instrumentationtechnologist in the School of Information Technology andEngineering (SITE), University of Ottawa. He has designedand built many mobile robotic platforms, some of whichare being used in the capstone design projects of SITE stu-dents. During the last 12 years, he has served as a volunteermember of the Hazardous Material Emergency ResponseTeam of the University of Ottawa. He has also been servingfor eight years as a volunteer firefighter for the city ofClarence-Rockland, Ontario, Canada.

emil m. petriu, thom e. whalen, rami abielmona, and alan ...petriu/mag04-...

Documents