active security system for an industrial robot based...

7
ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED ON ARTIFICIAL VISION AND FUZZY LOGIC PRINCIPLES B. Fevery, B. Wyns, L. Boullart Department of Electrical Energy, Systems and Automation, Ghent University, Ghent, Belgium [email protected], [email protected], [email protected] J. R. Llata Garc´ ıa, C. Torre Ferrero Control Engineering Group, Electronic Technology and Automatic Systems Department University of Cantabria, Santander, Spain [email protected], [email protected] Keywords: Active security, application, artificial vision, fuzzy logic, real-time, robot control. Abstract: An active security system assures that interacting robots don’t collide or that a robot operating independently doesn’t hit any obstacle that is encountered in the robots workspace. In this paper, an active security system for a FANUC industrial robot is introduced. The active security problem where one robot needs to avoid a moving obstacle in its workspace is considered. An obstacle detection and localization mechanism based on stereoscopic vision methods was successfully developed. To connect the vision system, an operator’s pc and the robot environment a real-time communication is set up over Ethernet using socket messaging. We used fuzzy logic for intelligent trajectory planning. A multitask oriented robot application in the KAREL programming language of FANUC Robotics was implemented and tested. 1 INTRODUCTION In industrial settings, robots often work on valuable products and with expensive tools. When more robots are working together on one assignment, a collision free interaction of the robot arms needs to be guar- anteed at all time. Systems that establish collision free robot interaction are identified as Active Security Systems (ASSYS). ASSYSs can also be situated in the domain of interaction between industrial manipu- lators and human operators, where the physical safety of the operator, rather than an economical concern, constitutes the necessity for the design of an appro- priate security system. The key principle of ASSYSs is the vigilance of the work area of cooperating robots and the streaming of information about events that are unexpected for each robot. This contrasts with a strategy where every robot is seperately programmed to execute its task and where interaction signals are sent between robots over rigid communication media. Safe robot motion is typically guaranteed by the use of a sensor system. A camera network based human-robot coexistence system was already pro- posed in (A.J. Baerveldt, 1992) and a safety system also using a network of cameras and with path re- planning in an on-line manner was presented in (D. Ebert et al., 2005). In this paper, we present the setup of a basic vision system to detect and positionally reconstruct obstacles in the robot’s workspace. The stereoscopic vision techniques applied in the design of the vision system will be presented in section 2. In literature, some researchers focus on the direct kinematics of robot manipulators. Using the differ- ences between actual and goal angular configuration of every axis, output actions for every axis’s motor are produced taking into account an obstacle’s configura- tion in the two or three dimensional space. In this paper, we will present a security system that controls the motional actions of an industrial FANUC Robot Arc Mate 100iB with six degrees of freedom and a circular range of 1800 millimeters. Instead of giving commands to every axis’s motor, positional and ro- tational configuration of the robot arm will be calcu- lated along the nodes of an alternative path around one detected obstacle. Intelligent path planning is done by using a fuzzy logic control system. A rule base com- posed of linguistic if-then implications is used to sim- ulate human reasoning in decision taking. The fuzzy system produces a set of alternative positions and ro- tational configurations that assure collision free mo- tion continuation towards a final location. Fuzzy logic 17

Upload: others

Post on 26-Mar-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOTBASED ON ARTIFICIAL VISION AND FUZZY LOGIC PRINCIPLES

B. Fevery, B. Wyns, L. BoullartDepartment of Electrical Energy, Systems and Automation, Ghent University, Ghent, Belgium

[email protected], [email protected], [email protected]

J. R. Llata Garcıa, C. Torre FerreroControl Engineering Group, Electronic Technology and Automatic Systems Department

University of Cantabria, Santander, [email protected], [email protected]

Keywords: Active security, application, artificial vision, fuzzy logic, real-time, robot control.

Abstract: An active security system assures that interacting robots don’t collide or that a robot operating independentlydoesn’t hit any obstacle that is encountered in the robots workspace. In this paper, an active security systemfor a FANUC industrial robot is introduced. The active security problem where one robot needs to avoida moving obstacle in its workspace is considered. An obstacle detection and localization mechanism basedon stereoscopic vision methods was successfully developed. To connect the vision system, an operator’s pcand the robot environment a real-time communication is set up over Ethernet using socket messaging. Weused fuzzy logic for intelligent trajectory planning. A multitask oriented robot application in the KARELprogramming language of FANUC Robotics was implemented and tested.

1 INTRODUCTION

In industrial settings, robots often work on valuableproducts and with expensive tools. When more robotsare working together on one assignment, a collisionfree interaction of the robot arms needs to be guar-anteed at all time. Systems that establish collisionfree robot interaction are identified as Active SecuritySystems (ASSYS). ASSYSs can also be situated inthe domain of interaction between industrial manipu-lators and human operators, where the physical safetyof the operator, rather than an economical concern,constitutes the necessity for the design of an appro-priate security system.

The key principle of ASSYSs is the vigilance ofthe work area of cooperating robots and the streamingof information about events that are unexpected foreach robot. This contrasts with a strategy where everyrobot is seperately programmed to execute its task andwhere interaction signals are sent between robots overrigid communication media.

Safe robot motion is typically guaranteed by theuse of a sensor system. A camera network basedhuman-robot coexistence system was already pro-posed in (A.J. Baerveldt, 1992) and a safety systemalso using a network of cameras and with path re-

planning in an on-line manner was presented in (D.Ebert et al., 2005). In this paper, we present the setupof a basic vision system to detect and positionallyreconstruct obstacles in the robot’s workspace. Thestereoscopic vision techniques applied in the designof the vision system will be presented in section 2.

In literature, some researchers focus on the directkinematics of robot manipulators. Using the differ-ences between actual and goal angular configurationof every axis, output actions for every axis’s motor areproduced taking into account an obstacle’s configura-tion in the two or three dimensional space. In thispaper, we will present a security system that controlsthe motional actions of an industrial FANUC RobotArc Mate 100iB with six degrees of freedom and acircular range of 1800 millimeters. Instead of givingcommands to every axis’s motor, positional and ro-tational configuration of the robot arm will be calcu-lated along the nodes of an alternative path around onedetected obstacle. Intelligent path planning is done byusing a fuzzy logic control system. A rule base com-posed of linguistic if-then implications is used to sim-ulate human reasoning in decision taking. The fuzzysystem produces a set of alternative positions and ro-tational configurations that assure collision free mo-tion continuation towards a final location. Fuzzy logic

17

Page 2: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

is popular due to its simplicity and hands-on, intuitivedesign of the control strategy and was successfully ap-plied by preceding authors, e. g. in (Bischoff, 1999),to build active safety systems for robots. In this paper,a three dimensional obstacle avoidance strategy willbe introduced that is founded on the idea of repellingand attracting forces (P. Zavlangas et al., 2000). Thedesign of a fuzzy logic controller will be highlightedin section 3.

Although specialised solutions exist for each com-ponent of the proposed ASSYS, the goal in this paperwas to build such a system using only basic compo-nents communicating over an Ethernet network. Nomultiple robot interaction was assumed and due to theearly stage of the investigation, the vision system wasonly designed to make the robot avoid collision witha single, however dynamically moving, obstacle. At-tention was also given to the time performance of thevision system.

To make the industrial FANUC robot move alongan alternative path in an on-line manner, a robot ap-plication needed to be programmed in the proper pro-gramming language KAREL of FANUC Robotics. Amultitask oriented design in the KAREL languageassures that alternative positions can be read in bythe robot’s system and subsequently moved to by therobot arm. The architecture of the robot application,as well as details on the real-time communication sys-tem established over Ethernet, will be commented insection 4. In section 5 results and drawbacks of thedesigned ASSYS are commented.

2 ARTIFICIAL VISION

2.1 3D Object Reconstruction

Stereoscopic vision applications intent to reconstructthe 3D location of characteristic object points. From(Torre Ferrero, 2002) an analytical method was takenthat allows for a unique 3D reconstruction of an objectpoint P, knowing the pixel sets (u1, v1) and (u2, v2) ofP’s projection into two different image planes I1 andI2. The camera’s projection matrices, that are com-posed of the camera’s extrinsic and intrinsic parame-ters (Gonzalez Jimenez, 1999), are also needed for re-construction. These parameters were obtained for ev-ery camera by applying a camera calibration methodbased on (J. Heikkila et al., 1997). For more details oncamera projection principles and reconstruction meth-ods, please consult (Gonzalez Jimenez, 1999) and(Torre Ferrero, 2002).

2.2 Camera Setup of the Vision System

A triplet of network cameras was installed to watchthe robot’s workspace. Camera images can be ob-tained by sending an image request signal to their IPaddress over a Local Area Network (LAN). For ev-ery camera, a video stream of images using ActiveXcomponents is activated. Images are taken out of thevideo stream and saved as image matrices of dimen-sion 480x640x3 in the Red Green Blue (RGB) imagespace. A pc is used to perform image processing op-erations. The cameras were collocated in a triangularpattern and mounted on the ceiling above the robot’sworkspace.

2.3 Object Detection andReconstruction Methods

In industrial settings, image processing times need tobe small. Preliminary knowledge about the object’scolor and shape is therefore often used to detect obsta-cles in the robot’s workspace as quickly as possible.For the experimental setup of our vision system, weworked with a foam obstacle of parallelepiped struc-ture. The motion of the obstacle is achieved by simplydragging the foam into the robot’s workspace with arope. Because it is not within the scope of this paper,no attention was given to the detection of the robot’sarm, nor to the detection of humans or objects of otherform than a parallelepiped. In the next sections, wewill introduce the vision techniques that were usedfor the detection of a moving obstacle and for the re-construction of its 3D position. The reconstructionmethod is based on the technique of epipolar lines,which form a useful geometric restriction in visionapplications.

2.3.1 Obstacle Observation

The obstacle of parallelepiped form is detected in animage by converting this image to binary form andsubsequently check for the presence of contours ofsquared form, using a simple criterion that relates asquare’s perimeter to its area. The presence of the ob-stacle is checked by drawing the image of one cameraout of the activated video stream every 50 millisec-onds and by applying the square detection criterion.

When a moving obstacle is detected for the firsttime in the workspace, the ASSYS halts all robot mo-tion. Only if the obstacle stops moving within a cer-tain number of time frames after it had first been de-tected, the robot will resume its motion, now movingaround the obstacle. By taking subsequent images outof the video stream of the same camera and resting

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

18

Page 3: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

. .

.

. .

.

.

. C 3

I l

p1

C 1 C 2

p2

P21

P(x, y, z) P2m

E 2

E 3 p3

E 31

E 3m

E 3j

I 3

I 2

Figure 1: Pixel correspondence algorithm.

two subsequent image matrices, the system checks ifthe object has stopped moving. As soon as this condi-tion holds, images are drawn out of the video streamof the two other cameras and the 3D reconstructionof the obstacle is initiated. We typically reconstructthe 3D location of the obstacle’s four upper corners.These characteristic object points are determined byapplying a Canny edge detector (Gonzalez Jimenez,1999) and a corner detection operator on the three im-ages.

Once characteristic points -true and also false ob-ject corners due to image noise or nearby objects- aredetected, an algorithm is applied to determine corre-sponding corners. This problem boils down to findingfor every upper corner of the obstacle in a first image,the location of the same corner in the second and thirdimage.

2.3.2 Calculation of Pixel Correspondences

The applied algorithm is based on the geometric re-striction of the epipolar line: an image point P thatis projected onto a pixel in a first image can only beprojected onto one line of pixels in a second image.We aim to find three pairs of pixel coordinates of onepoint in space that is projected into three images I1,I2 and I3. This problem is identified as the search forpixel correspondences. The algorithm can briefly beexplained as follows, according to the notations of fig-ure 1. The image point p2 in the second image I2 thatcorresponds to the point p1 in the first image I1 hasto be situated on the epipolar line E2 associated to p2.Characteristic pixels of I2 that are located sufficientlyclose to this epipolar line are selected as correspon-dence candidates P2i in I2. When the epipolar linesE3 j associated to all correspondence candidates P2iare constructed in a third image I3, this results in anumber of intersections p3k in I3 between the epipolar

lines associated to the set P2i and the epipolar line E3associated to p1. The intersection that coincides or islocated sufficiently close to one of the characteristicpoints in I3, results in the unique corresponding pixeltriplet {p1, p2, p3}. Epipolar line pixel equations from(Torre Ferrero, 2002) were used to construct epipolarlines.

As soon as the corresponding corner pixels arefound in the three camera images, the pixel correspon-dences are used to calculate the obstacle’s 3D locationin space, as described in section 2.1. False pixel cor-respondences, that originated from detected cornersnot belonging to the object, can be discarded becausethe resulting 3D positions don’t fall within the rangein which the obstacle is expected to be encountered.The obstacle’s 3D location in space is used as an inputof the fuzzy logic controller that calculates an alterna-tive trajectory.

3 FUZZY LOGIC CONTROL

3.1 Introduction

A Fuzzy Logic Controller (FLC) is a useful tool totransform linguistic control strategies based on exper-tise into an automatic control strategy (O. Cordon etal., 2001). The basic idea is to assign linguistic la-bels to physical properties. The process that convertsa numerical value into a linguistic description is thefuzzification process. Using a rule base that simulateshuman reasoning in decision taking, a number of lin-guistic control actions is computed and subsequentlydefuzzified or converted to numerical control actions.For more information and a detailed description onFLCs, please consult (O. Cordon et al., 2001).

A pneumatically controlled tool was mounted on

ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED ON ARTIFICIAL VISION AND FUZZYLOGIC PRINCIPLES

19

Page 4: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

the robot arm. The term End Effector is used to indi-cate this tool, that is depicted in figure 2 for differentconfigurations. In robot terminology, the central pointof the End Effector is called the Tool Center Point(TCP). The dots between the ends of the End Effectorin figure 2 represent this TCP.

3.2 Fuzzy Avoidance Strategy

A fuzzy rule containing two types of actuating forceswas designed. An attracting force proportional to the1D distance differences between actual TCP coordi-nates and final location coordinates causes the FLC tooutput distance increments towards the final location.A repelling force describing the distance to the obsta-cle’s sides deactivates the attracting force and invokesspecific avoidance actions that have to be undertakenby the robot’s End Effector to avoid collision with theobstacle.

Further on, it will be explained how safety zonesare constructed around the obstacle, based on the dis-tance differences between the TCP and the obstacle’ssides. When the robot’s TCP enters one of thesesafety zones around the obstacle, two types of avoid-ance actions are undertaken. Rotational actions guar-antee the End Effector’s orthogonal position to the ob-stacle’s sides and translational actions assure an accu-rate avoidance in position. This idea is depicted infigure 2.

5 4

3

2 .

.

.

.

.

1

+90° y

-90° y

Very Close

Figure 2: Fuzzy avoidance strategy.

3.3 Inputs to the Fuzzy Logic Controller

The inputs of the FLC consist of two types. A firsttype describes 1D distance differences between actualTCP coordinates and final location coordinates, whilethe second input indicates if the TCP is near to one ofthe obstacle’s sides. The first input is related to the at-tracting influence and the second one to the repellinginfluence. It will now be explained how both types ofinputs to the FLC are composed.

The distance between final location and TCP canbe described in linguistic terms as e.g. Close or Far.

CloseNeg

FarNeg

VCloseNeg

VClosePos

ClosePos

FarPos

∆r

Figure 3: MFs for fuzzy sets of attracting influence.

For a given numeric value of distance to the final lo-cation, each of the linguistic labels will be true witha certain value in the range [0, 1]. This value willbe determined by the Membership Function (MF) ofthe specified linguistic distance label. Figure 3 illus-trates the MFs of the labels that describe the distancedifference ∆r in x, y and z direction between the coor-dinates of the Tool Center Point and the final locationcoordinates. MFs of triangular and open trapezoidalform were chosen because they are easy to implementin programming applications and require small evalu-ation times. The central triangular represents the MFfor Contact with the obstacle. Table 1 indicates the1D distance descriptions in coordinates r = x, y and ztowards the final desired configuration.

Table 1: Labels for attracting influence.

Linguistic label Short notationGoal Far Negative GFar Neg r

Goal Close Negative GCl Neg rGoal Very Close Negative GVCl Neg r

Goal Reached Goal rGoal Very Close Positive GVCl Pos r

Goal Close Positive GCl Pos rGoal Far Positive GFar Pos r

The second FLC input is related to the repellingforce. To understand how these FLC inputs originate,we make the following consideration. If the robot’sTCP is Very Close to the positive x side of the obsta-cle, this means it is close to the positive x bound ofthe obstacle AND: within the y and z range OR withinthe y range and very close to the positive z bound OR... . Figure 4 illustrates this idea for the constructionof the label Very Close Positive x.

The distance differences ∆s (s = x, y and z-dimension) represent the distances of the TCP to theclosest obstacle bound in the considered coordinate.For the example of figure 4, the considered labelneeds to be evaluated using AND and OR logical oper-ators. Table 2 represents distance difference descrip-tions towards the sides of the obstacle.

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

20

Page 5: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

Z

Y

X

VCl Pos ∆yVCl Neg ∆z

VCl Pos ∆yContact ∆z

VCl Pos ∆yVCl Pos ∆z

Contact ∆yVCl Neg ∆z

Contact ∆yContact ∆z

Contact ∆yVCl Pos ∆z

VCl Neg ∆yVCl Neg ∆z

VCl Neg ∆yContact ∆z

VCl Neg ∆yVCl Pos ∆z

VCl Pos ∆x

Figure 4: Construction of label Very Close Positive x.

Table 2: Labels for repelling influence.

Linguistic label Short notationFar Negative Far Neg ∆s

Not Close Negative NCl Neg ∆sClose Negative Cl Neg ∆s

Very Close Negative VCl Neg ∆sContact Contact ∆s

Very Close Positive VCl Pos ∆sClose Positive Cl Pos ∆s

Not Close Positive NCl Pos ∆sFar Positive Far Pos ∆s

Note that, according to the idea of figure 4, thelabels of table 2 have three-variable MFs, becausethey are all composed of one dimensional MFs forcoordinate differences ∆r (r = x, y and z) towards theobstacle’s boundary coordinates. These 1D MFs aresimilar to the ones for the attracting forces depictedin figure 3. Next step is to construct safety zonesaround the obstacle, as shown in figure 2 for the labelVery Close. Analogously, zones Close and Not Close,and an outer region Far, complementary to the innerzones, are constructed.

To fuzzify the entrances of the Fuzzy InferenceSystem, we used a singleton fuzzificator (J. R. LlataGarcıa et al., 2003).

3.4 Design of a Rule Base

The basic principle of the rule base is the deactivationof the attracting influence -determined by the distanceto the final location- when the repelling influence istriggered. Taking this into account, logical rules forclosing up to the final location can be constructed.

For the rules related to the repelling influence, wecan state that the designer of the rule base is free tochoose the direction and sense of the avoidance ac-

tions. We decided to undertake an avoidance actionin positive z direction when the TCP closes up to the(Very) Close x or y, Negative or Positive side of theobstacle.

The avoidance action for the (Very) Close z, Pos-itive or Negative side, is decided upon by a criterionthat checks the distance difference in x and y coordi-nate of the TCP’s current position and the final loca-tion coordinates. If the distance difference is biggerin the x direction, then an avoidance action in x is un-dertaken, otherwise in y direction.

As soon as the TCP enters the safety zone NotClose, a rotation of -90◦ or +90◦ around the appro-priate axis of a fixed coordinate system needs to beundertaken, to prevent the End Effector from hittingthe obstacle (see figure 2).

To resolve the fuzzy intersection operator used forthe composition of rule premises and for the impli-cation on rule consequents, we used a T-norm of theproduct type. In the aggregation of rule consequentsan S-norm for the fuzzy union operator was chosen.We implemented a maximum operator as this S-normto save in processing time.

Given an initial and final position and an obsta-cle’s location supplied by the vision system, the FLCoutputs a set of positional and rotational configura-tions that guarantee collision free motion towards thefinal location.

3.5 Outputs of the FLC

Fuzzy outputs of the Sugeno singleton type (J. R.Llata Garcıa et al., 2003) were used for defuzzifica-tion. Depending on the output of a rule, a specificvalue can be assigned to the considered system out-put. The designer of the FLC is free to determine thesize of the output actions.

Upon detection of an obstacle and halting of robotmotion, the TCP’s current position is sent by therobot’s operational system over a socket connectionto the artificial intelligence system as a start pointfor the calculation of an alternative path. Alternativepositions and rotational configurations are then sentback over the socket in data packages that contain thedesired coordinates of the TCP and the desired rota-tional configuration of the End Effector with respectto the fixed coordinate system.

ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED ON ARTIFICIAL VISION AND FUZZYLOGIC PRINCIPLES

21

Page 6: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

4 EXPERIMENTAL SETUP

4.1 Real-time Communication

Robotic control applications often have cycle timesof typically hundreds of microseconds. When oper-ational data needs to be exchanged between a robotand an operator’s pc, the fastness and the guaranteeof data transmission is of utmost importance. Formany years, Ethernet was banned as a communicationmedium from the industrial work floor, for data pack-ages that are sent over the Ethernet by devices con-nected to a same Local Area Network can collide andbe lost, due to the network’s media access control pro-tocol CSMA/CD (Van Moergestel, 2007). Nowadays,Fast Ethernet switches can be used to isolate networkdevices into their own collision domain, hereby to-tally eliminating the chance for collision and loss ofdata packages. Ethernet switches together with thedevelopment of Fast Ethernet (100Mbps) and Giga-bit Ethernet (1Gbps) have made Ethernet popular asa real-time communication medium in industrial set-tings (Decotignie, 2005).

To establish the Ethernet communications in theASSYS, we used so called Ethernet sockets. Socketsare software entities that are assigned to a combina-tion of communication port and IP address, so thatthey can be used by a client and a server device tocommunicate over a LAN. In our setup, this LAN wascreated by a Fast Ethernet switch. The exchange of alldata packages between the industrial FANUC RobotArc Mate 100iB and a pc running the vision and fuzzylogic applications is performed by socket messaging.

4.2 Multitask Robot Application

A multitask oriented active security application wasdeveloped and tested in KAREL, the programminglanguage of FANUC Robotics for advanced user ap-plications. A motion task executes a normal opera-tion trajectory until a condition handler is triggeredby the detection signal that was received through anEthernet socket by a concurrently running communi-cation task. When this condition handler is triggered,robot motion is halted within a time that is accept-ably small and the TCP’s current position is sent bythe communication task to the operator’s pc, wherethe FLC calculates the first alternative positions andsends them back over the opened socket connection tothe communication task. An interrupt routine for mo-tion along the alternative path is then invoked in themotion task. The robot axes’s motors start accelerat-ing immediately. Meanwhile, the communication taskcompletes the reading in of subsequent alternative po-

sitions and rotational configurations. Motion contin-ues until the original final location is reached. TheKAREL application is written in a non-cyclic way.Upon reaching the final position, program executionis aborted. Coordination between the motion task andthe communication task was realised by the use ofsemaphores. The artificial vision system, the FLC andthe robot control application in KAREL were tested inan integrated way.

5 RESULTS AND DISCUSSION

The obstacle is dragged into the robot’s workspacewhen the robot arm is close to the leftmost or right-most point of its regular trajectory. Absence of therobot’s arm in the central zone of the workspace isnecessary for correct obstacle detection because therobot arm would deform the binary image of the ob-stacle’s squared contour. Further development of thevision system is therefore needed to distinguish therobot arm from the obstacle in order to be able to sig-nal obstacle presence in all operational situations. Inan advanced stadium, detection criteria for human op-erators can also be elaborated.

However, if the robot arm is occulting one of theobstacle’s upper corners in one of the three images,performing an accurate reconstruction of the obsta-cle’s 3D location is still possible, since a free view onthree of the four upper corners in all images is suffi-cient for the reconstruction. The parallelepiped in fig-ure 5 depicts the result of the vision system and fuzzylogic path planning.

Figure 5: Alternative trajectory around reconstructed obsta-cle.

With distance increments of 50 millimeter in theFLC we typically obtained a number of 40 alterna-tive positions. The designer of the FLC is free tochoose the size of the translational increments largeror smaller and in a rather intuitive way. However, a

ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics

22

Page 7: ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED …vigir.missouri.edu/~gdesouza/Research/Conference_CDs/IFAC_ICINCO_2008... · ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT

thorough study can be performed making a trade-offbetween small increments and thus larger calculationtimes and larger robot processing times or large dis-tance increments and thus smaller calculation timesand robot processing times. This last option impli-cates however that the safety zones around the obsta-cle need to be bigger and that longer trajectories haveto be completed by the robot tool before it reachesthe final location. For industrial settings, where smallrobot motion execution times are of utmost impor-tance, this trade-off study is an interesting topic forfuture research. More specifically, a time efficient anddistance optimal path construction algorithm can bedesigned.

The FLC only takes the TCP’s position as an in-put. Collision of the robot’s arm is prevented by ro-tating the End Effector +90◦ or -90◦ when it entersthe first safety zone Not Close. For the majority ofpractically executable robot trajectories, this preven-tive action has proven to be sufficient. In future re-search however, the distance to the obstacle of extrapoints on the robot’s arm will have to be monitored toguarantee safer motion.

In this design, a parameter to take into accountis the processing time needed by the robot’s systemto handle new motion instructions. The robot systemis able to continue program execution after launchinga motion instruction. Moreover, a continuous transi-tion between two separate motion instructions is pos-sible using the appropriate clauses in the motion com-mands. Nevertheless, we chose to keep the numberof motion commands as limited as possible and de-cided to only send every fourth alternative position asan effective motion instruction to the robot. Given thefact that alternative positions are situated close to eachother (see figure 5), this strategy still results in accu-rate obstacle avoidance and in a smooth, continuousrobot motion.

The time needed to draw images out of the videostream and save them as pixel matrices for furtherimage processing could be restricted to 15 millisec-onds. The computational time to identify pixel corre-spondences and make a 3D reconstruction is also verysmall. Regarding processing time, the bottleneck ofthe vision system, and thus of the entire ASSYS, hasproven to be the identification of characteristic objectpixels, in our case corner pixels of the parallelepiped.Improvements have to be made. Remark that during3D reconstruction of the obstacle the robot is motion-less, thus no unsafe situation is created due to the highprocessing times of the vision system.

6 CONCLUSIONS

An active security system for an industrial FANUCrobot was designed. With special attention for real-time performance of the constituting subsystems, sat-isfying experimental results were obtained. The setupof the Ethernet communication through sockets, thefuzzy obstacle avoidance mechanism and the visionsystem open wide perspectives for future investiga-tion on active security. More attention can be givento distinguishing the robot arm from foreign objects,to optimizing image processing times, to searchingcost optimized alternative paths and to automating therobot application in a cyclic way.

REFERENCES

A. J. Baerveldt (1992). A safety system for close interactionbetween man and robot. Proceedings of IFAC Confer-ence on Safety Security Reliability SAFECOMP 1992.

Bischoff, A. (1999). Echtzeit Kollisionsvermeidung fureinen Industrieroboter durch 3d-Sensoruberwachung.

D. Ebert et al. (2005). Safe human-robot-coexistence:Emergency-stop using a high-speed vision-chip. 2005IEEE/RSJ International Conference on IntelligentRobots and Systems, pages 1821–1826.

Decotignie, J.-D. (2005). Ethernet-based real-time and in-dustrial communications. IEEE, 93:1102–1117.

Gonzalez Jimenez, J. (1999). Vision por computador.Paraninfo.

J. Heikkila et al. (1997). A four-step camera calibra-tion procedure with implicit image correction. 1997IEEE Computer Society Conference on Computer Vi-sion and Pattern Recognition.

J. R. Llata Garcıa et al. (2003). Introduccion a las Tecnicasde Inteligencia Artificial. Grupo de Ingenierıa de Con-trol, Departamento de Tecnologıa Electronica e In-genierıa de Sistemas y Automatica, Universidad deCantabria.

O. Cordon et al. (2001). Genetic fuzzy systems: Evolu-tionary tuning and learning of fuzzy knowledge bases.World Scientific.

P. Zavlangas et al. (2000). Industrial robot navigation andobstacle avoidance employing fuzzy logic. Journal ofIntelligent and Robotic Systems, 27:85–97.

Torre Ferrero, C. (2002). Reconstruccion de piezas en 3dmediante tecnicas basadas en vision estereoscopica.

Van Moergestel, L. J. M. (2007). Computersystemen en Em-bedded Systemen, 2nd reviewed print. Academic Ser-vice, Den Haag.

ACTIVE SECURITY SYSTEM FOR AN INDUSTRIAL ROBOT BASED ON ARTIFICIAL VISION AND FUZZYLOGIC PRINCIPLES

23