real-time obstacle avoidance for telerobotic systems based on

International Journal of Advanced Robotic Systems

Real-Time Obstacle Avoidance for Telerobotic Systems Based on Equipotential Surface Regular Paper

Xin Li*, Aiguo Song, Huijun Li, Wei Lu and Chen Mao School of Instrument Science and Technology, Southeast University, Nanjing, China * Corresponding author E-mail: [email protected] Received 29 May 2012; Accepted 12 Jun 2012 DOI: 10.5772/50652 © 2012 Li et al.; licensee InTech. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract Redundant manipulators offer a dual advantage of flexibility and dexterity and can be used in many civilian and military areas. However, operating such systems by teleoperation is challenging because of the redundancy and unstructured task environment, which result in the human operator suffering a huge burden when telemanipulator is facing the complicated obstacles. The existing methods usually use some off‐line algorithms to solve the problem of obstacle avoidance. It is difficult for them to meet the requirements of real‐time teleoperation in some unknown environment. This paper presents an on‐line method for a telerobotic system to take advantage of redundancy to avoid obstacle, which is based on real‐time sensor information. With this method, the human operator can focus attention on the end‐effector operation regardless of the obstacle avoidance of other parts. The effectiveness and advantage of the method are well demonstrated by experiments. Keywords Equipotential surface, obstacle avoidance, real‐ time, telerobot.

1. Introduction Telerobotic systems offer an attractive opportunity to combine human intelligence with robotic proficiency. In the last decade, the telerobotic systems can be used in critical and hazardous environments, including military, energy, space, and others. In order to improve the ability of interacting with the real world, the manipulators which possess redundant degree of freedom are introduced in the telerobotic system to deal with the various challenges such as obstacle avoidance, joint limits and better manipulability. Telerobotic control is different from the normal robotic control. The normal robotic control need path planning automatically, while the path of telerobot is planned by the command information provided by human operator. In general telerobotic systems, the human operator is saddled with two basic tasks: 1) moving the robot arm to its desired position, and 2) avoiding obstacles collide with the telemanipulator [1]. Practically, when disposing a grasping task by redundant telerobotic systems, human operator only focuses attention on the end‐effector to track the command trajectory, while movement of other links of

1Xin Li, Aiguo Song, Huijun Li, Wei Lu and Chen Mao: Real-Time Obstacle Avoidance for Telerobotic Systems Based on Equipotential Surface

www.intechopen.com

ARTICLE

www.intechopen.com Int J Adv Robotic Sy, 2012, Vol. 9, 71:2012

manipulator are usually ignored and in unsupervised situation. In other words, it is difficult for human operator to control all links of manipulator to avoid obstacles, especially in a complex environment. Hence, the links with the risk of collision should have the ability to automatically avoid collision. The previous researches for the collision avoidance of robotic system refer to a structured environment, so the solution is based on off‐line in most cases. However, due to the variable environment, the telerobotic system needs the on‐line ability to avoid the collision in order to ensure the security of manipulator. In order to reduce the burden of human operator, therefore, there are two practically difficult problems need to solved. One is that the telerobotic system has to detect the obstacle and judge the possibility of collision in real‐time; the other one is the telerobotic system need a suitable scheme to avoid obstacles efficiently. The redundant telemanipulator leads to infinite possible joint positions for the same pose of the end‐effector. Accord‐ ingly, this naturally leads to the research that a robot can achieve a primary task with its end‐effector and simultaneously satisfy certain requirement with the supplementary degrees of freedom. In this paper, an obstacle avoidance approach based on an equipotential surface is proposed for a 7‐DOF robot manipulator in telerobotic system, which takes advantage of redundant freedom of robot manipulator to achieve obstacle avoidance, and results of relative experiments are illustrated. The rest of this paper is organized as follows. The relevant research is discussed in Section II, which is followed by the control architecture of our telerobotic system in Section III. We present the sensor‐based obstacle avoidance is studied in Section IV. In Section V, experimental results are presented to demonstrate the efficacy of the proposed algorithms. Finally, conclusions are made in Section VI. 2. Related research Traditional researches related to robotic obstacle avoidance strategies can be divided into reactive control, global planning and mixed approaches [2]. The fact that real robotic systems rarely have an exact description of the environment, coupled with a desire for real‐time planning, led to the development of reactive control approaches. Reactive control approaches are mainly based on potential field methods that rely on local information [3] [4]. The drawback of reactive control is that the robot might get stuck in a local minimum. In teleoperation, however, because of the path planning by human operator, the above problem could be solved. Hence, the reactive control is fit for teleoperation. Global planning approaches on the other hand take global information into account. These methods focused on the robot operated in the invariable environment, such as development of the PRM planner [5] [6]. Due to the assumption of a static environment and poor ability of

real‐time, global planning approaches are not applicable to telerobotic system in unknown environment. After the reactive control and global planning, various mixed method were proposed. Elastic Bands [7] and Elastic Strips [8] are mixed method, which combine global information and local one. Initially, they made a planned path, and then the path is adapted to a changing environment which uses reactive control. However, calculating distances from the robot to a point cloud might be time consuming. Dynamic Roadmaps (DRM) introduced by Leven and Hutchinson [9] adopt the method of increasing on‐line efficiency through preprocessing from classical PRM approaches to plan path in changing environments. Due to adapting the classical PRM to get a preprocessed roadmap, DRM still relies on preprocessing to enable real time plan. In other word, DRM also need preprocessing, hence it is not suitable for teleoperation. Especially, some researches on obstacle avoidance for telerobot in Jet Propulsion Laboratory were more professional [10] [11] [12]. Due to the specific characteristic of telerobotic system, the global path planning is controlled by human operator, the idea of reactive control is more suitable for telerobot to avoid obstacles. Seraji introduced configuration control for redundant robot [13], and then he proposed the virtual spring‐plus‐damper method with proximity sensor to implement the security of robot based on configuration control. Nonetheless, because the cost of proximity sensor is too high, this method which measure the distance from robot to the obstacle is unsuitable for normal telerobotic system. Instead, our approach uses general visual sensor or range sensor. Additionally, our focus is on avoiding collision as in real time as possible. The contributions in our work are: (1) a method based on equipotential surface is proposed to calculate the distance from the robot to the obstacle. (2) a control scheme taking advantage of redundant degree of freedom is presented to achieve obstacle avoidance. 3. Control architecture In general, a telerobotic system can be represented by the block diagram in Figure 1 and consists of five subsystems: the human operator, the master, the communication media, the slave (telemanipulator), and the environment [14]. The human operator provides a position/velocity and force command, through the master, communication media, and slave, to interact with the environment; likewise, the information sensed at the environment is transmitted backward through these blocks, to the operator [15]. This is a bilateral process.

Figure 1. Concept map of the telerobotic system

2 Int J Adv Robotic Sy, 2012, Vol. 9, 71:2012 www.intechopen.com

In this paper, we construct a telerobotic system, which consists of three parts: teleoperation station, communication media and robot system, as shown in Fig. 2. The teleoperation station includes the human operator and the master, where the joystick is used as master. The communication media is the Internet. The telerobot system is composed of a mobile manipulator and a sensor system. The mobile manipulator is the executor that implements the mission, which contains a 4‐wheel mobile base and a dexterous 7‐DOF arm robot, which exist one redundant joint for the task 6‐DOF task‐ space control. In other words, there are infinite different arm postures which yield the same end‐effector configuration. In this paper, the kinematic control of manipulator is to employ the configuration control method proposed in [13]. The sensor system is used to detect the information of environment. In the control process, the human operator sends the six dimensional velocity commands to the end‐effector, through the Internet to the remote robot system. Then, the robot system responds to implement the task. The end‐effector of manipulator is supervised by human operator, which ensures the collision avoidance. At the same time, the sensor system detects the information of task space in real‐time and controls the links of manipulator to achieve collision avoidance autonomously. Next, we introduce the proposed method in detail.

Figure 2. Architecture of the telerobotic system 4. Sensor‐based obstacle avoidance In this section, the method of sensor‐based obstacle avoidance is discussed. The aim of this method is to make telemanipulator accomplish collision avoidance automatically through a sensor. 4.1 Modeling of the manipulator The configuration control, which is also called the ‘augmented Jacobian method’, is used in modelling the manipulator in this paper. For the 7‐DOF redundant manipulator in Figure 3, let 6X , denotes the position and orientation of the manipulator end‐effector. Let 7 denotes the vector the 7 joint angles. Hence, the forward kinematics and speed kinematics of the manipulator can be written as:

( )X f (1)

( )eeX J (2)

where 7 6:f is the forward kinematic function, and 6 7

eeJ is Jacobian matrix of end‐effector. Then X is

augmented by the user‐defined kinematic function to yield the 7 1 configuration vector [ , ]T TY X . In this paper, the user‐define function is distributed by ‘arm angle’, which describes the angle from reference plane and the shoulder‐elbow‐wrist plane SEW , as shown in Figure 4. The reference plane is consisted of y‐axis of the basic frame and line SW [16]. The kinematic function and the differential function of the arm angle as:

( )g (3)

J (4)

where 7 1:g is kinematic function of arm angle. J is the Jacobian matrix associated with the arm angle. Thus the kinematic model of whole manipulator is:

( )( )

( )X f

Y hg

(5)

where 7 7:h , and the differential kinematic model:

( )( )

( )

ee

augJ

Y JJ

(6)

where 7 1J is Jacobian matrix of arm angle, 7 7augJ is the augmented Jacobian. In other words, the basic task of the manipulator control has been augmented by the additional task. For the teleoperation, the collision of end‐effector can be avoided by human operator because of supervision in the control process. When the human operator forcuses attention on the end‐effector, the forearm part, EW , is in the unsupervised case, so it is dangerous and the collision might happen. Hence, an idea that takes advantage of additional task of redundant manipulator to avoid potential danger is considered. Intuitively, the collision avoidance is achieved by rotating the elbow E with respect to the shoulder‐wrist axis SW .

Figure 3. Model of 7‐DOF manipulator


www.intechopen.com

Figure 4. Arm angle 4.2 Sensor system The information we need is the 3D shape data of the obstacle in the environment, so it is feasible for any kind of sensor or combination of several sensors as long as they possess the ability to obtain 3D information of obstacle, such as stereo vision, 3D laser measurement sensor, and others. Before detection, the coordinate frame of sensor system needs to be calibrated with the base coordinate frame of robot system. Calibration gives all necessary parameters which relate the sensor system frame ( , , )cr c d to the coordinates of a point ( , , )wx y z in detection field. The subscript c and w indicate the sensor coordinate frame and the world coordinate frame respectively. So, there is:

( , , ) ( , , )w ww c c cx y z R r c d O (7)

where w

cR is the transformation matrix from sensor frame to base frame of robot system. w

cO is the coordinate of the origin of the sensor frame in the base frame. After calibrating, each point in sensor coordination is transformed into the corresponding position in world coordinate frame. Then, the position of obstacle needs to be located. The method proposed in this paper take advantage of projection on a certain surface. In detail, this method is to project the dangerous link of manipulator to the moving direction of the end‐effector of manipulator, and the positional relationship of projections is used as the criteria. In this paper, an equipotential surface which is perpendicular to the moving direction of the end‐effector, namely is determined by the x‐value of the end‐effector, is used to be projective plane in the front of the manipulator, as shown in Figure 5. The effect of the equipotential surface is to evaluate the positional relationship between the link of manipulator and the obstacle. According to the transformation obtained by calibration, the distance between the equipotential surface and the r c plane of the sensor frame can be calculated. Suppose the distance between the equipotential surface and the r c plane of the sensor frame is D , and the distance from a 3D data point in the environment to the r c plane of the sensor frame is id . If

id D , he point are defined as obstacle point, otherwise the point is non‐obstacle one, as shown in Figure 6. Then, the method requires to get the projection of the link of

manipulator on the equipotential surface, such as i iEW in Figure 6. However, it is different from the above method. The method is:

weq eq wL R L (8)

where eqL is the coordinate of link in the equipotential surface, wL is the coordinate of link in world frame, w

eqR is transformation matrix from the world frame to the equipotential surface. Thus, the projection of link on the equipotential surface can be obtained. A Laser Measurement System with a 2‐D pan‐tilt is used as 3D laser detection system in this paper. In Figure 7, we can find that the obstacle is detected using projection method by the point cloud data obviously. Figure 7(a) shows the original image; Figure 7(b) shows the result of detection. In Figure 7(b), the red closed curve indicates the edge of the obstacle.

ZY

X

r

cd

Figure 5. Equipotential surface

iE

iW

Figure 6. Obstacle and forearm are projected on Equipotential surface

(a) Original image of laser system


(b) Detection image by laser system

Figure 7. The result of laser detection 4.3 Obstacle avoidance scheme The obstacle avoidance scheme was inspired by the idea of artificial potential field [3]. Because the control is based on velocity, the link of manipulator needs to get velocity value to avoid obstacles. In other words, the link of manipulator would keep away from the obstacle at a certain speed in a artificial potential field. Additionally, there are following properties in projection method: 1) When the projection of link i iEW overlaps or intersects the obstacle projection, the collision may happen; 2) When the projection of link i iEW does not overlap the obstacle projection, the collision never happen. Accordingly, the link is safe as long as the projection of link and the projection of obstacle would not intersect. According to the 3D data of the environment, the potential field of obstacle on the equipotential surface can be obtained by shell. 1 and 2 represent the influence distance of the potential field influence to avoid undesirable disturbance when the obstacle approaches, and hence they form two envelopes, as shown in Figure 8. denotes the shortest distance between the projection

i iEW , and the edge of obstacle can be used to construct the potential field relation. In addition, in Figure 8, the point P represents the closest point on the surface of an obstacle to the projection of the link i iEW . The point Q is the point on

i iEW that is closest to P .

PQ

iE

iW

2

1

Figure 8. Distance relation between the link projection and the obstacle projection

Thus, there is a double layers protection for the arm. 1 is absolute security distance, which defines the inner layer. When the forearm link is close to the obstacle, the is less than 1 possibly, even the i iEW may overlap the obstacle projection. In this case, the arm angle needs to rotate as a fast constant angular velocity, which is denoted as maxV . This case is dangerous, so we can employ an approach to cease the motion of the end‐effector and only to rotate the arm angle with predefined velocity until the shortest distance equals 1 . Namely, once the projection i iEW enters into this layer, only arm angle rotates at a certain angular velocity, while other kinematic joints are static. Here, the velocity of arm angle in this case innerv is defined by

max 1innerv V (9)

where maxV is an user‐specified constant velocity. And, 2 is the relative security distance, which defines the outer layer. Thus, when the projection i iEW enter into the outer layer, namely 1 2 , the link EW might be affected by the virtual velocity. The virtual velocity is designated outerv

1 22

1 1outerv K

(10)

where K is an user‐specified proportional coefficient. Note that the direction of outerv is from P to Q and it ensures that the velocity of motion is continuous from

maxV to 0. However, the outerv is the linear velocity of the corresponding point Q on the projection i iEW , so we still need to transfer it into angular velocity of arm angle. According to the position of Q on the projection i iEW , the corresponding point on the forearm link EW can be defined, which is denoted by EWQ Though the proportional relation /EW i i iQ W QW EW EW , we can obtain the position of EWQ . In order to obtain the radius of rotation, we let h be the distance from the point EWQ to the line SW , as shown in Figure 9. Thus, we can get the angular velocity of arm angel with respect to EWQ

2

1 1A outerouter

v Kvh h

(11)

Therefore, the virtual velocity v on arm angle is

max 1

1 22

2

1 1( )

0

Vkvh

(12)

Note that the virtual velocity is zero when 2 . For the case of multiple obstacle, v is sum of several virtual velocity generated by the corresponding obstacles,


www.intechopen.com

namely 1

nn

v v , where n is amount of obstacles, nv is

the virtual velocity generated by nth obstacle. In the same way, if the human operator inputs a velocity humanv on the arm angle, the output velocity outputv of arm angle will be changed by integrating the obstacle avoiding velocity, namely

1

output human nn

v v v (13)

In this way, the forearm link of the telemanipulator can implement a teleoperation task safely.

Obstacle Projection

P

Q

EWQ

Elbow

Shoulder WristH

Equapotential

Surface

h

Y

Z

X

Figure 9. Radius of rotation h 5. Experiment To perform the experiment, a 7‐DOF manipulator named LWA3 which is produced by Schunk Company was used. The LWA3 manipulator is controlled by a configuration controller which ensures that end‐effector tracks user‐specified trajectories to catch a target. During the manipulation, the human operator only focused on the motion of end‐effector, while the detection system continuously computed the distances between the forearm link projection and the obstacle projection. The obstacle avoidance strategy was implemented in the real‐time controller so that the forearm link was always in safe situation. There were two experiments to illustrate the effectiveness and real‐time ability of the proposed method respectively.

Figure 10. Experiment Setting

Let 2K for experiment. Additionally, 1 50 mm,

2 150 mm and max 0.04 /V rad s are adopted. The experiment setting is shown in Figure 10. 5.1 Experiment 1 – Obstacle is still The goal of this experiment is to demonstrate the effectiveness of the proposed method. In the experiment, the end‐effector is commanded to move forward, namely x‐axis direction, and the obstacle is in the front of forearm link EW . If the arm angle keep still, the collision must happen as the end‐effector moves. However, the proposed method make the arm angle rotate according to the detected information, namely use redundancy resolution to achieve collision avoidance based on sensor system. Figure 11 shows the velocity command of x‐axis direction, which indicates end‐effector keep moving forward in x‐axis direction. In Figure 12, the blue real line indicates the change of the closest distance between the forearm link projection i iEW and the obstacle projection, i.e. . The red dashed line illustrates the position of the 1 . The intersection of the blue real line and the red dashed line indicates the time that the link i iEW leaves the inner layer, which is 15.6s. So, the initial part shows the i iEW overlaps the obstacle projection and the inner layer before 15.6s; the following part shows the i iEW is moving in outer layer after 15.6s. Figure 13 shows the angular velocity of arm angle. Correspondingly, we can find the angular velocity keeps to be 0.04 rad/s before 15.6s and v is decreasing as the is increasing. The virtual velocity can ‘push’ the forearm link away from the obstacle. Thus, this experiment illustrates the forearm link can achieve collision avoidance by the proposed method. In addition, Figure 14 shows the whole process of collision avoidance.

Figure 11. X‐axis velocity of end effector


Figure 12. Closest distance between the forearm link and the obstacle

Figure 13. Angular velocity of arm angle

Figure 14. Process of collision avoidance (sampled every 5s)

5.2 Experiment 2 – Obstacle is mobile In this experiment, the aim is to demonstrate the real‐time ability of the proposed method. We made the obstacle keep moving in the front of forearm link randomly, namely changed the randomly and checked the response of the angular velocity of arm angle. Similarly, the human operator sent task space command of the end‐effector in x‐axis direction of the base frame, then the angular velocity of the arm angle was autonomously calculated based on the sensor information. Figure 15 shows the velocity command of x‐axis direction in this experiment. The value of the blue real line indicates end‐effector keep moving forward in x‐axis direction. Figure 16 indicates the change of (the red dashed line indicates 50 mm), and Figure 17 shows the corresponding change of v (the red dashed line indicates the

0.04 /v rad s ). Obviously, the v becomes 0.04 rad/s once the is less than 50 mm, e.g. from 0s to 3.4s and from 8.7s to 10.1s. It is seen that the velocity of the arm angle changes accordingly and correspondingly to the change of the closest distance between the forearm link and the obstacle, as shown in Figure 16 and Figure 17. The ability of real‐time autonomous obstacle avoidance is demonstrated.

Figure 15. X‐axis velocity of end effector

Figure 16. Closest distance between the forearm link and the obstacle


www.intechopen.com

Figure 17. Angular velocity of arm angle 6. Conclusion The autonomy and real‐time ability of collision avoidance are the important problem in the telerobotic system. A new approach based on real‐time redundancy resolution of dexterous 7‐DOF manipulator is developed and demonstrated in this paper. First, the projection method is used to judge the proximity between the link of manipulator and the obstacle by the real‐time sensor information. Then the virtual velocity at the redundant arm angle is employed to implement the collision avoidance. The two supportive experimental results are presented and illustrate the effectiveness and real‐time ability of the proposed method respectively. 7. Acknowledgments This work was supported by Major Projects of the Ministry of Education (No. 708045), Key Project of Jiangsu Province Natural Science Foundation (No. BK2010063), Sci. and Tech. Project of Changzhou (No. CE20100022). 8. References [1] V. J. Lumelsky and E. Cheung (1993), Real‐Time

Collision Avoidance in Teleoperated Whole‐Sensitive Robot Arm Manipulators, IEEE Trans. On Systems, Man and Cybernetics, vol. 23, pp. 194‐203.

[2] T. Kunz, U. Reiser, M. Stilman and A. Verl (2010), Real‐time Path Planning for a Robot Arm in Changing Environments, IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems (IROS 10), pp. 5906‐5911.

[3] O. Khatib (1998), Real‐Time Obstacle Avoidance for Manipulators and Mobile Robots, The International Journal of Robotics Research, pp. 90‐98.

[4] D. E. Koditschek, Robot planning and control via Potential Function, The Robotics Review, MIT Press, Cambridge, MA, pp. 349‐367.

[5] N. M. Amato, O. B. Bayazit, L. K. Dale, C. Jones and D. Vallejo (1998), OBPRM: An Obstacle‐based PRM for 3D workspaces, In Proc. Of Workshop on Algorithmic Foundations of Robotics, pp. 155‐168.

[6] T. Horsch, F. Schwarz, and H. Tolle (1994), Motion Planning with Many Degress of Freedom – Random Reflection at C‐space Obstacles, In Proc. Of IEEE Conf. on Robotics and Automation, pp. 3318‐3323.

[7] S. Quinlan, O. Khatib (1993), Elastic Bands: Connecting Path Planning and Control, In Proc. of IEEE Intl. Conf. on Robotics and Automation, pp. 802‐807.

[8] O. Brock, O. Khatib (2002), Elastic Strips: A Framework for Motion Generation in Human Environments, The International Journal of Robotics Research, vol. 12, no. 21, pp. 1031‐1052.

[9] P. Leven and S. Hutchinson (2002), A Framework for Real‐time Path Planning in Changing Environments, The International Journal of Robotics Research, vol. 21, no. 12, pp. 999‐1030.

[10] H. Seraji, R. Steele, and R. Ivlev (1996), Sensor‐Based Collision Avoidance: Theory and Experiment, Journal of Robotic Systems, vol. 13, no. 9, pp. 571‐586.

[11] H. Seraji, B. Bon, R, Steele (1997), Experiments in Real‐Time Collision Avoidance for Dexterous 7‐DOF Arms, Proc. of IEEE Intl. Conf. on Robotics and Automation, pp. 569‐574.

[12] H. Seraji, B. Bon (1999), Real‐Time Collision Avoidance for Position‐ Controlled Manipulators, IEEE Transactions On Robotics and Automation, vol. 15, no. 4, pp. 670‐677.

[13] H. Seraji (1989), Configuration Control of Redundant Manipulators: Theory and Implementation, IEEE Transactions On Robotics and Automation, vol. 5, no. 4, pp. 472‐490.

[14] W. Yu, R. Alqasemi, R. Dubey and N. Pernalete (2005), Telemanipulation Assistance Based on Motion Intention Recognition, In Proc. Intl. Conf. on Robotics and Automation, ICRA’05, Barcelona, Spain, pp. 1121‐1126.

[15] H. Li and A. Song (2007), Virtual‐Environment Modeling and Correction for Force‐Reflecting Teleoperation With Time Delay, IEEE Trans. Ind. Electron., vol. 54, no. 2, pp. 1227‐1233.

[16] M. K. Long (1992), Task‐Directed Inverse Kinematics for Redundant Manipulators, Journal of Intelligent and Robotic Systems, vol. 6, pp. 241‐261


real-time obstacle avoidance for telerobotic systems based on

Documents