Systems & Control Letters 59 (2010) 381–389

Contents lists available at ScienceDirect

Systems & Control Letters

journal homepage: www.elsevier.com/locate/sysconle

Distributed control of cooperative target enclosing based on reachability andinvariance analysisI

Ying Lan, Gangfeng Yan, Zhiyun Lin ∗Asus Intelligent Systems Lab, Department of Systems Science and Engineering, Zhejiang University, 38 Zheda Road, 310027 Hangzhou, PR China

a r t i c l e i n f o

Article history:Received 27 November 2009Received in revised form19 April 2010Accepted 22 April 2010Available online 4 June 2010

Keywords:Cooperative controlTarget enclosingHybrid controlUnicycles

a b s t r a c t

The paper presents a hybrid control approach to the problem of steering a group of unicycle-type mobilerobots to reach desired relative positions and orientations with respect to a specific target and othergroup-mates, which is referred to as the cooperative target enclosing problem. With the idea of havingindependent motion towards the target without inter-individual interactions in the further range andswitching to coordinated motion control in the closer range to the target, reachability and invarianceanalysis is recalled to yield a hybrid control law using only local available information such that a groupof unicycle-type mobile robots achieves a uniform circular motion around the target at equal angulardistances from each other.

© 2010 Elsevier B.V. All rights reserved.

1. Introduction

In recent years,multi-robot systems have attractedmuch atten-tion [1–12] due to their various applications and due to the advan-tage that they are able to execute a wide variety of missions withincreased robustness as compared to their single vehicle counter-part. For multi-robot systems, one main challenge stems from thedistributed and local nature. That is, each robot must take controlactions based only on information from local sensors and limitedcommunication. Under these assumptions, one problem arises forexample when multiple robots are required to cooperatively cap-ture a target, for which it is expected that the robots reach desiredrelative positions and orientations with respect to a specific tar-get, obstacles, and other group-mates. This problem was first in-vestigated in [13] using a group of point-mass like mobile robots.More recently, the problem has been addressed via cyclic pursuitstrategies [7,16] where a collective circular motion around the tar-get is achieved. With a more general interaction topology, collec-tive circular motions are also explored in [15] but no target existsin the setup. On the other hand, the problem has been extended tounicycle-typemobile robots [5,6,17,18,14] though it is challengingin feedback control synthesis due to nonholonomic constraints. Inaddition, there are also some other attractive approaches such asformulating the target-tracking problem in the framework of non-cooperative games [19–21].

I The work was supported in part, by National Natural Science Foundation ofChina (60875074) and Qianjiang Talents Program (2009R10027).∗ Corresponding author. Tel.: +86 571 8795 1637; fax: +86 571 8795 2152.E-mail addresses: linz@zju.edu.cn, zhiyun.lin@gmail.com (Z. Lin).

0167-6911/$ – see front matter© 2010 Elsevier B.V. All rights reserved.doi:10.1016/j.sysconle.2010.04.003

Inspired by the progress of the field, this paper tackles the prob-lem of steering a group of unicycles to form a collective uniformcircular motion around a fixed target with equal angular distancesfrom each other. But unlike some approaches [7,17] assuming thatrobots are labeled and the interaction topology is pre-specified,robots in the paper are indistinguishable and the interaction topol-ogy may change depending on their states. Also, unlike [6,22] thatrequire full information of all other robots so that the group cen-troid is accessible, we are interested in having a decentralized anddistributed solution that relies only on local information (relativedistances and bearing angles). With this objective, [5,18] devel-oped a scalable distributed control law for a group of indistinguish-able unicycles using only local information to achieve a collectivecircular motion around a fixed target. However, the solution doesnot lead to a uniform distribution on the enclosing circle and theattraction domain is unknown though local stability is analyzed. Incontrast, this paper presents a newhybrid control solution that notonly achieves a uniform circular motion around the target but alsoattains an evenly spaced configuration. In addition, attraction do-main and even global convergence results are essentially obtainedvia reachability and invariance analysis. In the paper, it is assumedthat if a robot is close enough to the target, it can access local in-formation of the target and other robots around the target, and ifit is far away from the target, it may not sense the target due toits limited sensing range. Consequently, the primary objective ofthe robots in the further range to the target is to get closer with-out inter-individual interactions, and for the robots at the nearbyplaces, their primary objective should coordinate their motions inorder to cooperatively capture the target. Thus, the cooperative tar-get enclosing problem is divided into two sub-problems: one is to

382 Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389

Fig. 1. Local measurement (distance dij and bearing angle αij).

achieve a coordinated motion in the close range and the other is tonavigate robots closer to the target if they are far away initially. Forthe first sub-problem, a switching control law is designed to reachthe desired behavior and also ensure that no robot moves furtheraway from the target. That means, as long as a robot sees the tar-get, it never loses the visibility of the target under the proposedcontrol law. For the second sub-problem, several existent methodsand solutions are discussed.

2. Preliminaries and problem statement

2.1. Vehicle kinematic model

Consider a group of n unicycles labeled 1 through n in the plane.For any unicycle i (i = 1, . . . , n), its configuration is described byq(i)c = (x(i)c , y

(i)c , θ

(i)c )

T∈ R2 × [−π, π), where (x(i)c , y

(i)c ) denotes

the position of its representational point in an inertial coordinateframe W , the angle θ (i)c is its orientation with respect to x-axis ofW . Thus, in the inertial coordinate frameW , the kinematic modelfor unicycle iwith pure rolling and non-slipping is given as follows:

q(i)c =

x(i)cy(i)cθ (i)c

=v(i)c cos θ (i)cv(i)c sin θ

(i)c

ω(i)c

. (1)

where v(i)c and ω(i)c stand for the linear speed and angular speed,

and satisfy the physical constraints∣∣∣v(i)c ∣∣∣ ≤ v and

∣∣∣ω(i)c ∣∣∣ ≤ ω forsome constants v and ω. In addition, suppose there is a stationarytarget (or beacon) in the plane, which is labeled 0. Its position inWis denoted by z0.

2.2. Local sensing information

For each vehicle i, we construct a moving frame, the Frenet–Serret frame, that is fixed on the vehicle with its origin at therepresentational point and the x-axis coincident to the orientationof the vehicle. In our setup, the vehicles are indistinguishable andno vehicle can access the absolute positions of other vehicles or itsown. When vehicle i can sense vehicle j (or the target 0), vehiclei can only measure the distance dij (di0) between them and thebearing angle αij (αi0) with respect to its own Frenet–Serret frame(see Fig. 1). The bearing angle αij is defined in the range [−π, π).The distance and bearing angle can be measured by local sensorsequipped by the vehicles, e.g., onboard cameras.Let D0 be a disk-like region centered at the target with radius

2R (where R > 0 is a constant). In the paper, we assume that ifa vehicle is in D0, it can sense the target and other vehicles thatare also in D0, and if it is outside of D0, it can directly or perhapsindirectly sense (by hopping from one vehicle to another) thetarget or the vehicles that are inside the regionD0. The assumptionis quite mild as at least the first part can always be satisfied inpractice as long as the physical sensor radius is greater than 4R.

Fig. 2. Cooperative target enclosing.

2.3. Problem statement

The multi-robot cooperative target enclosing problem consistsof finding distributed control laws for each vehicle using onlylocally available information such that the group of vehicles caneventually converge to a uniform circularmotion around the targetand are evenly spaced (see Fig. 2). The radius of the circle, which isR in the paper, is pre-specified and is known by all the vehicles.In the paper, the problem of so-called cooperative target enclos-

ing inD0 will be investigated in detail, that is, to solve the cooper-ative target enclosing problem when vehicles’ initial positions arein the region D0. The global cooperative target enclosing problem,meaning the initial states can be anywhere in the state space, willalso be discussed by referring to some existing algorithms as a partof our control scheme.

3. Cooperative target enclosing in D0

In this section, we solve the cooperative target enclosing prob-lem in D0. Also, we want to guarantee that no vehicle leaves theregionD0 in the process of cooperative target enclosing for the fol-lowing reasons. On one hand, leaving the regionD0may imply thatthe vehicle loses visibility of other vehicles and the target due to itslimited sensing range, so the cooperative target enclosing problemmay not be solvable. On the other hand, when the vehicles in D0can be ensured to remain in D0 and are able to solve the cooper-ative target enclosing problem in D0, then for global cooperativetarget enclosing it suffices to navigate the vehicles outside of D0intoD0 and then switch the control.

3.1. Problem reformulation

In this subsection, we introduce a coordinate transformation,originally developed for path following [23,24], and reformulatethe cooperative target enclosing problem in the new coordinatesystem.Let γ be the clockwise circular orbit (circle with a direction of

motion) centered at the stationary target with radius R. For eachunicycle i = 1, . . . , n, we define a corresponding virtual vehicle iwhose position (x(i)r , y

(i)r ) is the orthogonal projection of the mov-

ing unicycle i’s position onto γ and whose orientation θ (i)r is tan-gent toγ in the direction ofmotion at the current location (x(i)r , y

(i)r )

(see Fig. 3 for an illustration). In other words, the position of thevirtual vehicle i is the nearest point on γ to vehicle i. Note that itis unique when vehicle i is not co-located with the target. Denoteq(i)r = (x(i)r , y

(i)r , θ

(i)r )

T as the configuration of the virtual vehicle iin W . Its kinematic model has the same form as (1) with controlinputs v(i)r and ω

(i)r .

Next we construct the virtual Frenet–Serret frame Σi that isfixed to the virtual vehicle i and defined in the same way as theFrenet–Serret frame on vehicle i. Then we are able to define theconfiguration of vehicle i inΣi as

Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389 383

Fig. 3. Virtual vehicle and coordinate variables.

q(i)e :=

cos θ (i)r sin θ (i)r 0− sin θ (i)r cos θ (i)r 00 0 1

(q(i)c − q(i)r ).Thus, the dynamics for q(i)e is

q(i)e =

x(i)ey(i)eθ (i)e

=ω(i)r y(i)e − v(i)r + v(i)c cos θ (i)e−ω(i)r x

(i)e + v

(i)c sin θ

(i)e

ω(i)c − ω(i)r

.By the definitions of virtual vehicle i and its associated virtualFrenet–Serret frame Σi, one obtains that x

(i)e and x

(i)e remain equal

to zero. Denote χ (i)r = −ω(i)r

v(i)ras the curvature of the circle. In our

setup, χ (i)r = 1/R for all i = 1, . . . , n. Then from the first equationof the above formula, it follows that

v(i)r =Rv(i)c cos θ

(i)e

R+ y(i)e

ω(i)r = −χ(i)r v

(i)r = −

v(i)c cos θ

(i)e

R+ y(i)e

under the constraint R + y(i)e 6= 0. Hence, it suffices to look at thereduced systemy(i)e = v

(i)c sin θ

(i)e

θ (i)e = ω(i)c +

v(i)c cos θ

(i)e

R+ y(i)e

i = 1, . . . , n. (2)

The dynamics (2) above describes the evolution of the distance andheading difference to the circular orbit γ .Notice that the constraint

R+ y(i)e 6= 0

is satisfied if vehicle i is not at the target’s location. From a practicalpoint of view, it is reasonable to assume that no robot is initially inZ0 (a disk-like region centered at the target with radius r0). Hence,from now on, we will only focus on the regionD0\Z0. Define

S0 := {(ye, θe)|ye ∈ [−R+ r0, R], θe ∈ [−π, π)}

and denote φ(i) = (y(i)e , θ(i)e )

T . It can be easily seen that φ(i) ∈ S0is equivalent to (x(i)c , y

(i)c ) ∈ D0\Z0. Hence, a necessary step to

solve the cooperative target enclosing problem in D0 is to devisedistributed control v(i)c andω

(i)c for each vehicle so that for all initial

states φ(i)(0) ∈ S0, i = 1, . . . , n, the following holds:

(i) φ(i)(t) ∈ S0 for all t;(ii) φ(i)(t)→ 0 as t →∞.

The first condition above guarantees that no vehicle leaves the re-gion D0\Z0 and the second one means that they eventually tendto move on the circle. However, these are not enough for solving

Fig. 4. Separation angles in an example of four vehicles.

the cooperative target enclosing problem.We need additional con-dition to assure that the vehicles are eventually evenly spaced onthe circle.Let ψij be the angle formed by rotating the ray (originating at

the target and pointing towards vehicle i) counterclockwise untilmeeting vehicle j. The angle ψij is called the separation angle fromvehicle i to j, which belongs to [0, 2π) by our definition. Moreover,a vehicle, that is inD0 and is firstmet by counterclockwise rotatingthe ray originating at the target and pointing towards vehicle i, iscalled a next-neighbor of vehicle i. In amathematical way, the next-neighbor set Ni is defined as

Ni :=

{j|ψij = min

k6=iψik and φ(j) ∈ S0

}.

Similarly, we define the pre-neighbor set Pi of vehicle i as

Pi :=

{j|ψij = max

k6=iψik and φ(j) ∈ S0

}.

A member in the pre-neighbor set Pi is called a pre-neighbor ofvehicle i. From the definitions, we can see that if there is a next-neighbor for vehicle i then there must be a pre-neighbor, and viceversa. Next, let

ψ−i := ψij|j∈Ni and ψ+i := ψji|j∈Pi .

That is, ψ−i and ψ+

i are the separation angle from vehicle i to itsnext-neighbor and the separation angle from its pre-neighbor tovehicle i, respectively. We know that ψ−i = ψ+i = 0 if and onlyif all vehicles are located in the same ray. An example is given inFig. 4, where the next-neighbor and pre-neighbor of vehicle i arethe vehicles j and l, respectively.Now we see that if the distributed control laws are designed to

additionally guarantee that

(iii) limt→∞ ψ−1 (t) = · · · = limt→∞ ψ−n (t) > 0,

then the vehicles are eventually evenly spaced on the circle. Thecondition limt→∞ ψ−i (t) > 0 for all i ensures that all the vehiclesdo not converge to the same moving point.It is worth pointing out that for any vehicle i, if there is only one

next-neighbor, sayNi = {j}, then the changing rate of ψ−i is

ψ−i = −

(v(j)r

R−v(i)r

R

)= ω(j)r − ω

(i)r . (3)

If there is more than one next-neighbor, then the changing rate ofψ−i can be derived as

ψ−i = minj∈Niω(j)r − ω

(i)r .

Now, we reformulate local sensing information in the new co-ordinate system,whichwill be used for control synthesis in the pa-per. Based on local available measurements in the real world, thestates y(i)e and θ

(i)e can be available for each vehicle (see Fig. 5) in

terms of the following relations:

384 Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389

Fig. 5. Local available information in the new state space.

Fig. 6. A partition of S0 .

y(i)e = di0 − R,

θ (i)e = θ(i)c − θ

(i)r =

−αi0 −

π

2, if αi0 ∈ [−π, π/2],

−αi0 +32π, if αi0 ∈ (π/2, π),

where di0 and αi0 are the distance and bearing angle of the targetrelative to vehicle i. Moreover, the separation angles ψ+i and ψ

−

iare also available to vehicle i in terms of a trigonometric relation-ship. As an example, in Fig. 5 the separation anglesψ+i andψ

−

i canbe calculated through

ψ+i = ψji = g(dij, di0, αij, αi0), ψ−i = ψik = g(dik, di0, αik, αi0),

where g(·) is a function related to trigonometric functions.Although in the region D0, each vehicle is able to sense all the

others and the target, we will use only local sensing information ofthe target and two neighbors (its pre-neighbor and next-neighbor)in cooperative target enclosing.Finally, we reformulate the cooperative target enclosing prob-

lem inD0 based on a hybrid control approach. It may not be possi-ble to have one smooth control that not only achieves cooperativetarget enclosing but also makes the set Sn0 (the Cartesian productof n copies of S0) positively invariant. Hence, we expect to find asubset of states in S0 containing the origin so that as long as a ve-hicle enters into the set, it remains in it thereafter under some dis-tributed control for cooperative target enclosing. And for vehicleswith initial states outside of this set, we expect to steer them intoit in finite time without getting out of S0. In terms of this idea, wepartitionS0 into a collection of setsS, S1, . . . , S4 (see Fig. 6), where

S = {(ye, θe) ∈ S0 : |ye| ≤ a, |θe| ≤ a, |ye + θe| ≤ a},S1 = {(ye, θe) ∈ S0\S : a− ε ≤ θe < π},

S2 = {(ye, θe) ∈ S0\S : − π ≤ θe ≤ −a+ ε},S3 = {(ye, θe) ∈ S0\S : ye > 0,−a+ ε < θe < a− ε},S4 = {(ye, θe) ∈ S0\S : ye < 0,−a+ ε < θe < a− ε},

where ε is a sufficiently small positive number and a is chosen tosatisfy 0 < a < min{π/2, R} and to make

b :=ω

v−

(a+

cos aR− a

)> 0

such that the linear and angular speed satisfy the physical con-straints.Thus, the cooperative target enclosing problem in the regionD0

can be solved by addressing the following two subproblems.

Problem 3.1. If φ(i)(0) ∈ S for all i = 1, . . . , n, devise vi andωi foreach vehicle using only locally available information such that forany i,(i) φ(i)(t) ∈ S for all t ≥ 0,(ii) φ(i)(t)→ 0 as t →∞,(iii) limt→∞ ψ−1 (t) = · · · = limt→∞ ψ

−n (t) > 0.

Problem 3.2. If φ(i)(0) ∈ S0\S for some i = 1, . . . , n, devise viand ωi for vehicle i using only locally available information suchthat(i) φ(i)(t) ∈ S0 for all t ≥ 0,(ii) φ(i)(t) enters S in finite time.

3.2. Solving Problem 3.1

We now construct a distributed control law for each vehiclewhen its state is currently in S. That is, as long as φ(i) ∈ S, (i =1, . . . , n), we let

v(i)c =

[c − Γ

(ψ−i − ψ

+

i

)] (R+ y(i)ecos θ (i)e

), ifNi 6= ∅,

c

(R+ y(i)ecos θ (i)e

), ifNi = ∅,

ω(i)c = v(i)c

(−1k(y(i)e + kθ

(i)e + 2 sin θ

(i)e )−

cos θ (i)eR+ y(i)e

),

(4)

where the function Γ : R→ R is defined as

Γ (x) ={0, if x ≥ 0,−γ0, if x < 0,

γ0 ∈ (0, v cos aR+a ), c ∈ (0,v cos aR+a − γ0], and k ≥ max{1,

a+2 sin ab } are

constants.

Remark 3.1. The control law (4) is used in order to obtain the fol-lowing closed-loop system

y(i)e = v(i)c sin θ

(i)e ,

θ (i)e = −v(i)c

k

(y(i)e + kθ

(i)e + 2 sin θ

(i)e

),

for which convergence to the desired orbit can be verified byLyapunov-based techniques. In addition, the function Γ is used toadjust the separation angles. The constraints on the control param-eters γ0, c , and k ensure that |vc | ≤ v and |ωc | ≤ ω under thiscontrol law.With the distributed control law above, we first show that if a

vehicle initially has its state inS, then its state remains inS nomat-ter where its neighbors are. In other words, the set S is positivelyinvariant for the closed-loop dynamics of vehicle i.

Theorem 3.1. For any vehicle i with the control law (4), if φ(i)(0) ∈S, then φ(i)(t) ∈ S for all t ≥ 0.

Proof. Substituting the expression of ω(i)c in (4) into (2), we obtainthe closed-loop system

φ(i) = f (φ(i)) :=

v(i)c sin θ(i)e

−v(i)c

k(y(i)e + kθ

(i)e + 2 sin θ

(i)e )

.

Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389 385

Describe the boundary of S (see Fig. 6) by

µ1 = {(ye, θe)|ye = −a, θe ∈ [0, a]},µ2 = {(ye, θe)|ye ∈ [−a, 0], θe = a},µ3 = {(ye, θe)|ye + θe = a, yeθe > 0},µ4 = {(ye, θe)|ye = a, θe ∈ [−a, 0]},µ5 = {(ye, θe)|ye ∈ [0, a], θe = −a},µ6 = {(ye, θe)|ye + θe = −a, yeθe > 0}.

Let nj, j = 1, . . . , 6, be a normal vector of µj pointing outsideof S. Then it can be inferred from Nagumo’s Theorem [25] that,if all the inner products of nj and the vector fields f (φ(i)) atthe corresponding boundary are non-positive, then the trajectoryφ(i)(t)never leavesS. From (4),we know that v(i)c is always positivefor all states on the boundary of S.For the boundary µ1, n1 = [−1, 0]T and

n1 · f (φ(i)) = −v(i)c sin θ(i)e ,

which is less than 0 for the states in µ1.For the boundary µ2, n2 = [0, 1]T and

n2 · f (φ(i)) = −v(i)c

k(y(i)e + kθ

(i)e + 2 sin θ

(i)e ).

For the states in µ2,

y(i)e + kθ(i)e ≥ −a+ ka ≥ 0

due to k ≥ max{1, a+2 sin ab }. Therefore, n2 ·f (φ(i)) ≤ 0 for the statesin µ2.For the boundary µ3, we know that n3 = [1, 1]T and

n3 · f (φ(i)) =v(i)c

k

[k(sin θ (i)e − θ

(i)e )− 2 sin θ

(i)e − y

(i)e

].

Recall that v(i)c > 0 and k > 0. Moreover, notice that for states onµ3, the following holds:

y(i)e ≥ 0, θ (i)e ≥ 0, and 0 < sin θ (i)e < θ (i)e .

Thus, one obtains

f (φ(i)) · n3 < 0 for all φ(i) on µ3.

Similarly, the condition of Nagumo’s Theorem can be verifiedfor µ4, µ5, and µ6. Thus, the conclusion follows. �

Next, we show that if all the vehicles initially have their statesin S and if the separation angle of any two vehicles is nonzero,then the separation angle can never be zero as the system evolvesunder the control law (4). Thatmeans, if all the vehicles are alreadyin the set S and no two vehicles are located in a same ray, thenthe neighboring relationship in terms of pre-neighbor and next-neighbor does not change at all.Suppose now that φ(i)(0) ∈ S for all i = 1, . . . , n and that no

two vehicles are on a same ray originated at the target (i.e.,ψ−i 6= 0for all i = 1, . . . , n). Since the labels of vehicles do not affect thecontrol strategy, for notation simplicity, we renumber the vehi-cles so that vehicle (i + 1) is the next-neighbor of vehicle i fori = 1, . . . , n − 1 and vehicle 1 is the next-neighbor of vehi-cle n (see Fig. 7 for an illustration). Thus, ψ+i = ψ−i−1 for i =2, . . . , n and ψ+1 = ψ−n . In what follows, we use a circular index.That is, we use the same notation (i − 1) for all i = 1, . . . , n, butwhen i = 1, it means that the index (i− 1) is just n.

Theorem 3.2. Suppose φ(i)(0) ∈ S for all i = 1, . . . , n and considerthe control law (4) for each vehicle. If ψ−i (0) > 0 for all i = 1, . . . , n,then ψ−i (t) > 0 for all i = 1, . . . , n and for all t ≥ 0.

Proof. Since φ(i)(0) ∈ S for all i = 1, . . . , n, it follows fromTheorem 3.1 that φ(i)(t) ∈ S forever. Note that φ(i) ∈ S implies

Fig. 7. Re-label the robots in order.

|θ(i)e | ≤ a < π/2 and |y(i)e | ≤ a < R. Then from (4), we knowthat v(i)c is always positive, meaning that the vehicle always movesforward.Now we show that ψ−i (t) > 0 for all i = 1, . . . , n and for all

t ≥ 0. Suppose by contradiction that at least ψ−i becomes 0 at Tfor the first time. That is, ψ−i (t) > 0 for all t ∈ [0, T ) and for alli ∈ I := {1, . . . , n}; ψ−i (T ) = 0 for i in some set I1 ⊂ I; andψ−i (T ) > 0 for i ∈ I2 := I\I1. Clearly, I1 is not empty by ourassumption, but I2 might be empty.First, suppose I2 is not empty. Thus, we can find an i ∈ I1 such

that (i − 1) is in I2. Hence, ψ−i (T ) = 0 and ψ−i−1(T ) > 0. Bycontinuity, we obtain that for small enough ε > 0,

ψ−i (t)− ψ−

i−1(t) < 0 for all t ∈ [T − ε, T ).

Note that

ω(i)r = −cos θ (i)eR+ y(i)e

v(i)c

= −c + Γ(ψ−i − ψ

+

i

)= −c + Γ

(ψ−i − ψ

−

i−1

).

Thus, from (3) we have

ψ−i = ω(i+1)r − ω(i)r

=[−c + Γ

(ψ−i+1 − ψ

−

i

)]−[−c + Γ

(ψ−i − ψ

−

i−1

)]= Γ

(ψ−i+1 − ψ

−

i

)− Γ

(ψ−i − ψ

−

i−1

). (5)

From the definition of Γ , we know Γ (·) is either 0 or −γ0. More-over, since ψ−i (t)− ψ

−

i−1(t) < 0 for t ∈ [T − ε, T ], it follows thatΓ(ψ−i (t)− ψ

−

i−1(t))= −γ0. Therefore, for t ∈ [T − ε, T ],

ψ−i (t) = Γ(ψ−i+1(t)− ψ

−

i (t))− Γ

(ψ−i (t)− ψ

−

i−1(t))≥ 0.

Hence,

ψ−i (T ) = ψ−

i (T − ε)+∫ T

T−εψ−i (τ )dτ ≥ ψ

−

i (T − ε) > 0,

a contradiction to the assumption that ψ−i (T ) = 0.Second, suppose that I2 is empty. That is, all ψ−i ’s become 0 at

T and ψ−i (t) > 0 for t ∈ [0, T ). Since∑ni=1 ψ

−

i (t) = 2π fort ∈ [0, T ), we can infer that there exists a j satisfying ψ−j (t) →2π as t → T . That means, for small enough ε > 0,

ψ−j+1(t)− ψ−

j (t) < 0 for all t ∈ [T − ε, T ).

Thus, by the same argument as for the first case, we obtainψ−j+1(T ) > 0, a contradiction to the assumption. �

Finally, we show that φ(i)(t) → 0 as t → ∞ (each vehicleconverges to the desired circular orbit) and that limt→∞ ψ−1 (t) =· · · = limt→∞ ψ−n (t) > 0 (they are evenly spaced). That is,the distributed control (4) solves Problem 3.1 and thus solves thecooperative target enclosing problem.

386 Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389

Theorem 3.3. If initially φ(i)(0) is in S for all i and no two vehiclesare on a same ray originated at the target, then the group of unicyclessolves the cooperative target enclosing problem using the distributedcontrol (4).Proof. First, consider a positive definite function

V (i) =12(y(i)e + kθ

(i)e )

2+12(y(i)e )

2, i = 1, 2, . . . , n.

Taking the derivative along the solution of (2) with the controllaw (4) and after several steps of calculation, one obtainsV (i) = (y(i)e + kθ

(i)e )(y

(i)e + kθ

(i)e )+ y

(i)e y

(i)e

= −v(i)c (y(i)e + kθ

(i)e )

2− kv(i)c θ

(i)e sin θ

(i)e .

Since φ(i)(0) is in S for all i, it follows from Theorem 3.1 that φ(i)(t)is in S for all t , which implies |θ (i)e (t)| ≤ a < π

2 ,∀t > 0.Hence, θ (i)e sin θ

(i)e ≥ 0. Furthermore, θ

(i)e sin θ

(i)e = 0 if and only

if θ (i)e = 0. Recall that v(i)c is always positive. Sowe know that V (i) is

negative definite with respect to the origin. Consequently, for anyi = 1, . . . , n,

φ(i)(t)→ 0 as t →∞. (6)Second, consider a function

W =12

n∑i=1

(ψ−i − ψ

−

i−1

)2.

The functionW is positive when the angles are not all equal and is0 when they are all equal. Taking the derivative leads to

W =n∑i=1

(ψ−i − ψ−

i−1)(ψ−

i − ψ−

i−1).

Denote βi = ψi − ψi−1. From (5), one getsβi = Γ (ψ

−

i+1 − ψ−

i )+ Γ (ψ−

i−1 − ψ−

i−2)− 2Γ (ψ−

i − ψ−

i−1).

Recall that Γ (·) is either 0 or−γ0. Hence, if ψ−i − ψ−

i−1 ≥ 0, thenΓ (ψ−i − ψ

−

i−1) = 0 and

βi = Γ (ψ−

i+1 − ψ−

i )+ Γ (ψ−

i−1 − ψ−

i−2) ≤ 0.

If ψ−i − ψ−

i−1 < 0, then Γ (ψ−

i − ψ−

i−1) = −γ0 and

βi = Γ (ψ−

i+1 − ψ−

i )+ Γ (ψ−

i−1 − ψ−

i−2)+ 2γ0 ≥ 0. (7)

Thus,(ψ−i − ψ

−

i−1)βi ≤ 0, i = 1, . . . , n,

which gives

W =n∑i=1

(ψ−i − ψ−

i−1)βi ≤ 0.

Furthermore, W = 0 implies ψ−i − ψ−i−1 ≥ 0 for all i. (Thiscan be seen by contradiction. Suppose that there is an i∗ such thatψ−i∗ −ψ

−

i∗−1 < 0. Then in order to make W = 0, βi∗ has to be zero,which means ψ−i∗+1 − ψ

−

i∗ < 0 from (7). Repeating this argument,it follows that ψ−i+1 − ψ

−

i < 0 for all i. Thus, the sum∑ni=1(ψ

−

i −

ψ−i−1) < 0, a contradiction to the fact∑ni=1(ψ

−

i − ψ−

i−1) = 0.)When W = 0, we just establish that ψ−i − ψ

−

i−1 ≥ 0 for all i.Taking the fact

∑ni=1(ψ

−

i −ψ−

i−1) = 0 into account, we obtain thatψ−i − ψ

−

i−1 = 0 for all i. Hence, it follows from Lyapunov theorythatlimt→∞

ψ−1 (t) = · · · = limt→∞ψ−n (t).

Moreover, we known ψ−i (t) > 0 all the time by Theorem 3.2 and∑ni=1 ψ

−

i = 2π . Hence, the vehicles are eventually spaced withequal angular distance.

Fig. 8. State transition graph.

Fig. 9. Phase portrait of the resulting linear closed-loop system.

In conclusion, the group of unicycles with the distributed con-trol (4) solves the cooperative target enclosing problem. �

Remark 3.2. From Theorem 3.2, we know ψi(t) > 0 all the time,which also guarantees that no collision occurs between vehicles.

3.3. Solving Problem 3.2

In the previous subsection, we devised a distributed control lawfor each vehicle to solve the cooperative target enclosing problemwhen they are initially satisfying φi(0) ∈ S. In other words, boththe initial orientations and locations of the vehicles should sat-isfy certain conditions. In this subsection, we expect to devise dis-tributed control to drive the vehicles into S in finite time (namely,solving Problem 3.2) if they initially do not satisfy φi(0) ∈ S. Basedon the partition in Fig. 6, we would like to construct controllersfor each vehicle such that the abstract state transition in Fig. 8happens.In the paper we use the notation X → Y to represent the

reachability specification from one state setX to its adjacent stateset Y. That is, find a control law such that for any initial stateψ (i)(0) ∈ X there exists T > 0 satisfying(a) ψ (i)(t) ∈ X for all 0 ≤ t < T ,(b) ψ (i)(t) ∈ Y when t = T .First, we solve the problem S1 → S. In the set S1, we use a

feedback-linearization technique to design a control law so that theresulting closed-loop system is a linear system in the plane withits equilibrium point at (0, a− ε) and its phase portrait as in Fig. 9.Thus, the control law is given byv(i)c = −k1

y(i)eSatδ(sin θ

(i)e )

ω(i)c = −k2(θ(i)e − a+ ε)−

v(i)c cos θ

(i)e

R+ y(i)e

(8)

where k1 ∈ (0, δvR ] and k2 ∈ (0,r0ω−v

r0(π−a+ε)] are constants satisfying

k1 > k2, and Satδ(·) is a saturation function defined as

Satδ(x) =

{x, |x| ≥ δ,δ, 0 ≤ x < δ,−δ, −δ < x < 0.

(Here, δ is chosen to be sin(ε)). The presence of saturation functionis to avoid the singularity using feedback linearization techniquewhen sin θ (i)e = 0.

Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389 387

Theorem 3.4. Under the control law (8), S1 → S.

Proof. In the set S1, if sin θ(i)e ≥ δ, (i.e., θ (i)e ∈ [a − ε, π − ε]),

the resulting closed-loop system with the control law (8) is thefollowing linear system{y(i)e = −k1y

(i)e ,

θ (i)e = −k2(θ(i)e − a+ ε),

whose phase portrait looks like Fig. 9. So it can be easily seen thatall trajectories initiated in S1 enter only S without crossing anyother boundaries.In the set S1, if sin θ

(i)e < δ, (namely, θ (i)e ∈ (π − ε, π)), then

the resulting closed system is of the following nonlinear formy(i)e = −k1 y(i)e sin θeδ

,

θ (i)e = −k2(θ(i)e − a+ ε).

With θ (i)e ∈ (π − ε, π), notice that on the boundary θe = π ,

θ (i)e = −k2(π − a+ ε) < 0,

on the boundary ye = −R+ r0,

y(i)e = k1(R− r0) sin θ

(i)e

δ> 0,

and on the boundary ye = R,

y(i)e = −k1R sin θ (i)e

δ< 0.

Hence, no trajectory leaves S0 with the above nonlinear dynamics.Moreover, we are able to find a vector ξ = [0,−1] such that

ξ · φ(i) = k2(θ (i)e − a+ ε) > 0 for all θ (i)e ∈ (π − ε, π),

which means all trajectories all the way flow down to the linearregion and then enter S. �

For the problem of S2 → S, due to the symmetrical properties,we can use the same technique as for S1 → S and then obtain thecontrol lawv(i)c = −k1

y(i)eSatδ(sin θ

(i)e ),

ω(i)c = −k2(θ(i)e + a− ε)−

v(i)c cos θ

(i)e

R+ y(i)e.

(9)

Corollary 3.1. Under the control law (9), S2 → S.Nowwe consider the states in S3. We simply choose the control

lawv(i)c = −k3θ

(i)e

ω(i)c = −k4 −v(i)c cos θ

(i)e

(R+ y(i)e )

(10)

with the constants k3 ∈ (0, va−ε ] and k4 ∈ (0, ω −

vR ]. So that the

closed-loop system in S3 is{y(i)e = −k3θ

(i)e sin θ

(i)e ,

θ (i)e = −k4.Then we have the following result.

Theorem 3.5. Under the control law (10), S3 → S ∪ S2.Proof. Notice that on the boundary ye = R,

y(i)e = −k3θ(i)e sin θ

(i)e ≤ 0

due to |θ (i)e | < π/2 in S3. So the trajectory starting in S3 neverleavesS0 through the boundary ye = R. Moreover, since θ

(i)e = −k4

for all states inS3, the trajectory flows downward in the state spaceall the time until it reaches S or S2. �

The similar argument applies to S4 so that the control lawv(i)c = k3θ

(i)e

ω(i)c = k5 −v(i)c cos θ

(i)e

(R+ y(i)e )

(11)

with constant 0 < k5 ≤ ω − vr0solves the reachability problem

S4 → S ∪ S1.

Corollary 3.2. Under the control law (11), S4 → S ∪ S1.

Remark 3.3. The control laws (8) and (9) are obtained via the feed-back linearization technique resulting in a desired linear closed-loop system with its equilibrium inside S. The saturation functionis used to avoid singularity. The control (10) and (11) are designedto induce in a simple linear closed-loop system for which the tra-jectories flows downward (upward) all the time until out of thethreshold of saturation. The constraints on constants k1, k2, k3, k4and k5 guarantee |vc | ≤ v and |ωc | ≤ ω.

3.4. Hybrid control for cooperative target enclosing inD0

Combining the above results, we are now able to construct adistributed hybrid control for each vehicle to solve the cooperativetarget enclosing problem in D0. For each vehicle, define a hybridcontrol based on the following table

(8) (9) (10) (11) (4)S1 S2 S3 S4 S

. (12)

The above table means that, when vehicle i’s state lies in S1, thenuse the control law (8), when vehicle i’s state lies in S2, then usethe control law (9), and so on.We now present the main result.

Theorem 3.6. The hybrid control (12) solves the cooperative targetenclosing problem for almost all initial states inD0.

Proof. For each unicycle i, the control law switches among severalcontrollers depending on its state φ(i)(t) at time t . By Theorems 3.4and 3.5 and Corollaries 3.1 and 3.2, it follows that every vehiclewillenter into S in finite time according to the state transition graph inFig. 8. Moreover, no vehicle will leave S when it gets into S at sometime by Theorem 3.1. Hence, without loss of generality, say at T1,all the vehicles’ states are in S. Thus, by Theorem 3.3 the group ofvehicles eventually achieve cooperative target enclosing if no twovehicles are in the same ray originated from the target. Note thatthe set of initial states which there are two vehicles on a same rayis of low dimension. So the conclusion follows. �

4. Global cooperative target enclosing

In the previous section, we have solved the cooperative targetenclosing problem in D0. Meanwhile, it is guaranteed that novehicle leaves the region D0. Hence, for the global cooperativetarget enclosing problem, it suffices to devise controllers to steerthe vehicles outside ofD0 intoD0 in finite time.Note that the vehicles in D0 remain in D0 forever and even-

tually move along the desired circle enclosing the target under thehybrid control (12). So if a vehicle outside ofD0 can sense a vehicleinD0 or the target, then it is easy to implement a tracking controllaw to make it get close to its tracking object and thus it entersD0.There are lots of methods that can be used to solve this, such as theartificial potential function approach [26], the input–output lin-earization technique, Lyapunov function based target tracking [27],the time-varying control approach [28], etc. Instead, if a vehicleoutside of D0 cannot directly see a vehicle in D0 or the target,

388 Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389

Fig. 10. Trajectories of five unicycles in the plane at t = 5s.

Fig. 11. Trajectories of five unicycles in the plane at t = 15s.

then by our assumption, it can indirectly sense (by hopping fromone vehicle to another) the target or the vehicles insideD0, whichmeans there is a path from this vehicle to the target or a vehicle inD0 in the sensing graph. In other words, it sees a vehicle i1 (whichmight be outside of D0), vehicle i1 sees a vehicle i2 (which mightalso be outside ofD0), and so on, but eventually a vehicle ik in thesequence sees the target or a vehicle in D0. Hence, every vehicleoutside of D0 makes use of any aforementioned method to trackits sensed neighbor vehicles and thus they form a forest-like inter-action topology with roots locating inside D0. Then, one and onevehicle is gradually brought into the regionD0 in finite time. Con-sequently, the global cooperative target tracking problem is solved.

5. Simulations

Wepresent a simulation of five unicycles achieving cooperativetarget enclosing. The stationary target is set at the origin withoutloss of generality. The radius of desired enclosing circle is R = 3m.So the setD0 is a disk-like region with radius 6m in the plane. Theinitial configurations of the five unicycles are randomly generated.They are (2, 0,−0.15π), (−6,−4, 0.2π), (−10,−10.8,−0.35π),(2.5, 3,−0.1π), and (−3, 0, 0.2π), respectively.For unicycles inD0, weuse the hybrid controller (12) depending

on their states and for unicycles outside of D0, we use a stabletarget tracking control law from [27]. When v = 12, ω = 50, thecontrol parameters for the hybrid controller are chosen as r0 =0.25, γ0 = 0.24, c = 0.5, a = 1.3 < π

2 , ε = 0.1, k = 4, k1 =0.4, k2 = 0.3, k3 = k4 = k5 = 1.

Fig. 12. Trajectories of five unicycles in the plane at t = 35s.

Fig. 13. The evolution of y(i)e (t) (i = 1, . . . , 5).

Fig. 14. The evolution of θ (i)e (t) (i = 1, . . . , 5).

Figs. 10–12 shows the trajectories of five unicycles in the planeunder our hybrid control, where the blank wedge represents theinitial configuration and the filled wedge represents the currentconfiguration at each sampling instant. Notice from Fig. 10 that,three unicycles are initially inD0, so they coordinate their motionand achieve cooperative target enclosing first. When the fourthone joins in, these four unicycles re-coordinate their motion andapproach an evenly spaced formation on the circle. Finally, all five

Y. Lan et al. / Systems & Control Letters 59 (2010) 381–389 389

Fig. 15. The evolution of separation angles ψi(t) (i = 1, . . . , 5).

unicycles come to run on the enclosing circle clockwise with equalseparation angles. Thus, they cooperatively capture the target.Figs. 13 and 14 provide the evolution curves of states y(i)e (t) andθ(i)e (t), i = 1, . . . , 5, respectively. In addition, the evolution curvesof the separation angles between any two neighbor vehicles aredepicted in Fig. 15, which converge to the value 2π/5. From there,it also can be seen that all the separation angles are greater than 0all the time as we showed.

6. Conclusion and future work

The paper addresses the cooperative target enclosing problemwith multiple unicycle-type robots. A novel approach based onreachability specification and hybrid control is proposed to synthe-size a controller for this problem. Vehicles far away from the tar-get are steered to a region near the target first, and vehicles nearbythe target coordinate their motion in order to cooperatively cap-ture the target through an enclosing circular motion around thetarget. The hybrid nature of independent motion towards the tar-get without inter-individual interactions in the further range andcoordinated motion in the closer range of target also meets therequirement of cooperative target enclosing in practice. In addi-tion, another important feature of the approach in the paper isthat nonholonomic constraints have been removed via a coordi-nate transformation. Limited sensing capability is also consideredfor the purposes of practical applications. However, the work nowis still at an early stage. Several challenging problems remain tobe considered in the future, including collision avoidance, trackingperformance with respect to a moving target, etc.

References

[1] W. Ren, R.W. Beard, Consensus seeking in multi-agent system under dynami-cally changing interaction topologies, IEEE Transactions on Automatic Control50 (5) (2005) 655–661.

[2] J.A. Fax, R.M. Murray, Information flow and cooperative control of vehicleformations, IEEE Transactions on Automatic Control 49 (9) (2004) 1465–1476.

[3] J. Lin, A.S. Morse, B.D.O. Anderson, The multi-agent rendezvous problem, in:Proceedings of the 42th IEEE Conference on Decision and Control, New Haven,CT, USA, 2003, pp. 1508–1513.

[4] J.A.Marshall, M.E. Broucke, B.A. Francis, Formation of vehicles in cyclic pursuit,IEEE Transactions on Automatic Control 49 (11) (2004) 1963–1974.

[5] N. Ceccarelli, M.D. Marco, A. Garulli, A. Giannitrapani, Collective circularmotion of multi-vehicle systems with sensory limitations, in: Proceedings ofthe 44th IEEE Conference on Decision and Control, and the European ControlConference, Seville, Spain, 2005, pp. 740–745.

[6] D.A. Paley, N.E. Leonard, R. Sepulchre, Oscillator models and collective mo-tion: splay state stabilization of self-propelled particles, in: Proceedings ofthe 44th IEEE Conference on Decision and Control, and the Euopean ControlConference, Seville, Spain, 2005, pp. 3935–3940.

[7] S. Hara, T.H. Kim, Y. Hori, Distributed formation control for target-enclosingoperations based on a cyclic pursuit strategy, in: Proceedings of 17th Interna-tional Federation of Automatic Control, Seoul, Korea, 2008, pp. 6602–6607.

[8] H. Tanner, G. Pappas, V. Kumar, Leader-to-formation stability, IEEE Transac-tions on Robotics and Automation 20 (3) (2004) 443–456.

[9] A. Jadbabaie, J. Lin, A.S. Morse, Coordination of groups of mobile autonomousagents using nearest neighbor rules, IEEE Transactions on Automatic Control48 (6) (2003) 988–1001.

[10] Z. Lin, M.E. Broucke, B.A. Francis, Local control strategies for groups of mobileautonomous agents, IEEE Transactions on Automatic Control 49 (4) (2004)622–629.

[11] E.W. Justh, P.S. Krishnaprasad, Steering laws and continuummodels for planarformations, in: Proceedings of the 42nd IEEE Conference on Decision andControl, Maui, Hawaii, USA, 2003, pp. 3609–3614.

[12] E.W. Justh, P.S. Krishnaprasad, Equilibria and steering laws for planar forma-tions, Systems and Control Letters 52 (1) (2004) 25–38.

[13] H. Yamaguchi, A cooperative hunting behavior by mobile robot troops,Robotics Research 18 (8) (1999) 931–940.

[14] H. Kawakami, T. Namerikawa, Consensus filter based target-enclosing strate-gies for multiple nonholonomic vehicles, in: Proceedings of the 47th IEEEConference on Decision and Control, Cancun, Mexico, 2008, pp. 2282–2287.

[15] W. Ren, Collectivemotion from consensus with cartesian coordinate coupling,IEEE Transactions on Automatic Control 54 (6) (2009) 1330–1335.

[16] T.H. Kim, T. Sugie, Cooperative control for target-capturing task based on acyclic pursuit strategy, Automatica 43 (8) (2007) 1426–1431.

[17] H. Yamaguchi, A distributed motion coordination strategy for multiple non-holonomic mobile robots in cooperative hunting operations, Robotics andAutonomous Systems 43 (4) (2003) 257–282.

[18] N. Ceccarelli, M.D. Marco, A. Garulli, A. Giannitrapani, Collective circularmotion of multi-vehicle systems, Automatica 44 (12) (2008) 3025–3035.

[19] K. Skrzypczyk, Game theory based on target following by a team of robots, in:Proceedings of the 4th International Workshop on Robot Motion and Control,Puszczykowo, Poland, 2004, pp. 91–96.

[20] I. Harmati, Multi-agent coordination for target tracking using fuzzy inferencesystem in game theoretic framework, in: Proceedings of the 2006 IEEEInternational Symposium on Intelligent Control, Munich, Germany, 2006,pp. 2390–2395.

[21] I. Harmati, K. Skrzypczyk, Robot team coordination for target tracking usingfuzzy logic controller in game theoretic framework, Robotics and AutonomousSystems 57 (1) (2009) 75–86.

[22] R. Sepulchre, D.A. Paley, N.E. Leonard, Stabilization of planar collective mo-tion: all-to-all communication, IEEE Transactions on Automatic Control 52 (5)(2007) 811–824.

[23] L. Aguilar, P. Soueres, M. Courdesses, S. Fleury, Robust path-following controlwith exponetial stability for mobile robots, in: Proceedings of the 1998 IEEEInternational Conference on Robotics and Automation, Leuven, Belgium, 1998,pp. 3279–3284.

[24] A. Micaelli, C. Samson, Trajectory tracking for unicycle-type and two-steering-wheels mobile robots, Tech. rep., Institut National de Recherche enInformatique et en Automatique (INRIA), France (1993).

[25] J.P. Aubin, Viability Theory, Birkhauser, 1991.[26] H.G. Tanner, A. Jadbabaic, G.J. Pappas, Flocking in fixed and switching

networks, IEEE Transactions on Automatic Control 52 (5) (2007) 863–868.[27] Y. Lan, G. Yan, Z. Lin, A hybrid control approach to multi-robot coordinated

path following, in: Proceedings of 48th IEEE Conference on Decision andControl, Shanghai, China, 2009, pp. 3032–3037.

[28] Z. Lin, Distributed Control and Analysis of Coupled Cell Systems, VDM-Verlag,2008.