1534 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 6, JUNE 2013

[12] M. Fliess, C. Join, and M. Mboup, “Algebraic change-point detec-tion,” Applicable Algebra Eng., Commun., Comp., vol. 21, no. 2, pp.131–143, 2010.

[13] E. Fridman, F. Gouaisbaut, M. Dambrine, and J.-P. Richard, “A de-scriptor approach to sliding mode control of systems with time-varyingdelays,” Int. J. Syst. Sci., vol. 34, no. 8–9, pp. 553–559, 2003.

[14] E. Fridman, “Stability of systems with uncertain delays: A new ‘com-plete’ Lyapunov-Krasovskii functional,” IEEE Trans. Autom. Control,vol. 51, no. 5, pp. 885–890, May 2006.

[15] J.-E. Fonde, “Delay Differential Equation Models in Mathematical Bi-ology,” Ph.D. dissertation, Univ. Michigan, Dearborn, 2005.

[16] J. P. Gauthier, H. Hammouri, and S. Othman, “A simple observer fornonlinear systems. Applications to bioreactors,” IEEE Trans. Autom.Control, vol. 37, no. 6, pp. 875–880, Jun. 1992.

[17] M. Ghanes, J. B. Barbot, J. DeLeon, and A. Glumineau, “A robustoutput feedback controller of the induction motor drives: New designand experimental validation,” Int. J. Control, vol. 83, no. 3, pp.484–497, 2010.

[18] A. Germani, C. Manes, and P. Pepe, “An asymptotic state observer fora class of nonlinear delay systems,” Kybernetika, vol. 37, no. 4, pp.459–478, 2001.

[19] M. Hou and R. T. Patton, “An observer design for linear time-delaysystems,” IEEE Trans. Autom. Control, vol. 47, no. 1, pp. 121–125,Jan. 2002.

[20] S. Ibrir, “Adaptive observers for time delay nonlinear systems in trian-gular form,” Automatica, vol. 45, no. 10, pp. 2392–2399, 2009.

[21] V. L. Kharitonov and D. Hinrichsen, “Exponential estimates for timedelay systems,” Syst. Control Lett., vol. 53, no. 5, pp. 395–405, 2004.

[22] N. N. Krasovskii, “On the analytical construction of an optimal controlin a system with time lags,” J. Appl. Math. Mech., vol. 26, no. 1, pp.50–67, 1962.

[23] V. Laskhmikanthan, S. Leela, and A. Martynyuk, “Practical stabilityof nonlinear systems,” in Proc. Word Scientific, 1990, [CD ROM].

[24] N. MacDonald, “Time lags in biological models,” in Lecture Notes inBiomath. New York: Springer, 1978.

[25] L. A. Marquez-Martinez, C. H. Moog, and V. V. Martin, “Observ-ability and observers for nonlinear systems with time delays,” Kyber-netika, vol. 38, no. 4, pp. 445–456, 2002.

[26] H. Mounier and J. Rudolph, “Flatness based control of nonlinear delaysystems: A chemical reactor example,” Int. J. Control, vol. 71, no. 5,pp. 871–890, 1998.

[27] K. Natori and K. Ohnishi, “A design method of communication distur-bance observer for time-delay compensation, taking the dynamic prop-erty of network disturbance into account,” IEEE T. Indust. Electron.,vol. 55, no. 5, pp. 2152–2168, 2008.

[28] S.-I. Niculescu, C.-E. de Souza, L. Dugard, and J.-M. Dion, “Robustexponential Stability of uncertain systems with time-varying delays,”IEEE Trans. Autom. Control, vol. 43, no. 5, pp. 743–748, May 1998.

[29] S.-I. Niculescu, Delay Effects on Stability: A Robust Control Ap-proach. New York: Springer LNCIS, 2001.

[30] P. Picard, O. Sename, and J. F. Lafay, “Observers and observabilityindices for linear systems with delays,” in Proc. IEEE Conf. Computat.Eng. Syst. Appl. (CESA 96) , 1996, vol. 1, pp. 81–86.

[31] J.-P. Richard, “Time-delay systems: An overview of some recentadvances and open problems,” Automatica, vol. 39, no. 10, pp.1667–1694, 2003.

[32] J.-P. Richard, F. Gouaisbaut, and W. Perruquetti, “Sliding mode con-trol in the presence of delay,” Kybernetica, vol. 37, no. 4, pp. 277–294,2001.

[33] O. Sename, “New trends in design of observers for time-delay sys-tems,” Kybernetica, vol. 37, no. 4, pp. 427–458, 2001.

[34] O. Sename and C. Briat, “H1 observer design for uncertain time-delaysystems,” in Proc. IEEE ECC’07, 2007, pp. 5123–4130.

[35] A. Seuret, T. Floquet, J.-P. Richard, and S. K. Spurgeon, “A slidingmode observer for linear systems with unknown time-varying delay,”in Proc. IEEE ACC’07, 2007, pp. 4558–4563.

[36] A. Seuret, T. Floquet, J.-P. Richard, and S. K. Spurgeon, Topics inTime-Delay Systems: Analysis, Algorithms and Control. Berlin, Ger-many: Springer Verlag, 2008.

[37] E. Shustin, L. Fridman, E. Fridman, and F. Castaos, “Robustsemiglobal stabilization of the second order system by relay feedbackwith an uncertain variable time delay,” SIAM J. Control Optim., vol.47, no. 1, pp. 196–217, 2008.

[38] R. Villafuerte, S. Mondie, and Z. Poznyak, “Practical stability oftime delay systems: LMI’s approach,” in Proc. IEEE CDC, 2008, pp.4807–4812.

[39] Z. Wang, B. Huang, and H. Unbehausen, “Robust H1 observer designfor uncertain time-delay systems: (I) the continuous case,” in Proc.IFAC 14th World Congress, Beijing, China, 1999, pp. 231–236.

[40] J. Zhang, X. Xia, and C. H. Moog, “Parameter identifiability of non-linear systems with time-delay,” IEEE Trans. Autom. Control, vol. 47,no. 2, pp. 371–375, Feb. 2006.

[41] G. Zheng, J. P. Barbot, D. Boutat, T. Floquet, and J. P. Richard, “Onobserability of nonlinear time-delay systems with unknown inputs,”IEEE Trans. Autom. Control, vol. 56, no. 8, pp. 1973–1978, Aug. 2011.

An Approximate Dual Subgradient Algorithm forMulti-Agent Non-Convex Optimization

Minghui Zhu and Sonia Martínez

Abstract—We consider a multi-agent optimization problemwhere agentssubject to local, intermittent interactions aim to minimize a sum of localobjective functions subject to a global inequality constraint and a globalstate constraint set. In contrast to previous work, we do not require thatthe objective, constraint functions, and state constraint sets are convex. Inorder to deal with time-varying network topologies satisfying a standardconnectivity assumption, we resort to consensus algorithm techniques andthe Lagrangian duality method. We slightly relax the requirement of exactconsensus, and propose a distributed approximate dual subgradient algo-rithm to enable agents to asymptotically converge to a pair of primal-dualsolutions to an approximate problem. To guarantee convergence, we as-sume that the Slater’s condition is satisfied and the optimal solution set ofthe dual limit is singleton. We implement our algorithm over a source lo-calization problem and compare the performance with existing algorithms.

Index Terms—Dual subgradient algorithm, Lagrangian duality.

I. INTRODUCTION

Recent advances in computation, communication, sensing and actu-ation have stimulated an intensive research in networked multi-agentsystems. In the systems and controls community, this has translatedinto how to solve global control problems, expressed by global objec-tive functions, by means of local agent actions. Problems consideredinclude multi-agent consensus or agreement [11], [17], coverage con-trol [4], formation control [7], [21] and sensor fusion [24].The seminal work [2] provides a framework to tackle optimizing

a global objective function among different processors where eachprocessor knows the global objective function. In multi-agent envi-ronments, a problem of focus is to minimize a sum of local objectivefunctions by a group of agents, where each function depends ona common global decision vector and is only known to a specificagent. This problem is motivated by others in distributed estimation[16], [23], distributed source localization [20], and network utilitymaximization [12]. More recently, consensus techniques have beenproposed to address the issues of switching topologies, asynchronouscomputation and coupling in objective functions; see for instance[14], [15], [27]. More specifically, the paper [14] presents the first

Manuscript received October 14, 2010; revised January 25, 2012; acceptedOctober 08, 2012. Date of publication November 16, 2012; date of current ver-sion May 20, 2013. This work was supported by the NSF CAREER AwardCMMI-0643673. Recommended by Associate Editor E. K. P. Chong.M. Zhu is with the Laboratory for Information and Decision Systems,

Massachusetts Institute of Technology, Cambridge MA 02139 USA (e-mail:mhzhu@mit.edu).S. Martínez is with the Department of Mechanical and Aerospace Engi-

neering, University of California, San Diego, La Jolla, CA 92093 USA (e-mail:soniamd@ucsd.edu).Digital Object Identifier 10.1109/TAC.2012.2228038

0018-9286/$31.00 © 2012 IEEE

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 6, JUNE 2013 1535

analysis of an algorithm that combines average consensus schemeswith subgradient methods. Using projection in the algorithm of [14],the authors in [15] further address a more general scenario that takeslocal state constraint sets into account. Further, in [27] we develop twodistributed primal-dual subgradient algorithms, which are based onsaddle-point theorems, to analyze a more general situation that incor-porates global inequality and equality constraints. The aforementionedalgorithms are extensions of classic (primal or primal-dual) subgra-dient methods which generalize gradient-based methods to minimizenon-smooth functions. This requires the optimization problems ofinterest to be convex in order to determine a global optimum.The focus of the current technical note is to relax the convexity

assumption in [27]. In order to deal with all aspects of our multi-agentsetting, our method integrates Lagrangian dualization, subgradientschemes, and average consensus algorithms. Distributed functioncomputation by a group of anonymous agents interacting intermit-tently can be done via agreement algorithms [4]. However, agreementalgorithms are essentially convex, and so we are led to the inves-tigation of nonconvex optimization solutions via dualization. Thetechniques of dualization and subgradient schemes have been popularand efficient approaches to solve both convex programs (e.g., in [3])and nonconvex programs (e.g., in [5], [6]).Statement of Contributions: Here, we investigate a multi-agent op-

timization problem where agents desire to agree upon a global decisionvector minimizing the sum of local objective functions in the presenceof a global inequality constraint and a global state constraint set. Agentinteractions are changing with time. The objective, constraint func-tions, as well as the state-constraint set, can be nonconvex. To dealwith both nonconvexity and time-varying interactions, we first definean approximate problem where the exact consensus is slightly relaxed.We then propose a distributed dual subgradient algorithm to solve it,where the update rule for local dual estimates combines a dual sub-gradient scheme with average consensus algorithms, and local primalestimates are generated from local dual optimal solution sets. This al-gorithm is shown to asymptotically converge to a pair of primal-dualsolutions to the approximate problem under the following assumptions:firstly, the Slater’s condition is satisfied; secondly, the optimal solutionset of the dual limit is singleton; thirdly, dynamically changing networktopologies satisfy some standard connectivity condition.A conference version of this manuscript was published in [26], and

an enlarged archived version of this paper is [25]. Main differences arethe following: i) by assuming that the optimal solution set of the duallimit is a singleton, and changing the update rule in the dual estimates,we are able to determine a global solution in contrast to an approximatesolution in [26]; ii) we present a simple criterion to check the new suffi-cient condition for nonconvex quadratic programs; iii) new simulationresults of our algorithm on a source localization example and a com-parison of its performance with existing algorithms are performed. Dueto space limitations, details of the technical proofs and simulations canbe found in [25].

II. PROBLEM FORMULATION AND PRELIMINARIES

Consider a networked multi-agent system where agents are labeledby . The multi-agent system operates in a syn-chronous way at time instants , and its topology will berepresented by a directed weighted graph ,for . Here, is the adjacency matrix,where the scalar is the weight assigned to the edgepointing from agent to agent , and is theset of edges with non-zero weights. The set of in-neighbors of agentat time is denoted by .Similarly, we define the set of out-neighbors of agent at time as

. We here make thefollowing assumptions on communication graphs:Assumption 2.1 (Non-Degeneracy): There exists a constant

such that , and , for , satisfies, for all .

Assumption 2.2 (Balanced Communication): It holds thatfor all and , and for

all and .Assumption 2.3 (Periodical Strong Connectivity): There is a

positive integer such that, for all , the directed graphis strongly connected.

The above network model is standard to characterize a networkedmulti-agent system, and has been widely used in the analysis of av-erage consensus algorithms; e.g. see [17], and distributed optimizationin [15], [27]. Recently, an algorithm is given in [9] which allows agentsto construct a balanced graph out of a non-balanced one under certainassumptions. The objective of the agents is to cooperatively solve thefollowing primal problem :

(1)

where is the global decision vector. The functionis only known to agent , continuous, and referred to as the objec-tive function of agent . The set , the state constraint set, iscompact. The function are continuous, and the in-equality is understood component-wise; i.e., , forall , and represents a global inequality constraint. Wewill denote and . Wewill assume that the set of feasible points is non-empty; i.e., .Since is compact and is closed, then we can deduce that iscompact. The continuity of follows from that of . In this way, theoptimal value of the problem is finite and , the set of primaloptimal solutionss, is non-empty. We will also assume the followingSlater’s condition holds:Assumption 2.4 (Slater’s Condition): There exists a vector

such that . Such is referred to as a Slater vector of theproblem .Remark 2.1: All the agents can agree upon a common Slater vectorthrough a maximum-consensus scheme. This can be easily imple-

mented as part of an initialization step, and thus the assumption thatthe Slater vector is known to all agents does not limit the applicabilityof our algorithm; see [25] for an algorithm solving this problem.In[27], inorder tosolvetheconvexcaseof theproblem (i.e.; andareconvexfunctionsand isaconvexset),wepropose twodistributedprimal-dual subgradient algorithms where primal (resp. dual) estimatesmove along subgradients (resp. supergradients) and are projected ontoconvex sets. The absence of convexity impedes the use of the algorithmsin [27] since, on the one hand, (primal) gradient-based algorithms areeasily trapped in local minima; on the other hand, projection maps maynot bewell-definedwhen (primal) state constraint sets are nonconvex. Inthesequel,wewill employLagrangiandualization, subgradientmethodsand average consensus schemes to design a distributed algorithmwhichcan find an approximate solution to the problem .Towards this end, we construct a directed cyclic graph

where . We assume that each agent has a uniquein-neighbor (and out-neighbor). The out-neighbor (resp. in-neighbor)of agent is denoted by (resp. ). With the graph , we willstudy the following approximate problem of problem :

(2)

1536 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 6, JUNE 2013

where , with a small positive scalar, and is the columnvector of ones. Problem (2) provides an approximation to , andwill be referred to as problem . In particular, the approximateproblem (2) reduces to the problem when . Its optimal valueand the set of optimal solutions will be denoted by and , respec-tively. Similarly to the problem , is finite and .Remark 2.2: The cyclic graph can be replaced by any strongly

connected graph . Given , each agent is endowed with two in-equality constraints: and , foreach out-neighbor . This set of inequalities implies that any feasiblesolution of problem satisfies the approximate con-sensus; i.e., . For simplicity, we will usethe cyclic graph , with a minimum number of constraints, as theinitial graph.

A. Dual Problems

Before introducing dual problems, let us denote by, , ,

and . The dual problemassociated with is given by

(3)

where , and .Here, the dual function is given as

, where is the Lagrangianfunction

We denote the dual optimal value of the problem by andthe set of dual optimal solutions by . We endow each agent withthe local Lagrangian function and the local dualfunction defined by

In the approximate problem , the introduction of, renders the and separable. As a result, the global

dual function can be decomposed into a simple sum of the local dualfunctions . More precisely, the following holds:

Notice that in the sum of , each forany appears in two terms: one is , and theother is . With this observation, we regroup theterms in the summation in terms of , and have the following:

(4)

Note that is not separable since depends onneighbor’s multipliers , .

B. Dual Solution Sets

The Slater’s condition ensures the boundedness of dual solution setsfor convex optimization; e.g., [10], [13]. We will shortly see that theSlater’s condition plays the same role in nonconvex optimization. Toachieve this, we define the function asfollows:

Let be a Slater vector for problem . Thenwith is a Slater vector of the problem . Similarly to (3)and (4) in [27], which employ Lemma 3.2 of [27], we have that for any

, it holds that

(5)

where . Let be zeroin (5). This leads to the upper bound on

(6)

where can be computed locally. We de-note

(7)

Since and are continuous and is compact, then is contin-uous; see Theorem 1.4.16 in [1]. Similarly, is continuous. Sinceis bounded, then .Remark 2.3: The requirement of exact agreement on in the

problem is slightly relaxed in the problem by introducing asmall . In this way, the global dual function is a sum of thelocal dual functions , as in (4); is non-empty and uniformlybounded. These two properties play important roles in the devise ofour subsequent algorithm.

C. Other Notation

Define the set-valued map as; i.e., given , the set are the so-

lutions to the following local optimization problem:

(8)

Here, is referred to as the marginal map of agent . Since is com-pact and are continuous, then in (8) for any . Inthe algorithm we will develop in next section, each agent is required toobtain one (globally) optimal solution and the optimal value the localoptimization problem (8) at each iterate. We assume that this can beeasily solved, and this is the case for problems of , or andbeing smooth (the extremum candidates are the critical points of theobjective function and isolated corners of the boundaries of the con-straint regions) or having some specific structure which allows the useof global optimization methods such as branch and bound algorithms.In the space , we define the distance between a point

to a set as ,and the Hausdorff distance between two sets as

.

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 6, JUNE 2013 1537

We denote by andwhere .

III. DADS ALGORITHM

In this section, we devise a distributed approximate dual subgradientalgorithm which aims to find a pair of primal-dual solutions to the ap-proximate problem .For each agent , let be the estimate of the primal solu-

tion to the approximate problem at timebe the estimate of the multiplier on the inequality constraint

(resp. )1 be the esti-mate of the multiplier associated with the collection of the localinequality constraints (resp. ),for all . We let ,for to be the collection of dual estimates of agent . Anddenote where

andare convex combinations of dual estimates of agent and its neighborsat time .At time , we associate each agent a supergradient vector

defined as , where, has components

, , and for, while the components of are given by:

, , and, for . For each agent , we define the set

for some where . Letto be the projection onto the set . It is easy to check that is

closed and convex, and thus the projection map is well-defined.The Distributed Approximate Dual Subgradient (DADS) Algorithm

is described in Table I.

Algorithm 1 The DADS Algorithm

Initialization: Initially, all the agents agree upon some in theapproximate problem . Each agent chooses a common Slatervector , computes and obtains through amax-consensus algorithm where is given in (7). After that, eachagent chooses initial states and .

Iteration: Every , each agent executes the following steps:1) For each , given , solve the local optimizationproblem (8), obtain a solution and the dualoptimal value .

2) For each , generate the dual estimate accordingto the following rule:

(9)

where the scalar is a step-size.3) Repeat for .

Remark 3.1: The DADS algorithm is an extension of the classicaldual algorithm, e.g., in [19] and [3] to the multi-agent setting and non-convex case. In the initialization of the DADS algorithm, the valueserves as an upper bound on . In Step 1, one solution inis needed, and it is unnecessary to compute the whole set .To assure primal convergence, we assume that dual estimates con-

verge to the set where each has a single optimal solution.

1We will use the superscript to indicate that and are estimatesof some global variables.

Definition 3.1 (Singleton Optimal Dual Solution Set): The set ofis the set of such that the set is a singleton,

where for each .The primal and dual estimates in the DADS algorithm converge to

a pair of primal-dual solutions to the approximate problem . Weformally state this in the following theorem:Theorem 3.1: (Convergence Properties of the DADS Algorithm):

Consider the problem and the corresponding approximate problemwith some . We let the non-degeneracy assumption 2.1, the

balanced communication assumption 2.2 and the periodic strong con-nectivity assumption 2.3 hold. In addition, suppose the Slater’s con-dition 2.4 holds for the problem . Consider the dual sequences of

, , and the primal sequence of of thedistributed approximate dual subgradient algorithm with sat-isfying , ,

.1) (Dual estimate convergence) There exists a dual solution

where and such that thefollowing holds for all

2) (Primal estimate convergence) If the dual solution satisfies, i.e. is a singleton for all , then there is

such that, for all

IV. DISCUSSION

Before outlining the technical proofs for Theorem 3.1, we wouldlike to make the following observations. First, our methodology is mo-tivated by the need of solving a nonconvex problem in a distributedway by a group of agents whose interactions change with time. Thisplaces a number of restrictions on the solutions that one can find. Time-varying interactions of anonymous agents can be currently solved viaagreement algorithms; however these are inherently convex operations,which does not work well in nonconvex settings. To overcome this, onecan resort to dualization. Admittedly, zero duality gap does not hold ingeneral for nonconvex problems. A possibility would be to resort tononlinear augmented Lagrangians, for which strong duality holds ina broad class of programs [5], [6], [22]. However, we find here an-other problem, as a distributed solution using agreement requires sepa-rability, as the one ensured by the linear Lagrangians we use here. Thus,we have looked for alternative assumptions that can be easier to checkand allow the dualization approach to work.More precisely, Theorem 3.1 shows that dual estimates always con-

verge to a dual optimal solution. The convergence of primal estimatesrequires an additional assumption that the dual limit has a single op-timal solution.We refer to this assumption as the singleton dual optimalsolution set (SD for short). The assumption may not be easy to checka priori, however it is of similar nature as existing algorithms for non-convex optimization. In [5] and [6], subgradient methods are definedin terms of (nonlinear) augmented Lagrangians, and it is shown thatevery accumulation point of the primal sequence is a primal solutionprovided that the dual function is required to be differentiable at thedual limit. An open question is how to resolve the above issues im-posed by the multi-agent setting with less stringent conditions on thenature of the nonconvex optimization problem.In the following, we study a class of nonconvex quadratic programs

for which a sufficient condition guarantees that the SD assumptionholds. Nonconvex quadratic programs hold great importance from both

1538 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 6, JUNE 2013

theoretic and practical aspects. In general, nonconvex quadratic pro-grams are NP-hard, and please refer to [18] for detailed discussion. Theaforementioned sufficient condition only requires checking the positivedefiniteness of a matrix.Consider the following nonconvex quadratic program:

(10)

where and are real and symmetricmatrices.The approximate problem of is given by

(11)

We introduce the dual multipliers as before. The local La-grangian function can be written as follows:

where the term independent of is dropped and is a linear functionof . The dual function and dual problem can be definedas before. Consider any dual optimal solution . If for all :

(P1) is positive definite;(P2) ;

then the SD assumption holds. The properties (P1) and (P2) are easyto verify in a distributed way once a dual solution is obtained. Wewould like to remark that (P1) is used in [8] to determine the uniqueglobal optimal solution via canonical duality when is absent.

V. CONVERGENCE ANALYSIS

This section outlines the analysis steps to prove Theorem 3.1. Pleaserefer to [25] for more details. Recall that is continuous and is com-pact. Then there are such that andfor all . We start our analysis from the computation of supergra-dients of .Lemma 5.1 (Supergradient Computation): If , then

is a supergradient ofat . That is, for any

(12)

A direct result of Lemma 5.1 is that the vectoris a supergradient of at

; i.e., the following supergradient inequality holds for any:

(13)

It can be seen that (9) of dual estimates in the DADS algorithm isa combination of a dual subgradient scheme and average consensusalgorithms. The following establishes that is Lipschitz continuouswith some Lipschitz constant .

Lemma 5.2 (Lipschitz Continuity of ): There is a constantsuch that for any , it holds that.In the DADS algorithm, the error induced by the projectionmap

is given by

A basic iterate relation of dual estimates in the DADS algorithm is thefollowing.Lemma 5.3 (Basic Iterate Relation): Under the assumptions in The-

orem 3.1, for any with for all ,we have for all

(14)

Asymptotic convergence of dual estimates is shown next.Lemma 5.4 (Dual Estimate Convergence): Under the assumptions

in Theorem 3.1, there exists such that, , and.

The remainder of the section is dedicated to characterizing the con-vergence properties of primal estimates.Lemma 5.5 (Properties ofMarginalMaps): The set-valuedmarginal

map is closed. In addition, it is upper semicontinuous at ;i.e., for any , there is such that for any ,it holds that .Lemma 5.6 (Primal Estimate Convergence): Under the assumptions

in Theorem 3.1, for each , it holds thatwhere .The main result of this technical note, Theorem 3.1, can be shown

next. In particular, we will show complementary slackness, primal fea-sibility of , and its primal optimality, respectively.Proof for Theorem 3.1:Claim 1: ,

and .Proof: Rearranging the terms related to in (14) leads to the fol-

lowing inequality holding for any withfor all :

(15)

IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 58, NO. 6, JUNE 2013 1539

Summing (15) over [0, ], and dividing by

(16)

We now proceed to show for each .Let , for and , in(16). Recall that is not summable but square summable, and

is uniformly bounded. Take , and then it followsfrom Lemma 5.1 in [27] that:

(17)

On the other hand, since , we have given thefact that is an upper bound of . Let where

. Then we could choose a sufficiently small andin (16) such that where is given in the definition ofand is given by: , for. Following the same lines toward (17), it gives that

. Hence, it holds that . Therest of the proof is analogous and thus omitted.The proofs of the following two claims can be found in [25].Claim 2: is a primal feasible solution to the approximate problem.

Claim 3: is a primal solution to the problem .

VI. CONCLUSION

We have studied a distributed dual algorithm for a class of multi-agent nonconvex optimization problems. The convergence of the algo-rithm has been proven under the assumptions that i) the Slater’s condi-tion holds; ii) the optimal solution set of the dual limit is singleton; iii)the network topologies are strongly connected over any given boundedperiod. An open question is how to address the shortcomings imposedby nonconvexity and multi-agent interactions settings.

REFERENCES[1] J. P. Aubin and H. Frankowska, Set-Valued Analysis. Boston, MA:

Birkhäuser, 1990.[2] D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Compu-

tation: Numerical Methods. Boston, MA: Athena Scientific, 1997.[3] D. P. Bertsekas, Convex Optimization Theory. Boston, MA: Anthena

Scietific, 2009.[4] F. Bullo, J. Cortés, and S. Martínez, Distributed Control of Robotic

Networks, ser. AppliedMathematics Series. Princeton, NJ: PrincetonUniv. Press, 2009 [Online]. Available: http://www.coordinationbook.info

[5] R. S. Burachik, “On primal convergence for augmented Lagrangianduality,” Optimization, vol. 60, no. 8, pp. 979–990, 2011.

[6] R. S. Burachik and C. Y. Kaya, “An update rule and a convergenceresult for a penalty function method,” J. Ind. Manag. Optim., vol. 3,no. 2, pp. 381–398, 2007.

[7] J. A. Fax and R. M. Murray, “Information flow and cooperative controlof vehicle formations,” IEEE Trans. Autom. Control, vol. 49, no. 9, pp.1465–1476, Sep. 2004.

[8] D. Y. Gao, N. Ruan, and H. Sherali, “Solutions and optimality criteriafor nonconvex constrained global optimization problems with connec-tions between canonical and Lagrangian duality,” J. Global Optim.,vol. 45, no. 3, pp. 474–497, 2009.

[9] B. Gharesifard and J. Cortés, “Distributed strategies for generatingweight-balanced and doubly stochastic digraphs,” Eur. J. Control, tobe published.

[10] J.-B. Hiriart-Urruty and C. Lemaréchal, Convex Analysis and Min-imization Algorithms: Part 1: Fundamentals. New York: Springer,1996.

[11] A. Jadbabaie, J. Lin, and A. S. Morse, “Coordination of groups of mo-bile autonomous agents using nearest neighbor rules,” IEEE Trans.Autom. Control, vol. 48, no. 6, pp. 988–1001, Jun. 2003.

[12] F. P. Kelly, A. Maulloo, and D. Tan, “Rate control in communicationnetworks: Shadow prices, proportional fairness and stability,” J. Oper.Res. Soc., vol. 49, no. 3, pp. 237–252, 1998.

[13] A. Nedic and A. Ozdaglar, “Approximate primal solutions and rateanalysis for dual subgradient methods,” SIAM J. Optim., vol. 19, no.4, pp. 1757–1780, 2009.

[14] A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multi-agent optimization,” IEEE Trans. Autom. Control, vol. 54, no. 1, pp.48–61, 2009.

[15] A. Nedic, A. Ozdaglar, and P. A. Parrilo, “Constrained consensus andoptimization in multi-agent networks,” IEEE Trans. Autom. Control,vol. 55, no. 4, pp. 922–938, Apr. 2010.

[16] R. D. Nowak, “Distributed EM algorithms for density estimation andclustering in sensor networks,” IEEE Trans. Signal Processing, vol. 51,pp. 2245–2253, 2003.

[17] R.Olfati-Saber andR.M.Murray, “Consensus problems in networks ofagents with switching topology and time-delays,” IEEE Trans. Autom.Control, vol. 49, no. 9, pp. 1520–1533, Sep. 2004.

[18] P. M. Pardalos and S. A. Vavasis, “Quadratic programming with onenegative eigenvalue is NP-hard,” J. Global Optim., vol. 1, no. 1, pp.15–22, 1991.

[19] B. T. Polyak, “A general method for solving extremum problems,” So-viet Math. Doklady, vol. 3, no. 8, pp. 593–597, Aug. 1967.

[20] M. G. Rabbat and R. D. Nowak, “Decentralized source localization andtracking,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing,May 2004, pp. 921–924.

[21] W. Ren and R. W. Beard, Distributed Consensus in Multi-vehicle Co-operative Control, ser. Communications and Control Engineering.New York: Springer, 2008.

[22] R. T. Rockafellar and R. J.-B. Wets, Variational Analysis. New York:Springer, 1998.

[23] S. Sundhar Ram, A. Nedic, and V. V. Veeravalli, “Distributed andrecursive parameter estimation in parametrized linear state-spacemodels,” IEEE Trans. Autom. Control, vol. 55, no. 2, pp. 488–492,Feb. 2010.

[24] L. Xiao, S. Boyd, and S. Lall, “A scheme for robust distributed sensorfusion based on average consensus,” in Proc. Symp. Inform. ProcessingSensor Networks, Los Angeles, CA, Apr. 2005, pp. 63–70.

[25] M. Zhu and Martínez, “An approximate dual subgradient algorithm formulti-agent non-convex optimization,” IEEE Trans. Autom. Control2013 [Online]. Available: http://arxiv.org/abs/1010.2732

[26] M. Zhu and S. Martínez, “An approximate dual subgradient algorithmfor multi-agent non-convex optimization,” inProc. IEEE Int. Conf. De-cision Control, Atlanta, GA, Dec. 2010, pp. 7487–7492.

[27] M. Zhu and S. Martínez, “On distributed convex optimization underinequality and equality constraints,” IEEE Trans. Autom. Control, vol.57, pp. 151–164, 2012.