online distributed optimization via dual averaging · 2017. 7. 29. · online distributed...
TRANSCRIPT
-
Online Distributed Optimizationvia Dual Averaging
Saghar Hosseini, Airlie Chapman and Mehran Mesbahi
Robotics, Aerospace, and Information Networks (RAIN) Lab
University of Washington
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 1 / 13
-
Motivation: Distributed Sensor Networks
Forest temperature detection Collect atmospheric data
Given accurate observation models (cost functions) and convergence time(offline) the problem is traditionally solved by distributed optimization
What if the observation models are largely uncertain and solutions arerequired in real-time?
... Online Distributed Optimization
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 2 / 13
-
Outline
Problem Statement
Previous Work
Online Distributed OptimizationAlgorithm
Main Results
Application: Estimation in aDistributed Sensor Network
Conclusion
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 3 / 13
-
Problem Statement
Problem: Minimize a global cost over a network of n agents:
ft(x) =1
n
n
∑i=1
ft,i (x) subject to x ∈ χ
Each ft,i (xi (t)) : Rd → R is a convex cost on agent i ’s x at time t, xi (t), andevolves over time in an unpredictable manner
Easily projectable constraint set χGraph G = (V ,E ) represent the communication constraints withV = {1,2, ...n} agents
How do we quantify performance over time for an agent’s choices of xi (t)?
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 4 / 13
-
Regret Definition
The regret is the difference between the cost of the sequence of decisions {xi (t)}generated by the algorithm and the performance of the best fixed decision inhindsight x∗
The regret due to agent i ’s action
RT (x∗,xi ) =
T
∑t=1
(ft(xi (t))− ft(x∗)) =T
∑t=1
n
∑i=1
(ft,i (xi (t))− ft,i (x∗))
Online Algorithm’s Objective:
Sublinear RT orRT/T → 0,
i.e., “on average” (xi (1),xi (2), . . . ,xi (T )) performs as well as (x∗,x∗, . . . ,x∗)
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 5 / 13
-
Previous Work
Online Distributed Gradient Descent Method
Yan et. al (2010): Distributed Autonomous Online Learning: Regrets andIntrinsic Privacy-Preserving Properties
Directed weighted graphsStrong convex cost functions : Regret = O(log(T ))Convex functions: Regret = O(
√T )
The effect of new information is diminishing over time
Distributed Dual Averaging Method
Duchi et. al (2012): Dual Averaging for Distributed Optimization:Convergence Analysis and Network Scaling
Offline problemEffect of different types of graph on convergenceNetwork and cost uncertainties
Raginsky et. al (2011): Decentralized Online Convex Programming withLocal Information
Chain graphs with radius of neighborhood r
Regret = O(√T ) for a growing graph (r ≥ (3log(T )+log(2))
log(2∗T 3/2/(2T 3/2−1)) )
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 6 / 13
-
Online Distributed Optimization Algorithm
Communication matrix (doubly stochastic): P = [Pj ,i ]
Non-increasing sequence of positive functions: α(t)Proximal function: ψ(x) : χ → R, strongly convex,ψ ≥ 0, and ψ(0) = 0Projection function:
Πψχ (z(t),α(t)) = arg min
x(t)∈χ
{〈z(t),x〉+ 1
α(t)ψ(x)
}
Online Distributed Dual Averaging (ODD) Algorithm
For t = 1 to T , and each agent i , provided a subgradient gi (t) ∈ ∂ ft,i (xi (t))
zi (t + 1) = ∑j∈N(i)
Pj ,izj(t) +gi (t) (Dual update)
xi (t + 1) = Πψχ (zi (t + 1),α(t)) (Primal Projection)
x̂i (t + 1) =1
t + 1
t+1
∑s=1
xi (s) (Optional: running average)
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 7 / 13
-
Main Results
Theorem
Given the ψ(x∗)≤ R2 and α(t) = k/√t,
supfT∈F
RT (x∗,xi ) =O
((kL2 +
R2
k+ 2kL2
(3√n
1−σ2(P)+ 6
))√T
),
where σ2(P) is the second largest singular value of P, n is the number of nodes,and fi ’s are L-Lipschitz
For P = I − 1ε diag(v)L(G) is doubly stochastic whereG is strongly connectedvTL(G) = 0 with positive vector v = [v1,v2, . . . ,vn]Tε ∈ (maxi∈V (vidi ) ,∞), where di is the in-degree of G
For special P, 1−σ2(P) ∝ λ2(G) a well known connectivity measure forundirected graphs
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 8 / 13
-
Application: Estimation in a Distributed Sensor Network
Sensors are estimating a random vector θ ∈ χ ={
θ ∈ Rd |‖θ‖2 ≤ θmax}
zt,i (θ) : Rd → Rpi is the observation vector for the ith sensor observing θ attime t
The sensor is modeled as hi (θ) = Hiθ where Hi ∈ Rpi×d is the observationmatrix of sensor i , and ‖Hi‖1 ≤ hmax, for all sensors i
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 9 / 13
-
Application: Estimation in a Distributed Sensor Network
Goal: Find the argument θ that minimizes thecost function
ft(θ) =1
n
n
∑i=1
ft,i (θ)
where
ft,i (θ) =1
2‖zt,i −Hiθ‖22
∂ ft,i (θ) = HTi (zt,i −Hiθ)
Best fixed strategy: The centralized optimalin hindsight is
θ ∗ =1
T
T
∑t=1
(n
∑i=1
HTi Hi
)−1(n
∑i=1
HTi zt,i
)
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 10 / 13
-
Simulation Run
100 sensor nodes distributed across map measuring a collection of local cells
Best Fixed in Hindsight θ ∗ ODD Algorithm θi (t)
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 11 / 13
temp_profile_sensor_50.aviMedia File (video/avi)
-
Results
100
101
102
103
104
10−2
10−1
100
101
102
103
T
RT(x
∗,x
1)
SimulationBounds
200 400 600 800 1000 1200 1400 1600 1800 2000
100
101
RT(x
∗ ,x1)/√T
T
PathDirected CycleRandom TreeRandomRandom regular
Graph σ2(P)Path 0.9993
Directed Cycle 0.9990Random Tree 0.9954
Erdos-Renyi, p = 0.08 0.8169Random k-regular, k = 6 0.5610
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 12 / 13
-
Conclusion
Extended dual averaging method to an distributed online formulation withO(√T ) regret
Future Work:
Investigate favorable graph characteristics for the online framework improvingthe regret bound
Apply online approach to traditionalproblems in multi-agent networks
Distributed estimation in adversarialenvironments, in the presence ofmistrust and jammingEnergy aware sensingOnline distributed energymanagement/pricing
Hosseini, Chapman and Mesbahi (Robotics, Aerospace, and Information Networks (RAIN) Lab)Online Distributed Optimization via Dual Averaging University of Washington 13 / 13