particle swarm optimization by dr. shubhajit roy chowdhury centre for vlsi and embedded systems...
Post on 17-Jan-2016
229 Views
Preview:
TRANSCRIPT
Particle Swarm Particle Swarm OptimizationOptimization
bybyDr. Shubhajit Roy Dr. Shubhajit Roy
ChowdhuryChowdhuryCentre for VLSI and Embedded Centre for VLSI and Embedded
Systems Technology, IIIT Systems Technology, IIIT HyderabadHyderabad
What is Optimization?What is Optimization?
Optimization can be defined as Optimization can be defined as the art the art of obtaining best policies to satisfy of obtaining best policies to satisfy certain objectives, at the same time certain objectives, at the same time satisfying fixed requirements.- satisfying fixed requirements.- GotfriedGotfried
Unconstrained OptimizationUnconstrained Optimization
Example: Example: Maximize Z,Maximize Z,
where Z= xwhere Z= x1122 x x22 –x –x22
22xx1 1 -2 x-2 x11 x x22
Unconstrained & Constrained Unconstrained & Constrained OptimizationOptimization
Unconstrained Approach:Unconstrained Approach: Set Set δδZ/ Z/ δδxx11= 0, & = 0, & δδZ/ Z/ δδxx22= 0.= 0.
Constrained OptimizationConstrained Optimization Example:Example: Design a box with maximum Design a box with maximum
volume and minimum surface area.volume and minimum surface area.
Constrained Optimization Constrained Optimization (Contd.)(Contd.)
Approach:Approach: LetLet L(x, y, z)= xyz – L(x, y, z)= xyz – λλ [2(xy + yz [2(xy + yz
+zx)]+zx)] volume Lag. Multiplier sur. volume Lag. Multiplier sur.
AreaArea Set Set δδL/ L/ δδx =0, x =0, δδL/ L/ δδy =0, y =0, δδL/ L/ δδz =0, z =0, and and δδL/ L/ δ δ λλ =0, =0, Solution:Solution: x= y= zx= y= z In most engineering system, the solution returns In most engineering system, the solution returns
numerical values of the variablesnumerical values of the variables..
Numerical Approach to Numerical Approach to OptimizationOptimization
Steepest DescentSteepest Descent Problem:Problem: Minimize f(x Minimize f(x11, x, x22, …, x, …, xnn))
Approach:Approach: x x11 := x := x11 - - δδf/ f/ δδxx11
xx22 := x := x2 2 - - δδf/ f/ δδxx22
…… ………… ……. …….. …… . …….. ……
xxnn:= x:= xnn - - δδf/ f/ δδxxnn
Loop through until Loop through until δδf/ f/ δδxxii, for all i, are zero., for all i, are zero.
But what about these multi-modal, noisy and even
discontinuous functions?
Gradient based methods get trapped in a local minima or the Function itself may be non differentiable.
How a single agent can find global optima by following gradient descent?
Way Out: Multi-Agent Optimization Way Out: Multi-Agent Optimization in Continuous Spacein Continuous Space
RandomlyInitialized Agents
Agents
Most Agents are nearGlobal Optima
After Convergence
Particle Swarm Particle Swarm Optimization Optimization
(PSO)(PSO)
(Kennedy and Eberhart, (Kennedy and Eberhart, 1995)1995)
Principles of Particle Swarm Principles of Particle Swarm OptimizationOptimization
Principles of Particle Swarm Principles of Particle Swarm OptimizationOptimization
Current direction
Direction of local maximum
global maximum
Resulting direction of motion
The PSO AlgorithmThe PSO Algorithm
1.1. Initialize position and velocity of n Initialize position and velocity of n particles randomly.particles randomly.
2.2. Evaluate fitness of all the particles.Evaluate fitness of all the particles.3.3. Adapt velocity of each particle by Adapt velocity of each particle by
taking into consideration of its current taking into consideration of its current velocity, Global best and local best velocity, Global best and local best positions, so far experienced.positions, so far experienced.
4.4. Evaluate new position from current Evaluate new position from current position and velocity.position and velocity.
5.5. Update Local best and global best Update Local best and global best positions based on the fitness of the positions based on the fitness of the new positions.new positions.
6.6. Repeat from (2) until most of the Repeat from (2) until most of the particles cease motions.particles cease motions.
PSO: Starting Situation
Randomly Scattered Particles over the fitness landscape and their randomly oriented velocities
All Parti
cles in
A close vicin
ity o
f the
Global o
ptimum The best Particle
Conquering the Peak
Situation after a few iterations
Best Position found so far By the particle
Globally Best positionFound by the swarm
Definitions
Best Position found By the agent so far (Plb)
Globally best position found so far (Pgb).Current Position
Vi(t)
Resultant VelocityVi(t+1)
Particle Swarm Optimization Kennedy & Eberhart(1995)
Vi(t+1)=φ.Vi(t)+C1.rand(0,1).(Plb-Xi(t))+C2.rand(0,1).(Pgb-Xi(t)) Xi(t+1)=Xi(t)+Vi(t+1)
C1.
rand
(0,1
).(Pl
b-Xi(t
))
C2.rand(0,1).(Pgb-Xi(t))
Example: f( x) = x(x-8)
Consider a small swarm of particles for the above single dimensional function:
Initial Position and velocities of the particles at time t=0: Randomly initialized In the range (-10, 10)
Particle number
Position x(0) at t = 0
Velocity v at t = 0
f (x)
1 7 3 -7
2 -2 5 20
3 9 6 9
4 -6 -4 84
So, the fittest particle is particle 1 and we set Pgb = -7and Plb =X1
Initial Distribution of the Particles over the fitness landscape
Change in position of the particles in next iteration:
For this small scale PSO problem, we set C1 = C2 = 2.0,ω = 0.5
Particle 1>V1(1) = 0.5*3 + 2*0.6*(7 – 7) + 2*0.4*(7 – 7) = 1.5X1(1) = 7+1.5 = 8.5 Fitness f (X(1)) =4.25
Vi(t+1)=φ.Vi(t)+C1.rand(0,1).(Plb-Xi(t))+C2.rand(0,1).(Pgb-Xi(t)) Xi(t+1)=Xi(t)+Vi(t+1)
Particle 2>V2(1) = 0.5*5 + 2*0.3*(-2 + 2 ) + 2*0.4*(7 – (-2)) = 6.5X2(1) = -2+6.5 = 3.5 Fitness f (X(1)) =-9.75
Particle 3>V3(1) = 0.5*6 + 2*0.8*(9 - 9 ) + 2*0.95*(7 – 9) = -0.8X3(1) = 6 – (-0.8) = 6.8 Fitness f (X(1)) =-8.16
Particle 4>V4(1) = 0.5*(-4) + 2*0.38*(-6 + 6 ) + 2*0.45*(7 – (-6)) = 9.7X4(1) = -6 + 9.3 = 3.7 Fitness f (X(1)) =-15.91
Here we go for the next iteration:
Particle number
Position at t = 1 Velocity at t = 1 f (x) Plb for
t = 2
Pgb for
t = 2
1 8.5 (at t = 0, 7) 1.5 (at t = 0, 3) 4.25 (att = 0, -7)
7 3.7
2 6.5 (-2) 3.5 (5) -9.75 (20) 6.5 3.7
3 6.8 (9) -0.8 (6) -8.16 (9) 6.8 3.7
4 3.7 (-6) 9.7 (-4) -15.91(84) 3.7 3.7
Distribution of the Particles over the fitness landscape at t = 1
Distribution of the Particles over the fitness landscape at t = 5Best particle at t = 5Pgb = 3.95f(Pgb) = -15.99
Optimization by PSO: Egg crate Function Minimize )sin(sin25)( 2
2
1
22
2
2
1 xxxxxf
X1 X2
f(x)Known global minima at [0,0] and optimum function value
0
Eggcrate Function Optimization by PSOPosition of the particles on a 2D parameter Space at different
instances
To know moreTo know moreTHE site:Particle Swarm Central, http://www.particleswarm.net
Clerc M., Kennedy J., "The Particle Swarm-Explosion, Stability, and Convergence in a Multidimensional Complex Space", IEEE Transaction on Evolutionary Computation, 2002,vol. 6, p. 58-73.
Problem with PSOProblem with PSO
If the fitness function is too wavy If the fitness function is too wavy and irregular, the particles will get and irregular, the particles will get trapped in some local minimum.trapped in some local minimum.
The result is that we get a sub The result is that we get a sub optimal solutionoptimal solution
Perceptive Particle Swarm Perceptive Particle Swarm OptimizationOptimization
The particles fly around in (n+1) dimensional search space The particles fly around in (n+1) dimensional search space for n dimensional optimization problemfor n dimensional optimization problem
The particles fly over a physical fitness landscape The particles fly over a physical fitness landscape observing its crests and trough from a far observing its crests and trough from a far
Particles observe the search space within their perception Particles observe the search space within their perception ranges by sampling a fixed number of directions to observe ranges by sampling a fixed number of directions to observe and sampling a finite number of points along those and sampling a finite number of points along those directions. directions.
The particles attempt to observe the search space for The particles attempt to observe the search space for landscape at several sampled distances from its position, landscape at several sampled distances from its position, in each direction. in each direction.
If the sampled point is within the landscape, the particle If the sampled point is within the landscape, the particle perceives the height of the landscape at that point. perceives the height of the landscape at that point.
The particles can observe neighboring particles in their The particles can observe neighboring particles in their perception range. perception range.
The particle randomly chooses the neighboring particles The particle randomly chooses the neighboring particles which will influence the particle to move towards them. which will influence the particle to move towards them.
The position of the chosen neighbor will be used as the The position of the chosen neighbor will be used as the local best position of the particle.local best position of the particle.
PPSO IllustrationPPSO Illustration
Adaptive Perceptive Adaptive Perceptive Particle Swarm Particle Swarm
Optimization (APPSO)Optimization (APPSO) In APPSO algorithm, if the local best In APPSO algorithm, if the local best
position of the particle at the current position of the particle at the current iteration does improve the performance iteration does improve the performance of the particle, then one or more of the of the particle, then one or more of the following things are done:following things are done:
(1)(1) Spacing between the sample points Spacing between the sample points along any direction within the along any direction within the perception radius is minimized perception radius is minimized
(2)(2) number of sampling directions is number of sampling directions is increased increased
(3)(3) Perception radius is minimizedPerception radius is minimized
APPSO IllustrationAPPSO Illustration
Different types of APPSODifferent types of APPSOAlgorithm Perception radius No. of directions No. of sample points
APPSO1 (PPSO) Fixed Fixed Fixed
APPSO2 Fixed Fixed Variable
APPSO3 Fixed Variable Fixed
APPSO4 Fixed Variable Variable
APPSO5 Variable Fixed Fixed
APPSO6 Variable Fixed Variable
APPSO7 Variable Variable Fixed
APPSO8 Variable Variable Variable
THANK YOUTHANK YOU
top related