Particle Swarm Optimization †

Spencer Vogel

† This presentation contains cheesy graphics and animations and they will be awesome

Particle Swarm Basics

✘ Each particle is trying to find the global optimum

✘ Each particle is moving✘ Each particle remembers where it’s

local optima was

Basic Idea

✘ Each particle in the swarm cooperates with all of the other particles

Each particle has a neighborhood associated with it

Neighborhoods

Neighborhoods

Social

Geographical

✘ Each particle in the swarm cooperates with all of the other particles

Each particle has a neighborhood associated with it

Each particle knows the fitness of all other particles in it’s neighborhood

҂ The best position from it’s neighborhood is used to adjust the particle’s velocity

Neighborhoods

✘ As each particle has to move, it has to move to a new position at each time step

It does this by adjusting it’s velocity It’s velocity is based off a random weight of:

҂ It’s current velocity҂ A random portion in the direction of it’s

personal optimal fitness҂ A random portion of the direction of the

neighborhood optimal fitness

Particle Action

Swarm Dynamics

Current MotionInfluence

SwarmInfluence

Particle MemoryInfluenceResulting

Vector

𝑥𝑖𝑘

𝑥𝑖𝑘+1

𝑝𝑘𝑔

𝑝𝑖

Projected Motion

Common Test Functions

Griewank RastriginRosenbrockSinenvsin

Parallelization

✘ Distributes all data to be processed amongst all processors

✘ Allows individual processors to complete all tasks for one set of data

✘ Applied widely to genetic algorithms

Data Parallelism

✘ All particles are sent to the environment and the algorithm waits for all analysis to complete before continuing

✘ Usually results in poor efficiency as you end up waiting for some tasks to complete before updating the particles

Synchronous

Synchronous

Evaluate Current Fitness

Check Convergence

Update Particle Position

Update Particle Velocity

Report Results

Evaluate Current Fitness

Check Convergence

Update Particle Position

Update Particle Velocity

Report Results

Evaluate Current Fitness

Check Convergence

Update Particle Position

Update Particle Velocity

Report Results

Evaluate Current State

✘ Separates updates actions for each point and the swarm

✘ Updates point actions after each point is analyzed, and swarm actions at the end of each iteration

Asynchronous

Asynchronous

Evaluate Current Fitness

Check Convergence

Update Particle Position

Update Particle Velocity

Report Results

Evaluate Current Fitness

Check Convergence

Update Particle Position

Update Particle Velocity

Report Results

Evaluate Current Fitness

Check Convergence

Update Particle Position

Update Particle Velocity

Report Results

Evaluate Current State

✘ Master Initialize all optimization parameters, positions,

and velocities Holds a queue of particles for slave processors

to evaluate Updates global positions and velocities Sends the position of the next particle in the

que to an available slave processor Receives cost function values from slave

processors Checks convergence

Asynchronous master/slave paradigm

✘ Slave Receives particle position from master

processor Evaluates the fitness function at the given

particle Updates particle position Updates particle velocity Sends a cost function to the master

processor

Asynchronous master/slave paradigm

Asynchronous vs Synchronous

(a) Asynchronous Implementation (b) Synchronous Implementation

✘ H(x) represent different fitness functions

✘ Values are the mean (standard deviation) of function evaluations required to reach the final solution

Performance Comparison

Efficiency Comparison

Efficiency Comparison

Thread communication

Loosely correlated parameters

Strongly correlated paramete

rs

Thread Communication

Unknown paramete

rcorrelatio

n

Performance Comparison

Communication style 1 with Rosenbrok

Communication style 1 with Rastrigin

Communication style 2 with Griewank

Communication style 3

My Project

✘ Written in C++ using OpenGL and boost

Uses a combination of glew and freeglut

✘ Allows variable initialization via a configuration file

✘ Particle’s “best fitness” value decreases over time

✘ Provides visual feedback of particles in the search space

My Implementation

✘ Parameter tuning✘ Getting social neighborhoods to set up

correctly✘ The swarm currently has moments of

“convergence” followed by moments of “oscillation”

Bug not currently fixed

Challenges

✘ Fix oscillation issue✘ Allow varying sizes of neighborhoods✘ Allow a non integer value for

✘ Implement different fitness functions to observe change of convergence

✘ Allow for live parameter tuning✘ Unlock FPS

Things to still implement

✘ All points seek out the green dot (user controllable via mouse)

✘ 3 axis search✘ Individual particle “depth” represented

by red – purple – blue dots Red represents + z axis Blue represents – z-axis Purple shades represent a neutral z axis

Live Demo

Questions?

✘Chu, Shu-Chuan, John F. Roddick, and Jeng-Shyang Pan. "A Parallel Particle Swarm Optimization Algorithm With Communication Strategies." Harbin Institute of Technology. 13 Sept. 2002. Web. 15 Oct. 2015.

✘Venter, Gerhard, and Jaroslaw Sobieszczanski-Sobiesk. "A Parallel Particle Swarm Optimization Algorithm Accelerated by Asynchronous Evaluations." EngOpt. 30 May 2005. Web. 10 Nov. 2015.

✘Pinto, Andry, Hugo Alves, Inês Domingues, Luís Rocha, and Susana Cruz. "The Particle Swarm Optimization Algorithm." University of Porto. 2011. Web. 15 Oct. 2015

✘"Particle Swarm Optimization Demo." YouTube. YouTube, 8 Feb. 2012. Web. 15 Oct. 2015.

References