a new hybrid particle swarm optimization with variable neighborhood search for solving unconstrained...

A new Hybrid Particle Swarm Optimization with Variable Neighborhood Search for Solving

Unconstrained Global Optimization Problems

The 5th International Conference on Innovations in Bio-Inspired Computing and Applications. June 23-25, 2014

Ahmed Fouad Ali

http://www.egyptscience.net

Scientific Research Group in Egyptwww.egyptscience.net

http://www.egyptscience.net/

Overview

Motivations Problem definition Introduction

Particle swarm optimization (PSO)Variable neighborhood search (VNS)

A hybrid particle swarm optimization with variable neighborhood search algorithm (HPSOVNS)Performance analysis

Numerical experiments Conclusion and future works


Motivations


• Many real-world problems in ( science, engineering and economics) may be formulated as optimization problems.

• In the past decades, many researches have been attracted to apply their works to solve the global optimization problems.

• The main key feature of designing any meta-heuristics algorithm is its capability of performing a wide exploration and a deep exploitation.

Problem definition


• We address the continuous non-linear function optimization problem in the form:

where f(x) is a nonlinear function, x = (x1, . . . , xn) is a vector of continuous and bounded variables, x, l, u ∈ ℜn.

Introduction


• In this work, a Particle swarm optimization PSO algorithm is combined with variable neighborhood search (VNS) algorithm in order to solve unconstrained global optimization problems.

• The proposed algorithm is called a hybrid particle swarm optimization with a variable neighborhood search algorithm (HPSOVNS).

• PSO with global variant represents the exploration process , while the VNS represents the exploitation process in the proposed algorithm

An exploitation process

An exploration process

Introduction Particle swarm optimization (PSO)


• Particle swarm optimization (PSO) is one of the most popular swarm intelligence method.

• The initial concept of PSO is to simulate the graceful and unpredictable choreography of a bird flock.

• In PSO, a swarm consists of a set particles, each particle represents a solution.

• The position of each particle is changed according to its own experience and its neighbors.

Introduction Particle swarm optimization (PSO) Cont.


• The initial swarm is generated randomly, each particle has position xi.

• At each iteration, the performance of each particle is evaluated by using the objective function.

• The performance of each particle is compared with its local best value pbesti and global best particle gbest.

Introduction Particle swarm optimization (PSO) Cont.


• The position of each particle xi is changed as shown in the following equation.

Where vi is a particle velocity, the velocity of each particle is changed as the following equation

c1, c2 are positive acceleration constant, r1, r2 are random vectors where r1 and r2 ϵ [0,1], w is the inertia weight .

(2)

(3)

Introduction Variable neighborhood search


• Variable neighborhood search (VNS) method has been proposed by P. Hansen and N. Mladenovic.

• In VNS method, a set of predefined neighborhoods are explored to provide a better solution.

• The main steps of VNS algorithm are shown the following Algorithm

Introduction Variable neighborhood search algorithm


Introduction Variable neighborhood search algorithm Cont.


• A set of neighborhood structure Nk are defined where k = 1, 2,…, n.

• At each iteration, an initial solution x is generated randomly. A random neighbor solution x` is generated in the current neighborhood Nk.

• The local search procedure is applied to the solution x` to generate the solution x``.

• If x`` is better than x then the solution x`` becomes the new current solution and the search starts from the current solution.

• Otherwise, the search moves to the next neighborhood Nk+1, generates a new solution in this neighborhood and try to improve it.

• These operations are repeated until a termination criteria satisfied.

A hybrid particle swarm optimization with variable neighborhood search algorithm (HPSOVNS) Cont.


• The proposed algorithm is called hybrid particle swarm optimization with variable neighborhood search algorithm (HPSOVNS).

• The main key feature of HPSOVNS is to combine particle swarm optimization with its capability of performing wide exploration and deep exploitation, with the variable neighborhood search with its capability of performing deep exploitation on the best obtained solution at each iteration.

• The formal detailed description of HPSOVNS is given in Algorithm 2 as fellow



• HPSOVNS starts with an initial swarm contains SS particles (solutions).

• HPSOVNS evaluates the solutions, the best global solution and the best local solutions are assigned.

• At generation t, the particles are updating there positions and velocities as shown in Equations 2, 3.

• The particles in the swarm are evaluated after updating their positions and velocities.

• Then, the variable neighborhood search is applied on the best global solution found at each generation in order to refine the best solution found so far.

• This scenario is repeated until the termination criteria are satisfied

Performance analysis


• In this work, nine benchmark functions f1- f9 with different properties have

been used to test the performance of HPSOVNS algorithm.

• The proposed HPSOVNS algorithm was programmed in MATLAB.

• The test function and their properties are reported in the following tables.

Performance analysis (test functions)


Performance analysis (test functions properties)


Performance analysis (HPSOVNS parameter setting)


Performance analysis (The general performance of HPSOVNS)


Numerical experiments


• HPSOVNS algorithm has been compared with the following four relevant particle swarm optimization algorithms:-

RWMPSOg (Random walk with direction exploitation and particle swarm optimization algorithm with global variants).

RWMPSOl (Random walk with direction exploitation and particle swarm optimization algorithm with local variants ).

PSOg (Standard particle swarm optimization with global variants).

PSOl (Standard particle swarm optimization with local variants).

Numerical experiments


• All results are averaged over 50 independent runs.

• The algorithm is terminated when it reach to the assigned error goal of each function.

• The best mean values are marked in bold face, and the standard deviation parameters are reported in parentheses.

• The experimental results for all comparative algorithms are taken from their original papers.

Numerical experiments Cont.


Conclusion and future works


• A new hybrid algorithm has been proposed in this paper in order to solve the unconstrained global optimization problems.

• The proposed algorithm is called a hybrid particle swarm optimization with a variable neighborhood search algorithm (HPSOVNS).

• The algorithm aims to combine the particle swarm optimization algorithm with its capability of exploring the search space and the variable neighborhood algorithm with its capability of exploiting the best found solution at each iteration.

Conclusion and future works


• In a comparison with four relevant PSO algorithms for several benchmark test functions, the numerical experiments verify that HPSOVNS presents significantly better results and faster than the other algorithms, in most of test functions.

• In the future work, we will apply the proposed algorithm with different optimization problems for example constrained, minimax, integer programming optimization problems.

Thanks and Acknowledgement

Authors: Ahmed Fouad Ali, Aboul Ella Hassanien, Vaclav Snasel and Mohamed F. Tolba

a new hybrid particle swarm optimization with variable neighborhood search for solving unconstrained...

Engineering

international conference

bioinspired computing

particle velocity

particle xi

global best particle

initial swarm

proposed algorithm

following algorithm