GRASP

Gre

edy R

andom

ized A

daptive

Searc

h P

roce

dure

Type o

f pro

blem

s

�Com

bin

ato

rial optim

ization p

roblem

: )(

)(

, :

Pro

ble

mon

Min

imis

ati

2:fu

nct

ion

O

bje

ctiv

e

2

solu

tions

fe

asib

le

of

Subse

t

},....

2,1{

ense

mble

F

init

e

**

Sf

Sf

FS

f

FnE

E

E

≤∈

ℜ→

⊆

=

Multi st

art m

eta

heurist

ic

�-C

reate

multip

le initial so

lutions (c

onst

ruct

ion)

�-R

epair solu

tion if it is in

feasible (re

pair)

�-R

un a

loca

l se

arc

h in its

neig

hborh

ood (lo

cal

searc

h)

�-Ite

rate

over th

ese

thre

e ste

ps until st

oppin

g

conditio

n is m

et

Histo

ry

�Conce

pt was fo

rmaliz

ed b

y F

eo a

nd

Rese

nde.

�Article p

ublis

hed in 1

989

GRASP o

verv

iew

�M

ost

of th

e litera

ture

is base

on the

const

ruct

ion p

hase

and im

pro

vem

ents

to

the c

onst

ruct

ion p

hase

�GRASP c

an b

e u

sed w

ith a

ny

com

bin

ation o

f co

nst

ruct

ion a

lgorith

ms

and loca

l se

arc

h a

lgorith

ms

Plan

�Const

ruct

ion p

hase

�Im

pro

vem

ents

to the c

onst

ruct

ion

phase

�GRASP e

xte

nsions:

�Path

relin

kin

g

�Overv

iew o

f a few o

thers

�Para

llel GRASP

Const

ruct

ion P

hase

�In

itial so

lutions are

built

ite

ratively b

y

addin

g e

lem

ents

.

�At each

sta

ge o

f th

e initial so

lution

const

ruct

ion, a R

est

rict

ed Candid

ate

List (R

CL) is c

onst

ruct

ed.

Const

ruct

ion c

ontinued

�The R

CL c

onta

ins an o

rdering o

f th

e

best

next elem

ent to

add to

the so

lution.

�The n

ext elem

ent is c

hose

n fro

m a

trunca

tion o

f th

is list

�Itera

te u

ntil we h

ave a

n initial so

lution

Const

ruct

ion a

lgorith

mp

roced

ure

Gre

ed

y R

ando

miz

ed

Con

str

uctio

n(S

eed

)

1

So

lution←

/0;

2

Init

iali

ze t

he

set

of

candid

ate

elem

ents

;

3

Eval

uat

e th

e in

crem

enta

l co

sts

of

the

candid

ate

elem

ents

;

4

wh

ile

ther

e ex

ists

at

leas

t one

candid

ate

elem

ent

do

5

Buil

d t

he

rest

rict

ed c

andid

ate

list

(R

CL

);

6

Sel

ect

an e

lem

ent

s fr

om

the

RC

L a

t ra

ndom

;

7S

olu

tion←

So

lution

+{s}

;

8

Updat

e th

e se

t of

candid

ate

elem

ents

;

9

Ree

val

uat

e th

e in

crem

enta

l co

sts;

10 e

nd

;

11 r

etu

rn S

olu

tion

;

end

Gre

ed

y R

andom

ized

Con

str

uction

.

RCL L

ist co

nst

ruct

ion

�Rank a

ppro

ach

�Keep the p

best

elem

ents

in the R

CL.

�Take random

elem

ent from

RCL

�If p�

N, pure

ly random

const

ruct

ion

�If p�

1, pure

ly g

reed c

onst

ruct

ion (and

only o

ne d

ete

rmin

ist in

itial so

lution)

RCL C

ontinued

�Relative T

hre

shold

Appro

ach

�Only k

eep e

lem

ents

in the R

CL that are

wors

t th

an b

est

elem

ent by a

fact

or

�Alp

ha is a h

yper para

mete

r

Alp

ha�

0, pure

ly g

reedy a

lgorith

m

Alp

ha�

1, pure

ly random

alg

orith

m

)](

,[

)(

:obey

that

el

emen

ts

kee

pO

nly

)(

cost

elem

ent

min

max

min

min

cc

cc

ec

ec

Ee

−+

∈

∈

α

RCL tra

deoff

�In

both

case

s, b

alance

must

occ

ur betw

een

pure

ly random

(m

ultip

le a

vera

ge initial

solu

tions)

and p

ure

ly g

reedy (only o

ne

solu

tion is tried).

�Also, co

mputa

tion c

ost

of lo

cal se

arc

h is

invers

ely p

roportio

nal to

qualit

y o

f in

itial

solu

tion a

nd c

om

puta

tion c

ost

of lo

cal se

arc

h

> initial so

lution c

onst

ruct

ion c

ost

RCL T

radeoff

RCL T

radeoff

�Typical alp

ha~0.2

Class

ic c

onst

ruct

ion p

roce

dure

sh

ortco

min

gs

�RCL c

onst

ruct

ion c

an b

e e

xpensive

since

it is c

onst

ruct

ed for every

elem

ent

added to the initial so

lution

�No h

isto

ry o

f in

itial so

lution is kept

�Past

good initial so

lutions sh

ould

influence

co

nst

ruct

ion o

f fu

ture

solu

tion

Const

ruct

ion p

hase

im

pro

vem

ents

�M

any a

lgorith

ms exist to

im

pro

ve

const

ruct

ion p

hase

(both

for qualit

y a

nd

com

plexity)

�Com

plexity reduct

ion

�React

ive G

rasp

�Bias fu

nct

ions

�In

telligent co

nst

ruct

ion

�POP p

rinciple

�Cost

perturb

ation

Com

plexity R

educt

ion

�Random

+ g

reedy

�Random

ly a

dd first

p e

lem

ents

to the initial

solu

tion

�Com

plete

the solu

tion w

ith g

reedy

appro

ach

�Sam

pled g

reedy

�Only sam

ple a

few e

lem

ents

to p

ut in

RCL

inst

ead o

f all

rem

ain

ing e

lem

ents

React

ive G

rasp

�Variable A

lpha

�At each

new initial so

lution c

onst

ruct

ion

phase

, ch

oose

Alp

ha fro

m a

set of

poss

ible A

lpha.

�The p

robability

of ch

oosing a

given

Alp

ha is pro

portio

nal to

the q

ualit

y o

f th

e p

ast

solu

tions with this A

lpha

React

ive G

RASP c

ontinued

�Form

ally

:

q

q

Azqp

i

i

i

ii

m

∑

==

:fo

r

Pro

bab

ilty

U

pd

ated

: g

iven

a o

f

sco

re

Rel

ativ

e

A :

wit

h

solu

tio

ns

o

f

val

ue

Av

erag

e

z

:so

luti

on

in

cum

ben

t

of

Co

st

:A

lph

aea

ch

o

fy

P

rob

abil

it

},...

,{

:A

lph

a

po

ssib

le

of

Set

i

*

i

ii

*

2

α

α

α

αα

αψ

Bias fu

nct

ions

�Next elem

ent to

take fro

m the R

CL to

add to solu

tion u

nder co

nst

ruct

ion is

usu

ally

chose

n random

ly fro

m R

CL

�Use

a b

ias fu

nct

ion b

ase

d o

n rank so

choice is no longer uniform

random

(s

am

e a

s gene select

ion thro

ugh rank)

Bias fu

nct

ion c

ontinued

�A few p

ropose

d b

ias fu

nct

ions

pro

pose

d:

∑=

==

=

=

−

−

))(

(

))(

(

R

CL

o

f

elem

ent

o

fy

p

rob

abil

it

)(

po

lyn

om

ial

)(

lex

po

nen

tia

/1

)(

lin

ear

1)

(u

nif

orm

σ

σσ

rb

iasr

bia

s

rr

bia

s

er

bia

s

rr

bia

s

rb

ias

n

r

Inte

lligent co

nst

ruct

ion

�Keep a

pool of elit

e solu

tion

�This p

ool co

nta

ins th

e b

est

solu

tions

that are

sufficiently d

iffe

rentbetw

een

each

oth

er

�W

hen c

onst

ruct

ing a

n initial so

lution,

choose

elem

ents

fro

m R

CL that will g

ive

a solu

tion w

hich c

onta

ins pattern

s in

elit

e solu

tions

Inte

lligent co

nst

ruct

ion

�Exam

ple in T

SP:

�Ass

um

e e

lite solu

tions co

nta

in the sam

e

3 lin

ked n

odes

�The c

onst

ruct

ion p

hase

should

favor th

e

use

of th

ese

sam

e thre

e n

odes so

they

are

lin

ked

Pro

xim

ate

Optim

alit

y P

rinciple

(POP)

�Gre

edy c

onst

ruct

ion is not alw

ays

optim

al. E

specially

on h

uge p

roblem

s.

�In

stead, ru

n a

few loca

l se

arc

h

alg

orith

ms during c

onst

ruct

ion p

hase

�Ex: Bin

ato

ran a

loca

l se

arc

h a

t 40%

and 8

0%

during c

onst

ruct

ion

Cost

Perturb

ation

�M

odify c

ost

funct

ion w

hen b

uild

ing R

CL

base

d o

n p

ast

solu

tions.

�Not nece

ssarily a

pplic

able to e

very

pro

blem

.

�Good resu

lts obta

ined u

sing this for th

e

Ste

iner tree p

roblem

(in

terc

onnect

poin

ts o

n a

plane)

Path

Re-lin

kin

g

�Given two solu

tions, e

xplo

re the p

ath

lin

kin

g these

two solu

tions in

the

solu

tion space

.

�Sta

rt fro

m o

ne solu

tion, go to b

est

neig

hbor th

at brings us close

r to

the

oth

er so

lution. Itera

te u

ntil re

ach

ing

oth

er so

lution.

Path

re-lin

kin

g

�Form

ally

:

end

s b

est

g

lob

al

bes

t)

glo

bal

if

(s

)m

in(

),,

(,

ss

of

cost

eval

uat

e

!s

wh

ile

),

(:

s in

to

s t

ran

sfo

rmto

nec

essa

ry

m

ov

es

calc

ula

te

1

1

1

21

1

, i

21

21

21

=

<

=

∀∆

∈+

=

=

∆

i

ii

ss

is

s

s

ss

δδ

Path

re-lin

kin

g a

nd G

RASP

�M

ultip

le w

ays of in

corp

ora

ting p

ath

re-

linkin

g w

ith G

RASP.

�M

ost

keep a

pool of elit

e solu

tions. A

s befo

re, th

is p

ool co

nta

ins th

e b

est

solu

tions th

at are

sufficiently d

iffe

rent

betw

een e

ach

oth

er

Path

re-lin

kin

g a

nd G

RASP

�Strate

gies:

�In

tensifica

tion: re

-lin

k a

ll lo

cal so

lution w

ith o

ne o

r m

ore

elit

e solu

tion

�In

tensifica

tion: re

-lin

k e

lites betw

een e

ach

oth

er

periodically

(akin

to a

n e

volu

tionary

pro

cess

)

�Post

-optim

ization: re

-lin

k e

very

elit

e solu

tion

�Post

-optim

ization: su

bm

it p

ool to

an e

volu

tionary

pro

cess

Inte

nsifica

tion

�Fin

d a

loca

l m

inim

a s thro

ugh loca

l se

arc

h

�Choose

a solu

tion g

fro

m e

lite p

ool. g

sh

ould

be c

hose

n so that it is diffe

rent

than s (diffe

rent ham

min

g d

ista

nce

for

exam

ple).

�Dete

rmin

e w

hich solu

tion is th

e initial

solu

tion a

nd w

hich is th

e d

est

ination

Inte

nsifica

tion c

ontinued

�Apply o

ne o

f th

e p

ath

re-lin

kin

g

alg

orith

ms

Inte

nsifica

tion v

ariations

�Forw

ard

path

re-lin

kin

g

�Go fro

m loca

l m

inim

a to e

lite solu

tion

�Back

ward

path

re-lin

kin

g

�Go fro

m e

lite solu

tion to loca

l m

inim

a

�(m

ore

logical since

elit

e n

eig

hborh

ood should

be o

n a

vera

ge b

etter)

�Back

and forw

ard

path

re-lin

kin

g�Do b

oth

. Expensive b

ut best

of both

solu

tions.

Inte

nsifica

tion v

ariations

�M

ixed p

ath

re-lin

kin

g

�Pin

g p

ong b

etw

een loca

l m

inim

a a

nd e

lite

solu

tion.

�Should

give the a

dvanta

ges of back

and

forw

ard

at half the c

ost

�Tru

nca

ted p

ath

re-lin

kin

g

�Only e

xplo

re c

lose

neig

hborh

ood o

f elit

e

solu

tion a

nd/o

r lo

cal m

inim

a

Inte

nsifica

tion v

ariations

�Gre

edy random

ized a

daptive p

ath

re-

linkin

g�

Num

ber of path

s betw

een solu

tion is

exponential in

�In

stead o

f gre

edily

explo

ring o

ne p

ath

, occ

asionally

choose

the n

ext elem

ent

random

ly�Leads to

more

path

bein

g e

xplo

red a

nd

divers

ifies th

e p

ath

s explo

red

),

(2

1s

s∆

Inte

nsifica

tion v

ariations

�Evolu

tionary

path

re-lin

kin

g

�Every

few G

RASP ite

ration, re

-lin

k the

solu

tions in

the e

lite p

ool to

cre

ate

a n

ew

population o

f elit

es.

�Use

gre

edy random

ized p

ath

re-lin

kin

g to

divers

ify the e

lite p

ool

Path

re-lin

kin

g a

dvanta

ges

�Path

re-lin

kin

g h

elp

s re

duce

the tim

e b

efo

re a

targ

et optim

um

is

reach

ed

GRASP e

xte

nsions and

implem

enta

tion ideas

�Use

of a h

ash

table o

f already e

xplo

red

solu

tion c

an lim

it w

ast

ing tim

e e

xplo

ring

multip

le tim

es th

e sam

e solu

tion

neig

hborh

ood.

�Not all

initial so

lutions need to b

e

explo

red. Only p

rom

isin

g o

nes sh

ould

be invest

igate

d

GRASP h

ybrids

�Class

ic G

RASP u

ses a sim

ple loca

l se

arc

h. But

it c

an u

se a

VNS inst

ead a

s th

e two c

an b

e

seen a

s co

mplim

enta

ry (co

nst

ruct

ion v

s.

explo

ration). T

abu searc

h c

an a

lso b

e u

sed.

In fact

, any searc

h m

eta

heurist

ic.

�Gra

sp c

onst

ruct

ion p

hase

can a

lso b

e u

sed to

create

an initial population for genetic

alg

orith

ms.

Gra

sp a

lgorith

m c

om

pariso

n

Para

llel Gra

sp

�M

ulti st

art h

eurist

ic p

articularly w

ell

suited for para

llel im

plem

enta

tion,

runnin

g o

n m

ultip

le C

PUs.

�Two types of st

rate

gies:

�M

ultip

le w

alk, in

dependent th

read

�M

ultip

le w

alk, co

opera

tive thre

ad

Independent th

read

�Each

CPU runs a subse

t of itera

tions of

GRASP. Best

solu

tion o

f all

CPUs is

kept. C

PUs do n

ot exch

ange a

ny o

ther

info

.

�Lin

ear gain

in tim

e.

�Does not work

with inte

nsifica

tion

stra

tegies and p

ath

re-lin

kin

g

Coopera

tive thre

ad

�Sam

e p

rinciple b

ut CPUs sh

are

info

.

�In

fo c

an b

e p

ool of elit

es, A

lpha b

ias

funct

ions etc

.

�Usu

ally

, all

share

d info

is m

anaged b

y a

ce

ntral pro

cess

or.

Coopera

tive v

s. independent

�Coopera

tive is better with m

any C

PUs

as we loose

1 C

PU for in

fo sharing.

Conclusion

�GRASP is a p

retty sim

ple c

once

pt to

im

plem

ent

�Easily m

odified b

y c

hangin

g

const

ruct

ion a

lgorith

m o

r lo

cal se

arc

h

alg

orith

m

�Low n

um

bers

of hyper para

mete

r to

tu

ne

�Efficient para

llel im

plem

enta

tion