combining exact and heuristic approaches for the multidimensional 0-1 knapsack problem
DESCRIPTION
Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem. Saïd HANAFI [email protected]. SYM-OP-IS 2009. Motivation. P : Hard Optimization Problem. Lower Bound v (P). Optimal Value v ( P ). Upper Bound v (P). Maximization. Exact - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/1.jpg)
Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem
Saïd HANAFI
SYM-OP-IS 2009
![Page 2: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/2.jpg)
2
Motivation
LowerBound
v(P)
UpperBoundv(P)
Heuristic &Metaheuristic
ExactMethod
OptimalValuev(P)
Maximization
Small sizeLarge size Large size
Relaxation &Duality
P : Hard Optimization Problem
![Page 3: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/3.jpg)
3
Outline
Variants of Multidimensional Knapsack problem (MKP)
Tabu Search & Dynamic Programming Relaxations of MKP Iterative Relaxation-based Heuristics Inequalities and Target Objectives
Conclusions
![Page 4: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/4.jpg)
4
Multidimensional Knapsack Problem
MKP is NP-Hard Dantzig 1957, Lorie & Savage 1955, Markowitz & Manne 1957 Martello & Toth, 1990; Kellerer & Pferschy & Pisinger 2004 Fréville & Hanafi 2005, Wilbaut & Hanafi & Salhi 2008; Hanafi & Wilbaut 2009 Boussier & Vasquez & Vimont & Hanafi & Michelon 2009 Hanafi & Lazic & Mladenovic & Wilbaut 2009
nx
bAts
c
MKP
1;0
x ..
x max
A(m,n),b(m),c(n) 0
![Page 5: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/5.jpg)
5
Applications Resource allocation problems Production scheduling problems Transportation management problems Adaptive multimedia system with QoS Telecommunication networks problems Wireless switch design Service selection for web services with QoS constraints Polyedral approach, (1,k)-configuration
Sub-problem : Lagrangean / Surrogate relaxation Benchmark
![Page 6: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/6.jpg)
6
Variants of Knapsack Problem
- Unidimensional Knapsack Problem m = 1
- Bi-Dimensional Knapsack Problem m = 2
- Quadratic- Hyperbolic- Min-Max- Multi-Objectives- Bi-Level
- Cardinality- Multiple-choice- Precedence - Demand- Quadratic
ObjectiveObjective ConstraintsConstraints
![Page 7: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/7.jpg)
7
Multidemand Multidimensional Knapsack Problem
Knapsack & Covering Problem
![Page 8: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/8.jpg)
8
,...,gj,...,gi,x
,...,gix
,...,mkbxats
xc
MMKP
iij
g
jij
kij
g
i
g
j
k
g
i
g
jijij
i
i
ij
i
1 ,11 0
11
1..
max
1
1 1
1 1
Multi-Choice Multidimensional Knapsack Problem
• g : Number of groups• gi : Number of items in group i• cij : Profit associated to item j of group i• : Consuming resource k by item j of group i• bk : Capacity of resource k
kija
![Page 9: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/9.jpg)
9
,...,gj,...,gi,x
,...,gix
,...,mkbxats
xc
GUBMKP
iij
g
jij
kij
g
i
g
j
k
g
i
g
jijij
i
i
ij
i
1 ,11 0
11
1..
max
1
1 1
1 1
Multidimensional Knapsack Problem with Generalized Upper Bound constraints
![Page 10: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/10.jpg)
10
Knapsack Constrained Maximum Spanning Tree Problem
KCMST problem arises in practice in situations where the aim is to design a mobile communication network under a strict limit on total costs.
Aa,x
x
bxwts
xc
KCMST
a
Aaaa
Aaaa
1 0
G of treecovering
..
max
Aggarwal et al (1982)
![Page 11: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/11.jpg)
11
Generalized Knapsack Problem
nj,yx
nj ybxya
cxts
xc
GKP
jj
jjjjj
n
jj
n
jjj
,...,11 0;0
,...,1
..
max
1
1
Suzuki (1978)
![Page 12: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/12.jpg)
12
Disjunctively Constrained Knapsack Problem
nj,x
Ii,jxx
bxats
xc
DCKP
j
ji
n
jjj
n
jjj
,...,11 0
)( 1
..
max
1
1
Yamada et al (2002)
![Page 13: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/13.jpg)
13
Max Min Multidimensional Knapsack
Problem
n
k
xbAx
ts
pkxcMinMax
MMMKP
1,0
..
,...,1:
Introduction to minimax, Demyanov Molozemov 1974
Minimax and applications, Du, Pardalos 1995
Robust optimization applications, Yu 1996
A virtual pegging approach, Taniguchi, Yamada, Kataoka 2009
Iterative Heuristics, Mansi, Hanafi, Wilbaut 2009
![Page 14: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/14.jpg)
14
Nj,x
liNNNN
bxats
mixc
KSP
j
li
m
ii
Njjj
Njjj
i
1 0
,
..
,...,1:minmax
1
Knapsack Sharing ProblemBrown (1979)Brown (1979)
![Page 15: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/15.jpg)
15
Moore, Bard 1990 => B2 = 0Dempe, Richter 2000Hanafi, Mansi, Brotcorne 2009
15
yxMax
, f 1 ( x , y ) = d 1 x + d 2 y
s.t. B1 x + B2 y ≤ b1 x 1nZ
y
Max f 2 ( y ) = c y
s.t. A1 x + A2 y ≤ b2
y 2n1,0
Leader objective function
Follower objective function
Follower problem
(BKP)
Bilevel Knapsack Problem
![Page 16: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/16.jpg)
16
Tabu Search & Dynamic Programming
Metaheuristic: Tabu Search (TS) Very powerful method on many optimization
problems Based on the exploration of a neighbourhood
Exact method: Dynamic Programming (DP) Efficient for small-sized problem Can be used for solving a series of "similar" problem Can be enhanced by adding reduction rules
![Page 17: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/17.jpg)
17
Tabu Search & Dynamic Programming
|N 1| small and depends on the instance and the machine. How defining N 1 and N 2 ?
N1
N2
Dynamic Programmingwith Fixation on N1
Tabu Search on N2
Partition
Recurrence
|N| = 8
![Page 18: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/18.jpg)
18
Dynamic Programming (Bellman, 1957)
Recurrence relation for the MKP
In our case all the subproblems concern the same variables and only g varies
![Page 19: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/19.jpg)
19
Reduction Rules
Let x0 be a feasible solution of the MKP. Order the variables such that:
Proposition 1 (Optimality):
Let ,
Proposition 2 (Fixation):
Balev, Fréville, Yanev and Andonov, 2001.
![Page 20: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/20.jpg)
20
Dynamic Programming as a Heuristic
P Initial solution: x0
Order the variables:uj upper bound of P(e-x0,{j})
Partition the set of variables: N 1 and N 2
Dynamic Programming
if cx1 > cx0
or if |F| > 0
N 1
N 2
n, m
the machine
List L (with all the optimal values)Fixation Fx1
![Page 21: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/21.jpg)
21
Global Intensification (GI) Mechanism
Dynamic Programming
List L (with all the optimal values)Fixation Fx1
Tabu Search:performed on N 2 and uses L
x*
If cx* > cx1 then restart the algorithm with x* as the initial solution
![Page 22: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/22.jpg)
22
? 0 1 ? ? 1 011 0 1 0 1 1 00
Tabu Search
TS on N2 (|N2| = n2) Neighbourhood of the current solution x2 on N2:
Evaluation of a solution over N from a solution over N2
N 1
N 2
Scan in L
![Page 23: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/23.jpg)
23
Computational Environment
Algorithm coded in C language The results have been obtained on a Solaris Ultra
Sparc workstation 300MHz n1 15% of n OR-Library : 270 correlated instances
n = 100, 250, 500 and m = 5, 10, 30 30 instances for each (n,m)
![Page 24: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/24.jpg)
24
A Summary of the Results
Each value = average over 30 instances
%: average gap wrt CPLEX (with the same computational time)
CPU: in seconds
TS alone: Tabu Search algorithm without global intensification mechanism
![Page 25: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/25.jpg)
25
RelaxationDefinition : Let
(P) max{ f(x) : x X}
(R) max{ g(x) : x Y}
Problem R is a relaxation of P if
1) X Y
2) x X : f(x) ≤ g(x).Properties
- v(P) ≤ v(R)
- Let x* an optimal solution of R, if x* is feasible for P and f(x*)=g(x*) then x* is an optimal solution of P
![Page 26: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/26.jpg)
26
LP-RelaxationDantzig (1957)
max max cxcx
(LP)(LP) s.t.s.t. Ax ≤ bAx ≤ b
x x [0, 1] [0, 1]nn
Properties :
- v(MKP) ≤ v(LP)
- Let x* an optimal solution of LP,
if x* {0,1}n then x* is an optimal solution of MKP
![Page 27: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/27.jpg)
27
MIP Relaxation
Let x0 {0, 1}n and J N
Remarks: MIP(x0,) = LP(P), MIP(x0,N) = P v(MIP(x0,J)) v(MIP(x0,J’) if J J’
Stronger bound : v(P) v(MIP(P,J)) v(LP(P))
Hanafi & Wilbaut (06), Glover (06)
![Page 28: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/28.jpg)
28
Lagrangean Relaxation
Lagrangean Relaxation : 0
LR() max{cx + (b – Ax) : x X} Lagrangean Dual :
(L) min{v(LR()) : 0}
Properties :
- v(MKP) ≤ v(L) ≤ v(LR())
- Let x* be an optimal solution of LR(*), if *(b – Ax*) = 0 and x* is feasible for P then x* is an optimal solution of P
Held & Krap (71), Geoffrion (74)
![Page 29: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/29.jpg)
29
Geometric Interpretation : v(L) = max{cx : Ax ≤ b, x co(X)}
eDxx
Conv nZxeDxx ,
bAxx
cx
LP
LR
P
nZxeDxxX ,
![Page 30: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/30.jpg)
30
Lagrangean Decomposition
nZx
eDx
bAx
cxZ
max
n
n
Zy
Zx
yx
eDy
bAx
cxZ
max
n
n
LD
Zy
Zx
eDy
bAx
xyvcxvZ
)(max)(
Guignard & Kim (87)
![Page 31: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/31.jpg)
31
Lagrangean Decomposition
)()(min 210
vZvZZ LDLDv
LD
n
n
LD
Zy
Zx
eDy
bAx
xyvcxvZ
)(max)(
n
LD
Zx
bAx
xvcvZ
)(max)(1
n
LD
Zy
eDy
vyvZ
max)(2
Sub-problem 1 Sub-problem 2
![Page 32: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/32.jpg)
32
P
Geometric InterpretationSub Pb 1
eDxx bAxx
cx
LP
LR
Conv nZxeDxx ,
Conv nZxbxAx ,
LD
Sub Pb 2
![Page 33: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/33.jpg)
33
Theorem
)()()()( LPvLRvLDvPv
![Page 34: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/34.jpg)
34
Surrogate Relaxation
Relaxation surrogate : 0SR()max{cx : Ax ≤ b, x X}
Dual surrogate corresponding : (S) min{v(SR()) : 0}.
Glover (65)
Propriétés :
- v(MKP) ≤ v(S) ≤ v(SR())
- Let x* be an optimal solution of SR(),
if x* is feasible for MKP then x* is an optimal solution of MKP
- (S) min{v(SR()) : U = { 0 : |||| = 1}}
![Page 35: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/35.jpg)
35
Composite Relaxation Composite Relaxation : , ≥ 0
CR(, ) max{cx + (b – Ax) : Ax ≤ b and x X}
Composite Dual :
(C) min{v(CR(, )) : , ≥ 0}
Greenberg & Pierskalla (70)
Remarks : - v(LR()) = v(CR(, 0)) - v(SR()) = v(CR(0, ))
![Page 36: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/36.jpg)
36
Comparisons
v(P) v(C) v(S) v(L) v(LP)
![Page 37: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/37.jpg)
37
Properties of Relaxation Functions
Lagrangean function v(LR()) is convex and piecewise-linear if X is finite
Surrogate function surrogate v(SR()) is quasi-convex and piecewise-linear if X is finite
Composite function v(CR(, )) is non convex, non quasi-convex
![Page 38: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/38.jpg)
38
Methods for Dual
(Quasi-)Sub-gradient : ’ = + t gg : sub-gradient of f( )=v(R())
t : Stepwise
x*() an optimal solution of R() g() = b – Ax*() a sub-gradient of f()
Bundle method, linearization, etc.
![Page 39: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/39.jpg)
39
Intersection Algorithmvv(LR((LR())))
11 2200 maxmax**
Fisher (75), Yu (90), (90), Hanafi (98)
![Page 40: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/40.jpg)
40
Extended Dichotomy Algorithmfor Bi-dimensional Knapsack
AA11xx≤≤bb11
AA22xx≤≤bb22
x*(u)x*(u)
uAxuAx≤u≤ubb
Fréville & Plateau (93), Fréville & Plateau (93), Hanafi (93)Hanafi (93)
![Page 41: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/41.jpg)
41
Relaxation-based Heuristics
u = u°, UB = g(u), LB = f(x0) Repeat
let x*(u) an optimal of the current relaxation R(u) let x(u) the projection of x*(u) an the feasible set of P UB = min(UB, v(R(u))) LB = max(LB, f(x(u))) Update the current multiplier u
![Page 42: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/42.jpg)
42
Relaxation-based Heuristics
Step 1: Solve one or more relaxations of the current problem P to generate one or more constraints.
Step 2: Solve one or more reduced problems induced by the optimal solution(s) of the previous relaxation(s) to obtain one or more feasible solution(s) of the initial problem.
Step 3: If a stopping criteria is satisfied then return the best lower bound. Else add a generated constraint(s) to the problem P and return to step 1.
Hanafi & Wilbaut (2006)
![Page 43: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/43.jpg)
43
Space Search
Binary Solution : x {0, 1}n
Improved Heuristic Continuous Solution : x [0, 1]n
LP-Relaxation Partial Solution : x {0, 1, *}n where * = free
Branch&Bound, Dynamic Programming, Heuristic Feasible and Infeasible Solutions
Oscillation, Resolution Search, Dynamic B&B
![Page 44: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/44.jpg)
44
Reduced Problem
Let x0 {0, 1}n and J N
P(x0, J) = (P| xJ = x0J)
Remarks : P(x0,) = P, v(P(x0,J)) v(P(x0,J’) if J J’ xJ = x0
J (eJ – 2 x0J)xJ + eJ x0
J = 0
|J| = 1 If v(P(e - x0,{j})) ≤ cx0 then x*j = x0
j , x* an optimal solution
![Page 45: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/45.jpg)
45
LP-based Heuristic
Step 1: Solve the LP-relaxation of P to obtain an optimal solution xLP
Step 2: Solve the reduced problem P(xLP,J(xLP)) to generate an optimal solution x0
where J (x) = {j N : xj {0, 1}}
Remarks: cx0 ≤ v(P) ≤ cxLP
Relaxation induced neighbourhoods search, Danna et al., 2003P((xLP+x*)/2,J((xLP+x*))/2) with cx > cx*
where x* the best feasible solution
![Page 46: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/46.jpg)
46
Pseudo-Cut
Let y be a vector in {0, 1}n, the following inequality (*) cuts off the solution y without cutting off any other solution in {0, 1}n
(*)
where J0(x) = {j N : xj = 0} J1(x) = {j N : xj = 1}.
Example:
(*) is called canonical cut in Balas and Jeroslow, 1972.
![Page 47: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/47.jpg)
47
Partial Pseudo cuts
Let x’ be a partial solution in {0, 1, *}n, the inequality
dJ(x, x’) = (eJ – 2 x’J)xJ + eJ x’J 1 (*)cuts off x’ and all partial solution dominated by x’.
J = {j N: x’j {0,1}}, xJ = (xj : j J)Remarks : (*) is called
Pseudo Cut by Glover, 2005. Canonical Cut by Balas & Jeroslow, 1972, if J = N
dN (x, x’) = Hamming distance over {0, 1}n
![Page 48: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/48.jpg)
48
Iterative LP-based heuristic (ILPH)
Let x* a feasible solution of Pv* = cx*; Q = P ; stop = False;while (stop = False) do Let xLP be an optimal solution of LP(Q); J = J(xLP);
Let x0 be an optimal solution of (Q|(eJ – 2 xLPJ)xJ k – eJ xLP
J) if cx0 > v* then x* = x0; v* = cx0
Q = (Q | (eJ – 2 xLPJ)xJ k + 1 – eJ xLP
J) if cxLP-v* < 1 then stop = True end while
![Page 49: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/49.jpg)
49
Theorem : Finite Convergence
The algorithm ILPH converges to an optimal solution of the problem in a finite number of iterations ( 3n).
Proof : v(P) = max(v(P|dx d0), v(P|dx d0 + 1))
d = (eJ – 2 xLPJ, 0) and d0 = – eJ xLP
J
Number of partial solutions = |{0, 1, *}n|=3n
![Page 50: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/50.jpg)
50
Convergence of IPLH
CPU 400sec (Cplex 10sec)
Replace the optimal condition by a total number of iterations
F(x): set of fractional variables of solution x
![Page 51: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/51.jpg)
51
Remark and Dominance
1 001 1 0 *01 * 01
1 001 1 0 *01 * * *
1 001 1 0 *01 * 01
1 001 1 0 *01 * * *x1
x2
J1 J0 F
Reduced problem included
![Page 52: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/52.jpg)
52
Improvements: Dominance Properties
Definition: Let x1 and x2 be two solutions in [0,1]n,x1 dominates x2 if F(x2) F(x1), J 1(x1) J 1(x2) and J 0(x1) J 0(x2)
Property 1: If x1 dominates x2 then v(P(x1,J(x1))) v(P(x2,J(x2)))
Property 2: The solution x defined bydominates x1 and x2, and we have :
v(P(x,J(x))) max { v(P(x1,J(x1))) , v(P(x2,J(x2))) }
Reduction of the number of reduced problems to solve
![Page 53: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/53.jpg)
53
Illustration of Dominance
ILPH_D: ILPH with property 1
ILPH_D*(t) : ILPH with properties 1 and 2
Reduction between 30 and 50%
![Page 54: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/54.jpg)
54
Iterative independent relaxations-based heuristic (IIRH)
xk optimal solution of MIP(Qk,F(xk-1))
k = k + 1
Pk+1 = (Pk | {f kx ≤ | J1(yk)| - 1}
yk optimal solution of LP(Pk)
wk optimal solution of P(yk,J(yk)) zk optimal solution of Q(xk,J(xk))
v*= max(v*,cwk,czk) u = min(cyk,cxk)
Q k+1 = (Q k | {f kx ≤ | J1(xk)| - 1}
Q = P, x0 optimal solution of LP(P); k = 1
Stop No Yes
If u–v*< 1
![Page 55: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/55.jpg)
55
Iterative Relaxations-based Heuristic (IRH)
k = k + 1
yk optimal solution of LP(Pk)
Pk+1 = (Pk | {f kx ≤ | J1(yk)| - 1 and/or f kx ≤ | J1(xk)| - 1}
xk optimal solution of MIP(Pk,F(yk))
wk optimal solution of P(yk,J(yk)) zk optimal solution of P(xk,J(xk))
v*= max(v*,cwk,czk) u = min(cxk,cyk)Stop
No
Yes
If u–v*< 1
![Page 56: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/56.jpg)
56
Comparison on a small instance
n = 30; m = 10
v* = 376
![Page 57: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/57.jpg)
57
Results for n = 500
1 line = average / 10 instances
Average computational time 70 hours 2 hours
![Page 58: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/58.jpg)
58
Summary (n = 500)
1 line = results for 30 instances
![Page 59: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/59.jpg)
59
Some Related Works
Canonical Cuts on the Unit Hypercube, Balas & Jeroslow, 1972 0-1 IP with many variables and few constraints, Soyster, Lev,
Slivka, 1978 A hybrid approach to discrete mathematical programming,
Marsten, Morin, 1978 A new hybrid method combining exact solution and local
search, Mautor, Michelon, 1997 Parametric Branch and Bound, Glover, 1978 Large neighborhood search, Shaw, 1998 Local branching, Fischetti, Lodi, 2002
![Page 60: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/60.jpg)
60
Some Related Works Short Hot Starts for B&B, Spielberg & Guignard, 2003 Relaxation induced neighbourhood search, Danna, Rothberg, Pape, 2003 Adaptive memory projection method, Glover, 2005 Feasibility Pump, Fischetti, Glover, Lodi, 2005 Relaxation guided VNS, Puchinger, Raidl, 2005 Global intensification using dynamic programming, Wilbaut, Hanafi,
Fréville, Balev, 2006 Convergent Heuristics for 0-1 MIP, Wilbaut, Hanafi, 2006 Variable Neighbourhood Decomposition Search, Lazic, Hanafi,
Mladenovic, Urosevic, 2009 Inequalities and Target Objectives in Metaheuristics for MIP, Glover,
Hanafi, 2009
![Page 61: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/61.jpg)
61
Inequalities and Target Objectives in Metaheuristics for MIP
Recent adaptive memory and evolutionary metaheuristics for MIP have included proposals for introducing inequalities and target objectives to guide the search.
Two types of search strategies Fix subsets of variables to particular values within
approaches for exploiting strongly determined and consistent variables,
Use of solution targeting procedures.
Glover & Hanafi (09)
![Page 62: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/62.jpg)
62
Inequalities and Target Objectives
Let a Target Solution x′, solveLP(x′, c′, M) min{Mfx + c′x : x X}
M is a large positive number. The coefficients c′j of the Target Objective seek to induce assignments xj = x′j for different variables with varying degrees of emphasis.An alternative to solve LP(LP(x′, c′, M) in two stages: 1- Solve LP(x′, c′, 0) = min{c′x : x X}2- Solve the problem min{fx : x X, cx′ = v(LP(x′, c′, 0))}Remark : The residual problem can be significantly smaller than the first stage problem, allowing it to be solved very efficiently
Glover & Hanafi (09)
![Page 63: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/63.jpg)
63
Strong Pseudo Cut
x {0, 1}n x X {x : (x, x) max{1, v(LP(x))}} = X
LP(x) min{(x, x) : x X}
We call x the target solution for LP(x’) Remarks : x X v(LP(x)) = 0 (x, x ) = v(LP(x)) fractional x X Parametric branch and bound, Glover (1978) Feasibility pump, Fischetti, Glover and Lodi (2005)
Nj
jjjj xxxxxx )1(')'1(),'(
![Page 64: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/64.jpg)
64
Illustration
MartinGrötsche
l
64
x
x
(x, x) = min{(x, x) : x X}d(x, x) d(x, x)
![Page 65: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/65.jpg)
65
Partial Pseudo Cuts
For all y s.t. (J, x, y) = 0, we have
y X {x : (J, x, x) max{1, v(LP(J, x))}}
LP(J, x) min{(J, x, x) : x X}
Remarks : Partial Hamming distance over {0, 1, *}n
Jj
jjjj xxxxxxJ )1(')'1(),',(
Let x [0,1]n, J N(x) = {j : xj {0, 1}}
![Page 66: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/66.jpg)
66
Theorem
Let x [0,1]n, N(x) = {j N: xj {0, 1}}, k an integer 0 k n - |N(x)|
The canonical hyperplane associated to x :H(x, k) = { x [0,1]n : (N(x), x, x) = k}We have :
Co(H(x, k) {0,1}n) = H(x, k)
where Co(X) is the convex hull of the set X.
Balas & Jeroslow 1972, Glover & Hanafi 2008
![Page 67: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/67.jpg)
67
Weighted Partial Pseudo Cuts
Nj
jjjjj xxxxcxxc )1(')'1(),',(Let x [0,1]n, c C(x) = {c INn : cjxj (1 – xj) = 0}
x {0, 1}n y : (c, x, y) = 0 y X {x : (c, x, x) max{1, v(LP(c, x))}}
LP(c, x) min{(c, x, x) : x X}Remarks : (e, x, x) = || x – x||1 = || x – x||2 (J, x, x) = (c, x, x) : cj = 1 if j J else cj = 0.
![Page 68: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/68.jpg)
68
Additional and Stronger Inequalities from Basic LP Solutions
Let x″ a basic feasible solution for
LP(c, x) min{(c, x, x) : x X}
Let d = c - rc where rc is reduced cost to x″, consider the pseudo cut
(d, x, x) v(LP(d, x)) (*)
LP(d, x) min{(d, x, x) : x X}
The inequality (*) is satisfied by all binary vectors x X, and excludes the solution x = x″ when (c, x,x″) is fractional.
If the basic solution x″ for LP(x, c) is optimal then the inequality (*) dominates
(c, x, x) v(LP(c, x))
![Page 69: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/69.jpg)
69
Intensification Procedure
Pp
PpsXx
xxdsxxdts
sws
p
pppp
ppPp
pp
,
,0,
)",',(),',(..
min 0
Pp
Xx
xxdxxdsts
sppppp
,),',()",',(..
min
0
0
Min-Sum
Min-Max
![Page 70: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/70.jpg)
70
Diversification Procedure
Pp
PpsXx
xxdsxxdts
sws
p
pppp
ppPp
pp
,
,0,
)",',(),',(..
max 0
Pp
Xx
xxdxxdsts
sppppp
,),',()",',(..
max
0
0
Max-Sum
Max-Min
![Page 71: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/71.jpg)
71
Conclusions
Many MIP problems resist to be solved by the best current B&B and B&C methods.
Metaheuristics can likewise profit from generating inequalities to supplement their basic functions.
This framework can be exploited in generating target objectives, employing both adaptive memory ideas and newer ones proposed here.
![Page 72: Combining Exact and Heuristic Approaches for the Multidimensional 0-1 Knapsack Problem](https://reader036.vdocuments.us/reader036/viewer/2022062323/568158c1550346895dc60ba3/html5/thumbnails/72.jpg)
72
Some Prospects
Extension to other problems Improve the implementation of dominance rules Integrate intensification and diversification Strengthen the use of memory Parallel versions