some recent advances in mixed-integer nonlinear programming · mixed-integer nonlinear programming...
TRANSCRIPT
![Page 1: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/1.jpg)
Some Recent Advances in Mixed-IntegerNonlinear Programming
Andreas Wächter
IBM T.J. Watson Research Center
Yorktown Heights, New York
SIAM Conference on Optimization 2008Boston, MA
May 12, 2008
Andreas Wächter (IBM) MINLP SIOPT 2008 1 / 30
![Page 2: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/2.jpg)
An MINLP Research Initiative
CMU-IBM research collaboration, started in 2004
The Team:
CMU◮ Pietro Belotti◮ Lorenz T. Biegler◮ Gérard Cornuéjols◮ Ignacio E. Grossmann◮ Carl D. Laird (Texas A&M)◮ François Margot◮ Nick Sawaya◮ Nick Sahinidis
IBM◮ Pierre Bonami (CNRS Marseilles)◮ Andrew R. Conn◮ Claudia D’Ambrosio (U Bologna)◮ John J. Forrest◮ Joao Goncalves◮ Oktay Günlük◮ Laszlo Ladanyi◮ Jon Lee◮ Andrea Lodi (U Bologna)◮ Andreas Wächter
Andreas Wächter (IBM) MINLP SIOPT 2008 2 / 30
![Page 3: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/3.jpg)
Mixed-Integer Nonlinear Programming (MINLP)
min f (x , y)
s.t . c(x , y) ≤ 0
yL ≤ y ≤ yU
x ∈ {0, 1}n, y ∈ R
p
f , c sufficiently smooth(e.g., C 2)
Often in practice: Simplify original problem to obtain◮ NLP by relaxing integrality conditions (rounding)◮ MILP by approximating nonlinearities (piece-wise linear)
Andreas Wächter (IBM) MINLP SIOPT 2008 3 / 30
![Page 4: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/4.jpg)
Mixed-Integer Nonlinear Programming (MINLP)
min f (x , y)
s.t . c(x , y) ≤ 0
yL ≤ y ≤ yU
x ∈ {0, 1}n, y ∈ R
p
f , c sufficiently smooth(e.g., C 2) and convex
Often in practice: Simplify original problem to obtain◮ NLP by relaxing integrality conditions (rounding)◮ MILP by approximating nonlinearities (piece-wise linear)
Goal: Design exact algorithms
In this talk: Convex MINLP (f , c convex)
Andreas Wächter (IBM) MINLP SIOPT 2008 3 / 30
![Page 5: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/5.jpg)
The Power Of MILP
MILP has been extensively explored for decades◮ Based on branch-and-bound [Dakin (1965)]◮ Very powerful algorithms, techniques, and codes◮ Can solve very large problems◮ Used heavily in practice
Andreas Wächter (IBM) MINLP SIOPT 2008 4 / 30
![Page 6: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/6.jpg)
The Power Of MILP
MILP has been extensively explored for decades◮ Based on branch-and-bound [Dakin (1965)]◮ Very powerful algorithms, techniques, and codes◮ Can solve very large problems◮ Used heavily in practice
How can this be used for MINLP?
Use MILP solvers directly:◮ Piece-wise linear approximation (SOS constraints)◮ Outer approximation
Andreas Wächter (IBM) MINLP SIOPT 2008 4 / 30
![Page 7: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/7.jpg)
The Power Of MILP
MILP has been extensively explored for decades◮ Based on branch-and-bound [Dakin (1965)]◮ Very powerful algorithms, techniques, and codes◮ Can solve very large problems◮ Used heavily in practice
How can this be used for MINLP?
Use MILP solvers directly:◮ Piece-wise linear approximation (SOS constraints)◮ Outer approximation
In a “nonlinear” branch-and-bound algorithm:◮ Try to learn from MILP tricks
Andreas Wächter (IBM) MINLP SIOPT 2008 4 / 30
![Page 8: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/8.jpg)
The Power Of MILP
MILP has been extensively explored for decades◮ Based on branch-and-bound [Dakin (1965)]◮ Very powerful algorithms, techniques, and codes◮ Can solve very large problems◮ Used heavily in practice
How can this be used for MINLP?
Use MILP solvers directly:◮ Piece-wise linear approximation (SOS constraints)◮ Outer approximation
In a “nonlinear” branch-and-bound algorithm:◮ Try to learn from MILP tricks
Andreas Wächter (IBM) MINLP SIOPT 2008 4 / 30
![Page 9: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/9.jpg)
Outer Approximation (Duran, Grossmann [1986])
min z (linear objective)
s.t . f (x , y) ≤ z
c(x , y) ≤ 0
x ∈ {0, 1}n, y ∈ R
p, z ∈ R
Andreas Wächter (IBM) MINLP SIOPT 2008 5 / 30
![Page 10: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/10.jpg)
Outer Approximation (Duran, Grossmann [1986])
min z (linear objective)
s.t . f (x , y) ≤ z
c(x , y) ≤ 0
x ∈ {0, 1}n, y ∈ R
p, z ∈ R
Approximate by MILP (hyperplanes)
min z
s.t . ∇f (x k, yk )T
(
x − x k
y − yk
)
+ f (x k, yk ) ≤ z
∇c(x k, yk )T
(
x − x k
y − yk
)
+ c(x k, yk ) ≤ 0
for all (x k, yk ) ∈ T
x ∈ {0, 1}n, y ∈ R
p, z ∈ R
T contains linearization points
Andreas Wächter (IBM) MINLP SIOPT 2008 5 / 30
![Page 11: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/11.jpg)
Outer Approximation (Duran, Grossmann [1986])
min z (linear objective)
s.t . f (x , y) ≤ z
c(x , y) ≤ 0
x ∈ {0, 1}n, y ∈ R
p, z ∈ R
Approximate by MILP (hyperplanes)
min z
s.t . ∇f (x k, yk )T
(
x − x k
y − yk
)
+ f (x k, yk ) ≤ z
∇c(x k, yk )T
(
x − x k
y − yk
)
+ c(x k, yk ) ≤ 0
for all (x k, yk ) ∈ T
x ∈ {0, 1}n, y ∈ R
p, z ∈ R
T contains linearization points
◮ augmented during algorithm
Algorithm: Repeat1 solve current MILP → (x l , y l)2 solve NLP with x l fixed → y l
3 add (x l , y l) to T
Andreas Wächter (IBM) MINLP SIOPT 2008 5 / 30
![Page 12: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/12.jpg)
Outer Approximation Discussion
Original algorithm:◮ Alternatingly solve NLPs and MILPs◮ Finite termination◮ Advantage: Simple to implement; uses all MILP techniques◮ Disadvantage: Solve every MILP from scratch
Andreas Wächter (IBM) MINLP SIOPT 2008 6 / 30
![Page 13: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/13.jpg)
Outer Approximation Discussion
Original algorithm:◮ Alternatingly solve NLPs and MILPs◮ Finite termination◮ Advantage: Simple to implement; uses all MILP techniques◮ Disadvantage: Solve every MILP from scratch
Improvement [Quesada, Grossmann (1992)]:◮ Build only one MILP enumeration tree
Andreas Wächter (IBM) MINLP SIOPT 2008 6 / 30
![Page 14: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/14.jpg)
Quesada-Grossmann
LP LB=4
LP LB=5
LP
UB=7
LP LB=6
LP LP
LB=8
x3=0 x3=1
x2=0 x2=1
x1=0 x1=1
integerfeasible
infeasible
Andreas Wächter (IBM) MINLP SIOPT 2008 7 / 30
![Page 15: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/15.jpg)
Quesada-Grossmann
LP LB=4
LP LB=5
NLP
UB=7.5
LP LB=6
LP LP
LB=8
x3=0 x3=1
x2=0 x2=1
x1=0 x1=1
integerfeasible
infeasible
Andreas Wächter (IBM) MINLP SIOPT 2008 7 / 30
![Page 16: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/16.jpg)
Outer Approximation Discussion
Original algorithm:◮ Alternatingly solve NLPs and MILPs◮ Finite termination◮ Advantage: Simple to implement; uses all MILP techniques◮ Disadvantage: Need to solve every MILP from scratch
Improvement [Quesada, Grossmann (1992)]:◮ Build only one MILP enumeration tree◮ Solve NLP for every MILP integer feasible solution◮ Add new outer approximation cuts to current MILP
Andreas Wächter (IBM) MINLP SIOPT 2008 8 / 30
![Page 17: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/17.jpg)
Outer Approximation Discussion
Original algorithm:◮ Alternatingly solve NLPs and MILPs◮ Finite termination◮ Advantage: Simple to implement; uses all MILP techniques◮ Disadvantage: Need to solve every MILP from scratch
Improvement [Quesada, Grossmann (1992)]:◮ Build only one MILP enumeration tree◮ Solve NLP for every MILP integer feasible solution◮ Add new outer approximation cuts to current MILP
“Hybrid” approach [Bonami et al. (2005)]◮ Solve NLPs also at non-integer nodes◮ For example, solve NLP in every 10th node
+ Includes information about nonlinear geometry more quickly− Requires solution of more NLPs
◮ Abhishek, Leyffer, Linderoth (2007) (FilMINT code):⋆ Don’t solve NLP, just add linearization (Extended cutting plane)
Andreas Wächter (IBM) MINLP SIOPT 2008 8 / 30
![Page 18: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/18.jpg)
Preliminary Numerical Experiments
Software implementation◮ Bonmin (Open source software on COIN-OR)
http://www.coin-or.org/Bonmin
◮ Based on other COIN-OR projects (Cbc, Clp, Cgl, Ipopt, . . . )- Essential for fast development: Availability of open source
◮ NLP solvers: FilterSQP [Fletcher, Leyffer] and Ipopt
Andreas Wächter (IBM) MINLP SIOPT 2008 9 / 30
![Page 19: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/19.jpg)
Preliminary Numerical Experiments
Software implementation◮ Bonmin (Open source software on COIN-OR)
http://www.coin-or.org/Bonmin
◮ Based on other COIN-OR projects (Cbc, Clp, Cgl, Ipopt, . . . )- Essential for fast development: Availability of open source
◮ NLP solvers: FilterSQP [Fletcher, Leyffer] and Ipopt
Test problems◮ Representative selection of 44 convex MINLPs from
- CMU/IBM library
http://egon.cheme.cmu.edu/ibm/page.htm
- MacMinlp [Leyffer]◮ Difficult, but mostly solvable within 3 hour time limit◮ Problem statistics
⋆ # total vars: 42–1796 (289.8); # discrete vars: 14–432 (93.7)⋆ # constraints: 42–3190 (395.4)
Andreas Wächter (IBM) MINLP SIOPT 2008 9 / 30
![Page 20: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/20.jpg)
Bonmin 0.1.4 with Ipopt (CPU)
0
20
40
60
80
100
1 10 100 1000
% o
f pro
ble
ms
not more than x times worse than best
Performance
Hybrid_0.1.4QG_0.1.4OA_0.1.4
Andreas Wächter (IBM) MINLP SIOPT 2008 10 / 30
![Page 21: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/21.jpg)
Developer Version with FilterSQP (CPU)
0
20
40
60
80
100
1 10 100 1000
% o
f pro
ble
ms
not more than x times worse than best
Performance
HybridQGOA
Andreas Wächter (IBM) MINLP SIOPT 2008 11 / 30
![Page 22: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/22.jpg)
The Success Story Of MILP
In: Bixby, Fenelon, Gu, Rothberg, Wunderling (2004)Mixed-Integer Programming: A Progress Report
What lead to the dramatic improvement of MILP solvers?
Andreas Wächter (IBM) MINLP SIOPT 2008 12 / 30
![Page 23: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/23.jpg)
The Success Story Of MILP
In: Bixby, Fenelon, Gu, Rothberg, Wunderling (2004)Mixed-Integer Programming: A Progress Report
What lead to the dramatic improvement of MILP solvers?
Very efficient node solvers
Andreas Wächter (IBM) MINLP SIOPT 2008 12 / 30
![Page 24: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/24.jpg)
The Success Story Of MILP
In: Bixby, Fenelon, Gu, Rothberg, Wunderling (2004)Mixed-Integer Programming: A Progress Report
What lead to the dramatic improvement of MILP solvers?
Very efficient node solvers
Variable/node selection
Primal heuristics
Presolve
Cutting planes
Andreas Wächter (IBM) MINLP SIOPT 2008 12 / 30
![Page 25: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/25.jpg)
The Success Story Of MILP
In: Bixby, Fenelon, Gu, Rothberg, Wunderling (2004)Mixed-Integer Programming: A Progress Report
What lead to the dramatic improvement of MILP solvers?
Very efficient node solvers
Variable/node selection
Primal heuristics
Presolve
Cutting planes
What can we learn from this for a B&B-based method for MINLP?
Andreas Wächter (IBM) MINLP SIOPT 2008 12 / 30
![Page 26: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/26.jpg)
Branch-and-bound: Variable Selection
LB=4
LB=5
UB=7
LB=6
LB=8
x3=0 x3=1
x2=0 x2=1
x1=0 x1=1
integerfeasible
infeasible
Andreas Wächter (IBM) MINLP SIOPT 2008 13 / 30
![Page 27: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/27.jpg)
Variable Selection
Some possible options:
Random
Most-fractional (most integer-infeasible)- used in MINLP-BB [Fletcher, Leyffer]
Andreas Wächter (IBM) MINLP SIOPT 2008 14 / 30
![Page 28: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/28.jpg)
Variable Selection
Some possible options:
Random
Most-fractional (most integer-infeasible)- used in MINLP-BB [Fletcher, Leyffer]
Strong branching [Applegate et al. (1995)]
Pseudo costs [Benichou et al. (1971), Forrest et al. (1974)]- optional in SBB [GAMS]
Reliability branching [Achterberg et al. (2005)]
Andreas Wächter (IBM) MINLP SIOPT 2008 14 / 30
![Page 29: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/29.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
x? =0 x? =1
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 30: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/30.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
Idea: Try some candidatesxi1 ,
xi1 =0 xi1 =1
LB0i1
LB1i1
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 31: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/31.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
Idea: Try some candidatesxi1 , xi2 , . . .
xi2 =0 xi2 =1
LB0i2
LB1i2
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 32: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/32.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
Idea: Try some candidatesxi1 , xi2 , . . .
Choose candidate with largestLB0
i and LB1i
xi2 =0 xi2 =1
LB0i2
LB1i2
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 33: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/33.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
Idea: Try some candidatesxi1 , xi2 , . . .
Choose candidate with largestLB0
i and LB1i
If candidate’s child infeasible:fix variable
xi2 =0 xi2 =1
LB1i2
infeasible
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 34: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/34.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
Idea: Try some candidatesxi1 , xi2 , . . .
Choose candidate with largestLB0
i and LB1i
If candidate’s child infeasible:fix variable
If LB0/1
i > UB : fix variable
xi2 =0 xi2 =1
LB1i2
LB0i2
> UB
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 35: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/35.jpg)
Strong Branching
Q: Which variable xi should bebranched on?
Idea: Try some candidatesxi1 , xi2 , . . .
Choose candidate with largestLB0
i and LB1i
If candidate’s child infeasible:fix variable
If LB0/1
i > UB : fix variable
Requires to solve many relaxations
xi2 =0 xi2 =1
LB1i2
LB0i2
> UB
Andreas Wächter (IBM) MINLP SIOPT 2008 15 / 30
![Page 36: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/36.jpg)
Strong Branching Improvements
Approximate node solutions
For MILP: Limit the number of simplex iterations◮ Dual simplex algorithm gives valid bounds
Andreas Wächter (IBM) MINLP SIOPT 2008 16 / 30
![Page 37: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/37.jpg)
Strong Branching Improvements
Approximate node solutions
For MILP: Limit the number of simplex iterations◮ Dual simplex algorithm gives valid bounds
For MINLP: Solve approximation problem◮ LP: Linearize functions at parent solution◮ QP: Use QP from last SQP iteration (BQPD [Fletcher])
Andreas Wächter (IBM) MINLP SIOPT 2008 16 / 30
![Page 38: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/38.jpg)
Strong Branching Improvements
Approximate node solutions
For MILP: Limit the number of simplex iterations◮ Dual simplex algorithm gives valid bounds
For MINLP: Solve approximation problem◮ LP: Linearize functions at parent solution◮ QP: Use QP from last SQP iteration (BQPD [Fletcher])
Can use hot-starts (reuse factorization)◮ Only one bound changes
Andreas Wächter (IBM) MINLP SIOPT 2008 16 / 30
![Page 39: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/39.jpg)
Strong Branching Improvements
Pseudo costs
Idea: Collect statistical data about the effect of fixing each xi :◮ Average change in LB0
iand LB1
iper unit change in xi
(up and down change separately)
Use to estimate LB0i and LB1
i of child nodes
Andreas Wächter (IBM) MINLP SIOPT 2008 17 / 30
![Page 40: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/40.jpg)
Strong Branching Improvements
Pseudo costs
Idea: Collect statistical data about the effect of fixing each xi :◮ Average change in LB0
iand LB1
iper unit change in xi
(up and down change separately)
Use to estimate LB0i and LB1
i of child nodes
Initialize with strong branching
Update each time a node has been solved
Andreas Wächter (IBM) MINLP SIOPT 2008 17 / 30
![Page 41: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/41.jpg)
Strong Branching Improvements
Pseudo costs
Idea: Collect statistical data about the effect of fixing each xi :◮ Average change in LB0
iand LB1
iper unit change in xi
(up and down change separately)
Use to estimate LB0i and LB1
i of child nodes
Initialize with strong branching
Update each time a node has been solved
Reliability branching
Pseudo costs, but do strong-branching on non-trusted variables
Limit the number of strong-branching solves
Andreas Wächter (IBM) MINLP SIOPT 2008 17 / 30
![Page 42: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/42.jpg)
Variable Selection
Comparative experiments in literature:
MILP
◮ Linderoth, Savelsbergh (1999):- Pseudo costs work very well
◮ Achterberg, Koch, Martin (2005):- Reliability branching best- Most-fractional about as good as Random
Andreas Wächter (IBM) MINLP SIOPT 2008 18 / 30
![Page 43: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/43.jpg)
Variable Selection
Comparative experiments in literature:
MILP
◮ Linderoth, Savelsbergh (1999):- Pseudo costs work very well
◮ Achterberg, Koch, Martin (2005):- Reliability branching best- Most-fractional about as good as Random
MINLP
◮ Gupta, Ravindran (1985)- Most-fractional works best
Andreas Wächter (IBM) MINLP SIOPT 2008 18 / 30
![Page 44: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/44.jpg)
Branch-And-Bound Comparison (# Nodes)
0
20
40
60
80
100
1 10 100 1000
% o
f pro
ble
ms
not more than x times worse than best
Performance
RandomMostFra
StrongNLPStrongQP
PseudoNLPPseudoQP
Andreas Wächter (IBM) MINLP SIOPT 2008 19 / 30
![Page 45: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/45.jpg)
Branch-And-Bound Comparison (CPU time)
0
20
40
60
80
100
1 10 100 1000
% o
f pro
ble
ms
not more than x times worse than best
Performance
RandomMostFra
StrongNLPStrongQP
PseudoNLPPseudoQP
Andreas Wächter (IBM) MINLP SIOPT 2008 20 / 30
![Page 46: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/46.jpg)
B&B and Hybrid Comparison
0
20
40
60
80
100
1 10 100 1000
% o
f pro
ble
ms
not more than x times worse than best
Performance
PseudoQPHybrid
Andreas Wächter (IBM) MINLP SIOPT 2008 21 / 30
![Page 47: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/47.jpg)
Experiments Summary
Strong-branching, pseudo-costs work for nonlinear B&B◮ Hot-started QP approximations improve performance◮ LP approximation not efficient◮ In these experiments: Reliability branching not helpful
Andreas Wächter (IBM) MINLP SIOPT 2008 22 / 30
![Page 48: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/48.jpg)
Experiments Summary
Strong-branching, pseudo-costs work for nonlinear B&B◮ Hot-started QP approximations improve performance◮ LP approximation not efficient◮ In these experiments: Reliability branching not helpful
B&B competitive to OA-based Hybrid method◮ Methods should “learn from each other”
- e.g., use nonlinear strong-branching in Hybrid approach
Best choice depends on problem instance
◮ Need to identify relevant problem characteristics
Andreas Wächter (IBM) MINLP SIOPT 2008 22 / 30
![Page 49: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/49.jpg)
Experiments Summary
Strong-branching, pseudo-costs work for nonlinear B&B◮ Hot-started QP approximations improve performance◮ LP approximation not efficient◮ In these experiments: Reliability branching not helpful
B&B competitive to OA-based Hybrid method◮ Methods should “learn from each other”
- e.g., use nonlinear strong-branching in Hybrid approach
Best choice depends on problem instance
◮ Need to identify relevant problem characteristics
Number of nodes for solved problems:
Min Max GeoMean
Hybrid 8 436393 6226.5StrongQP 14 2033352 1685.8
Andreas Wächter (IBM) MINLP SIOPT 2008 22 / 30
![Page 50: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/50.jpg)
Node SolversIn MILP:
Very efficient implementation of dual simplex◮ Tailored to B&B: Changes in bounds; added cuts
Hot-starts (reusing factorization) extremely efficient
Andreas Wächter (IBM) MINLP SIOPT 2008 23 / 30
![Page 51: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/51.jpg)
Node SolversIn MILP:
Very efficient implementation of dual simplex◮ Tailored to B&B: Changes in bounds; added cuts
Hot-starts (reusing factorization) extremely efficient
In MINLP:
NLP solvers now much more robust and efficient than in the past◮ For trimloss4: Solved >2,000,000 NLPs! (105 [85] var, 64 con)
Andreas Wächter (IBM) MINLP SIOPT 2008 23 / 30
![Page 52: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/52.jpg)
Node SolversIn MILP:
Very efficient implementation of dual simplex◮ Tailored to B&B: Changes in bounds; added cuts
Hot-starts (reusing factorization) extremely efficient
In MINLP:
NLP solvers now much more robust and efficient than in the past◮ For trimloss4: Solved >2,000,000 NLPs! (105 [85] var, 64 con)
Large-scale problems:◮ Large-scale active-set methods?◮ Combine interior-point and active-set methods?
Andreas Wächter (IBM) MINLP SIOPT 2008 23 / 30
![Page 53: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/53.jpg)
Node SolversIn MILP:
Very efficient implementation of dual simplex◮ Tailored to B&B: Changes in bounds; added cuts
Hot-starts (reusing factorization) extremely efficient
In MINLP:
NLP solvers now much more robust and efficient than in the past◮ For trimloss4: Solved >2,000,000 NLPs! (105 [85] var, 64 con)
Large-scale problems:◮ Large-scale active-set methods?◮ Combine interior-point and active-set methods?
Hot-starts possible?
Andreas Wächter (IBM) MINLP SIOPT 2008 23 / 30
![Page 54: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/54.jpg)
Node SolversIn MILP:
Very efficient implementation of dual simplex◮ Tailored to B&B: Changes in bounds; added cuts
Hot-starts (reusing factorization) extremely efficient
In MINLP:
NLP solvers now much more robust and efficient than in the past◮ For trimloss4: Solved >2,000,000 NLPs! (105 [85] var, 64 con)
Large-scale problems:◮ Large-scale active-set methods?◮ Combine interior-point and active-set methods?
Hot-starts possible?
Storing warm-start information more memory intensive◮ In experiments: Use optimal solution of root node
Andreas Wächter (IBM) MINLP SIOPT 2008 23 / 30
![Page 55: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/55.jpg)
Node SolversIn MILP:
Very efficient implementation of dual simplex◮ Tailored to B&B: Changes in bounds; added cuts
Hot-starts (reusing factorization) extremely efficient
In MINLP:
NLP solvers now much more robust and efficient than in the past◮ For trimloss4: Solved >2,000,000 NLPs! (105 [85] var, 64 con)
Large-scale problems:◮ Large-scale active-set methods?◮ Combine interior-point and active-set methods?
Hot-starts possible?
Storing warm-start information more memory intensive◮ In experiments: Use optimal solution of root node
Need fast detection of infeasibility
Andreas Wächter (IBM) MINLP SIOPT 2008 23 / 30
![Page 56: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/56.jpg)
Cuts
Approximate convex hull of integer-feasible points◮ Strengthen the relaxation
Andreas Wächter (IBM) MINLP SIOPT 2008 24 / 30
![Page 57: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/57.jpg)
Cuts
Approximate convex hull of integer-feasible points◮ Strengthen the relaxation
MILP: (hot topic over past 30 years)◮ Many cut generators available (many easy to compute)
Andreas Wächter (IBM) MINLP SIOPT 2008 24 / 30
![Page 58: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/58.jpg)
Cuts
Approximate convex hull of integer-feasible points◮ Strengthen the relaxation
MILP: (hot topic over past 30 years)◮ Many cut generators available (many easy to compute)
MINLP:◮ For linear parts, can use MILP machinery directly
- Hybrid method works with linear formulation- B&B: could work with linearizations
Andreas Wächter (IBM) MINLP SIOPT 2008 24 / 30
![Page 59: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/59.jpg)
Cuts
Approximate convex hull of integer-feasible points◮ Strengthen the relaxation
MILP: (hot topic over past 30 years)◮ Many cut generators available (many easy to compute)
MINLP:◮ For linear parts, can use MILP machinery directly
- Hybrid method works with linear formulation- B&B: could work with linearizations
◮ Some research specific for nonlinear case:- Stubbs, Mehrotra (1999, 2002)- Atamtürk, Narayanan (2007)- . . .
◮ Can also use nonlinear cuts
◮ Ideally: Need access to problem representation (expression tree)
Andreas Wächter (IBM) MINLP SIOPT 2008 24 / 30
![Page 60: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/60.jpg)
Other MILP techniques
Primal heuristics (quickly finding good integer feasible points)
Have answer when time limit exceeded
Improve upper bounds (e.g, for strong branching)
Andreas Wächter (IBM) MINLP SIOPT 2008 25 / 30
![Page 61: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/61.jpg)
Other MILP techniques
Primal heuristics (quickly finding good integer feasible points)
Have answer when time limit exceeded
Improve upper bounds (e.g, for strong branching)
MILP: A dozen generic heuristics (root node and in tree)(hot topic over last 7 years)
MINLP: Preliminary work, e.g.,- Nonlinear feasibility pump [Bonami et al. (2006)]
Andreas Wächter (IBM) MINLP SIOPT 2008 25 / 30
![Page 62: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/62.jpg)
Other MILP techniques
Node selection
In experiments: Use “best-bound” (node with smallest LB)
Diving- Quickly find integer solution- Allows hot-starts when proceeding to child nodes
Andreas Wächter (IBM) MINLP SIOPT 2008 26 / 30
![Page 63: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/63.jpg)
Other MILP techniques
Node selection
In experiments: Use “best-bound” (node with smallest LB)
Diving- Quickly find integer solution- Allows hot-starts when proceeding to child nodes
Presolve (tighten and simplify formulation)
At root node and in search tree
MILP: Just look at coefficients of linear functions
MINLP: General nonlinear functions difficult to predict- Requires access to problem representation
(e.g., expression tree)
Andreas Wächter (IBM) MINLP SIOPT 2008 26 / 30
![Page 64: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/64.jpg)
What is Good Modeling?
Example: Uncapacitated facility location problem
min∑n
i=1 cixi +∑n
i=1
∑mj=1 dijyij
s.t .∑m
j=1 yij = 1 (i = 1, . . . , n)
Weak :∑n
i=1 yij ≤ n · xi (j = 1, . . . , m)
Strong : yij ≤ xi (i = 1, . . . , n; j = 1, . . . , m)
x ∈ {0, 1}n, y ∈ R
m+
MILP MINLP
n = 30, m = 100 nodes time nodes time
weak formulation 46,294 143.16strong formulation 0 0.18
Andreas Wächter (IBM) MINLP SIOPT 2008 27 / 30
![Page 65: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/65.jpg)
What is Good Modeling?
Example: Uncapacitated facility location problem
min∑n
i=1 cixi +∑n
i=1
∑mj=1 dijy2
ij
s.t .∑m
j=1 yij = 1 (i = 1, . . . , n)
Weak :∑n
i=1 yij ≤ n · xi (j = 1, . . . , m)
Strong : yij ≤ xi (i = 1, . . . , n; j = 1, . . . , m)
x ∈ {0, 1}n, y ∈ R
m+
MILP MINLP
n = 30, m = 100 nodes time nodes time
weak formulation 46,294 143.16 46,384 8117.52strong formulation 0 0.18 30,112 7888.24
Andreas Wächter (IBM) MINLP SIOPT 2008 27 / 30
![Page 66: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/66.jpg)
What is Good Modeling?
Example: Uncapacitated facility location problem
min∑n
i=1 cixi +∑n
i=1
∑mj=1 dijy2
ij
s.t .∑m
j=1 yij = 1 (i = 1, . . . , n)
Weak :∑n
i=1 yij ≤ n · xi (j = 1, . . . , m)
Strong : yij ≤ xi (i = 1, . . . , n; j = 1, . . . , m)
x ∈ {0, 1}n, y ∈ R
m+
MILP MINLP
n = 30, m = 100 nodes time nodes time
weak formulation 46,294 143.16 46,384 8117.52strong formulation 0 0.18 30,112 7888.24
weak with cuts/presolve 25 2.71
Andreas Wächter (IBM) MINLP SIOPT 2008 27 / 30
![Page 67: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/67.jpg)
The Nonconvex Case
Global optimization already very difficult◮ Spatial branch-and-bound with convex under-estimators◮ Incorporation of discrete variables natural◮ Several algorithms and codes:
Alpha-BB [Adjiman et al.], BARON [Sahinidis, Tawarmalani],Couenne [Belotti et al.], LaGO, [Nowak, Vigerske], . . .
◮ Limitation in problem size
Andreas Wächter (IBM) MINLP SIOPT 2008 28 / 30
![Page 68: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/68.jpg)
The Nonconvex Case
Global optimization already very difficult◮ Spatial branch-and-bound with convex under-estimators◮ Incorporation of discrete variables natural◮ Several algorithms and codes:
Alpha-BB [Adjiman et al.], BARON [Sahinidis, Tawarmalani],Couenne [Belotti et al.], LaGO, [Nowak, Vigerske], . . .
◮ Limitation in problem size
Heuristics based on convex MINLP algorithms◮ Outer-approximation based (e.g., DICOPT [Grossmann et al.])
- use one side of equality constraints based on multipliers- allow penalized slack in OA cuts- delete violated OA cuts
Andreas Wächter (IBM) MINLP SIOPT 2008 28 / 30
![Page 69: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/69.jpg)
The Nonconvex Case
Global optimization already very difficult◮ Spatial branch-and-bound with convex under-estimators◮ Incorporation of discrete variables natural◮ Several algorithms and codes:
Alpha-BB [Adjiman et al.], BARON [Sahinidis, Tawarmalani],Couenne [Belotti et al.], LaGO, [Nowak, Vigerske], . . .
◮ Limitation in problem size
Heuristics based on convex MINLP algorithms◮ Outer-approximation based (e.g., DICOPT [Grossmann et al.])
- use one side of equality constraints based on multipliers- allow penalized slack in OA cuts- delete violated OA cuts
◮ Nonlinear branch-and-bound- resolve NLPs from different starting points- do not trust lower bounds or infeasibilities
Andreas Wächter (IBM) MINLP SIOPT 2008 28 / 30
![Page 70: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/70.jpg)
Conclusions
Encouraging progress◮ New algorithms and implementations (e.g., Bonmin, FilMINT)◮ Outer-approximation based algorithms
- MILP framework with NLP solves◮ Nonlinear branch-and-bound
- Pseudo costs, QP-based strong branching
Andreas Wächter (IBM) MINLP SIOPT 2008 29 / 30
![Page 71: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/71.jpg)
Conclusions
Encouraging progress◮ New algorithms and implementations (e.g., Bonmin, FilMINT)◮ Outer-approximation based algorithms
- MILP framework with NLP solves◮ Nonlinear branch-and-bound
- Pseudo costs, QP-based strong branching
Many open questions◮ Can we repeat the success of MILP?
- Further explore MILP techniques in the nonlinear case- Robust large-scale NLP solvers with hot starts?- Devise specific nonlinear techniques (e.g., cuts)
Andreas Wächter (IBM) MINLP SIOPT 2008 29 / 30
![Page 72: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/72.jpg)
Conclusions
Encouraging progress◮ New algorithms and implementations (e.g., Bonmin, FilMINT)◮ Outer-approximation based algorithms
- MILP framework with NLP solves◮ Nonlinear branch-and-bound
- Pseudo costs, QP-based strong branching
Many open questions◮ Can we repeat the success of MILP?
- Further explore MILP techniques in the nonlinear case- Robust large-scale NLP solvers with hot starts?- Devise specific nonlinear techniques (e.g., cuts)
◮ Nonconvex problems
Andreas Wächter (IBM) MINLP SIOPT 2008 29 / 30
![Page 73: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/73.jpg)
Conclusions
Encouraging progress◮ New algorithms and implementations (e.g., Bonmin, FilMINT)◮ Outer-approximation based algorithms
- MILP framework with NLP solves◮ Nonlinear branch-and-bound
- Pseudo costs, QP-based strong branching
Many open questions◮ Can we repeat the success of MILP?
- Further explore MILP techniques in the nonlinear case- Robust large-scale NLP solvers with hot starts?- Devise specific nonlinear techniques (e.g., cuts)
◮ Nonconvex problems◮ Implementation
- Collaboration essential (through open source?)- “Accessible” nonlinear problem representation- Parallel implementation
Andreas Wächter (IBM) MINLP SIOPT 2008 29 / 30
![Page 74: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/74.jpg)
Conclusions
Encouraging progress◮ New algorithms and implementations (e.g., Bonmin, FilMINT)◮ Outer-approximation based algorithms
- MILP framework with NLP solves◮ Nonlinear branch-and-bound
- Pseudo costs, QP-based strong branching
Many open questions◮ Can we repeat the success of MILP?
- Further explore MILP techniques in the nonlinear case- Robust large-scale NLP solvers with hot starts?- Devise specific nonlinear techniques (e.g., cuts)
◮ Nonconvex problems◮ Implementation
- Collaboration essential (through open source?)- “Accessible” nonlinear problem representation- Parallel implementation
Need representative real-world test problems
Andreas Wächter (IBM) MINLP SIOPT 2008 29 / 30
![Page 75: Some Recent Advances in Mixed-Integer Nonlinear Programming · Mixed-Integer Nonlinear Programming (MINLP) min f (x,y) s.t. c(x,y) ≤ 0 yL ≤ y ≤ yU x ∈ {0,1}n,y ∈ Rp f,c](https://reader034.vdocuments.us/reader034/viewer/2022042909/5f3af26f9a63780da87f5b48/html5/thumbnails/75.jpg)
Thank you!
Andreas Wächter (IBM) MINLP SIOPT 2008 30 / 30