analyzing the computational impact of individual minlp solver … · 2019-12-19 · analyzing the...

Analyzing the computational impact ofindividual MINLP solver components

Stefan Vigerskejoint work with Ambros M. Gleixner

Zuse Institute Berlin · GAMS

MIP 2014, July 21, The Ohio State University

1

Outline

Analyzing MINLP solver components

Software, Hardware, Methodology

Separation

Reformulation

Primal Heuristics

Tree search

Propagation

2

Outline



Separation

Reformulation

Primal Heuristics

Tree search

Propagation

3

The Solver: SCIP

. a branch-cut-and-price framework

. a full-scale MIP and MINLP solver

. free for academic purposes, source code available, http://scip.zib.deMIP

. LP relaxation

. cutting planes

. column generation

MIP, GO, CP, and SAT

. branch-and-bound

GO

. spatial branching

CP

. domain propagation

SAT

. conflict analysis

. periodic restarts

SCIP PrimalHeuristic

actconsdiving

coefdiving

crossover dins

feaspump

fixandinfer

fracdiving

guideddiving

intdiving

intshifting

linesearchdiving

localbranching

mutation

subnlp

objpscostdiving

octane

oneopt

pscostdiving

rensrins

rootsoldiving

rounding

shifting

shift&prop

simplerounding

trivial

trysol

twooptundercover

veclendiving

zi round

Variable

Event

default

Branch

allfullstrong

fullstrong

inference

leastinf

mostinf

pscostrandom

relpscost

Conflict

ConstraintHandler

and

bounddisjunc.

countsols cumu

lative

indicator

integral

knapsack

linear

linking

logicor

ororbitope

quadratic

setppc

soc

sos1

sos2

varbound

xor

Cutpool

LP

clp

cpxmsk

none

qso

spx

xprs

Dialog

default

Display

default

Nodeselector

bfs

dfs

estimate

hybridestim

restartdfs

· · ·

Presolver

boundshift

dualfix

implics

inttobinary

probing

trivial

Implications

Tree

Reader

ccg

cip

cnf

fix

lp

mps opb

ppm

rlp

sol

sos

zpl

Pricer

Separator

clique

cmir

flowcover

gomory

impliedbounds intobj

mcf

oddcycle

rapidlearn

redcost

strongcg

zerohalf

Propagator

pseudoobj

rootredcost

vbound

Relax

4

http://scip.zib.de

The Instances: MINLPLib 1 → 2

MINLPLib. a collection of MINLP instances (trivial . . . challenging). GAMS scalar format, part of GAMS World / MINLP WorldNext version (in development). more instances, more file formats, more statistics, . . .. currently 822 publicly available MINLP instances. collected from MINLPLib 1, minlp.org, POLIP, . . .. see http://www.gamsworld.org/minlp/minlplib2/html/

2002 2004 2006 2008 2010 2012 2014100

200

300

400

500

600

700

800

900MINLPLib - Number of Instances

MINLPLib 1new in MINLPLib 2

If you have interesting instances, please consider contributing.5

http://www.gamsworld.org/minlp/minlplib2/html/

789 choose 475

The Testset. take MINLPLib2α (as of April’14): 789 instances. run SCIP with default settings. 475 instances solved within 2 hours. 455 instances solved within 1 hour⇒ subsequent experiments: the set of 475 instances, 1 hour time limit

Hardware. Dell PowerEdge M1000e, 48GB RAM, Intel Xeon [email protected]

Software. SCIP 3.1.0.1. SoPlex 2.0. Ipopt 3.11.8. CppAD 20140000.1

6

Averaging over heterogeneous test sets

Instances vary widely in size, nonlinearity, . . .

, time to optimality. arithmetic average: dominated by large times. geometric average: weights trivial and hard instances equally. shifted geometric average: which shift?

Some results are not distinguished by performance profiles alone:

inst A B

1 10s 2s2 10s 2s3 10s 50s4 10s 50s

7


Instances vary widely in size, nonlinearity, . . . , time to optimality. arithmetic average: dominated by large times. geometric average: weights trivial and hard instances equally. shifted geometric average: which shift?


inst A B

1 10s 2s2 10s 2s3 10s 50s4 10s 50s

7




inst A B

1 5x 1x2 5x 1x3 1x 5x4 1x 5x

7




inst A B

1 10s 2s2 20s 100s3 50s 10s4 100s 500s

7

The Method: Filtered Performance Diagrams

Gradually exclude instances solved by A and B and compute speedup:

t 7→µ({tA,i : max{tA,i , tB,i} ≥ t})µ({tB,i : max{tA,i , tB,i} ≥ t})

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1

time

In the following: µ = geometric mean [See also Achterberg and Wunderling 2013]

8

The Method: Filtered Performance Diagrams

Gradually exclude instances solved by A and B and compute speedup:

t 7→µ({NA,i : max{tA,i , tB,i} ≥ t})µ({NB,i : max{tA,i , tB,i} ≥ t})

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1

time nodes

In the following: µ = geometric mean [See also Achterberg and Wunderling 2013]

8

Number of unsolved instances by time (default settings)

t 7→ |{i : ti ≥ t}|

0 600 1,200 1,800 2,400 3,000 3,6000

100

200

300

400

9

Outline



Separation

Reformulation

Primal Heuristics

Tree search

Propagation

10

Separation: MIP cutting planes

. General: Gomory, cMIR, {0, 1/2}-cuts, . . .

. Problem-specific: knapsack, clique, multi commodityflow, . . .

Default Settings. run certain separators during root node. no separation during tree search

Alternative Setting I: off

Alternative Setting II: aggressive. run separators also during tree search. run previously disabled separators during root node

11


all maxtime ≥ 100

setting solved time nodes time nodes

MIP cuts off −39 +65% +107% +333% +395%MIP cuts aggr −11 −7% −10% −18% −23%

0 600 1,200 1,800 2,400 3,000 3,600

0.25

0.5

0.75

1

1.25

1.5MIP cuts off MIP cuts aggr

12


Distribution of Speedups

<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

200

250

time(default)/time(MIP cuts off)

#instances

MIP cuts off

<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

200

250

time(default)/time(MIP cuts aggr)

MIP cuts aggressive

13

Separation: Approximation of Nonlinearities

Gradient cuts for convex terms. feasibility enforced without branching. exploit integer information for univariate

convex terms-1.0 -0.5 0.0 0.5 1.0

-1.0

-0.5

0.0

0.5

1.0

Linear underestimators for nonconvex termsconcave functions

0.5 1.0 1.5 2.0 2.5 3.0

-2.0

-1.5

-1.0

-0.5

0.5

1.0

x |x |n, n ≥ 0

-1.0 -0.5 0.5 1.0

-1.0

-0.5

0.5

1.0

x · y

Alternative setting:. off during fractional branching. thus, weak relaxation of nonlinearities while branching on fractionalities

14


all maxtime ≥ 100


nonlin sepa off −102 +302% +695% +1964% +5569%

0 600 1,200 1,800 2,400 3,000 3,6000

5 · 10−2

0.1

0.15

0.2nonlin sepa off

15



<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

time(default)/time(nonlin sepa off)

#instances

nonlin sepa off

16

Outline



Separation

Reformulation

Primal Heuristics

Tree search

Propagation

17

Reformulation

Expression graph reformulation. merge expressions, e.g., polynomials. replace subexpressions with new variables. when switched off, only a very simple relaxation

based on interval gradients is generatedi1

[200,200]

x3

[0,100]

i2

[200,200]

900+c0^2

[40900,40900]

c0

7200+c0^2

[47200,47200]

c0

2960.88+18505.5*c0^2

[7.4e+08,7.4e+08]

c0

-c0c1c3+420.169c2^0.5

[-3.91503e+06,84974]

c3c0 c1

c2

/

[15682.7,15682.7]

c1 c0

0.0471c0c1^0.5

[1906.04,1906.04]

c0

c1

Products with binary variables. linearize using big-M

x ·∑

k akyk with x ∈ {0, 1}↓

MLx ≤ w ≤ MUx ,∑k akyk −MU(1− x) ≤ w ≤

∑k akyk −ML(1− x)

18

Reformulation

all maxtime ≥ 100


expr reform off −69 +160% +322% +1386% +3631%bin reform off −9 +8% −11% +20% −21%

0 600 1,200 1,800 2,400 3,000 3,6000

0.25

0.5

0.75

1

1.25

expr reform off bin reform off

19

Reformulation


<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

time(default)/time(expr reform off)

#instances

expr reform off

<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

time(default)/time(binary reform off)

binary reform off

20

Outline



Separation

Reformulation

Primal Heuristics

Tree search

Propagation

21

Primal Heuristics

Besides waiting for feasible LP solutions . . .

Standard MIP heuristics applied to MIP relaxation. rounding, diving, feasibility pump, . . .. large neighborhood search (RENS, RINS, ...)

NLP local search. for integer and LP feasible solutions. fix integers and solve remaining NLP (Ipopt)

MINLP heuristics. NLP diving. RENS [Berthold 2013]. Undercover [Berthold and Gleixner 2013]. . . .

min

22

Primal Heuristics

all maxtime ≥ 100


all heur off −19 +7% +36% +84% +144%only NLP −11 −4% +22% +33% +22%LNS heur off −10 +4% +20% +51% +71%

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1

all heur off only NLP LNS heur off

23

Primal Heuristics


<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

200

time(default)/time(heur off)

#instances

heuristics off

<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

200

time(default)/time(only NLP)

only NLP

24

Primal Heuristics

all maxtime ≥ 100


heur aggressive −2 +27% −4% +28% +86%

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1heur aggressive

25

Primal Heuristics


<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

time(default)/time(heur aggr)

#instances

heuristics aggressive

26

Outline



Separation

Reformulation

Primal Heuristics

Tree search

Propagation

27

Branching

Reliability (MIP) Inference1 (CP) VSIDS2 (SAT)

1. on “fractional” integer vars

2. on vars in violated nonlinear terms

pseudo-cost (GO)-1.0 -0.5 0.5 1.0

0.2

0.4

0.6

0.8

1.0

Alternative settings for spatial branchingI inference1, most infeasible, random

[See Tawarmalani and Sahinidis 2002, Achterberg and Berthold 2009, Belotti et al. 2009, . . . ]

1 Inference branching: prefer variables where branching resulted in high number of domain propagation before

2 VSIDS: prefer variables used to produce recent conflict constraints

28

Spatial Branching

all maxtime ≥ 100


inference −27 +31% +34% +167% +176%most inf −24 +30% +38% +165% +209%random −24 +30% +28% +145% +130%

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1 inference most inf random

29

Spatial Branching


<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

400

time(default)/time(most infeas)

#instances

Most-Infeasible Branching

<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

400

time(default)/time(random)

Random Branching

30

Node selection

TasksI improve primal boundI keep computational effort smallI improve global dual bound

Best estimate with plungingI select node Q with best/minimal

(pseudo cost) estimate value forfeasible solution objective value

z̄Q +∑

k:x̄k fractional

min{Ψ−f −,Ψ+f +}

I plunge (diving with single backtrack)

Alternative setting: breadth first search

31

Node selection

all maxtime ≥ 100


breadth first −22 +42% +29% +136% +81%

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1breadth first

32

Node Selection


<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

200

time(default)/time(breadth first)

#instances

Breadth First Node Selection

33

Conflict analysis / “nogood” learning

Analyse reason for pruning a nodeI branchings and propagationsI infeasible and bound exceeding

LP relaxation: dual ray heuristicI derive short nogoods/conflict

constraintsI most nonlinear constraints do not

participate in conflict analysis yet

Use subsequentlyI to cut off other nodesI to enable further propagationsI for VSIDS in branching

x1 − x3 ≤ 0

34

Conflict analysis / “nogood” learning

all maxtime ≥ 100


conflict off −2 +2% +9% +11% +27%

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1conflict off

35

Conflict Analysis / “nogood” learning


<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

time(default)/time(conflict off)

#instances

Conflict Analysis Off

36

Outline



Separation

Reformulation

Primal Heuristics

Tree search

Propagation

37

Bound tightening/propagation

Particularly important for nonconvex MINLP. branching on continuous variables/infinite domains. tight domains tight relaxation

Primal and dual reductions. reduced cost. probing on binaries. FBBT: feasibility-based bound tightening. OBBT: optimization-based bound tightening

and Lagrangian variable bounds:

min xk

λ2

λ1

xk ≥∑

i :ri>0

ri x i +∑

i :ri<0

ri x i + µ cT x∗ + λT b

[Ryoo and Sahinidis 1996, Belotti et al. 2009, Gleixner and Weltge 2013, . . . ]

38

Propagating Lagrangian Variable Bounds (LVBs)

The right-hand side of xk ≥ rT x + rT x + µcT x∗ + λTb is tightenedI if some variable lower bound x i increases for r i > 0I if some variable upper bound x i decreases for r i < 0I if a better primal solution x∗ is found and µ < 0

Learn LVBs during root OBBT and propagate againI locally at nodes of the branch-and-bound treeI globally if a better primal solution is foundI compare “duality-based reduction” [Tawarmalani and Sahinidis 2004]

Computational ExperienceI on every other MINLP, at least one nontrivial LVB from every 2nd

OBBT LPI LVB propagation typically ≤ 2% of total running time, when

implemented efficiently

This promises a computationally cheap approximation of OBBT in the tree.

[Gleixner and Weltge 2013]

39

Propagating Lagrangian Variable Bounds (LVBs)

The right-hand side of xk ≥ rT x + rT x + µcT x∗ + λTb is tightenedI if some variable lower bound x i increases for r i > 0I if some variable upper bound x i decreases for r i < 0I if a better primal solution x∗ is found and µ < 0

Learn LVBs during root OBBT and propagate againI locally at nodes of the branch-and-bound treeI globally if a better primal solution is foundI compare “duality-based reduction” [Tawarmalani and Sahinidis 2004]

Computational ExperienceI on every other MINLP, at least one nontrivial LVB from every 2nd

OBBT LPI LVB propagation typically ≤ 2% of total running time, when

implemented efficiently

This promises a computationally cheap approximation of OBBT in the tree.[Gleixner and Weltge 2013]

39


all maxtime ≥ 100


propagation off −48 +90% +129% +332% +378%OBBT off −25 +47% +93% +198% +396%LVB off −4 +6% +9% +18% +17%

0 600 1,200 1,800 2,400 3,000 3,6000

0.2

0.4

0.6

0.8

1

propagation off OBBT off LVB off

40



<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

50

100

150

200

time(default)/time(prop off)

#instances

Propagation off

41



<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

time(default)/time(OBBT off)

#instances

OBBT off

<0.01

(0.01, 0.1)

(0.1,0.5

)

(0.5,0.9

)

(0.9,1.1

)

(1.1,2)

(2, 10)

(10,100)

≥100

0

100

200

300

time(default)/time(LVB off)

LVB off

42

Summaryall maxtime ≥ 100


nonlin sepa off −102 +302% +695% +1964% +5569%expr reform off −69 +160% +322% +1386% +3631%propagation off −48 +90% +129% +397% +461%MIP cuts off −39 +65% +107% +333% +395%inference branching −27 +31% +34% +167% +176%OBBT off −25 +47% +93% +303% +607%most inf branching −24 +30% +38% +165% +209%random branching −24 +30% +28% +145% +130%breadth first search −22 +42% +29% +136% +81%all heur off −19 +7% +36% +84% +144%MIP cuts aggr −11 −7% −10% −18% −23%only NLP heur −11 −4% +22% +33% +22%LNS heur off −10 +4% +20% +51% +71%bin reform off −9 +8% −11% +20% −21%LVB off −4 +6% +9% +20% +19%heur aggressive −2 +27% −4% +28% +86%conflict off −2 +2% +9% +11% +27%

43

analyzing the computational impact of individual minlp solver … · 2019-12-19 · analyzing the...

Documents