carla p. gomes school on optimization cpaior02 exploiting structure and randomization in...

Carla P. Gomes

School on Optimization CPAIOR02

Exploiting Structure and Randomization

in Combinatorial Search

Carla P. Gomesgomes@cs.cornell.edu

www.cs.cornell.edu/gomes

Intelligent Information Systems InstituteDepartment of Computer Science

Cornell University

Exploiting Structure and Randomization

in Combinatorial Search

Carla P. Gomesgomes@cs.cornell.edu

Intelligent Information Systems InstituteDepartment of Computer Science

Cornell University

Carla P. Gomes

OutlineOutline

A Structured Benchmark Domain

Randomization

Conclusions

Carla P. Gomes

OutlineOutline

Randomization

Conclusions

Carla P. Gomes

Given an N X N matrix, and given N colors, a quasigroup of order N is a a colored matrix, such that:

-all cells are colored.

- each color occurs exactly once in each row.

- each color occurs exactly once in each column.

Quasigroup or Latin Square(Order 4)

Quasigroups or Latin Squares:An Abstraction for Real World Applications

Carla P. Gomes

Quasigroup Completion Problem (QCP)

Given a partial assignment of colors (10 colors in this case), can the partial quasigroup (latin square) be completed so we obtain a full quasigroup?

Example:

32% preassignment

(Gomes & Selman 97)

Carla P. Gomes

Quasigroup Completion Problem A Framework for Studying SearchQuasigroup Completion Problem

A Framework for Studying Search

NP-Complete.

Has a structure not found in random instances,

such as random K-SAT.

Leads to interesting search problems when structure is perturbed (more about it later).

Good abstraction for several real world problems: scheduling and timetabling, routing in fiber optics, coding, etc(Anderson 85, Colbourn 83, 84, Denes & Keedwell 94, Fujita et al. 93, Gent et al. 99, Gomes & Selman 97, Gomes et al. 98, Meseguer & Walsh 98, Stergiou and Walsh 99, Shaw et al. 98, Stickel 99, Walsh 99 )

Carla P. Gomes

Fiber Optic Networks

Nodesconnect point to point

fiber optic links

Carla P. Gomes

Fiber Optic Networks

Nodesconnect point to point

fiber optic links

Each fiber optic link supports alarge number of wavelengths

Nodes are capable of photonic switching --dynamic wavelength routing --

which involves the setting of the wavelengths.

Carla P. Gomes

Routing in Fiber Optic Networks

Routing Node

How can we achieve conflict-free routing in each node of the network?

Dynamic wavelength routing is a NP-hard problem.

Input Ports Output Ports1

preassigned channels

Carla P. Gomes

QCP Example Use: Routers in Fiber Optic Networks

Dynamic wavelength routing in Fiber Optic Networks can be directly mapped into the Quasigroup Completion Problem.

(Barry and Humblet 93, Cheung et al. 90, Green 92, Kumar et al. 99)

•each channel cannot be repeated in the same input port (row constraints);• each channel cannot be repeated in the same output port (column constraints);

CONFLICT FREELATIN ROUTER

Output ports

Input Port Output Port

Carla P. Gomes

Traditional View of Hard Problems - Worst Case View

“They’re NP-Complete—there’s no way to do anything but try heuristic approaches and hope for the best.”

Carla P. Gomes

New Concepts in ComputationNew Concepts in Computation

Not all NP-Hard problems are the same!

We now have means for discriminating easy from hard instances

---> Phase Transition concepts

Carla P. Gomes

NP-completeness is a worst-case notion – what about average

complexity?

Structural differences between instances of the same NP- complete problem (QCP)

NP-completeness is a worst-case notion – what about average

complexity?

Structural differences between instances of the same NP- complete problem (QCP)

Carla P. Gomes

Are all the Quasigroup Instances(of same size) Equally Difficult?

1820150

Time performance:

What is the fundamental difference between instances?

Carla P. Gomes

Are all the Quasigroup Instances

Equally Difficult?

1820 165

40% 50%

Time performance:

Fraction of preassignment:

Carla P. Gomes

Complexity of Quasigroup Completion

Fraction of pre-assignment

Critically constrained area

Overconstrained areaUnderconstrained

42% 50%20%

Carla P. Gomes

Phase Transition

Almost all unsolvable area

Fraction of pre-assignmentFra

Almost all solvable area

Complexity Graph

Phase transition from almost all solvableto almost all unsolvable

Carla P. Gomes

These results for the QCP - a structured domain, nicely complement previous results on phase transition and computational complexity for random instances such as SAT, Graph Coloring, etc.

(Broder et al. 93; Clearwater and Hogg 96, Cheeseman et al. 91, Cook and Mitchell 98, Crawford and Auton 93, Crawford and Baker 94, Dubois 90, Frank et al. 98, Frost and Dechter 1994, Gent and Walsh 95, Hogg, et al. 96, Mitchell et al. 1992, Kirkpatrick and Selman 94, Monasson et 99, Motwani et al. 1994, Pemberton and Zhang 96, Prosser 96, Schrag and Crawford 96, Selman and Kirkpatrick 97, Smith and Grant 1994, Smith and Dyer 96, Zhang and Korf 96, and more)

Carla P. Gomes

QCPDifferent Representations /

Encodings

QCPDifferent Representations /

Encodings

Carla P. Gomes

Cubic representation of QCP

Columns

Colors

Carla P. Gomes

QCP as a MIPQCP as a MIP

• Variables -

• Constraints -

}1,0{ijk

....,,2,1,,;, nkjikcolorhasjicellijk

....,,2,1,,1,

nkjii ijk

....,,2,1,,1,, nkjik ijk

....,,2,1,,1,

nkjij ijkx

Row/color line

Column/color line

Row/column line

Carla P. Gomes

QCP as a CSPQCP as a CSP

• Variables -

• Constraints -

}...,,2,1{, njix

....,,2,1,;,, njijicellofcolorjix

....,,2,1);,,...,2,

( ninixix

ixalldiff

....,,2,1);,,...,,2

( njjnxj

xalldiff

column

[ vs. for MIP])3(nO

[ vs. for MIP])2(nO

Carla P. Gomes

Exploiting Structure for Domain Reduction

• A very successful strategy for domain reduction in CSP is to exploit the structure of groups of constraints and treat them as global constraints.

Example using Network Flow Algorithms:

• All-different constraints

(Caseau and Laburthe 94, Focacci, Lodi, & Milano 99, Nuijten & Aarts 95, Ottososon & Thorsteinsson 00, Refalo 99, Regin 94 )

Carla P. Gomes

Exploiting Structure in QCPALLDIFF as Global Constraint

Two solutions:

we can update the domains of the column

variables

Analogously, we can update the domains of the other variables

Matching on a Bipartite graph

All-different constraint

(Berge 70, Regin 94, Shaw and Walsh 98 )

Carla P. Gomes

Exploiting StructureArc Consistency vs. All Diff

Arc ConsistencySolves up to order 20

Size search space 40020

AllDiffSolves up to order 33

Size searchspace 108933

Carla P. Gomes

Quasigroup as SatisfiabilityQuasigroup as Satisfiability

Two different encodings for SAT:

2D encoding (or minimal encoding);

3D encoding (or full encoding);

Carla P. Gomes

2D Encoding or Minimal Encoding2D Encoding or Minimal Encoding

Variables:

Each variables represents a color assigned to a cell.

Clauses:

• Some color must be assigned to each cell (clause of length n);

• No color is repeated in the same row (sets of negative binary clauses);

• No color is repeated in the same column (sets of negative binary clauses);

( ijnxij

....,,2,1,,;, nkjikcolorhasjicellijk

}1,0{ijk

Carla P. Gomes

3D Encoding or Full Encoding3D Encoding or Full Encoding

This encoding is based on the cubic representation of the quasigroup: each line of the cube contains exactly one true variable;

Variables:

Same as 2D encoding.

Clauses:

• Same as the 2 D encoding plus:

• Each color must appear at least once in each row;

• Each color must appear at least once in each column;

• No two colors are assigned to the same cell;

Carla P. Gomes

Capturing Structure - Performance of SAT Solvers

State of the art backtrack and local search and complete SAT solvers using 3D encoding are very competitive with specialized CSP algorithms.

In contrast SAT solvers perform very poorly on 2D encodings (SATZ or SATO);

In contrast local search solvers (Walksat) perform well on 2D encodings;

Carla P. Gomes

SATZ on 2D encoding(Order 20 -28)

SATZ and SATO can only solve up to order 28 when using 2D encoding;When using 3D encoding problems of the same size take only 0 or 1 backtrack and much higher orders can be solved;

1,000,000Order 28

Order 20

Carla P. Gomes

Walksat on 2D and 3D encoding(Order 30-33)

1,000,0002D order 333D order 33

Walksat shows an unsual pattern - the 2D encodings are somewhat easier than the 3D encoding

at the peak and harder in the undereconstrained region;

Carla P. Gomes

Quasigroup - SatisfiabilityQuasigroup - Satisfiability

Encoding the quasigroup using only

Boolean variables in clausal form using

the 3D encoding is very competitive.

Very fast solvers - SATZ, GRASP,

SATO,WALKSAT;

Carla P. Gomes

Structural features of instances provide insights into their hardness namely:

Backbone

Inherent Structure and Balance

Carla P. Gomes

Backbone

This instance has4 solutions:

Backbone

Total number of backbone variables: 2

Backbone is the shared structure of all the solutions to a given instance.

Carla P. Gomes

Phase Transition in the Backbone

• We have observed a transition in the backbone from a phase where the size of the backbone is around 0% to a phase with backbone of size close to 100%.

• The phase transition in the backbone is sudden and it coincides with the hardest problem instances.

(Achlioptas, Gomes, Kautz, Selman 00, Monasson et al. 99)

Carla P. Gomes

New Phase Transition in BackboneQCP (satisfiable instances only)

% Backbone

Sudden phase transition in Backbone

Fraction of preassigned cells

Computationalcost

Carla P. Gomes

Inherent Structure and Balance

Carla P. Gomes

Quasigroup Patterns and Problems Hardness

Rectangular Pattern Aligned Pattern Balanced Pattern

Tractable Very hard

(Kautz, Ruan, Achlioptas, Gomes, Selman 2001)

Carla P. Gomes

SATZSATZ

Balanced QCP

Rectangular QCP

Aligned QCP

Carla P. Gomes

Walksat Walksat

aligned

rectangular

Balanced filtered QCPBalance QWH

QCPQWH

We observe the same ordering in hardness when using Walksat,SATZ, and SATO – Balacing makes instances harder

Carla P. Gomes

Phase Transitions, Backbone, Balance

Summary

The understanding of the structural properties of problem instances based on notions such as phase transitions, backbone, and balance provides new insights into the practical complexity of many computational tasks.

Active research area with fruitful interactions between computer science, physics (approaches

from statistical mechanics), and mathematics (combinatorics / random structures).

Carla P. Gomes

OutlineOutline

Randomization

Conclusions

Carla P. Gomes

Randomized Backtrack Search Randomized Backtrack Search ProceduresProcedures

Carla P. Gomes

BackgroundBackground

Stochastic strategies have been very successful in the area of local search.

Simulated annealingGenetic algorithmsTabu SearchGsat and variants.

Limitation: inherent incomplete nature of local search methods.

Carla P. Gomes

BackgroundBackground

We want to explore the We want to explore the addition of aaddition of a

stochastic elementstochastic element to a systematic search to a systematic search

procedure procedure without losing completeness.without losing completeness.

Carla P. Gomes

We introduce stochasticity in a backtrack search method, e.g., by randomly breaking ties in variable and/or value selection.

Compare with standard lexicographic tie-breaking.

Randomization

Carla P. Gomes

RandomizationRandomization

At each choice point break ties (variable selection and/or value selection) randomly or:

“Heuristic equivalence” parameter (H) - at every choice point consider as “equally” good H% top choices; randomly select a choice from equally good choices.

Carla P. Gomes

Randomized StrategiesRandomized Strategies

Strategy Variable sel. Value sel.

DD deterministic deterministic

DR deterministic random

RD random deterministic

RR random random

Carla P. Gomes

Quasigroup DemoQuasigroup Demo

Carla P. Gomes

Distributions of Randomized Backtrack Search

Key Properties:

I Erratic behavior of mean

II Distributions have “heavy tails”.

Carla P. Gomes

Median = 1!

samplemean

Erratic Behavior of Search CostQuasigroup Completion ProblemErratic Behavior of Search Cost

Quasigroup Completion Problem

number of runs

Carla P. Gomes

75%<=30

Number backtracks Number backtracks

5%>100000

Carla P. Gomes

Heavy-Tailed DistributionsHeavy-Tailed Distributions

… … infinite variance … infinite meaninfinite variance … infinite mean

Introduced by Pareto in the 1920’s

--- “probabilistic curiosity.”

Mandelbrot established the use of heavy-tailed distributions to model real-world fractal phenomena.

Examples: stock-market, earth-quakes, weather,...

Carla P. Gomes

Decay of DistributionsDecay of Distributions

Standard --- Exponential Decay

e.g. Normal:

Heavy-Tailed --- Power Law Decay

e.g. Pareto-Levy:

Pr[ ] , ,X x Ce x for some C x 2 0 1

Pr[ ] ,X x Cx x 0

Carla P. Gomes

Standard Distribution(finite mean & variance)

Power Law Decay

Exponential Decay

Carla P. Gomes

Normal, Cauchy, and LevyNormal, Cauchy, and Levy

Normal - Exponential Decay

Cauchy -Power law DecayLevy -Power law Decay

Carla P. Gomes

Tail Probabilities (Standard Normal, Cauchy, Levy)

c Normal Cauchy Levy0 0.5 0.5 11 0.1587 0.25 0.68272 0.0228 0.1476 0.52053 0.001347 0.1024 0.43634 0.00003167 0.078 0.3829

Carla P. Gomes

Example of Heavy Tailed Model(Random Walk)

Random Walk:•Start at position 0

•Toss a fair coin:

• with each head take a step up (+1)

• with each tail take a step down (-1)

X --- number of steps the random walk takes to return to position 0.

Carla P. Gomes

The record of 10,000 tosses of an ideal coin

(Feller)

Zero crossing Long periods without zero crossing

Carla P. Gomes

Random Walk

Heavy-tails vs. Non-Heavy-TailsHeavy-tails vs. Non-Heavy-Tails

Normal(2,1000000)

Normal(2,1)

O,1%>200000

Median=2

lved f

X - number of steps the walk takes to return to zero (log scale)

Carla P. Gomes

How to Check for “Heavy Tails”?How to Check for “Heavy Tails”?

Log-Log plot of tail of distribution

should be approximately linear.

Slope gives value of

infinite mean and infinite varianceinfinite mean and infinite variance

infinite varianceinfinite variance

Carla P. Gomes

319.0153.0

Number backtracks (log)

1 => Infinite mean

Heavy-Tailed Behavior in QCP Domain

18% unsolved

0.002% unsolved

Carla P. Gomes

Formal Models of Heavy-Tailed Behavior in Combinatorial Search

Chen, Gomes, Selman 2001

Carla P. Gomes

MotivationMotivationMotivationMotivation

Research on heavy-tails has been largely based on empirical studies of run time distribution.

Goal: to provide a formal characterization of tree search models and show under what conditions heavy-tailed distributions can arise.

Intuition: Heavy-tailed behavior arises:

• from the fact that wrong branching decisions may lead the procedure to explore an exponentially large subtree of the search space that contains no solutions;

• the procedure is characterized by a large variability in the time to find a solution on different runs, which leads to highly different trees from run to run;

Carla P. Gomes

Balanced vs. ImbalancedBalanced vs. Imbalanced Tree Model Tree Model

Balanced Tree Model:

• chronological backtrack search model;• fixed variable ordering;• random child selection with no propagation

mechanisms;

(show demo)

Carla P. Gomes

221)]([

122)]([n

The run time distribution of chronological backtrack search ona complete balanced tree is uniform (therefore not heavy-tailed).Both the expected run time and variance scale exponentially

Carla P. Gomes

Balanced Tree ModelBalanced Tree ModelBalanced Tree ModelBalanced Tree Model

• The expected run time and variance scale exponentially, in the height of the search tree (number of variables);

• The run time distribution is Uniform, (not heavy tailed ).

• Backtrack search on balanced tree model has no restart strategy with exponential polynomial time.

221)]([

122)]([n

Chen, Gomes & Selman 01

Carla P. Gomes

How can we improve on the balanced serach tree model?

Very clever search heuristic that leads quickly to the solution node - but that is hard in general;

Combination of pruning, propagation, dynamic variable ordering that prune subtrees that do not contain the solution, allowing for runs that are short.

---> resulting trees may vary dramatically from run to run.

Carla P. Gomes

T - the number of leaf nodes visited up to and including the successful node; b - branching factor

0)1(][ iippibTP

Formal Model Yielding Heavy-Tailed BehaviorFormal Model Yielding Heavy-Tailed Behavior

(show demo)

Carla P. Gomes

Expected Run Time(infinite expected time)

Variance

(infinite variance)

(heavy-tailed)

][1 TEb

bLpLTPb

Carla P. Gomes

Bounded Heavy-Tailed BehaviorBounded Heavy-Tailed Behavior

(show demo)

Carla P. Gomes

No Heavy-tailed behavior for Proving Optimality

Carla P. Gomes

Proving OptimalityProving Optimality

Carla P. Gomes

Small-World Vs. Heavy-Tailed Behavior

Does a Small-World topology (Watts & Strogatz) induce heavy-tail behavior?

The constraint graph of a quasigroup exhibits a small-world topology(Walsh 99)

Carla P. Gomes

Exploiting Heavy-Tailed BehaviorExploiting Heavy-Tailed Behavior

Heavy Tailed behavior has been observed in several domains: QCP, Graph Coloring, Planning, Scheduling, Circuit synthesis, Decoding, etc.

Consequence for algorithm design:

Use restarts or parallel / interleaved runs to exploit the extreme variance performance.

Restarts provably eliminate heavy-tailed behavior.

(Gomes et al. 97, Hoos 99, Horvitz 99, Huberman, Lukose and Hogg 97, Karp et al 96, Luby et al. 93, Rish et al. 97, Wlash 99)

Carla P. Gomes

X XX XX

solved10 101010 10

Sequential: 50 +1 = 51 seconds

Parallel: 10 machines --- 1 second 51 x speedup

Super-linear Speedups

Interleaved (1 machine): 10 x 1 = 10 seconds 5 x speedup

Carla P. Gomes

RestartsRestarts

70%unsolved

no restarts

restart every 4 backtracks

250 (62 restarts)

0.001%unsolved

Carla P. Gomes

Example of Rapid Restart Speedup(planning)

100000

1000000

1 10 100 1000 10000 100000 1000000

log( cutoff )

2000 ~100 restarts

Cutoff (log)

~10 restarts

100000

Carla P. Gomes

Sketch of proof of elimination of heavy tails

Let’s truncate the search procedure after m backtracks.

Probability of solving problem with truncated version:

Run the truncated procedure and restart it repeatedly.

pm X m Pr[ ]

X numberof backtracks to solve the problem

Carla P. Gomes

Y total number backtracks with restarts

F Y y pmY m

c e c y

Pr[ ] ( )

Number of starts Y m Geometric pmRe / ~ ( )

Y - does not have Heavy Tails

Carla P. Gomes

Decoding in Communication Systems

Source Encoder Decoder DestinationChannel

Voice waveform, binary digits from a cd, output of a set of sensors in a space probe, etc.

Telephone line, a storage medium, a space communication link, etc.

usually subject to NOISE

Processing prior to transmission,e.g., insertion of redundancy to combat the channel noise. Processing of the channel output with the

objective of producing at the destinationan acceptable replica of the source output.

Decoding in communication systems is NP-hard.

(Berlekamp, McEliece, and van Tilborg 1978, Barg 1998)

Carla P. Gomes

Retransmissions in Sequential Decoding

without retransmissions

with retransmissions

Gomes et al. 2000 / 20001

Carla P. Gomes

Paramedic Crew AssignmentParamedic Crew Assignment

Paramedic crew assignment is the problem of assigning paramedic crews from different stations to cover a given region, given several resource constraints.

Carla P. Gomes

Deterministic SearchDeterministic Search

Carla P. Gomes

RestartsRestarts

Carla P. Gomes

Deterministic

Logistics Planning 108 mins. 95 sec.

Scheduling 14 411 sec 250 sec

(*) not found after 2 days

Scheduling 16 ---(*) 1.4 hours

Scheduling 18 ---(*) ~18 hrs

Circuit Synthesis 1 ---(*) 165sec.Circuit Synthesis 2 ---(*) 17min.

Results on Effectiveness of Restarts

Carla P. Gomes

Algorithm Portfolio DesignAlgorithm Portfolio Design

Gomes and Selman 1997 - Proc. UAI-97;

Gomes et al 1997 - Proc. CP97.

Carla P. Gomes

MotivationMotivation

The runtime and performance of randomized algorithms can vary dramatically on the same instance and on different instances.

Goal: Improve the performance of different algorithms by combining them into a portfolio to exploit their relative strengths.

Carla P. Gomes

Branch & Bound:Best Bound vs. Depth First Search

Carla P. Gomes

Branch & Bound(Randomized)

Standard OR approach for solving Mixed Integer Programs (MIPs)• Solve linear relaxation of MIP• Branch on the integer variables for which the solution of the LP relaxation is non-integer:

apply a good heuristic (e.g., max infeasibility) for variable selection ( + randomization ) and create two new nodes (floor and ceiling of the fractional value)

• Once we have found an integer solution, its objective value can be used to prune other nodes, whose relaxations have worse values

Carla P. Gomes

Branch & BoundDepth First vs. Best bound

Critical in performance of Branch & Bound:

the way in which the next node to be expanded is selected.

Best-bound - select the node with the best LP bound

(standard OR approach) ---> this case is equivalent to A*, the LP relaxation provides an admissible search heuristic

Depth-first - often quickly reaches an integer solution

(may take longer to produce an overall optimal value)

Carla P. Gomes

Portfolio of AlgorithmsPortfolio of Algorithms

A portfolio of algorithm is a collection of algorithms and / or copies of the same algorithm running interleaved or on different processors.

Goal: to improve on the performance of the component algorithms in terms of:

expected computational cost“risk” (variance)

Efficient Set or Efficient Frontier: set of portfolios that are best in terms of expected value and risk.

Carla P. Gomes

Depth-First: Average - 18000;St. Dev. 30000

Brandh & Bound for MIP Depth-first vs. Best-bound

Brandh & Bound for MIP Depth-first vs. Best-boundC

Number of nodes

30%Best bound

Best-Bound: Average-1400 nodes; St. Dev.- 1300 Optimal strategy: Best Bound

45%Depth-first

Carla P. Gomes

Depth-First and Best and Bound do not dominate each other overall.

Carla P. Gomes

Heavy-tailed behavior of Depth-firstHeavy-tailed behavior of Depth-first

Carla P. Gomes

Portfolio for heavy-tailed search procedures (2 processors)

0 DF / 2 BB

2 DF / 0 BB

Standard deviation of run time of portfolios

Expect

Carla P. Gomes

Portfolio for 6 processorsPortfolio for 6 processors

0 DF / 6 BB

6 DF / 0BB

5 DF / 1BB

3 DF / 3 BB

4 DF / 2 BB

Efficient set

Carla P. Gomes

Portfolio for 20 processorsPortfolio for 20 processors

0 DF / 20 BB

20 DF / 0 BBExp

The optimal strategy is to run Depth First on the 20 processors!

Optimal collective behavior emerges from suboptimal individual behavior.

Carla P. Gomes

Compute Clusters and Distributed Agents

With the increasing popularity of compute clusters and distributed problem solving / agent paradigms, portfolios of algorithms --- and flexible computation in general --- are rapidly expanding research areas.

(Baptista and Marques da Silva 00, Boddy & Dean 95, Bayardo 99, Davenport 00, Hogg 00, Horvitz 96, Matsuo 00, Steinberg 00, Russell 95, Santos 99, Welman 99. Zilberstein 99)

Carla P. Gomes

Portfolio for heavy-tailed search procedures (2-20 processors)

Carla P. Gomes

A portfolio approach can lead to substantial improvements in the expected cost and risk of stochastic algorithms, especially in the presence of heavy-tailed phenomena.

Carla P. Gomes

Summary of RandomizationSummary of Randomization

Considered randomized backtrack search.

Showed Heavy-Tailed Distributions.

Suggests: Rapid Restart Strategy.

--- cuts very long runs

--- exploits ultra-short runs

Experimentally validated on previously unsolved planning and scheduling problems.

Portfolio of Algorithms for cases where no single heuristic dominates

Carla P. Gomes

Research Direction:Learning Restart Policies

Carla P. Gomes

Bayesian Model Structure LearningBayesian Model Structure Learning

(Horvitz, Ruan, Gomes, Kautz, Selman, Chickering 2001)

Learning to infer predictive models from data and to identify key variables==> restarts, cutoffs and other adaptive behavior of search algorithms.

Carla P. Gomes

Green - long runsGray - short runs

Variance in number of uncolored cells across rows and columns

Number uncolored cells per column

Min depth Avg Depth

Max number of uncolored cells across rows and columns

Quasigroup Order 34 (CSP)

Model accuracy 96.8% vs 48% for the marginal model

Carla P. Gomes

Analysis of different solver features and problem features

Carla P. Gomes

OutlineOutline

Randomization

Conclusions

Carla P. Gomes

Summary Summary

The understanding of the structural properties of problem instances based on notions such as

phase transitions, backbone, and balance provides new insights into the practical complexity of many

computational tasks.

Active research area with fruitful interactions between computer science, physics (approaches

from statistical mechanics), and mathematics (combinatorics / random structures).

Carla P. Gomes

Stochastic search methods (complete and incomplete) have been shown very effective.

Restart strategies and portfolio approaches can lead to substantial improvements in the expected runtime and variance, especially in the presence of heavy-tailed phenomena.

Randomization is therefore a tool to improve algorithmic performance and robustness.

Machine Learning techniques can be used to learn predicitive models.

Summary

Carla P. Gomes

General Solution Methods

Real WorldProblems

Exploiting Structure:Tractable Components

Transition Aware Systems(phase transitionconstrainedness

backbone resources)

RandomizationExploits variance

to improve robustness and performance

Bridging the GapBridging the Gap

Carla P. Gomes

Check also:

www.cis.cornell.edu/iisi

Check also:

www.cis.cornell.edu/iisi

Demos, papers, etc.

carla p. gomes school on optimization cpaior02 exploiting structure and randomization in...

Documents

cp - 2001 1 formal models of heavy-tailed behavior in...

carla gomes info 2950 1 info 2950 mathematical methods for...

carla p. gomes, ashish sabharwal cornell university...

computational methods for management and economics carla...

aem412 computational methods for management and economics...

chris barrett and carla gomes cornell center for a...

computational methods for management and economics carla...

1 cs 4700: foundations of artificial intelligence carla p....

computational methods for management and economics carla...

1 info 2950 prof. carla gomes gomes@cs.cornell.edu module...

1 cs 4700: foundations of artificial intelligence carla p....

dynamic restarts optimal randomized restart policies with...

computational methods for management and economics carla p....

1 info 2950 prof. carla gomes gomes@cs.cornell.edu module...

computational methods for management and economics carla...

beyond satisfiability: model counting, quantification, and...

carla p. gomes info372 cs-info 372: explorations in...

developing electronic resources and computational techniques...

controlling computational cost: structure and phase...

1 info 2950 prof. carla gomes gomes@cs.cornell.edu module...