![Page 2: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/2.jpg)
Outline
Introduction to Combinatorial Optimization
Modelling
Solving Technologies
Modelling and Solving with Numberjack
Summary
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 2
![Page 3: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/3.jpg)
Section 1
Introduction to CombinatorialOptimization
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 3
![Page 4: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/4.jpg)
About this Tutorial
MotivationCombinatorial optimization provides powerful support fordecision-making; many interesting real-world problems arecombinatorial and can be approached using similartechniques.
The PromiseThis tutorial will teach attendees how to develop interestingmodels of combinatorial problems and solve them usingconstraint programming, satisfiability, and mixed integerprogramming techniques.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 4
![Page 5: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/5.jpg)
About this Tutorial
MotivationCombinatorial optimization provides powerful support fordecision-making; many interesting real-world problems arecombinatorial and can be approached using similartechniques.
The PromiseThis tutorial will teach attendees how to develop interestingmodels of combinatorial problems and solve them usingconstraint programming, satisfiability, and mixed integerprogramming techniques.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 4
![Page 6: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/6.jpg)
Combinatorial Optimisation
What is Combinatorial Optimisation?Optimization problems divide neatly into two categories,those involving:
• continuous variables
• discrete variables -- these are combinatorial.
Usually we’re concerned with finding a least-cost solution to aset of constraints.
Our focus in this tutorial
• Constraint Programming
• Satisfiability
• Mixed Integer Programming
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 5
![Page 7: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/7.jpg)
What is a Combinatorial Problem?Variables, Domains, and ConstraintsGiven a set of variables, each taking a value from a domain ofpossible values, find an assignment to all variables that satisfythe constraints.
Variables Domains Constraints
Adjacent countiesmust be coloureddifferently.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 6
![Page 8: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/8.jpg)
What is a Combinatorial Problem?Variables, Domains, and ConstraintsGiven a set of variables, each taking a value from a domain ofpossible values, find an assignment to all variables that satisfythe constraints.
Variables Domains Constraints
Adjacent countiesmust be coloureddifferently.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 6
![Page 9: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/9.jpg)
Well known example: Sudoku
Variables and Domains
• Each cell must take a valuefrom 1 to 9.
Constraints
• All cells in a row must bedifferent.
• All cells in a column mustbe different.
• Each 3× 3 block must bedifferent.
83 6
7 9 25 7
4 5 71 3
1 6 88 5 1
9 4
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 7
![Page 10: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/10.jpg)
Applications: Optical Network Design
• Configuration of thebackhaul network
• Every exchange must beconnected to a metronode
• Every exchange must bereachable along two paths
• Paths are bounded inlength
• Links are of limitedcapacity
• The amount of cableshould be minimised
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 8
![Page 11: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/11.jpg)
Applications: Rosetta/Philae Mission
Scheduling Scientific Experiments on theRosetta/Philae MissionGilles Simonin, Christian Artigues,Emmanuel Hebrard, Pierre Lopez
Image: ESA
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 9
![Page 12: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/12.jpg)
price = [215, 275, 335, 355, 420, 580]apitizers = ["Mixed Fruit","French Fries","Side Salad", "Hot Wings","
Mozzarella Sticks","Sample Plate"]
quantities = VarArray(len(apitizers), 0, max_qty)model = Model( Sum(quantities, price) == 1505 )
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 10
![Page 13: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/13.jpg)
Types of Applications
Satisfaction
• Find a single solution
• Example: conference schedule
Optimization
• Find the best/high quality solution
• Minimize/maximize: cost, time,profit Fig. 2. A domino portrait of George Boole generated by our approach using 49 sets of
“double nine” dominoes, i.e. 49⇥ 55 = 2695 individual dominoes.
Many many othersScheduling; planning; configuration; nurse rostering;timetabling; vehicle routing; electricity, water, and oilnetworks; sourcing optimization; bioinformatics; . . .
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 11
![Page 14: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/14.jpg)
Section 2
Modelling
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 12
![Page 15: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/15.jpg)
Modelling Languages
FeaturesDeclarative specification of the problem, separating (in so faras possible) the formulation and the search strategy.
A Constraint Model of the Sudoku Puzzle
matrix = Matrix(N∗N, N∗N, 1, N∗N)
sudoku = Model([AllDiff(row) for row inmatrix.row],
[AllDiff(col) for col inmatrix.col],
[AllDiff(matrix[x:x+N, y:y+N])for x in range(0, N∗N, N)for y in range(0, N∗N, N)] )
83 6
7 9 25 7
4 5 71 3
1 6 88 5 1
9 4
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 13
![Page 16: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/16.jpg)
Modelling Languages
FeaturesDeclarative specification of the problem, separating (in so faras possible) the formulation and the search strategy.
A Constraint Model of the Sudoku Puzzle
matrix = Matrix(N∗N, N∗N, 1, N∗N)
sudoku = Model([AllDiff(row) for row inmatrix.row],
[AllDiff(col) for col inmatrix.col],
[AllDiff(matrix[x:x+N, y:y+N])for x in range(0, N∗N, N)for y in range(0, N∗N, N)] )
83 6
7 9 25 7
4 5 71 3
1 6 88 5 1
9 4
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 13
![Page 17: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/17.jpg)
Basic Process
Problem
Human
Model
Solver
Solution
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 14
![Page 18: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/18.jpg)
Modelling
• This is onemodel, not themodel of the problem.
• Many possible alternatives
• Not always clear which is the bestmodel
• Often: not clear what is the problem
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 15
![Page 19: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/19.jpg)
More Realistic
Problem
Human
Model
Solver
SolutionHangs Wrong Solution
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 16
![Page 20: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/20.jpg)
Modelling Frameworks
Benefits
• High-level, solver independent model
• Use of multiple back-end solvers
• Automatic translation for each solver
Some Modelling Frameworks
• Numberjack (Insight, Ireland)
• Savile Row (St Andrews, UK)
• Minizinc (NICTA, Australia)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 17
![Page 21: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/21.jpg)
Framework Process
Problem
Human
Model
Compile/Reformulate
CP MIP SAT Other
Solution Solution Solution Solution
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 18
![Page 22: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/22.jpg)
Section 3
Solving Technologies
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 19
![Page 23: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/23.jpg)
Solving a Combinatorial Problem
Many possibilities
• Constraint Programming (CP)
• Satisfiability (SAT)
• Mixed Integer Programming (MIP)
• Local Search -- heuristic guess with heuristic repair
• Large Neighbourhood Search -- systematic and localsearch
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 20
![Page 24: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/24.jpg)
Solving a Combinatorial Problem
Many possibilities
• Constraint Programming (CP)• Satisfiability (SAT)• Mixed Integer Programming (MIP)• Local Search -- heuristic guess with heuristic repair
• Large Neighbourhood Search -- systematic and localsearch
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 20
![Page 25: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/25.jpg)
Constraint Programming (CP)In a nutshell
• Domains typically represented as integers (others: realvalued, set, graph)
• Typically solved using backtracking-search
• Polynomial-time inference reduces the size of the searchspace
Inference/PropagationThe All-Different constraint specifies that variables must beassigned distinct values.x1 ∈ {1, 2, 5}x2 ∈ {1, 2, 5}x3 ∈ {1, 2, 5}x4 ∈ { 2, 3, 4 }x5 ∈ {1, 2, 3, 4, 5}
x1 ∈ {1, 2, 5}x2 ∈ {1, 2, 5}x3 ∈ {1, 2, 5}x4 ∈ { 3, 4 }x5 ∈ { 3, 4 }
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 21
![Page 26: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/26.jpg)
Constraint Programming (CP)In a nutshell
• Domains typically represented as integers (others: realvalued, set, graph)
• Typically solved using backtracking-search
• Polynomial-time inference reduces the size of the searchspace
Inference/PropagationThe All-Different constraint specifies that variables must beassigned distinct values.x1 ∈ {1, 2, 5}x2 ∈ {1, 2, 5}x3 ∈ {1, 2, 5}x4 ∈ { 2, 3, 4 }x5 ∈ {1, 2, 3, 4, 5}
x1 ∈ {1, 2, 5}x2 ∈ {1, 2, 5}x3 ∈ {1, 2, 5}x4 ∈ { 3, 4 }x5 ∈ { 3, 4 }
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 21
![Page 27: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/27.jpg)
Sudoku: simple example of CP propagation
83 6
7 9 25 7
4 5 71 3
1 6 88 5 1
9 4
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
Initial domains withpreset clues.
83 6
7 9 25 7
4 5 71 3
1 6 88 5 1
9 4
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
1 2 34 5 67 8 9
After propagating thepreassigned cells.
8 1 2 7 5 3 6 4 99 4 3 6 8 2 1 7 56 7 5 4 9 1 2 8 31 5 4 2 3 7 8 9 63 6 9 8 4 5 7 2 12 8 7 1 6 9 5 3 45 2 1 9 7 4 3 6 84 3 8 5 2 6 9 1 77 9 6 3 1 8 4 5 2
Complete solution aftersearch.
Core IdeaRemove inconsistent values from domains which cannot bepart of a solution.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 22
![Page 28: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/28.jpg)
Satisfiability Problem (SAT)
• Set of Boolean variables x1, x2, . . . , xn
• A literal is a variable x1 or its negation ¬x1
• A clause is a disjunction of literals (x2 ∨ ¬x3)
• Formula e.g. (x1 ∨ x3 ∨ ¬x4) ∧ (x4) ∧ (x2 ∨ ¬x3)
• A clause is satisfied when at least one of its literals is true.
• A formula is satisfied when all clauses are satisfied.
• Goal: Find an assignment to the variables that satisfiesthe formula
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 23
![Page 29: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/29.jpg)
SAT Solving
• Backtracking search• 1st decision: a
• (¬a ∨ ¬b)• 2nd decision: c
• (b ∨ ¬c ∨ d)• (¬d ∨ e)
• 3rd decision: f• (b ∨ ¬f ∨ g)• (¬a ∨ ¬f ∨ h)
• Conflict-driven clause learning• (¬d ∨ ¬i)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 24
![Page 30: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/30.jpg)
Mixed Integer Programming
Standard MIP Model has the following form
min cx + dy (1)s.t. Ax + By ≥ 0 (2)
x , y ≥ 0 (3)y integer (4)
InformallyThe objective function is linear, e.g. a weighted sum,expression over the variables. Each constraint is linear. Somevariables take integer values, other can take real values.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 25
![Page 31: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/31.jpg)
MIP Solving
1
2
3
4
5
6
1 2 3 4 5 6
x2
x1
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 26
![Page 32: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/32.jpg)
MIP Solving
1
2
3
4
5
6
1 2 3 4 5 6
x2
x1
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 27
![Page 33: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/33.jpg)
MIP Solving
1
2
3
4
5
6
1 2 3 4 5 6
x2
x1
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 28
![Page 34: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/34.jpg)
MIP Solving
1
2
3
4
5
6
1 2 3 4 5 6
x2
x1
Fractional solution
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 29
![Page 35: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/35.jpg)
MIP Solving
1
2
3
4
5
6
1 2 3 4 5 6
x2
x1
Fractional solution
Cut
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 30
![Page 36: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/36.jpg)
CP, SAT, MIP: choice is good!
Operationally different
• Algorithms are different• CP: Constraint propagation + Search• SAT: Unit propagation + Clause Learning + Search• MIP: Linear relaxation + Cutting planes + Branch & Bound
• Often not clear which is best for a problem
• Encodings matter
Numberjack
• Write a single model in Python
• Numberjack constructs: union of SAT, MIP, and CP
• Encoded depending on the choice of solver
• Rapidly evaluate different solvers and paradigms
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 31
![Page 37: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/37.jpg)
Summary of Solving Technologies
Core goal
• The user states the problem and a general purpose solveris used to find a solution
• Generic tools to solve a broad range of applications
Solving Approaches
• Multiple complimentary approaches: CP, SAT, MIP
• Often not clear which is best for a particular problem
• Choice is good, modelling frameworks allow to try manypossibilities
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 32
![Page 38: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/38.jpg)
Section 4
Modelling and Solving with Numberjack
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 33
![Page 39: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/39.jpg)
Numberjack Overview
What is Numberjack?
• A Python platform for combinatorial optimization
• Open source project (Github, LGPL)
• Common platform for diverse paradigms (CP, SAT, MIP)
• Fast backend solvers
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 34
![Page 40: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/40.jpg)
Backend Solvers Available
CP Solvers
• Mistral v1 & v2
• Toulbar2
SAT Solvers
• MiniSat
• Walksat
• Lingeling
• CryptoMiniSat
• Glucose
• . . . any CNF solver
MIP Solvers
• IBM ILOG CPLEX
• Gurobi
• SCIP• Through COIN-OR OpenSolver Interface:• COIN-OR CBC• COIN-OR CLP• COIN-OR DyLP• COIN-OR SYMPHONY• COIN-OR Volume Algorithm• GNU LP Toolkit
• Soplex
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 35
![Page 41: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/41.jpg)
Installation
Python Package Index (PyPI)
pip install Numberjack
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 36
![Page 42: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/42.jpg)
Subsection 1
Modelling Constructs
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 37
![Page 43: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/43.jpg)
Variables in Numberjack
Different Variable constructors
Constructor DescriptionVariable() Boolean variableVariable(’x’) Boolean variable called ’x’Variable(N) Variable in the domain of [0 . . .N − 1]Variable(N, ’x’) Variable in the domain of [0 . . .N − 1] called ’x’Variable(l,u) Variable in the domain of [l . . . u]Variable(l,u, ’x’) Variable in the domain of [l . . . u] called ’x’Variable(list) Variable with domain specified as a listVariable(list, ’x’) Variable with domain as a list called ’x’
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 38
![Page 44: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/44.jpg)
More Variables in Numberjack
Similarily for arrays and matrices
Constructor DescriptionVarArray(N) Array of N Boolean variablesVarArray(N, u) Array of N variables with domains [0 . . . u − 1]VarArray(N, l, u) Array of N variables with domains [l . . . u]
...Matrix(N, M) N ×M matrix of Boolean variablesMatrix(N, M, u) N ×M matrix of variables with domains [0 . . . u − 1]Matrix(N, M, l, u) N ×M matrix of variables with domains [l . . . u]
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 39
![Page 45: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/45.jpg)
Constraints in Numberjack
Constraints can be specified by arithmetic operators ofvariables
x > yy <= zx != zx + y == z
x + 4 > z ∗ 3z == (x < y)z <= (x < y)(x == y) != (a == b)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 40
![Page 46: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/46.jpg)
Hello World Example
from Numberjack import ∗
x, y, z = VarArray(3, 5) # 3 variables with domains 0..4model = Model(
x != y, x != z, y != z,x == y + z,z > y,
)
solver = model.load("Mistral") # Load the model into a solversolver.solve() # Tell the solver to solve
print x.get_value(), y.get_value(), z.get_value()
Output:
3 1 2
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 41
![Page 47: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/47.jpg)
Hello World Example
from Numberjack import ∗
x, y, z = VarArray(3, 5) # 3 variables with domains 0..4model = Model(
x != y, x != z, y != z,x == y + z,z > y,
)
solver = model.load("Mistral") # Load the model into a solversolver.solve() # Tell the solver to solve
print x.get_value(), y.get_value(), z.get_value()
Output: 3 1 2
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 41
![Page 48: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/48.jpg)
Global Constraints
Many benefits
• Constraints over multiple variables• More concise modelling• More expressive• Often specialized algorithms to achievehigher levels of inference.
All-DifferentEach variable must take a distinct value.
#We should write:AllDiff([w, x, y, z])
# instead of:w != x, w != y, w != z, x != y, x != z, y != z
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 42
![Page 49: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/49.jpg)
Concise Models
VarArray and Matrix allow for concise modelling.
Rows and Columns
for row inmatrix.row:m += AllDiff(row)
for col inmatrix.col:m += AllDiff(col)
SlicesGiven a 9× 9 Sudoku matrix, wecould get a 3× 3 cell:
AllDiff( matrix[x:x+3, y:y+3] )
83 6
7 9 25 7
4 5 71 3
1 6 88 5 1
9 4
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 43
![Page 50: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/50.jpg)
Global Constraints: Sum
Sum(row) == 1
Sum([a, b, c, d]) >= e
Weighted Sum
2∗a + b + 0.5∗c + 3∗d == e
Sum([2∗a, b, 0.5∗c, 3∗d]) == e
Sum([a,b,c,d], [2, 1, 0.5, 3]) == e
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 44
![Page 51: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/51.jpg)
Global Constraints: Element
ElementAllows indexing into a variable array (at solve time) by thevalue of another variable. Useful modelling construct.
Numberjack
x == Element(VarArray(10, 1, 10), Variable(10))
y == myvararray[myvariable] # Element
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 45
![Page 52: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/52.jpg)
Custom ConstraintsEasy to define your own custom constraints for modelling purposesand add custom decompositions for solvers which don’t support it.
Example
classMyAllDiff(Expression):
def decompose(self):return [var1 != var2 for var1, var2 in pair_of(self.
children)]
class NoOverlap(Predicate):
def decompose(self):task_i, task_j = self.childrenreturn [((task_i + task_i.duration) <= task_j) |
((task_j + task_j.duration) <= task_i)]
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 46
![Page 53: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/53.jpg)
Optimizing: Minimize and Maximize
Objectives
• Finding the best/good solutions.
• e.g. minimise cost, wastage, loss, time; maximize profit.
Numberjack
Minimize(2∗x + 3∗y)
Maximize(objectivevariable)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 47
![Page 54: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/54.jpg)
Subsection 2
Solving
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 48
![Page 55: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/55.jpg)
Framework Process
Problem
Human
Model
Compile/Reformulate
CP MIP SAT Other
Solution Solution Solution Solution
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 49
![Page 56: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/56.jpg)
One Model, Many Solvers
Constraint Solvers
mistral = model.load("Mistral")
toulbar = model.load("Toulbar2")
MIP Solvers
gurobi = model.load("Gurobi")
cplex = model.load("CPLEX")
scip = model.load("SCIP")
SAT Solvers
minisat = model.load("MiniSat")
walksat = model.load("Walksat")
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 50
![Page 57: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/57.jpg)
Configuring the Solver# Search strategies (solver specific)mistral.setHeuristic("DomainOverWDegree", "Lex")mistral.setHeuristic("Impact")
# Set Limitssolver.setTimeLimit(60) # Secondssolver.setNodeLimit(1000000)
# Solvesolver.solve()
# Find all solutionssolver.getNextSolution()
# Feasibility?solver.is_sat()solver.is_opt()solver.is_unsat()
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 51
![Page 58: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/58.jpg)
Complete solving example -- Sudokufrom Numberjack import ∗N = 3matrix = Matrix(N∗N, N∗N, 1, N∗N)model = Model(
[AllDiff(row) for row in matrix.row],[AllDiff(col) for col in matrix.col],[AllDiff(matrix[x:x+N, y:y+N])
for x in range(0, N∗N, N) for y in range(0, N∗N, N)] )
solver = model.load("Mistral") # Load the model into a solversolver.solve() # Tell the solver to solve
if solver.is_sat(): # Found a solutionprint matrix
elif solver.is_unsat(): # Solver proved that no solution existsprint "Unsatisfiable"
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 52
![Page 59: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/59.jpg)
Subsection 3
Modelling Examples
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 53
![Page 60: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/60.jpg)
Magic Square
• N × N grid
• Contains numbers 1 to N2
• Sum of rows, columns, anddiagonals equal the same value.
N × (N2 + 1)2 1
16611
413710
152125
14398
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 54
![Page 61: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/61.jpg)
Magic Square
The Model
# N x N matrix of variables from 1..N^2
square = Matrix(N,N,1,N∗N)
model = Model(
AllDiff(square),
[Sum(row) == sum_val for row in square.row],
[Sum(col) == sum_val for col in square.col],
Sum([square[a][a] for a in range(N)]) == sum_val,
Sum([square[a][N−a−1] for a in range(N)]) == sum_val)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 55
![Page 62: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/62.jpg)
N-Queens
Problem DefinitionPlace N queens on an N × N chessboard so that no queenattacks another. A queen attacks all cells in horizontal, verticaland diagonal direction.
Q
Q
Q
Q
Q
Q
Q
Q
One solution for N = 8
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 56
![Page 63: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/63.jpg)
Basic N-Queens Model
Cell based model
• A 0/1 variable for each cell to say if it is occupied or not
• Constraints on the rows, columns, and diagonals toenforce no attach
• N2 Boolean variables, 6N − 4 constraints
Column (row) based model
• Each column (row) must contain exactly one queen
• A 1..N variable for each column representing the positionof the queen in the column
• N variables, 3 global constraints (or N2/2 binaryconstraints)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 57
![Page 64: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/64.jpg)
N-Queens Model
# One variable for each queen (one per column) representing# the row in which that queen is placed.queens = VarArray(N, N)
model = Model(# Queens must be placed in different rowsAllDiff(queens),
# Queens are not on the same diagonalsAllDiff([queens[i] + i for i in range(N)]),AllDiff([queens[i] − i for i in range(N)])
)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 58
![Page 65: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/65.jpg)
Golomb Ruler
Problem definition
• Place N marks on a ruler
• Distance between each pair of marks is different
• Goal is to minimise the placement of the last mark
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 59
![Page 66: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/66.jpg)
Golomb Ruler
Create the Variables
marks = VarArray(nbMarks, rulerSize)
• Each variable represents the position of a mark
Now the model
• Modelling choices are important
• We will illustrate this with a series of models
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 60
![Page 67: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/67.jpg)
First Model
Naive Model
model = Model(Minimise( Max(marks) ),AllDiff(marks),AllDiff([first− second for first, second in pair_of(marks)])
)
• Each mark must be at a different position
• Each pair of distances must be different
• Minimise the position of the last mark
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 61
![Page 68: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/68.jpg)
First Model Analysis
This is not a very good model!
• The marks must all be different
• But they can also be totally ordered
• Instead of marks[0] being any mark on the ruler we canconstrain it to be the first mark
• Then we just have to minimise the position of the lastmark in the marks array
model = Model(Minimise( marks[−1] ),[marks[i] < marks[i+1] for i in range(nbMarks−1)],AllDiff([first− second for first, second in pair_of(marks)])
)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 62
![Page 69: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/69.jpg)
First Model Analysis
This is not a very good model!
• The marks must all be different
• But they can also be totally ordered
• Instead of marks[0] being any mark on the ruler we canconstrain it to be the first mark
• Then we just have to minimise the position of the lastmark in the marks array
model = Model(Minimise( marks[−1] ),[marks[i] < marks[i+1] for i in range(nbMarks−1)],AllDiff([first− second for first, second in pair_of(marks)])
)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 62
![Page 70: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/70.jpg)
Why does this work?
These extra constraints improve the speed
• These constraints reduce both the time taken and nodessearched
Adding in constraints can help
• Stronger constraints reduce the search space
• Still maintain (a set of) solutions
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 63
![Page 71: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/71.jpg)
More modeling advances
What can we reason about the first mark?
• It must always be at zero
• Proof: Any solution with the first mark at position n canbe be shifted n positions to the left while maintaining allthe constraints and reducing the objective function
• Obvious to us but not to the solver
model = Model(marks[0] == 0,Minimise( marks[−1] ),[marks[i] < marks[i+1] for i in range(nbMarks−1)],AllDiff([first− second for first, second in pair_of(marks)])
)
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 64
![Page 72: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/72.jpg)
Where are we searching?
What are the decision variables?
• By default the solvers will search on all variables in aproblem
• In the Golomb ruler most of these will be auxiliaryvariables representing the distance between the marks
• These are functionally dependent on the positions of themarks
• So why bother searching on these!?
s = model.load("Mistral", marks)
• Tell the solver what variables to search on
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 65
![Page 73: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/73.jpg)
Modelling and Solving Summary
Modelling
• Many possible modelling alternatives
• Not always clear which is the best model
• Often: not clear what is the problem
Solving
• Often not clear which solving technology to use
• Try each alternative, easy in a modelling framework
• Search strategy can have a huge impact on performance
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 66
![Page 74: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/74.jpg)
Map Colouring Demo
Demo
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 67
![Page 75: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/75.jpg)
Map Colouring Demo
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 67
![Page 76: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/76.jpg)
Summary
Numberjack
• A Python platform for combinatorial optimization
• Common platform for diverse paradigms (CP, SAT, MIP)
• Practical, easy to use, and intuitive
• Fast backend solvers
• Open source (Github, LGPL)
http://numberjack.ucc.ie
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 68
![Page 77: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/77.jpg)
Acknowledgements
This work is supported by Science Foundation Ireland (SFI) Grant10/IN.1/I3032.The Insight Centre for Data Analytics is supported by SFI GrantSFI/12/RC/2289.
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 69
![Page 78: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/78.jpg)
LinksLearn more
• Much more which we cannot cover here• ECLiPSe ELearning Coursehttp://4c.ucc.ie/~hsimonis/ELearning/index.htm
• Discrete Optimization on Coursera
Research
• Insight Centre for Data Analyticshttps://www.insight-centre.org
• UCC Research Vacancieshttp://www.ucc.ie/en/hr/vacancies/research/
Industrial Collaborations
• Chrys Ngwa, Business Development Manager, [email protected]
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 70
![Page 79: Numberjack - Combinatorial Optimization in Python](https://reader034.vdocuments.us/reader034/viewer/2022051008/58a2e4071a28ab26788b86f9/html5/thumbnails/79.jpg)
Exercise: Send More Money
A Crypt-Arithmetic Puzzle
• Each character stands for a digit from 0 to 9
• Numbers are built from digits in the usual, positionalnotation
• Repeated occurrence of the same character denote thesame digit
• Different characters denote different digits
Problem Definition
A Crypt-Arithmetic PuzzleWe begin with the definition of the SEND+MORE=MONEYpuzzle. It is often shown in the form of a hand-writtenaddition:
S E N D+ M O R EM O N E Y
Insight Centre for Data Analytics Slide 17September 1, 2014
Insight Centre for Data Analytics PyCon Ireland 2014 Slide 71