welcome to the ibm presentation template — ibm plex...

EWOSeminar

Eray Cakici, Ph.D.

January 23, 2020

IBMConstraint Programming (CP) — An Alternative Approach to Heuristics in Scheduling

Data Science & AI Elite Team

Agenda

1

Introduction

2

Computational Study

3

Conclusions

2Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation

Motivation

Overview of CP

How It Works?

Modeling

Scheduling Problems

CP vs MIP & Heuristics

Future Research

Motivation


• The manufacturing of integrated circuits (IC) on silicon wafers is a complex production process

• Companies strive to produce a large number of chips in the least amount of time possible at minimum cost

• An improvement in the semiconductor manufacturing can provide huge financial gains

• “TSMC invested $9.3 billion in its Fab15 300 mm wafer manufacturing facility in Taiwan. The same company estimations suggest that their future fab might cost $20 billion” --- Wikipedia

Constraint Programming (CP) can be an alternative approach to replacewidely used heuristics

Constraint Programming | Overview

4

• Solving complex problems with a set of constraints and unknown discretevariables

• constraint satisfaction (feasibility) and optimization problems

• Natively supports logical constraints, specialized constraints i.e.“all-different”, and has no limitations on the arithmetic constraints (i.e. non-linear)

• A rich domain for Artificial Intelligence, Computer Science, and OperationsResearch

• backtracking, constraint propogation, branching heuristics

• Very powerful on combinatorial feasibility and detailed scheduling problems

Data Science & AI Elite Team/ 23 January, 2020/ © 2020 IBM Corporation

IBM Cplex Optimization Studio


APIs• Java• .NET• C++

Optimization Engines

Math ProgrammingIBM ILOG CPLEX Optimizers (Simplex, Barrier, Mixed Integer)

Model Development ToolsIntegrated Development Environment (IDE)

ILOG Concert Technology (C++, .NET, Java)

Connectors• Microsoft Excel• MATLAB• AMPL• Python• MSF

(Callable libraries)

Optimization Programming Language (OPL) & OPLScript

Complete toolkit for prototyping, building and deploying analytical decision support applications using constraint programming and/or mathematical programming

Custom Algorithms

Constraint Programming (CP) IBM ILOG CPLEX CP Optimizer

How CP works?

6

• CP is a constructive approach

• Values are assigned to variables one at a time toextend a partial solution to a complete solution

• At a point, it may be useless to further extend a partial solution as at least one constraint is alreadyviolated by the partial solution

• The solver backtracks and tries a different value for a previously assigned variable

• All possible assignments of values to variables can be examined in this way


How CP works? (cont’d)

7

• In CP, the basic search behavior is tree search

• Including space reduction via domain filtering

• Domain filtering

• Before each value-variable assignment, domain filtering occurs

• Each value of a variable which cannot be usedin a solution (given the current assignment) can be removed

• Each constraint type has a specializedalgorithm which filters domains


X

Domain Filtering


1

2

3

4

5

1

3

1

3

4

1

4

1

2

4

5

A B C D E

All different

Domain Filtering


1

2

3

4

5

1

3

1

3

4

1

4

1

2

4

5

A B C D E

All different

X

X

X

X

X

Uses matching theory

adapted from https://www.slideshare.net/PaulShawIBM/cp-optimizer-may2013 – Paul Shaw – CP Optimizer Development Manager at IBM

https://www.slideshare.net/PaulShawIBM/cp-optimizer-may2013

CP Constructive Search Process


Select a decision variable

Assign a value after domain filtering

Constraint propagation (reduce domains of all other variables)

backtrack on failurecontinue on success

Selection StrategiesMost constrained variables first

…

Assignment StrategiesExplore branches

Try first, try last

…

A Complete Search Tree


E ≠ 4

S ∈ {9}E ∈ {4..7}N ∈ {5..8}D ∈ {2..8}M ∈ {1}O ∈ {0}R ∈ {2..8}Y ∈ {2..8}E = 4

E ≠ 6

S ∈ {9}E ∈ {6..7}N ∈ {7..8}D ∈ {2..8}M ∈ {1}O ∈ {0}R ∈ {2..8}Y ∈ {2..8}E = 6

S ∈ {9}E ∈ {5..7}N ∈ {6..8}D ∈ {2..8}M ∈ {1}O ∈ {0}R ∈ {2..8}Y ∈ {2..8}

infeasible

infeasible infeasible

S E N D+ M O R E= M O N E Y

E = 5 E ≠ 5

S ∈ {9}E ∈ {5}N ∈ {6}D ∈ {7}M ∈ {1}O ∈ {0}R ∈ {8}Y ∈ {2}

Solution found!

Assign distinct digits to letters such that equation holds

Agenda

1

Introduction

2

Computational Study

3

Conclusions


Motivation

Overview of CP

How It Works?

Modeling

Scheduling Problems


Future Research


Modeling | Optimization Programming Language (OPL)

Provides natural mathematical description of optimization models

High-level syntax for mathematical models • produces substantially simpler and

shorter code than general-purpose programming languages

• reduces effort and improves reliability of development, upgrades, and maintenance

Supports all expressions needed to model and solve problems using both mathematical programming and constraint programming


Modeling | Examples

Logical constraints

And (&&), Or (||), Imply (=>), …

Specialized constraints

i.e. alldifferent(x1,x2, ...,xn)

Compatibility constraints

{pair} possible = {<1,1>, <2,4>};{pair} forbidden = {<3,5>};allowedAssignments(possible, x, y);forbiddenAssignments(forbidden, x, y);


Modeling | Examples cont‘d

Interval variables

dvar interval Activity[a in Activities] optional in a.startmin..a.endmax size a.duration;

Expressions over interval variables

maximize sum(a in Activities) endOf(Activity[a]);

Precedence constraints

i.e. endBeforeStart(xi , xj , zij) => e(xi) + zij ≤ s(xj)

sequence, cumul, noOverlap, alternative, ...


Modeling | Classical Examples

Variables

𝑋𝑋1,1, … ,𝑋𝑋𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛,𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛

Constraints

AllDifferent(𝑋𝑋𝑛𝑛,1, … ,𝑋𝑋𝑛𝑛,9) ∀ 𝑟𝑟 ∈ 𝑅𝑅𝑅𝑅𝑅𝑅𝑅𝑅

AllDifferent(𝑋𝑋1,𝑛𝑛 , … ,𝑋𝑋9,𝑛𝑛) ∀ 𝑐𝑐 ∈ 𝐶𝐶𝑅𝑅𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝐶𝑅𝑅

AllDifferent(𝑋𝑋1,1, … ,𝑋𝑋3,3)

𝑋𝑋1,1 ≠ 8

…

SEND + MORE = MONEYVariables

S, E, N, D, M, O, R, Y

Constraints

0 ≤ S, E, N, D, M, O, R, Y ≤ 9

AllDifferent(S, E, N, D, M, O, R, Y)

1000 S + 100 E + 10 N + D +1000 M + 100 O + 10 R + E =10000 M + 1000 O + 100 N + 10 E + Y

S ≠ 0M ≠ 0


Scheduling Problems | Job-shop

Minimize makespan (the completion time of latest job),tardiness, weighted tardiness, …

op12

op41

op13

M1 M2 M3

J1

J2

J3

J4

op21 op22 op23

op31 op32

op11

op42 op43


Scheduling Problems | Job-shop cont’d

CP Optimizer model for Job-shop (10 lines of code)


Scheduling Problems | Flexible Job-shop

There are certain qualified machines for a given operation with different processing times

J1

J2

J3

J4

M1, M2 or M3147, 123, 145

M2 or M4130, 140

M4 or M5150, 160

M1, or M3214, 150

M2 or M366, 87

M3 or M5147, 145

M1 or M287, 62

M5173

M1, M2 or M4145, 136, 190

M1, M2 or M3128, 123, 145

M3, M4 or M586, 65, 47

M1 or M3110, 85

Scheduling Problems | Flexible Job-shop with Batching


• Ham. A and Cakici. E.,2016, Flexible job shop scheduling problem with parallel batch processing machine: MIP and CP approaches, Computers & IndustrialEngineering, 102,160-165

• Comparison of CP Optimizer vs different MIP formulations on a set of 20 instances from literature

• The batching allows multiple jobs to be simultaneously processed as long as the total size of the batch does not exceed machine capacity

• Commonly used in semiconductor manufacturing

• Minimize makespan

Scheduling Problems | Flexible Job-shop with Batching cont’d


MIP Model Revised MIP Model (Valid Inequalities) CP

Problem Objective ComputationalTime (sec) Objective Computational

Time (sec) Objective ComputationalTime (sec)

SFJS1 66 0.01 66 0.02 66 0.08

SFJS2 107 0.01 107 0.02 107 0.00

SFJS3 208 0.05 208 0.06 208 0.02

SFJS4 272 0.02 272 0.01 272 0.00

SFJS5 100 0.06 100 0.06 100 0.00

SFJS6 320 0.31 320 0.36 320 0.00

SFJS7 397 0.64 397 0.05 397 0.00

SFJS8 216 0.51 216 0.08 216 0.00

SFJS9 210 0.55 210 0.06 210 0.00

SFJS10 516 5.9 516 3.38 516 0.01

Average 0.8 0.4 0.01

CP is able to find optimal solution in all cases

Small size problems

Scheduling Problems | Flexible Job-shop with Batching cont’d


MIP Model Revised MIP Model (Valid Inequalities) CP

Problem Objective ComputationalTime (sec) Objective Computational

Time (sec) Objective ComputationalTime (sec)

MFJS1 410 300 410 300 410 0.01

MFJS2 410 300 410 300 410 0.01

MFJS3 420 300 420 300 420 0.01

MFJS4 506 300 506 300 503 0.04

MFJS5 488 300 488 300 488 0.03

MFJS6 631 300 614 82.84 614 0.05

MFJS7 916 300 848 300 789 0.67

MFJS8 896 300 804 300 774 0.88

MFJS9 nf 300 876 300 843 24.27

MFJS10 3418 300 1105 300 985 11.19

Average 300 278.3 3.72

CP is able to find better in 6 cases out of 10 with on average 3.72 seconds computational time

Medium/large size problems.

Scheduling Problems | Parallel batching machines with job release times, weights, and sizes


• Ham. A, Fowler. J.W., Cakici. E., 2017, Constraint programming approach for scheduling jobs with different job release times and incompatible families on parallel batching machines, IEEE Transactions on Semiconductor Manufacturing

• Comparison of CP Optimizer vs different heuristics & MIP models proposed byCakici et. al.,2013 on a set of 2560 instances

• Minimize Total Weighted Completion Time

[1] Cakici, E., Mason, S.J., Fowler, J.W., Geismar, H.N., 2013, Batch Scheduling on Parallel Machines with Dynamic Job Arrivals and Incompatible Job Families, International Journal of Production Research



15 jobs MIP Model 1 MIP Model 2 Heuristic 1 Heuristic 2 CP

Avg. TWCT 1347 (640)** 1350 (493) 1395 (95) 1396 (91) 1348 (639)Avg. PR 1.0014 1.0347 1.0355 1.0000Avg. Run-time 22.23 299.02 19.14 19.08 1.79

15-job instances *

25-job instances *25 jobs Heuristic 1 Heuristic 2 CP

Avg. TWCT 3085 (24) 3086 (18) 2925 (633)Avg. PR 1.0545 1.0549 1.0001Avg. Run-time 39.83 40.03 21.00

* 640 different instances for 15-job, 25-job, 50-job, and 100-job instances** ( ) denotes number of best solutions achieved



50-job instances *

100-job instances *100 jobs Heuristic 1 Heuristic 2 CP


50 jobs Heuristic 1 Heuristic 2 CP


CP is able to find best solution in 98.5% of all cases whereas heuristics can find the best only in 4% of all cases


Scheduling Problems | Others

• See [2] for some comparisons against more sophisticated MIP models for Job-shop scheduling (with similar conclusions)

• Comparison of CP Optimizer vs a disjunctive MIP formulation on a set of 140 classical Job-shop instances (50-2000 tasks), time-limit: 2mn, 4 threads

• See [3] for recent review of CP Optimizer (modeling concepts, examples, performance,…)

[2] W.Y. Ku and J. C. Beck. Mixed Integer Programming Models for Job Shop Scheduling: A Computational Analysis. Computers & Operations Research, 2016[3] P. Labroire, J. Rogerie, P. Shaw, and P. Vilim, IBM ILOG CP Optimizer for scheduling. Constraints, 2018


3+ times faster as compared to 5 years ago

Constraint Programming Optimizer | Performance

Agenda

1

Introduction

2

Computational Study

3

Conclusions


Motivation

Overview of CP

How It Works?

Modeling

Scheduling Problems


Future Research


Conclusions | CP vs MIP

• IBM algorithms outperforms classical MIP approaches drastically in both solutiontime and quality

• Large and hard problems which cannot be solved by MIPs are now solvable

• Difficult business rules can also be modeled easily• i.e. Certain lots/activities must observe a maximum time between 2 steps (consecutive or not)

• Certain steps of oven-type (diffusion step) and batching with size constraints


Conclusions | CP vs Heuristics

• There is again significant performance difference between IBM CP Optimizer’sunique algorithms and widely used heuristic approaches

• Better solutions can be achieved much more quickly

• Heuristics’ development and maintenance is not easy• Solution quality not guaranteed as data and few parameters change

• Can require a lot of coding


Conclusions | Industrial Scheduling Problems

The classical job-shop scheduling problem

• Resource/machines are over-simplified• In reality: setup-times, production modes, activities incompatibilities, batching, cumulative resources,

inventories (reservoirs), execution conditions (e.g. resource safety levels, auxiliary resources),...• All operations are performed in a unique way

• In reality: resource allocation, optional operations, alternative recipes, hierarchical decomposition• The single objective function is completely unrealistic

• In reality: combination of earliness/tardiness costs, nonexecution cost, resource related costs, constraint

violation, job/customer priorities…• Real problems are often much larger than the size of current benchmarks

adapted from https://www.slideshare.net/PhilippeLaborie/planningscheduling-with-cp-optimizer - Philippe Laborie, Principal Scientist at IBM

https://www.slideshare.net/PhilippeLaborie/planningscheduling-with-cp-optimizer

https://www.slideshare.net/PhilippeLaborie?utm_campaign=profiletracking&utm_medium=sssite&utm_source=ssslideview


Future Research | CP & MIP & Heuristics together

A Hybrid Approach for Machine Scheduling

Job assignment to machines with MIP or Heuristics (Master Problem)

+

Job sequencing & scheduling with CP (Subproblem)


Future Research | Application Areas

• Workforce Scheduling

• Vehicle Routing

• Advertisement Scheduling

• Sports League Scheduling

• Bridge Building

• Gate Allocation

• Timetabling

Data Science & AI Elite Team / © 2020 IBM Corporation

Thank you !

© Copyright IBM Corporation 2020. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. Any statement of direction represents IBM’s current intent, is subject to change or withdrawal, and represent only goals and objectives. IBM, the IBM logo, and ibm.com are trademarks of IBM Corp., registered in many jurisdictions worldwide. Other product and service names might be trademarks ofIBM or other companies. A current list of IBM trademarks is available at Copyright and trademark information.

To Learn More www.ibm.com/community/datascience/elite/

https://www.ibm.com/legal/copytrade

welcome to the ibm presentation template — ibm plex...

Documents