Simulation Optimization for Discrete-event Systems:

Ordinal Optimization and Beyond

Chun-Hung Chen

Dept. of Systems Engineering & Operations Research

George Mason University

Fairfax, VA, USA

• Studied at Harvard from 1991-94

• #39

• One year overlapping with Leyuan Shi

GREAT APPRECIATIONS TO PROF. HO

• Arrive before the queue starts to build up

• Avoid short flights‒ Longer flights have higher priority

‒ GDP: Ground Delay Program

• Avoid congested areas‒ Enough capacity is needed for the entire flight

‒ Shanghai, for example

FLIGHT DELAY IN CHINA

>

• Separation: 45 sec vs. 2 min

• Safety standard: 10-9 vs. 0.0

CAN BE DRAMATICALLY INCREASED, BUT…

Continuous systems Discrete event dynamic systems

• From 1990, students worked on a new subject – OO

• Fast solution for hard optimization problem

• Extremely important in the new era of big data

• IOT / Internet Plus

• Industry 4.0

• Cyber Physical Systems

NETWORKED SYSTEMS WITH BIG DATA

• Challenges and Opportunities‒ Connection and integration

system becomes larger and more complicated

‒ Global sensors

more factors (input variables) to include for decision making

‒ Real-time dynamic data

data keep arriving and changing less time to make a decision

‒ Smarter decision

smart city, smart grids, smartcare, smart buildings

• Useful Methodology‒ Large-scale Optimization

‒ Efficient Simulation

IN THE NEW ERA OF BIG DATA

Ordinal Opt.

SIMULATION OPTIMIZATION

Stochastic

SimulationX J(X)

)(min XJX

Control policy,

Decision,

Design alternative

System performance,

Output

COMPUTATIONAL ISSUES

),(1

)],([)(1

i

N

iX

WXfN

WXfEXJMin

Many Alternatives in Design Space

Multiple Simulation Runs (replications)Simulation

Optimization

EFFICIENCY CONCERN FOR SIMULATING K DESIGNS

# of simulation runs

1

Alternative Design

2

3

4

5

6

K-1

K

Challenges:

K*N can be

very large

N

METHODOLOGIES

# of simulation runs

1

Alternative Design

23456

K-1K

N

• Ordinal Optimization (Ho et al. 1992)1. Focus on good enough solutions

2. Concentrate on relative order comparison

only need to conduct a very small fraction of simulations

• Optimal Computing Budget Allocation (OCBA)Further enhance efficiency via optimal control of simulation

• Ordinal Transformation

OO1: GOOD ENOUGH SOLUTION

• Basic Idea‒ Instead of asking the best design, OO focuses on a good enough solution

‒ Only need to simulate a very small fraction of designs

• Conservative Case – Blind Picking

‒ Assume simulation estimation noise is extremely large (e.g., before simulation)

‒ If we are willing to accept a good design, say within top-0.1% (99.9 percentile), with a confidence probability Psat

• Utilize Simple Analytic Approximation Model

– Can do much better than blind picking in the above worst cast analysis

• See Lau & Ho (1997), Luo, Chen, Guignard-Spielberg (2001), Lin & Ho

(2002), Lin et al. (2004)

Psat 90% 99% 99.99% 99.999%

# of designs for simulation 2301 4603 9206 11,508

OO2: ORDINAL COMPARISON

• Basic Idea

‒ Instead of accurately estimating the performance measures for all

designs, OO concentrates on relative order comparison

‒ Can obtain exponential convergence rate (vs. O( ) for confidence

interval)

‒ Only need to conduct a smaller number of simulation runs

• Correct Selection Probability

P{CS} P{ The selected design is indeed better than others }

= P{ Correct Selection of the Best Alternative }

= 1 - e- N (, > 0)

(Dai 1996, Dai & Chen 1997, Ho et al. 2000)

N

1

OO: MUCH SMALLER SUBSET OF SIMULATION

Ordinal

Optimization

# of simulation runs

1

Alternative Design

23456

K-1K

N

# of simulation runs

1

Alternative Design

2

k-1k

Ranking & Selection

RANKING & SELECTION: TRADITIONAL PROCEDURES

• Many developments in simulation society :‒ Rinott (1978), Dudewicz and Dalal (1975), Goldsman and Nelson

(1994), Matejcik and Nelson (1993, 1995), Bechhofer, et al. (1995), …

• Ideas‒ Based on least favorable configuration

• Main results‒ Find the required Ni to asymptotically guarantee a desired P{CS}

‒ Conservative

‒ Simulation allocation is proportional to variance: Ni = ci2i

SMART SIMULATION ALLOCATION

x1 x2 x3 x4 x5

99% Confidence

Intervals for J(X)

after some simulations

• Which designs should we simulate more?‒ 2 & 3 are clearly superior

‒ 1, 4 & 5 have larger variances

• Chen (1995) & Chen (1996) propose smarter allocations for efficiency

• Maximize the Probability of Correctly Selecting the Best Design

OPTIMAL COMPUTING BUDGET ALLOCATION (OCBA)

Ni (b, j / j)2

Nj (b,i / i)2 for i j b

Nb = b ib (Ni2 / i

2)

• Asymptotically Optimal Solution

𝐦𝐚𝐱𝑁1,…,𝑁𝑠

𝑷{𝐂𝐒}

s.t. 𝑁1 + 𝑁2 +⋯+𝑁𝑠 = 𝑇 (total number of runs)

SOME INSIGHTS OF OCBA RULE

2

2

2,1

2

3

3,1

3

2

N

N

inversely proportional to

the square of the signal to noise ratio

Signal to Noise Ratio

Designs1 2 3

c2

1,3

1,2

c3

• Non-normal Distributions

- P. Glynn (Stanford Univ.)

- S. Juneja (Columbia Univ.)

• Heavy-tailed Distributions

- M. Broadie, M. Han, and A. Zeevi (Columbia University)

• Minimizing Variance Instead of Minimizing Mean

- Lucy Pao & Lidija Trailovic (U. of Colorado)

• Correlated Sampling

- Michael Fu (U. of Maryland at College Park)

- J.Q. Hu & Y.J. Peng (Fudan Univ.)

• Finding both Simple and Good Designs

- E. Zhou (Georgia Tech)

- Q.S. Jia (Tsinghua University)

SELECTED GENERALIZATIONS & EXTENSIONS (1)

• Multiple Objectives

- L. Lee & E. Chew (National University of Singapore)

• Small Computing Budget

- J. LaPorte (US Military Academy at West Point)

- J. Branke (Warwick Univ.)

• Transient Simulation

- D. J. Morrice (University of Texas at Austin)

• Expected opportunity cost instead of the probability of correct selection

- S. Gao (City U of HK)

- W. Chen (Rutgers Univ.)

- L. Shi (Peking Univ.)

• Optimal Subset Selection

- S. Zhang (Shanghai University)

SELECTED GENERALIZATIONS & EXTENSIONS (2)

EFFICIENCY USING MULTI-FIDELITY MODELS

Full Simulation Model Simplified Model

Some examples:High-fidelity model Low fidelity model

Discrete-event simulation Queueing theoryFine model Coarse model

Capturing uncertainty Ignoring uncertainty

Complex Much simpler

Good accuracy, butvery time consuming

Biased, but fast

MULTI-FIDELITY TIME-SENSITIVE DATA

Data from last month

Data from last week

Data from last hour

Fast Time

Data from yesterday

Freshness

of Data

Time Availability to Decision Point

Decision pointNow

• Flexible Manufacturing System• 2 product types

• 5 workstations

• Non-exponential service times

• Re-entrant manufacturing process

• Product 1 has higher priority than product 2

• Decision variable: number of machines allocated to each workstation

𝐌𝐢𝐧𝐢𝐦𝐢𝐳𝐞 Expected Total Processing Time𝐒𝐮𝐛𝐣𝐞𝐜𝐭 𝐭𝐨

𝟓 ≤ # of machines at each workstation ≤ 𝟏𝟎

Total # of machines at all workstatiosn = 𝟑𝟖

• # of alternatives: 780

EXAMPLE: RESOURCE ALLOCATION PROBLEM

Workstation 1

Workstation 2

Workstation 3

Workstation 4

Workstation 5

P2P1

• Bias is non-homogeneous and can be quite large

FULL SIMULATION VS. QUEUEING APPROXIMATION

ORDINAL TRANSFORMATION

• Quickly evaluate each alternative using low-fidelity model

• Transform the decision space into an ordinal space

High Fidelity Low Fidelity

ORDINAL TRANSFORMATION

OT

BENEFITS

OT

– Non-smooth response can become much smoother

– Neighborhood connection is strengthened

– Designs with similar performance are grouped together

– Search/optimization efficiency is enhanced

• Theorem 1. Ordinal transformation can reduce the variability of each group by at least

100 𝟏 −𝟑

𝒎+𝟐+

𝟔

𝒌𝟐𝝆𝟐%

‒ is the correlation between original and ordinal models

• Theorem 2. The differences between the means of two neighboring groups can be increased by

100𝟏𝟐𝒎

𝒌(𝒎+𝟏)𝝆%

SOME PROPERTIES

• 10 Groups (k=10)

MACHINE ALLOCATION PROBLEM

• Full simulation (high-fidelity)

MULTI-FIDELITY DATA AND MODELS

Product 1 Product 2

New Demand 250 150

• Low-fidelity model: queueing approximation‒ Very fast but poor approximation

• Multi-fidelity data (with old simulations)‒ Minimum additional cost but poor approximation

Product 1 Product 2

Old Demand 1 210 170

Old Demand 2 280 140

Product 1 Product 2

New Demand 250 150

• Suppose we have 2 approximation models: g1 and g2

𝑔 𝑋 = 𝑎1𝑔1 𝑋 + 𝑎2𝑔2 𝑋

• To maximize the correlation with the true model

𝑚𝑎𝑥𝑎1,𝑎2 𝜌 ≡𝐶𝑜𝑣 𝑔 𝑋 , 𝑓 𝑋

𝑉𝑎𝑟 𝑔 𝑋 𝑉𝑎𝑟 𝑓 𝑋

• Optimal weighting factors

𝑎1∗

𝑎2∗ =

𝜌1 − 𝜌12𝜌2 𝜎2𝜌2 − 𝜌12𝜌1 𝜎1

OPTIMAL LINEAR COMBINATION OF TWO APPROXIMATION MODELS

** Joint work with Si Zhang

• Ordinal Transformation using single model/data

LEAD TO HIGH CORRELATION & ALIGNMENT

RankCorrelation

Alignment (top-5)

Old Demand 1 only 0.623 2

Old Demand 2 only 0.596 2

• Ordinal Transformation with intelligent combining use of two models

RankCorrelation

Alignment (top-5)

Old 1 + Old 2Demand Models

0.875 5

DOES A SECOND MODEL ALWAYS HELP?

2

12

• Consider a case where 1 = 0.6

If 2 = 0.6

helpful area, i.e., 12 < 0.2

SUMMARY

• Ordinal Optimization (Ho et al. 1992)1. Focus on good enough solutions

2. Concentrate on relative order comparison

only need to conduct a very small fraction of simulations

• Optimal Computing Budget Allocation (OCBA)‒ Further enhance OO efficiency via optimal control of simulation

• Ordinal Transformation‒ Utilize low-fidelity models/data to transform the decision space

into a better space

‒ Enhance search efficiency