Optimizing Sensor Type and Location for Rapid Restoration of Power Grids

Lina Al-Kanj, PhD

Warren B. Powell

Operations Research and Financial Engineering DepartmentPrinceton University

Princeton, NJ

Federal Energy Regulatory Commission Technical ConferenceWashington, DC

Outline

Problem Description

Probability Model for Grid Faults

Sequential Stochastic Optimization Model for Utility Crew Routing Multistage Lookahead Policy (Monte Carlo Tree Search)

Stochastic Optimization Model for Sensor and Protective Device Placements

2

Problem Description – Power System

3

Problem Description – Power System

4

Problem Description – Power System

XX

X

X

5

Problem Description – Power System

XX

X

X

6

Problem Description – Power System

X

X

XX

X

X

7

Problem Description - Emergency Storm Response

XX

X

X

X

X

8

Problem Description - Emergency Storm Response

XX

X

X

X

X

9

Problem Description - Emergency Storm Response

XX

X

X

X

X

10

Problem Description - Emergency Storm Response

XX

X

X

X

X

11

Problem Description - Emergency Storm Response

XX

X

X

X

X

12

Problem Description - Emergency Storm Response

XX

X

X

X

callcall

X

call

call

call

call

call

13

XX

X

X

X

X

Problem Description - Emergency Storm Response

call

call

callcall

call

call

call

14

Known customers in outage

Unknown outages

Outage calls(known)

Network outages(unknown)

Storm

15

How to create a belief about outages?

How to reconstruct the grid to minimize customer outage-minutes?

What is the value of a sensor/re-closer in terms of reducing customer outage-minutes?

What is the best place to put a sensor or re-closer?

What is the value of upgrades to SCADA systems?

What is the value of investments in reconfigurable grids?

Problem Description – Research Goals

16

Outline

Problem Description

Probability Model for Grid Faults

Sequential Stochastic Optimization Model for Utility Crew Routing Multistage Lookahead Policy (Monte Carlo Tree Search)

Stochastic Optimization Model for Sensor and Protective Device Placements

17

Exploiting the information from phone calls We have to blend the following…. What we knew before the phone calls came in – this is called the prior

belief.

The outage calls – this is called information.

…to produce the updated estimates of outage – this is called the posterior.

To compute this we have to use Bayes theorem:

Prob[lights-out calls | segment is out]Prob[ is out]Prob[segment is out|lights-out calls]Prob[lights-out calls]

l ll

Probability Model for Outage Identification

[1] L. Al-Kanj, B. Bouzaeini-Ayari and W. Powell, “A Probability Model for Grid Faults Using Incomplete Information”, IEEE Transactions on Smart Grids, July 2015.

18

Exploiting the information from phone calls We have to blend the following…. What we knew before the phone calls came in – this is called the prior

belief.

The outage calls – this is called information.

…to produce the updated estimates of outage – this is called the posterior.

To compute this we have to use Bayes theorem:

Prob[lights-out calls | segment is out]Prob[ is out]Prob[segment is out|lights-out calls]Prob[lights-out calls]

l ll

The prior

Probability Model for Outage Identification

19

Exploiting the information from phone calls We have to blend the following…. What we knew before the phone calls came in – this is called the prior

belief.

The outage calls – this is called information.

…to produce the updated estimates of outage – this is called the posterior.

To compute this we have to use Bayes theorem:

Prob[lights-out calls | segment is out]Prob[ is out]Prob[segment is out|lights-out calls]Prob[lights-out calls]

l ll

The prior

The information

Probability Model for Outage Identification

20

Exploiting the information from phone calls We have to blend the following…. What we knew before the phone calls came in – this is called the prior

belief.

The outage calls – this is called information.

…to produce the updated estimates of outage – this is called the posterior.

To compute this we have to use Bayes theorem:

Prob[lights-out calls | segment is out]Prob[ is out]Prob[segment is out|lights-out calls]Prob[lights-out calls]

l ll

The prior

The informationThe posterior

Probability Model for Outage Identification

21

Exploiting the information from phone calls We have to blend the following…. What we knew before the phone calls came in – this is called the prior

belief.

The outage calls – this is called information.

…to produce the updated estimates of outage – this is called the posterior.

To compute this we have to use Bayes theorem:

Prob[lights-out calls | segment is out]Prob[ is out]Prob[segment is out|lights-out calls]Prob[lights-out calls]

l ll

The prior

The informationThe posterior

The conditional outage distribution

Probability Model for Outage Identification

22

Exploiting the information from phone calls We have to blend the following…. What we knew before the phone calls came in – this is called the prior

belief.

The outage calls – this is called information.

…to produce the updated estimates of outage – this is called the posterior.

To compute this we have to use Bayes theorem:

Prob[lights-out calls | segment is out]Prob[ is out]Prob[segment is out|lights-out calls]Prob[lights-out calls]

l ll

The prior

The informationThe posterior

The phone call distribution

The conditional outage distribution

Probability Model for Outage Identification

23

Probability Model – Prior Probabilities

0.2 0.3 0.6 0.6 0.6

0.2

0.2

0.2

0.2

0.3

0.4

0.4

0.5

0.5

0.50.5 0.5

0.5 0.5

0.5 0.5 0.5

0.6 0.6

0.50.50.5

0.5

0.50.5

0.7

0.50.5

0.50.5

0.5

0.50.5

24

XX

X

X

X

X

Posterior Probabilities

callcall

call

call

call

call

call

0.26 0.38 0.78 0.6 0.6

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

30 Customers

30 Customers

30 Customers

25

Outline

Problem Description

Probability Model for Grid Faults

Sequential Stochastic Optimization Model for Utility Crew Routing Multistage Lookahead Policy (Monte Carlo Tree Search)

Stochastic Optimization Model for Sensor and Protective Device Placements

26

Grid Restoration Model

Objective: Develop an optimal policy that routes the utility truck in order to minimize the number of customers in outage at any point in time.

[4] L. Al-Kanj, W. Powell and B. Bouzaeini-Ayari, “The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response”, Submitted to Operations Research, http://arxiv.org/abs/1605.05711, 2016.

, … ,00 0

0 0

: Physical State

: Information State

: Knowledge State

?

?

?

??

? ?

Information-Collection utility truck routing

27

Grid Restoration Model

Objective: Develop an optimal policy that routes the utility truck in order to minimize the number of customers in outage at any point in time.

[4] L. Al-Kanj, W. Powell and B. Bouzaeini-Ayari, “The Information-Collecting Vehicle Routing Problem: Stochastic Optimization for Emergency Storm Response”, In preparation.

Information-Collection utility truck routing

, … ,00 0

0 0

?

0

??

? ?: Physical State

: Information State

: Knowledge State

28

Sequential Stochastic Optimization Model

Five fundamental elements of sequential stochastic optimization:

State - information capturing what we know at time t;, , where represents the physical state of the network.

Decision - captures the decision made at time ; Let be the policy that determines ∈ given .

Exogenous information - new information that arrives between 1and ;

Includes new arriving calls, travel time, repair time and outages.

Transition function - , , represents the evolution of the states

e.g., , , 1| , 1 0 in addition to .

Objective function - min ∑ , ;minimizes the number of customers in outage at any point in time.

29

Sequential Stochastic Optimization Model

Cost function -

Figure 6: Objective function; Customer outage-minute is represented by the shaded areaunder the curve.

,

30

Sequential Stochastic Optimization Model

min ,

Two fundamental strategies for designing policies [5]: Policy Search Lookahead Approximations Value function approximation

Direct lookahead

where

, ,

Optimization problem

[2] W. B Powell, A Unified Framework for Optimization under Uncertainty, Informs Tutorials, November 2016.31

Lookahead approximations – Approximate the impact of a decision now on the future: An optimal policy (based on looking ahead):

Lookahead Approximations

*' ' ' 1

' 1( ) arg max ( , ) max ( , ( )) | | ,

t

T

t t x t t t t t t t tt t

X S C S x C S X S S S x

32

Lookahead approximations – Approximate the impact of a decision now on the future: An optimal policy (based on looking ahead):

2a) Approximating the value of being in a downstream state using machine learning (“value function approximations”)

2b) Approximate lookahead models – Optimize over an approximate model of the future:

*1 1

1 1

( ) arg max ( , ) ( ) | ,

( ) arg max ( , ) ( ) | ,

arg max ( , ) ( )

t

t

t

t t x t t t t t t

VFAt t x t t t t t t

x xx t t t t

X S C S x V S S x

X S C S x V S S x

C S x V S

' ' , !' 1

( ) arg max ( , ) max ( , ( )) | | ,

T

LAt t t t tt tt t t t t

t t

X S C S x C S X S S S x

*' ' ' 1

' 1( ) arg max ( , ) max ( , ( )) | | ,

t

T

t t x t t t t t t t tt t

X S C S x C S X S S S x

Lookahead Approximations

33

We can then simulate this lookahead policy over time:

. . . .

t 1t 2t 3t

The

look

ahea

dm

odel

The base model

Multistage Lookahead Approximations

34

. . . .

t 1t 2t 3t

The

look

ahea

dm

odel

The base model

Multistage Lookahead Approximations

We can then simulate this lookahead policy over time:

35

. . . .

t 1t 2t 3t

The

look

ahea

dm

odel

The base model

Multistage Lookahead Approximations

We can then simulate this lookahead policy over time:

36

. . . .

t 1t 2t 3t

The

look

ahea

dm

odel

The base model

Multistage Lookahead Approximations

We can then simulate this lookahead policy over time:

37

Multistage Lookahead Policy

∗ argmin ∈ , , ∈ , min, ∈ , , , , ,

, ∈ , … , ∈ , , | , … | , |

The optimal policy is computationally intractable, requiring approximations:

where , , , , , , … , 1

Discretizing the time, states and decision

Limiting the horizon from , to , Dimensionality reduction e.g., by limiting some variables (e.g., fixing the set of calls,

limiting the fault types and travel times)

Aggregating the outcome or sampling by using Monte Carlo sampling

38

Monte Carlo Tree Search (MCTS)

MCTS is a recent research area; the first MCTS algorithm has been developed by Chang et al [2005].

MCTS growth [5]

[3] H. S. Chang, M. C. Fu, J. Q. Hu, and S. I. Marcus. An adaptive sampling algorithm for solving Markov decision processes. Operations research, 53(1):126139, 2005.[4] Kocsis Levente and Csaba Szepesvári. "Bandit based monte-carlo planning“, Machine Learning: ECML 2006. pp. 282-293.[5] R. Coquelin, Pierre-Arnaud and Munos, “Bandit Algorithms for Tree Search,” in Proc. Conf. Uncert. Artif. Intell. Vancouver,Canada: AUAI Press, 2007, pp. 67–74.[6] A. Coutoux, J. B. Hoock, N. Sokolovska, O. Teytaud, and N. Bonnard. Continuous upper confidence trees. In International Conference on Learning and Intelligent Optimization, pages 433445. Springer Bernlin Heidelberg, 2011.

MCTS biases the growth of the tree towards the most promising moves which decreases the search space [3].

MCTS mainly applied for deterministic problems but Coutoux et al [2011] extended it to stochastic problems based on double progressive widening.

39

Monte Carlo Tree Search

Decision selection based on UCB (Upper Confidence Bounds for Trees) [7]: Visit count ofstate

Visit count of decision from state Exploitation Exploration

MCTS Construction Steps

[7] D.Auger, A. Couetoux, and O. Teytaud, “Continuous upper confidence trees with polynomial exploration consistency,” in Proc. ofthe European Conference on Machine Learning and Knowledge Discovery in Databases, (Prague, Czech Republic), pp. 194–209, 2013.

∗ argmax ∈ ,log

,

40

Monte Carlo Tree Search – Convergence Theory

For deterministic MCTS, the UCB policy samples the actions infinitely often and Kocsis and Szepesvari [2006] exploit this to show that the probability of selecting a suboptimal action converges to zero at the root of the tree.

Auger et al. [2013] provides convergence results for stochastic MCTS with double progressive widening under an action sampling assumption. The asymptotic convergence of also MCTS relies on some form of “exploring every node infinitely often“.

In Jiang et al. [2017], we design a version of stochastic MCTS that asymptotically does not expand the entire tree, yet is still optimal!

[6] Kocsis Levente and Csaba Szepesvári. "Bandit based monte-carlo planning“, Machine Learning: ECML 2006. pp. 282-293.[7] D.Auger, A. Couetoux, and O. Teytaud, “Continuous upper confidence trees with polynomial exploration consistency,” in ProcThe European Conference on Machine Learning and Knowledge Discovery in Databases, (Prague, Czech Republic), pp. 194–209,[8] D. Jiang, L. Al-Kanj, W. B. Powell, “Monte Carlo Tree Search with Information Relaxation Dual Bounds”, In preparation for Operations Research, 2017.

41

Simulation Policy

Generate sample path from Ω , that determines all random events (faults, travel time, repair time).

Utility truck should visit each fault once to repair it.

Objective: find the optimal route that minimizes the customer-outage minute.

In the worst case, computational complexity ! .

X

X

X

X

42

Simulation Policy – Optimal Solution

Define the graph , with connection costs .

: subset of nodes visited by the truck.

, : customer-outage minute starting from node 1 and ending at node . .

: function returning the number of customers in outage after visiting the nodes of .

The cost of moving from node to node is, , ∗

Dynamic Program

For all ∈ do 1, , ∑For 3 to

For all Subsets of of size doFor all ∈ , 1 do

min ,

, min∈ ,

, ∗ Computational Complexity

243

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.26 0.38 0.78 0.6 0.6

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

44

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.26 0.38 0.78 0.6 0.6

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

45

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.715 0.715

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

46

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.715 0.715

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

47

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

48

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

49

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

00.032

0.048

0.056

0.056

0.08

0.0604

0.06040.0604 0.0604

0.51 0.51

0.51 0.51 0

0.62 0.62

00.990.08

0.08

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

50

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

00.032

0.048

0.056

0.056

0.08

0.0604

0.06040.0604 0.0604

0.51 0.51

0.51 0.51 0

0.62 0.62

00.990.08

0.08

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

51

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

00.0

0.0

0.0

0.0

0.0

0.0603

0.06030.0603 0.0603

0.52 0.52

0.52 0 0

0.63 0.63

000.0

0.0

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

52

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

00.0

0.0

0.0

0.0

0.0

0.0603

0.06030.0603 0.0603

0.52 0.52

0.52 0 0

0.63 0.63

000.0

0.0

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

53

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

00.0

0.0

0.0

0.0

0.0

0.0603

0.06030.0603 0.0603

0.59 0

0.59 0

0 0

000.0

0.0

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

0

54

XX

X

X

X

X

Lookahead Policy

callcall

call

call

call

call

call

0.0 0.0 0.0 0.0 0.0

0

0

00.0

0.0

0.00.0

0.06

0.060.06 0.06

0 0

0 0

0 0

000.0

0.0

0

0

0

00

00

0

0.670.67

00.0

55

Industrial Heuristics

Escalation Algorithm

Step 1. For each circuit do

Step 1a. Collect all calls and back trace to find the first node that is

common to all calls say node .

Step 1b. Send the truck to node and then back trace to the substation

to make sure that there is no upstream fault

Step 1c. From node , perform down tracing to reach the first segment

from which a call was initiated and place it in set .

Step 2. For each segment in do

Step 2a. Perform down tracing to cover all nodes that called.

56

XX

X

callcall

call

call

call

Escalation Algorithm

call

call

X

X

X

57

XX

X

callcall

call

call

call

Escalation Algorithm

call

call

X

X

X

58

XX

X

callcall

call

call

call

Escalation Algorithm

call

call

X

X

X

59

XX

X

callcall

call

call

call

Escalation Algorithm

call

call

X

X

X

60

XX

X

X

X

callcall

call

call

call

Escalation Algorithm

call

call

X

61

X

callcall

call

call

Escalation Algorithm

call

call

call

X

X

X

X

X

62

X

callcall

call

call

Escalation Algorithm

call

call

call

X

X

X

X

X

63

Numerical Results – Customer Outage-Minutes

36150

55110

Escalation algorithm

Lookahead policy

Performance metrics Total outage minutes SAIDI, CAIDI, SAIFI, …

64

Studies

Possible questions:

What is the effect of higher call-in rates?

What is the best place to locate a sensor/protective device?

What is the value of a sensor vs. protective device?

65

Numerical Results

Figure 10: Average customer outage-hours vs. MCTS budget for ten networks.

66

Comparison to Industrial Heuristics

67

Outline

Problem Description

Probability Model for Grid Faults

Sequential Stochastic Optimization Model for Utility Crew Routing Multistage Lookahead Policy (Monte Carlo Tree Search) Monte Carlo Tree Search with Information Relaxation Dual Bounds

Stochastic Optimization Model for Sensor and Protective Device Placements

68

Optimal Protective Device and Sensor Placement

Protective Device vs Sensor Placement to minimize Customer-Outage Minute:

A protective device prevents power flow to the downstream segments when a fault is detected.

A sensor feeds back information to the utility center whether power is on or off.

: a binary vector; 1if a protective device is placed at node and 0 otherwise.

combinations of protective devices which results in high computation complexity.

Solved sequentially by placing one protective device at a time.

Optimization problem for optimal protective device and sensor placements:

69

XX

X

X

X

X

callcall

call

call

call

call

call

0.26 0.38 0.78 0.6 0.6

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

?

?

?

?

Optimal Protective Device and Sensor Placement

?

70

XX

X

X

X

X

Case study A: sensor vs. protective device

callcall

call

call

call

call

call

0.26 0.38 0.78 0.6 0.6

0

0

0

0.02

0.03

0.04

0.04

0.05

0.064

0.0640.064 0.064

0.502 0.502

0.502 0.502 0.502

0.603 0.603

0.670.670.05

0.05

0.5030.5

0.76

0.5030.503

0.540.54

0.503

0.5030.503

Customer Outage-Minute Reduction:Value of sensor = 0Value of protective device = 225

30 Customers

30 Customers

30 Customers

71

XX

X

X

X

X

callcall

call

call

call

call

call

0.26 0.38 0.78 0.6 0.6

0.01

0.01

0.01

0.18

0.27

0.36

0.36

0.45

0.47

0.470.47 0.47

0.501 0.501

0.501 0.501 0.501

0.601 0.601

0.580.580.45

0.45

0.5010.5

0.71

0.5010.501

0.5070.507

0.501

0.5010.501

10 Customers

10 Customers

10 Customers

Customer Outage-Minute Reduction:Value of sensor = 3360Value of protective device = 0

Case study B: sensor vs. protective device

72

Summary

We have developed: A detailed storm and grid simulator that simulates storms, outages and lights-out

calls

Models the behavior of protective devices and grid reconfigurations

An industry-standard escalation policy and a probabilistic lookahead model that uses prior knowledge and anticipates likely outages

Shown that the lookahead policy produces near-optimal performance when compared against optimal on a deterministic model

Simulations We have quantified the value of sensors and protective devices at different

locations

We can find the best locations for each type of device

Conclusion We believe that our simulator can be used to develop effective rate cases for the

BPU

73

Summary

We have developed: A detailed storm and grid simulator that simulates storms, outages and lights-out

calls

Models the behavior of protective devices and grid reconfigurations

An industry-standard escalation policy and a probabilistic lookahead model that uses prior knowledge and anticipates likely outages

Shown that the lookahead policy produces near-optimal performance when compared against optimal on a deterministic model

Simulations We have quantified the value of sensors and protective devices at different

locations

We can find the best locations for each type of device

Conclusion We believe that our simulator can be used to develop effective rate cases for the

BPUThank you for your attention!

74

[1] L. Al-Kanj, B. Bouzaeini-Ayari and W. Powell, “A Probability Model for Grid Faults Using Incomplete Information”, IEEE Transactions on Smart Grids, July 2015.

Problem Description – Grid Restoration

I set of poles, i.e., I , 1, … ,

U set of circuits, i.e., U , 1, … ,

I set of power lines on circuit u, i.e., I , 1, … ,

Nu Set of nodes on circuit u, i.e., N , ∀ ∈

Number of customers attached to node

, Number of customers that called on node

Ω sample space at time ; each scenariorepresents phone calls, faults, travel & repair times

random vector representing the realizations of received calls at time

random vector representing the realizations of power line faults at time

random variable representing the travel time from node to node at timerandom variable representing the repair time of power line on circuit

random vector representing the trajectory of the truck at time ;

, ∈

Bayes’ Theorem:Illustration of distribution grid

Prior

Likelihood of callsgiven faults

Probability of calls

|| |

∑ ∈ |

Posterior

?

?

??

?

?

call

call

call

? ??

?

75

Features of the simulator: Complete model of the distribution grid: Substations Sensors, protective devices, reconfiguration rules, …

Able to simulate: Many storm trajectories with varying sizes Process of outages occurring as storm progresses Model of who loses power (from grid configuration) Process of lights-out calls (with variable call-in rates)

Probabilistic model of outages Captures prior knowledge and probabilistic model of lights-out calls

Dispatch policies: Industry-standard escalation policy – Uses lights-out calls and grid

configuration Stochastic lookahead policy – Exploits uncertainty model of outage

probabilities

Problem Description – Simulator

76

Monte Carlo Tree Search

Primal-Dual MCTS Algorithm Propose a Primal-Dual MCTS, that takes advantage of the information relaxation bound idea that

asymptotically converges [8] to the optimal solution while ignoring suboptimal parts of the tree Explore a set of actions:

Smooth with previous estimates:

Expand it if the value is greater than

[8] D. Jiang, L. Al-Kanj, W. B. Powell, “Monte Carlo Tree Search with Information Relaxation Dual Bounds”, In preparation for Operations Research, 2017. 77

Simulation Policy – Integer programming

Minimizes the customer-outage minute

Computes number of customers with restored power

Indicates whether power line is still in outage

Computes required travel and repair time by going from node to node

Guarantee that the required travel and repair times are met

Guarantees that each arc is visited at most once in one direction

Indicates whether the truck can move to the next node

The problem can be formulated as a non-linear integer program as follows:

Domains of the variables 78