stochastic optimization algorithms for smart gridhome.iitk.ac.in/~amritbd/docs/mtech_defense.pdf ·...

81
Stochastic Optimization Algorithms for Smart Grid M.Tech. Defense Amrit Singh Bedi Supervisor: Dr. Ketan Rajawat SPiN Lab, EE Dept., IIT Kanpur Sep. 5, 2017 1/63

Upload: buithuan

Post on 07-Apr-2019

225 views

Category:

Documents


0 download

TRANSCRIPT

Stochastic Optimization Algorithms for Smart Grid

M.Tech. Defense

Amrit Singh Bedi

Supervisor: Dr. Ketan Rajawat

SPiN Lab, EE Dept., IIT Kanpur

Sep. 5, 2017

1/63

2/63

Outline

1 Introduction and Motivation

2 Part I: Online Load SchedulingDemand Side Management (DSM)Related Work and ContributionSystem Model and Problem FormulationStochastic Dual Descent Based SolutionConvergence ResultsProof OutlineSimulation Results

3 Part II : Online Energy OptimizationRelated Work and ContributionsSystem model and DefinitionsProblem Formulation and solutionWith forcasted PricesSimulation Results

4 Conclusion and Future work

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 2 / 63

3/63

Introduction and Motivation

Outline

1 Introduction and Motivation

2 Part I: Online Load SchedulingDemand Side Management (DSM)Related Work and ContributionSystem Model and Problem FormulationStochastic Dual Descent Based SolutionConvergence ResultsProof OutlineSimulation Results

3 Part II : Online Energy OptimizationRelated Work and ContributionsSystem model and DefinitionsProblem Formulation and solutionWith forcasted PricesSimulation Results

4 Conclusion and Future work

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 3 / 63

4/63

Introduction and Motivation

Why Smart Grid ??

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 4 / 63

5/63

Introduction and Motivation

Introduction and Motivation

I The conventional power grid suffers from reliability and stability issuesresulting from an unpredictable events occurring throughout the grid1,2

Table 1: A vision for Smart Grid1

Existing Grid Smart Grid

One-way flow Two-way flowCentralized generation Distributed generationFew sensors Sensors throughoutManual monitoring Self monitoringFailure and blackouts AdaptiveNo customer involvement Customer involvement

1Xi Fang et al. “Smart grid - The new and improved power grid: A survey”. In: IEEE Commun. Surveys Tuts. 14.4 (2012), pp. 944–980.

2Ye Yan et al. “A survey on smart grid communication infrastructures: Motivations, requirements and challenges”. In: IEEE Commun.Surveys Tuts. 15.1 (2013).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 5 / 63

6/63

Introduction and Motivation

Introduction and Motivation

There is a popular comparison that underscores the pace of change - or lackthereof - regarding conventional grid

The story goes like this:

“If Alexander Graham Bell were somehow transported to the 21st century,he would not begin to recognize the components of modern telephony cellphones, texting, cell towers, PDAs, etc. while Thomas Edison, one of thegrids key early architects, would be totally familiar with the grid.”

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 6 / 63

6/63

Introduction and Motivation

Introduction and Motivation

There is a popular comparison that underscores the pace of change - or lackthereof - regarding conventional grid

The story goes like this:

“If Alexander Graham Bell were somehow transported to the 21st century,he would not begin to recognize the components of modern telephony cellphones, texting, cell towers, PDAs, etc. while Thomas Edison, one of thegrids key early architects, would be totally familiar with the grid.”

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 6 / 63

6/63

Introduction and Motivation

Introduction and Motivation

There is a popular comparison that underscores the pace of change - or lackthereof - regarding conventional grid

The story goes like this:

“If Alexander Graham Bell were somehow transported to the 21st century,he would not begin to recognize the components of modern telephony cellphones, texting, cell towers, PDAs, etc. while Thomas Edison, one of thegrids key early architects, would be totally familiar with the grid.”

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 6 / 63

7/63

Introduction and Motivation

What is Smart Grid ??

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 7 / 63

8/63

Introduction and Motivation

What is Smart Grid?

Figure 1: Two way information flow

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 8 / 63

9/63

Introduction and Motivation

Smart Grid

I Smart grid is an electrical grid equipped with automation, advancedmetering, and communication systems to control generation,distribution and consumption of electrical energy3

I Smart Grid seeks to employ state-of-the-art tools from communications,information technology, and signal processing

3“Smart grid vision and roadmap for India”. In: (2013).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 9 / 63

9/63

Introduction and Motivation

Smart Grid

I Smart grid is an electrical grid equipped with automation, advancedmetering, and communication systems to control generation,distribution and consumption of electrical energy3

I Smart Grid seeks to employ state-of-the-art tools from communications,information technology, and signal processing

3“Smart grid vision and roadmap for India”. In: (2013).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 9 / 63

9/63

Introduction and Motivation

Smart Grid

I Smart grid is an electrical grid equipped with automation, advancedmetering, and communication systems to control generation,distribution and consumption of electrical energy3

I Smart Grid seeks to employ state-of-the-art tools from communications,information technology, and signal processing

3“Smart grid vision and roadmap for India”. In: (2013).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 9 / 63

9/63

Introduction and Motivation

Smart Grid

I Smart grid is an electrical grid equipped with automation, advancedmetering, and communication systems to control generation,distribution and consumption of electrical energy3

I Smart Grid seeks to employ state-of-the-art tools from communications,information technology, and signal processing

3“Smart grid vision and roadmap for India”. In: (2013).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 9 / 63

10/63

Introduction and Motivation

Different Challenges

From Signal Processing Perspective:

I Two-way flow of energy and information throughout the system

I Plug-in hybrid electric vehicles integration

I Observing the Power Grid

I State Estimation in Electric Power Grids

I Demand-Side Management (Demand Response) with Dynamic Pricing

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 10 / 63

11/63

Introduction and Motivation

Different Challenges

From Signal Processing Perspective:

I Two-way flow of energy and information throughout the system

I Plug-in hybrid electric vehicles integration

I Observing the Power Grid

I State Estimation in Electric Power Grids

I Demand-Side Management (Demand Response) with Dynamic Pricing

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 11 / 63

12/63

Introduction and Motivation

Part I :Online Load Scheduling Under Price &Demand Uncertainty in Smart Grid4

4Amrit S Bedi and Ketan Rajawat. “Online load scheduling under price and demand uncertainty in smart grid”. In: Signal Processing andCommunications (SPCOM), 2016 International Conference on. IEEE. 2016, pp. 1–5.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 12 / 63

13/63

Part I: Online Load Scheduling

Outline

1 Introduction and Motivation

2 Part I: Online Load SchedulingDemand Side Management (DSM)Related Work and ContributionSystem Model and Problem FormulationStochastic Dual Descent Based SolutionConvergence ResultsProof OutlineSimulation Results

3 Part II : Online Energy OptimizationRelated Work and ContributionsSystem model and DefinitionsProblem Formulation and solutionWith forcasted PricesSimulation Results

4 Conclusion and Future work

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 13 / 63

14/63

Part I: Online Load Scheduling Demand Side Management (DSM)

DSM

I What is DSM?I Demand side management5 is adjustment of the demand for electricity

at user side during peak periods and reducing their overall energyconsumption cost

I It comprises of two main principle activitiesI Demand responseI Energy storage management

I Advantages

• Cutting costs for service providers, Saving money for households• Helping utilities to operate more efficiently and in return reducing

emission of greenhouse gases

5Brandon Davito, Humayun Tai, and Robert Uhlaner. “The smart grid and the promise of demand-side management”. In: McKinsey onSmart Grid 3 (2010), pp. 8–44.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 14 / 63

14/63

Part I: Online Load Scheduling Demand Side Management (DSM)

DSM

I What is DSM?I Demand side management5 is adjustment of the demand for electricity

at user side during peak periods and reducing their overall energyconsumption cost

I It comprises of two main principle activitiesI Demand responseI Energy storage management

I Advantages

• Cutting costs for service providers, Saving money for households• Helping utilities to operate more efficiently and in return reducing

emission of greenhouse gases

5Brandon Davito, Humayun Tai, and Robert Uhlaner. “The smart grid and the promise of demand-side management”. In: McKinsey onSmart Grid 3 (2010), pp. 8–44.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 14 / 63

15/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Motivation

Figure 2: Sector wise electricity consumption in India6

6“Residential Electricity Consumption in India”. In: Ministry of Statistic and Programme Implementation Dec. (2015).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 15 / 63

16/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Motivation

Figure 3: Residential load profile in Gujrat7

7“Residential Electricity Consumption in India”. In: Ministry of Statistic and Programme Implementation Dec. (2015).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 16 / 63

17/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Demand Response

Figure 4: Demand Response8

8John S Vardakas, Nizar Zorba, and Christos V Verikoukis. “A survey on demand response programs in smart grids: Pricing methods andoptimization algorithms”. In: IEEE Communications Surveys & Tutorials 17.1 (2015), pp. 152–178.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 17 / 63

18/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Demand Response

Figure 5: Main participants in DR program9

9John S Vardakas, Nizar Zorba, and Christos V Verikoukis. “A survey on demand response programs in smart grids: Pricing methods andoptimization algorithms”. In: IEEE Communications Surveys & Tutorials 17.1 (2015), pp. 152–178.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 18 / 63

19/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Demand Response

Figure 6: Example of scheduling in DR program10

10John S Vardakas, Nizar Zorba, and Christos V Verikoukis. “A survey on demand response programs in smart grids: Pricing methods andoptimization algorithms”. In: IEEE Communications Surveys & Tutorials 17.1 (2015), pp. 152–178.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 19 / 63

20/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Dynamic Pricing11

Dynamic pricing means that electricity prices are time varying

I Time of use pricing (ToU)

I Critical peak pricing (CPP)

But in these schemes, the prices are set well in advance and do not reallyreflects the system state

I Real time pricing (RTP)

11Brandon Davito, Humayun Tai, and Robert Uhlaner. “Smart grid: An approach to dynamic pricing in India”. In: April (2014).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 20 / 63

20/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Dynamic Pricing11

Dynamic pricing means that electricity prices are time varying

I Time of use pricing (ToU)

I Critical peak pricing (CPP)

But in these schemes, the prices are set well in advance and do not reallyreflects the system state

I Real time pricing (RTP)

11Brandon Davito, Humayun Tai, and Robert Uhlaner. “Smart grid: An approach to dynamic pricing in India”. In: April (2014).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 20 / 63

20/63

Part I: Online Load Scheduling Demand Side Management (DSM)

Dynamic Pricing11

Dynamic pricing means that electricity prices are time varying

I Time of use pricing (ToU)

I Critical peak pricing (CPP)

But in these schemes, the prices are set well in advance and do not reallyreflects the system state

I Real time pricing (RTP)

11Brandon Davito, Humayun Tai, and Robert Uhlaner. “Smart grid: An approach to dynamic pricing in India”. In: April (2014).

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 20 / 63

21/63

Part I: Online Load Scheduling Related Work and Contribution

Related Work and Contribution

I Load uncertainty, day ahead pricing, and utility-side problem of powergeneration scheduling has been considered in12,13

I Price uncertainty alone has been considered via robust optimization14

I Stochastic dynamic programming framework is used in15 for schedulingbut assumes the statistical knowledge of future prices

I Load scheduling under price uncertainty (forecasted prices) isconsidered in16 (temporarily coupled constraints)

12M. Parvania and M. Fotuhi-Firuzabad. “Demand response scheduling by stochastic Security Constrained Unit Commitment SCUC”. . In:IEEE Trans. on Smart Grid 1.1 (2010), pp. 89–98.

13J. Zhang, J. D. Fuller, and S. Elhedhli. “A stochastic programming model for a day-ahead electricity market with real-time reserve shortagepricing”. In: 25.2 (2010), pp. 703–713.

14A.J. Conejo, J.M. Morales, and L. Baringo. “Real-Time Demand Response Model”. In: IEEE Trans. on Smart Grid 1.3 (2010),pp. 236–242.

15T.T. Kim and H.V. Poor. “Scheduling Power Consumption With Price Uncertainty”. In: IEEE Trans. on Smart Grid 2.3 (2011),pp. 519–527.

16R. Deng et al. “Load scheduling with price uncertainty and temporally-coupled constraints in smart grids”. In: 29.6 (2014), pp. 2823–2834.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 21 / 63

22/63

Part I: Online Load Scheduling Related Work and Contribution

Related Work and Contribution

I A dual-descent based algorithm was proposed in17

I Online scheduling algorithm of17 utilizes price statistics for loadscheduling, and is therefore susceptible to large variations in prices

I In contrast, the proposed algorithm18 allows the prices and loads tohave different average values for different times of the day

I The proposed algorithm is inspired from the incremental stochasticsubgradient algorithm of19

17R. Deng et al. “Load scheduling with price uncertainty and temporally-coupled constraints in smart grids”. In: 29.6 (2014), pp. 2823–2834.

18Amrit S Bedi and Ketan Rajawat. “Online load scheduling under price and demand uncertainty in smart grid”. In: Signal Processing andCommunications (SPCOM), 2016 International Conference on. IEEE. 2016, pp. 1–5.

19S. S. Ram, A. Nedic, and V. V. Veeravalli. “Incremental stochastic subgradient algorithms for convex optimization”. In: SIAM Journal onOptimization 20.2 (2009), pp. 691–717.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 22 / 63

23/63

Part I: Online Load Scheduling System Model and Problem Formulation

System Model Settings

Power

Grid

Utility

Company

Smart

Meter

LOAD

Controller

Battery

AC/DC AC/DC

Solar

PV

Energy Transfer Link

Control Signals

Consumer

Power Bus

Loa

d s

ch

SoC

Iref

Pri

ce

Battery

AC/DCSmart

Meter

Figure 7: System model description

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 23 / 63

24/63

Part I: Online Load Scheduling System Model and Problem Formulation

Definitions and Goal

I Time is slotted, T = 24 slots per day

I Prices pt is assumed to be stochastic with unknown distribution

I Load demand yt` is uncertain

I Goal is to find optimal scheduling policy xt` for all appliances

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 24 / 63

25/63

Part I: Online Load Scheduling System Model and Problem Formulation

Basic Idea

I Selective operational time slots

xt`(d) = 0 ∀ t /∈ T`, d ≥ 1 (1)

I Minimum (Xmin` ) and Maximum (Xmax

` ) energy consumption, i.e.,

Xmin` ≤ xt`(d) ≤ Xmax

` , ∀ t ∈ T`, d ≥ 1 (2)

• Examples: PHEVs and batteries etc., variable charging/discharging rate

I Appliances only with ON-OFF control, that is,

xt`(d) ∈ {0, P`} ∀ t ∈ T`, d ≥ 1 (3)

I Together, constraints in (1)-(3) can be collectively expressed as

xt`(d) ∈ X t` ∀ ` ∈ L, d ≥ 1 (4)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 25 / 63

25/63

Part I: Online Load Scheduling System Model and Problem Formulation

Basic Idea

I Selective operational time slots

xt`(d) = 0 ∀ t /∈ T`, d ≥ 1 (1)

I Minimum (Xmin` ) and Maximum (Xmax

` ) energy consumption, i.e.,

Xmin` ≤ xt`(d) ≤ Xmax

` , ∀ t ∈ T`, d ≥ 1 (2)

• Examples: PHEVs and batteries etc., variable charging/discharging rate

I Appliances only with ON-OFF control, that is,

xt`(d) ∈ {0, P`} ∀ t ∈ T`, d ≥ 1 (3)

I Together, constraints in (1)-(3) can be collectively expressed as

xt`(d) ∈ X t` ∀ ` ∈ L, d ≥ 1 (4)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 25 / 63

25/63

Part I: Online Load Scheduling System Model and Problem Formulation

Basic Idea

I Selective operational time slots

xt`(d) = 0 ∀ t /∈ T`, d ≥ 1 (1)

I Minimum (Xmin` ) and Maximum (Xmax

` ) energy consumption, i.e.,

Xmin` ≤ xt`(d) ≤ Xmax

` , ∀ t ∈ T`, d ≥ 1 (2)

• Examples: PHEVs and batteries etc., variable charging/discharging rate

I Appliances only with ON-OFF control, that is,

xt`(d) ∈ {0, P`} ∀ t ∈ T`, d ≥ 1 (3)

I Together, constraints in (1)-(3) can be collectively expressed as

xt`(d) ∈ X t` ∀ ` ∈ L, d ≥ 1 (4)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 25 / 63

25/63

Part I: Online Load Scheduling System Model and Problem Formulation

Basic Idea

I Selective operational time slots

xt`(d) = 0 ∀ t /∈ T`, d ≥ 1 (1)

I Minimum (Xmin` ) and Maximum (Xmax

` ) energy consumption, i.e.,

Xmin` ≤ xt`(d) ≤ Xmax

` , ∀ t ∈ T`, d ≥ 1 (2)

• Examples: PHEVs and batteries etc., variable charging/discharging rate

I Appliances only with ON-OFF control, that is,

xt`(d) ∈ {0, P`} ∀ t ∈ T`, d ≥ 1 (3)

I Together, constraints in (1)-(3) can be collectively expressed as

xt`(d) ∈ X t` ∀ ` ∈ L, d ≥ 1 (4)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 25 / 63

26/63

Part I: Online Load Scheduling System Model and Problem Formulation

Basic idea

I Constraint on meeting daily demand is only imposed on an average

E

[T∑t=1

xt`(d)− yt`(d)

]≥ 0 ∀` ∈ L (5)

here expectation is with respect to the random prices and demands

I User satisfaction at time slot t is∑`

ω`U`(xt`(d), yt`(d)) (6)

• weights ω` quantify the importance of the load `

I Note: Success of demand response algorithms depends almost entirelyon the demand side flexibility afforded by the users

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 26 / 63

27/63

Part I: Online Load Scheduling System Model and Problem Formulation

Problem Formulation

Full-day schedule design problem

P = max{xt

`(·)}E

T∑t=1

∑`∈L

ω`U`(xt`(d), yt`(d))︸ ︷︷ ︸Satisfaction

− pt(d)xt`(d)︸ ︷︷ ︸Cost

(7)

s.t. E

[T∑t=1

xt`(d)− yt`(d)

]≥ 0, ∀` ∈ L (8)

xt`(d) ∈ X t` , ∀t ∈ T (9)

Why difficult to solve?

I The statistics of the random quantities in unknown

I Scheduling policies {xt`(·)}t,` makes the problem infinite dimensional

I We need a solution with less per iteration complexity

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 27 / 63

27/63

Part I: Online Load Scheduling System Model and Problem Formulation

Problem Formulation

Full-day schedule design problem

P = max{xt

`(·)}E

T∑t=1

∑`∈L

ω`U`(xt`(d), yt`(d))︸ ︷︷ ︸Satisfaction

− pt(d)xt`(d)︸ ︷︷ ︸Cost

(7)

s.t. E

[T∑t=1

xt`(d)− yt`(d)

]≥ 0, ∀` ∈ L (8)

xt`(d) ∈ X t` , ∀t ∈ T (9)

Why difficult to solve?

I The statistics of the random quantities in unknown

I Scheduling policies {xt`(·)}t,` makes the problem infinite dimensional

I We need a solution with less per iteration complexity

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 27 / 63

27/63

Part I: Online Load Scheduling System Model and Problem Formulation

Problem Formulation

Full-day schedule design problem

P = max{xt

`(·)}E

T∑t=1

∑`∈L

ω`U`(xt`(d), yt`(d))︸ ︷︷ ︸Satisfaction

− pt(d)xt`(d)︸ ︷︷ ︸Cost

(7)

s.t. E

[T∑t=1

xt`(d)− yt`(d)

]≥ 0, ∀` ∈ L (8)

xt`(d) ∈ X t` , ∀t ∈ T (9)

Why difficult to solve?

I The statistics of the random quantities in unknown

I Scheduling policies {xt`(·)}t,` makes the problem infinite dimensional

I We need a solution with less per iteration complexity

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 27 / 63

28/63

Part I: Online Load Scheduling Stochastic Dual Descent Based Solution

Dual Problem

I The Lagrangian function is written as

L(x, λ) =E

[T∑t=1

∑`∈L

[ω`U`(x

t`(d), yt`(d))− pt(d)xt`(d)

]]

+∑`∈L

λ`

(E

[T∑t=1

xt`(d)− yt`(d)

])(10)

I The resulting dual problem can therefore be written as

D = minλ�0

T∑t=1

Dt(λ) (11)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 28 / 63

29/63

Part I: Online Load Scheduling Stochastic Dual Descent Based Solution

Classical Subgradient Algorithm

I Primal update

{x̂t`(d)}Tt=1 := arg max{xt

`∈Xt` }E

[T∑t=1

∑`∈L

ω`U`(xt`(d), yt`(d))

−(pt(d)− λ`(d))xt`(d)

]−

T∑t=1

∑`∈L

λ`(d)E[yt`(d)

](12)

I Dual update

λ`(d+ 1) =

[λ`(d) + ε

T∑t=1

x̂t`(d)− yt`(d)

]+(13)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 29 / 63

30/63

Part I: Online Load Scheduling Stochastic Dual Descent Based Solution

Proposed Stochastic algorithm

I Stochastic dual descent:

{x̂t`(d)}Tt=1 = arg maxxt`∈X

t`

ω`U`(xt`, y

t`(d))− (pt(d)− λ`(d))xt` (14)

λ`(d+ 1) =

[λ`(d)− ε

T∑t=1

(x̂t`(d)− yt`(d)

) ]+(15)

I Stochastic Incremental algorithm: Within this framework, the primaland dual updates become

x̂t`(d) = arg maxxt`∈X

t`

ω`U`(xt`, y

t`(d))− (pt(d)− λt−1` (d))xt` ;∀ ` ∈ L (16)

λt`(d) =

[λt−1` (d)− ε

[x̂t`(d)− yt`(d)

] ]+;∀ ` ∈ L

λ0`(d+ 1) = λT` (d) (17)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 30 / 63

30/63

Part I: Online Load Scheduling Stochastic Dual Descent Based Solution

Proposed Stochastic algorithm

I Stochastic dual descent:

{x̂t`(d)}Tt=1 = arg maxxt`∈X

t`

ω`U`(xt`, y

t`(d))− (pt(d)− λ`(d))xt` (14)

λ`(d+ 1) =

[λ`(d)− ε

T∑t=1

(x̂t`(d)− yt`(d)

) ]+(15)

I Stochastic Incremental algorithm: Within this framework, the primaland dual updates become

x̂t`(d) = arg maxxt`∈X

t`

ω`U`(xt`, y

t`(d))− (pt(d)− λt−1` (d))xt` ;∀ ` ∈ L (16)

λt`(d) =

[λt−1` (d)− ε

[x̂t`(d)− yt`(d)

] ]+;∀ ` ∈ L

λ0`(d+ 1) = λT` (d) (17)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 30 / 63

31/63

Part I: Online Load Scheduling Convergence Results

Convergence Results

Theorem 1

Assuming that |x̂t`(d)− yt`(d)| ≤ G for all t, `, d, and that set X is compact,the following results hold

(a). Average primal-near optimality

The schedules are near-optimal on an average, that is,

limD→∞

E

[1

D

D∑d=1

T∑t=1

wtd(x̂t`(d))

]≥ P− εT 2G2 (18)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 31 / 63

31/63

Part I: Online Load Scheduling Convergence Results

Convergence Results

Theorem 1

Assuming that |x̂t`(d)− yt`(d)| ≤ G for all t, `, d, and that set X is compact,the following results hold

(a). Average primal-near optimality

The schedules are near-optimal on an average, that is,

limD→∞

E

[1

D

D∑d=1

T∑t=1

wtd(x̂t`(d))

]≥ P− εT 2G2 (18)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 31 / 63

32/63

Part I: Online Load Scheduling Convergence Results

Convergence Results

Theorem 1

Assuming that |x̂t`(d)− yt`(d)| ≤ G for all t, `, d, and that set X is compact,the following results hold

(b). Asymptotic constraint violation

Given arbitrary α > 0, it holds for all ` ∈ L that

lim infD→∞

1

D

D∑d=1

T∑t=1

E[x̂t`(d)− yt`(d)

]≥ −α (19)

lim supD→∞

1

D

D∑d=1

T∑t=1

E[x̂t`(d)− yt`(d)

]≤ α (20)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 32 / 63

33/63

Part I: Online Load Scheduling Proof Outline

Proof Outline

The proof is done for the following general stochastic optimization problem

maxxi,pi

K∑i=1

f i(xi) (21)

s.t.K∑i=1

ui(xi) + E[vi(hi,pihi)

]� 0 (22)

xi ∈ X i, pi ∈ Pi (23)

I First we need the following sequence of distance from the optimal as

E∥∥λKT − λ?

∥∥2 ≤ E∥∥λ0

1 − λ?∥∥2 +

∑t,i

ε2E∥∥git∥∥2 − 2

∑t,i

εE[〈git,λi−1t −λ?〉

]I Developing the lower bound for

E

[K∑i=1

f i(x̄i)]≥ 1

TE

[T∑t=1

K∑i=1

f i(xit)

](24)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 33 / 63

33/63

Part I: Online Load Scheduling Proof Outline

Proof Outline

The proof is done for the following general stochastic optimization problem

maxxi,pi

K∑i=1

f i(xi) (21)

s.t.K∑i=1

ui(xi) + E[vi(hi,pihi)

]� 0 (22)

xi ∈ X i, pi ∈ Pi (23)

I First we need the following sequence of distance from the optimal as

E∥∥λKT − λ?

∥∥2 ≤ E∥∥λ0

1 − λ?∥∥2 +

∑t,i

ε2E∥∥git∥∥2 − 2

∑t,i

εE[〈git,λi−1t −λ?〉

]I Developing the lower bound for

E

[K∑i=1

f i(x̄i)]≥ 1

TE

[T∑t=1

K∑i=1

f i(xit)

](24)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 33 / 63

33/63

Part I: Online Load Scheduling Proof Outline

Proof Outline

The proof is done for the following general stochastic optimization problem

maxxi,pi

K∑i=1

f i(xi) (21)

s.t.K∑i=1

ui(xi) + E[vi(hi,pihi)

]� 0 (22)

xi ∈ X i, pi ∈ Pi (23)

I First we need the following sequence of distance from the optimal as

E∥∥λKT − λ?

∥∥2 ≤ E∥∥λ0

1 − λ?∥∥2 +

∑t,i

ε2E∥∥git∥∥2 − 2

∑t,i

εE[〈git,λi−1t −λ?〉

]I Developing the lower bound for

E

[K∑i=1

f i(x̄i)]≥ 1

TE

[T∑t=1

K∑i=1

f i(xit)

](24)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 33 / 63

34/63

Part I: Online Load Scheduling Simulation Results

Results

I Two appliances are considered, ` = 1 mimic the demand for load otherthan PHEV, ` = 2 is 10kWhr battery rating PHEV operating fromevening 6pm to 6am

I U t` (x`(t)) := −0.5 ∗ (xt` − yt`)2

I Actual price data from20 is used

20Comed. url: https://hourlypricing.comed.com/hp-api/.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 34 / 63

35/63

Part I: Online Load Scheduling Simulation Results

Results

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 240

1

2

3

4

5

6

7

8

9

Time slots (in hrs)

Dem

and(inKW

hr)

Average demand profile, ℓ = 1

Average demand profile, ℓ = 2

Figure 8: Average demand profile

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 35 / 63

36/63

Part I: Online Load Scheduling Simulation Results

Results

1 2 3 4 5 6 7 8 9−400

−300

−200

−100

0

100

200

300

400

500

σ2

Averagecost

(positive)

AverageUtility

(negative)

Proposed utiProposed billOnline uti [13]Online bill [13]Dual descent utiDual descent bill

Figure 9: Comparison of different techniques

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 36 / 63

37/63

Part I: Online Load Scheduling Simulation Results

Results

2 4 6 8 10 12 14 16 18 20 22 240

1

2

3

4

5

6

7

8

9

Time slots (in hours)

Dem

and (

in K

Whr)

Average demandDual descentIncremental

Figure 10: Scheduled vector x1

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 37 / 63

38/63

Part I: Online Load Scheduling Simulation Results

Results

101

102

1030

10

20

30

40

50η90

ω210

110

210

30

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Ave

rage

und

erch

arge

(in

%)

Average undercharge

η90

Figure 11: Performance analysis for the critical load (6 pm to 6 am)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 38 / 63

39/63

Part II : Online Energy Optimization

Outline

1 Introduction and Motivation

2 Part I: Online Load SchedulingDemand Side Management (DSM)Related Work and ContributionSystem Model and Problem FormulationStochastic Dual Descent Based SolutionConvergence ResultsProof OutlineSimulation Results

3 Part II : Online Energy OptimizationRelated Work and ContributionsSystem model and DefinitionsProblem Formulation and solutionWith forcasted PricesSimulation Results

4 Conclusion and Future work

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 39 / 63

40/63

Part II : Online Energy Optimization

Part II :Optimal Utilization of Storage Systems underReal-time Pricing21

21Amrit S Bedi et al. “Optimal utilization of storage systems under real-time pricing”. In: Communications Workshops (ICC Workshops),2017 IEEE International Conference on. IEEE. 2017, pp. 1141–1146.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 40 / 63

41/63

Part II : Online Energy Optimization

System Model

Power

Grid

Utility

Company

Smart

Meter

LOAD

Controller

Battery

AC/DC AC/DC

Solar

PV

Energy Transfer Link

Control Signals

Consumer

Power Bus

Loa

d s

ch

SoC

Iref

Pri

ce

Battery

AC/DCSmart

Meter

Figure 12: System model with dynamic pricing

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 41 / 63

42/63

Part II : Online Energy Optimization Related Work and Contributions

Related Work

I Storage technologies advancements, large scale batteries22,23

I From utility’s perspective, studied in24, minimize grid operational cost

I From customer’s perspective, studied in25,26, cost minimization

I Threshold based structure of optimal storage policy27,28,29

22“Clean Energy Storage and Onions”. In: GillsOnions PrudentEnergy WhitePaper (2012), Battery Energy Storage for Smart GridApplications. url: http://www.eurobat.org/sites/default/files/eurobat_smartgrid_publication_may_2013.pdf.

23Alexander E Emanuel. “Peak shaving using energy storage at the residential level”. PhD thesis. Worcester Polytechnic Institute.

24Iordanis Koutsopoulos, Vassiliki Hatzi, and Leandros Tassiulas. “Optimal energy storage control policies for the smart power grid”. In:IEEE SmartGridComm. 2011, pp. 475–480.

25Peter M van de Ven et al. “Optimal control of end-user energy storage”. In: IEEE Trans. Smart Grid 4.2 (2013), pp. 789–797.

26Alexander E Emanuel. “Peak shaving using energy storage at the residential level”. PhD thesis. Worcester Polytechnic Institute.

27Peter M van de Ven et al. “Optimal control of end-user energy storage”. In: IEEE Trans. Smart Grid 4.2 (2013), pp. 789–797.

28Iordanis Koutsopoulos, Vassiliki Hatzi, and Leandros Tassiulas. “Optimal energy storage control policies for the smart power grid”. In:IEEE SmartGridComm. 2011, pp. 475–480.

29Rahul Urgaonkar et al. “Optimal power cost management using stored energy in data centers”. In: Proc. ACM Int’l Conf. Measurementand Modeling of Computer Systems (SIGMETRICS ’11). 2011, pp. 221–232.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 42 / 63

43/63

Part II : Online Energy Optimization Related Work and Contributions

Related Work

I Utilize MDP approach, determining optimal threshold is not trivial

I Requires a priori knowledge of the price, demand statistics,discrete-valued price

I Lyapunov optimization based approach is adopted in30

I Computational complexity issue is not there but algorithms still need tosolve a linear programming problem at each time slot

30Longbo Huang, Jean Walrand, and Kannan Ramchandran. “Optimal demand response with energy storage management”. In: Smart GridCommunications (SmartGridComm), 2012 IEEE Third International Conference on. IEEE. 2012, pp. 61–66.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 43 / 63

44/63

Part II : Online Energy Optimization Related Work and Contributions

Contributions

I Different from the standard MDP-based approaches

I A novel online incremental algorithm for solving the batteryoptimization problem

I Proposed scheme is threshold-based, and works without any a prioriknowledge about the price or demand statistics

I A practical version of the algorithm is introduced that directly utilizesthe battery state-of-charge (SoC) for making the charge/dischargedecisions

I Simple to implement, works with continuous-valued prices, demands,and SoC

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 44 / 63

45/63

Part II : Online Energy Optimization System model and Definitions

Definitions

I Time is divided into slots and is indexed by t, τ ∈ N

I At t, each customer has the knowledge ofI price pt ∈ P ⊂ R+

I battery SoC bt−1

I load Lt ≤ Pmax

I Based on this, decision of charge discharge by an amount xt is to betaken

I From battery specification, it is required to hold that

xt ∈ [−βout, βin] (25)

bt ∈ [βmin, βmax] (26)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 45 / 63

46/63

Part II : Online Energy Optimization Problem Formulation and solution

Optimization Problem

The battery optimization problem over a horizon of T time slots is

minxt∈X

1

T

T∑t=1

pt(Lt + xt) +x2t2α

(27)

s. t.1

T

T∑t=1

xt = 0 (28)

I Set X := [−βout, βin] provides safe limits for optimization variable{xt}Tt=1

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 46 / 63

47/63

Part II : Online Energy Optimization Problem Formulation and solution

Solution via Incremental Dual Ascent

I The Lagrangian is given by

L({xt}, λ) =1

T

T∑t=1

[(pt − λ)xt +

x2t2α

+ptLt

](29)

I The incremental dual updates are given by:

xt =[α(λt−1 − pt)

]βin

−βout

(30)

λt = λt−1 − δxt (31)

I δ > 0 is the step-size parameterI and [x]ba := min(max(x, a), b)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 47 / 63

48/63

Part II : Online Energy Optimization Problem Formulation and solution

Theoretical Results

I Assumptions:

(A1) The stochastic process {pt} is bounded, i.e., pt ≤ pmax <∞ for someconstant pmax.

(A2) The time-average of {pt} exists, i.e., p̄ = limT→∞1T

∑Tt=1 pt almost

surely (a.s.), where p̄ <∞ is a constant.

I Result:

Theorem

Under (A1)-(A2) and for αδ < 1, the charge/discharge process {xt}Tt=1

obtained from (15) is a.s. asymptotically feasible.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 48 / 63

48/63

Part II : Online Energy Optimization Problem Formulation and solution

Theoretical Results

I Assumptions:

(A1) The stochastic process {pt} is bounded, i.e., pt ≤ pmax <∞ for someconstant pmax.

(A2) The time-average of {pt} exists, i.e., p̄ = limT→∞1T

∑Tt=1 pt almost

surely (a.s.), where p̄ <∞ is a constant.

I Result:

Theorem

Under (A1)-(A2) and for αδ < 1, the charge/discharge process {xt}Tt=1

obtained from (15) is a.s. asymptotically feasible.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 48 / 63

49/63

Part II : Online Energy Optimization Problem Formulation and solution

Proposed Controller Settings

I Battery dynamics evolves as

bt =

[bt−1 + xt

]βmax

βmin

(32)

I For γ = pmaxβmax

βmax−βmin, ε = pmax/(βmax − βmin), α = min(βin, βout)/pmax

we get −βout ≤ α(λt−1 − pt) ≤ βin, making limits in (30) redundant

I Finally, (32) can be written as

bt−1 + xt = (1− αδ)bt−1 + αδγ − ptε

(33)

I In summary

xt = α(γ − εbt−1 − pt) (34)

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 49 / 63

50/63

Part II : Online Energy Optimization Problem Formulation and solution

Algorithm:

(S1) Given parameters βmax, βmin, initial SoC b0, and pmax, selectγ = pmaxβmax

βmax−βmin, ε = pmax

βmax−βmin, and α = min(βin, βout)/pmax

Repeat for t ≥ 1:

(S2) Input: price pt, battery SoC bt−1;

(S3) Calculate xt using xt = α(γ − εbt−1 − pt);

(S4) Charge/discharge battery by xt units of energy;

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 50 / 63

51/63

Part II : Online Energy Optimization With forcasted Prices

With forcasted Prices

I Consider the following problem

minxt∈X

1

T

T∑t=1

pt(Lt + x̃t) +x̃2t2α

(35)

s. t.1

T

T∑t=1

xt = 0 (36)

optimization variable is x̃t := xt − x̂tI Using incremental dual descent, we get

xt =[α(λt−1 − pt) + x̂t

]βin

−βout

(37)

λt = λt−1 − δxt (38)

Since x̂t is known ahead of time, the limits in (37) can again beeliminated by choosing αt = min((βin, βout)− x̂t)/pmax

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 51 / 63

52/63

Part II : Online Energy Optimization Simulation Results

Simulation parameters

I Customer has a 5 kW peak power requirement, 3 kW of energy storagecapacity, Solar PV of peak power rating 2.5 MW

I The average load profile is obtained from31

I Two price data sets considered (Fig. 2)I Averaged wholesale prices, for the duration 03/06/2015 to 29/06/201632

I Hourly residential prices33

I Real time prices by adding zero mean Gaussian noise with standarddeviation of 5

I Other associated parameters for the algorithm are pmax = 15,βmax = 3MWhr, βin = βout = 0.5βmax, βmin = 20%, and α = 0.1416

31Load profiles and their use in electricity settlement.http://www.uea.ac.uk/~e680/energy/energy_links/electricity/load_profiles.pdf.

32“Alberta Electric System Operator”. In: (). url: http://ets.aeso.ca/.

33Comed. url: https://hourlypricing.comed.com/hp-api/.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 52 / 63

53/63

Part II : Online Energy Optimization Simulation Results

Performance results

5 10 15 200

1

2

3E

nerg

y (i

n M

Whr

)

Solar energyAverage load

5 10 15 200

5

10

Days

Pric

e (i

n $/

MW

hr)

Average price (data set 1)Average price (data set 2)

× 103

Figure 13: Average profile for price, load, and received solar energy

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 53 / 63

54/63

Part II : Online Energy Optimization Simulation Results

Performance results

Table 2: Performance Comparison

Avg.prices

MDP Prop.

% cost benefitper-day

5.36% 4.72% 12.03%

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 54 / 63

55/63

Conclusion and Future work

Outline

1 Introduction and Motivation

2 Part I: Online Load SchedulingDemand Side Management (DSM)Related Work and ContributionSystem Model and Problem FormulationStochastic Dual Descent Based SolutionConvergence ResultsProof OutlineSimulation Results

3 Part II : Online Energy OptimizationRelated Work and ContributionsSystem model and DefinitionsProblem Formulation and solutionWith forcasted PricesSimulation Results

4 Conclusion and Future work

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 55 / 63

56/63

Conclusion and Future work

Conclusion and Future work

I Online algorithm for load scheduling under price and load uncertainty

I Provided convergence guarantees

I Proposed an online algorithm for energy storage optimization

I Provided convergence guarantees

Future work:

I The proposed energy optimization algorithm can be extended fordistributed settings when each consumer is equipped with energygeneration and storage facilities.

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 56 / 63

57/63

Conclusion and Future work

Acknowledgments

I Dr. Sandeep Anand

I Mr. Waseem Ahmed

I Mr. Aditya P Prasad

I Mr. Swapnil Shinde

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 57 / 63

58/63

Thank You

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 58 / 63

59/63

Comparison

5 10 15 20 25 302

4

6

8

10

12

14

16

Days

Perc

enta

ge b

enef

it (i

n co

st/d

ay)

ProportionalAverage priceMDP

Figure 14: Percentage reduction in the electricity bill with storage optimization, without solarinstallation

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 59 / 63

60/63

Solar

5 10 15 20 25 301

1.2

1.4

1.6

1.8

2

2.2

Days

Cos

t/day

(in

$)

Conventional policy (without solar)Optimal storage policy (without solar)Conventional policy (with solar)Optimal storage policy (with solar)

× 103

Figure 15: Absolute costs incurred with and without storage optimization, both with and andwithout solar installation

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 60 / 63

61/63

Effect of battery size

3 4 5 6 7 8 9 10

5

10

15

20

Battery size (in MWhr)

Perc

enta

ge b

enef

it

(i

n co

st/d

ay)

With solar

With solar (20% backup)

Without solar

Figure 16: Percentage benefit in cost/day with different battery sizes

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 61 / 63

62/63

With forecasted prices

5 10 15 20 25 30−2

0

2

4

6

8

Days

Per

cent

age

bene

fit (

in c

ost/d

ay)

ProportionalProportional without forecastForecasted onlyMDP

Figure 17: Percentage reduction in the electricity bill with storage optimization, without solarinstallation

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 62 / 63

63/63

RTDS Simulation

S1

S2

S3

S4

LoiL

C

idc

S1 S2

-+

S4S3

-1

sin PLL

+

-

+

-

PIv

PIi

iL(ref)

Controller

+-vdc

vdc(ref)

Ibattery

Cin

L

SBattery

+

S

+ -

- PIi

Ibattery(ref)

Proposed Algorithm

Battery

SO

CV

gri

d

+I i

nduct

or

DC-AC ConverterDC-DC boost Converter DC-Link

GridFilter

Controller

_

Price Input

Figure 18: Inverter and dc-dc converter integrating battery with grid

M.Tech Defense Stochastic Optimization Algorithms for Smart Grid Sep. 5, 2017 63 / 63