Neeraj Jaggi

A SS I STA N T P R O F E SSO R

D E P T O F E L E C T R I C A L E N G I N E E R I N G A N D C O M P U T E R SC I E N C E

WICHITA STATE UNIVERSITY

1

Rechargeable Sensor Activation under Temporally Correlated

Events

Outline

Sensor Networks

Rechargeable Sensor System Design of energy-efficient algorithms Activation question – Single sensor scenario

Temporally correlated event occurrence Perfect state information

Structure of optimal policy Imperfect state information

Practical algorithm with performance guarantees

2

Neeraj Jaggi Dept of EECS Wichita State University

3

Sensor Nodes Tiny, low cost Devices Prone to Failures Redundant Deployment Rechargeable Sensor Nodes

Range of Applications

Important Issues Energy Management Quality of Coverage

Sensor Networks

Neeraj Jaggi Dept of EECS Wichita State University

4

Rechargeable Sensor System

Quality of Coverage

Discharge

Recharge

Event Phenomena

Renewable Energy

Randomness

Spatio-temporal Correlations

Activation Policy

Control

Rechargeable Sensors

Neeraj Jaggi Dept of EECS Wichita State University

Research Question5

How should a sensor be activated (“switched on”) dynamically so that the quality of coverage is maximized over time ?

A sensor became ready. What should it do ? Activate itself now :

Gain some utility in the short-term

Activate itself later : No utility in the short term Activate when the system “needs it more”

Neeraj Jaggi Dept of EECS Wichita State University

Temporal Correlations6

Event Process (e.g. Forest fire) On period (HOT) Off period (COLD) Correlation probabilities

0.5 < ( , ) < 1

( = = 0.8)

Performance Criteria – Single Sensor Node Fraction of Events Detected over time

oncp

offcp

oncp

offcp

Neeraj Jaggi Dept of EECS Wichita State University

Sensor Energy Consumption Model7

Discrete Time Energy Model Operational Cost (1) Detection Cost (2) Recharge Rate (qc)

Probability (q) Amount (c)

recharge

sensor not activated(no discharge)

qc

activation policy

K

δ1

discharge - Off period

discharge - On period

δ1+δ2sensor activated

Neeraj Jaggi Dept of EECS Wichita State University

System Observability8

Perfect State Information Sensor can always observe state of event process

(even while inactive)

Imperfect State Information Inactive sensor can not observe event process

Neeraj Jaggi Dept of EECS Wichita State University

Approach/Methodology9

Perfect State Information Formulate Markov Decision Problem (MDP) Structure of Optimal Policy

Imperfect State Information Formulate Partially Observable MDP (POMDP) Transform POMDP to equivalent MDP (Known techniques) Structure of Optimal Policy Near-optimal practical Algorithms

Neeraj Jaggi Dept of EECS Wichita State University

Perfect State Information10

Markov Decision Process State Space = {(L, E); 0 ≤ L ≤ K, E є [0, 1]}

L – Current Energy Level, E – On/Off period Reward r– one if event detected; zero otherwise Action u є [0, 1]; Transition probabilities p

Optimality equation (average reward criteria)

h* – state variables λ* – optimal reward

Neeraj Jaggi Dept of EECS Wichita State University

Perfect State Information (contd.)11

Approximate Solution

Closed form solution for h* does not seem to exist

Value Iteration

Activation Algorithm L << K

Sensitive to system parameters when L ~ K

Optimality equation (average reward criteria)

H* – variables Lambda* – optimal reward

Neeraj Jaggi Dept of EECS Wichita State University

Perfect State Information (contd.)12

Optimal Policy Structure Randomized algorithm P* is directly proportional to the recharge rate Energy balance

Average recharge rate equals average discharge rate in steady state

On Period ? ActivateYes

No

Sufficient Energy ?

No

Do Not Activate

Yes

Prob. ≤ P* ?YesNo

Neeraj Jaggi Dept of EECS Wichita State University

Imperfect State Information13

Partially Observable Markov Decision Process State Space Observation Space Optimal actions depend on current and past observations

(y) and on past actions (u)

Transformation to equivalent MDP 1

State – Information vector Zt of length |X|

Zt+1 is recursively computable given Zt, ut and yt+1

Zt forms a completely observable MDP

Equivalent rewards and actions 1 Neeraj Jaggi Dept of EECS Wichita State University

Equivalent MDP Structure14

Active Sensor – Observation = (L, 1) or (L, 0) State is the same as observation Zt has only one non-zero component

Inactive Sensor – Observation = (L, Φ) Let state last observed = E, number of time slots inactive =

i Zt has only two non-zero components Let pi= prob. that event process changed state from E to 1-

E in i time slots State = (L, E) with prob. 1 - pi

State = (L, 1 – E) with prob. pi

Zt is a function of (L, E, i)Neeraj Jaggi Dept of EECS Wichita State University

Transformed MDP State Space – (L, E, t) L – Current Energy Level E – State of Event process last observed t – Number of time slots spent in inactive state

Optimal Policy Structure f0 – (L, 0, t), f1

– (L, 1, t)

[1=c=1, 2= 2, = 0.6, = 0.9, q = 0.1]

Imperfect State Information (contd.)

15

Off Period – Reluctant Wakeup

On Period – Aggressive Wakeup

oncp

offcp

Neeraj Jaggi Dept of EECS Wichita State University

Practical Algorithm16

Correlation dependent Wakeup (CW) Activate during On Periods; Deactivate during Off

Sleep Interval (SI*) Derived using energy balance during a renewal

interval

-optimal ( ~ O(1/β)); β = 2/1

A – ActiveI – Inactive

Y – On, N – OffSI – sleep duration

t1, t2 – renewal instances

Y Y Y Y Y N

A A A A A A I

Y Y Y N

A A A A I

SI

I

tt2t1

Neeraj Jaggi Dept of EECS Wichita State University

Simulation Results17

Energy balancing Sleep Interval SI*

[ = 0.6, = 0.9, SI* = 7] [ = 0.7, = 0.8, SI* = 18]

[1 = c = 1, 1 = 6, q = 0.5, K = 2400]

oncp

offcp

oncp

offcp

Neeraj Jaggi Dept of EECS Wichita State University

Contributions18

Structure of Optimal Policy

EB Policy is Optimal for Perfect State Information

EB Policy is near Optimal for Imperfect State Information

Coauthors Prof. Koushik Kar , Rensselaer Polytechnic Institute Prof. Ananth Krishnamurthy, Univ. of Wisconsin

Madison 5th International Symposium on Modeling and

Optimization in Mobile Ad hoc and Wireless Networks (WIOPT) April 2007

ACM/KLUWER Wireless Networks 2008 (Accepted )

Neeraj Jaggi Dept of EECS Wichita State University

Q & A19

THANK YOU !!

Neeraj Jaggi Dept of EECS Wichita State University

Policies – AW, CW20

AW (Aggressive Wakeup) Policy Activate whenever L ≥ 2 + 1

Ignores temporal correlations Optimal if no temporal correlations

CW (Correlation dependent Wakeup) Policies Activate during On periods; deactivate during Off Upper Bound (U*

CW) State unobservable during inactive state Performance depends upon sleep duration

Neeraj Jaggi Dept of EECS Wichita State University

How long should sensor sleep ?

MDP – State Transitions21

State (L, 1): L ≥ 2 + 1Action u = 1 (activate)

Next state : (L + qc – δ1 – δ2, 1) with probability q.pc

on

(L + qc – δ1, 0) with probability q.(1 – pcon)

(L – δ1 – δ2, 1) with probability (1 – q ).pcon

(L – δ1, 0) with probability (1 – q ).(1 – pcon)

Reward r = 1 with probability pcon; 0 otherwise.

Action u = 0 (deactivate) Next state :

(L + qc, 1) with probability q.pcon

(L + qc, 0) with probability q.(1 – pcon)

(L, 1) with probability (1 – q).pcon

(L, 0) with probability (1 – q).(1 – pcon)

Reward r = 0

Neeraj Jaggi Dept of EECS Wichita State University