1 dymnd: robust adaptive coordination in dynamic meshes of networked devices nest pi meeting june...

1

DyMND:Robust Adaptive Coordination in

Dynamic Meshes of Networked Devices

NEST PI MeetingJune 29, 2004

Prasanta BoseAdvanced Technology CenterLockheed Martin Space Systems CompanyPalo Alto, CA

2DyMND: P. Bose, ATC, LMSSC

Outline

Robust DyMND Engineering– DyMNDSim and DyMNDES: The first generation– Lessons learned– DyMND-EE: The next generation

• Architecture• Implementation• Experiments

Adaptive Coordination in DyMND– Explicit local coordination for optimal control– Dynamic programming (reinforcement learning)– Design and validation strategy within DyMND-EE

Summary and Future Plan


DyMNDSim and DyMNDES – The First Generation

DyMNDSim– Discrete Event Simulation– Full-featured tool for implementing

sensing/radio models– Integration with Motes and Tossim

DyMNDES– Real-time injection and data retrieval– Scenario driven interaction – Script, save, and load emulation

event sequences – Generation of abstract mote and

target capabilities via virtual objects – 3D visualization of aggregate state

DyMNDSim server

Simulation Director

Client Proxy

Motes

Active message handler

Radio SensorsEffectors

Targetmodels

SvcCoord TaskQ


Lessons Learned

Translation of services from DyMNDSim to motes required manual recoding– DyMNDSim emulated TinyOS active messages and

event/task distinction, but executed Java classes

Hardware nodes led to noisy, non-repeatable experiments

Debugging and monitoring required source-code modifications

Principled integration of simulation and emulation components required for– Accurate validation of NEST systems


DyMND-EE – The Next Generation

Goal: End-to-end validation of sensor net software – Efficient execution of embedded source code– Realistic models for radio, sensing, actuation– Mixed models of execution for staged development

Goal: Accurate validation environment– Model of execution (concurrency) faithful to TinyOS

Goal: Scale to NEST challenges (e.g. EXSCAL)– Exploit tiered structuring via aggregate simulation and

interaction across aggregates


Evolving DyMNDSim To Realize DyMND-EE Goals

Based on principled composition of existing simulators Target-level simulators

– Emsim (Emstar):

cross-compiled nesC in

Linux user space High-level DE/

continuous-time simulators– Ptolemy II/VisualSense:

Multiple models of computation,

actor-oriented programming– Simulink: Mostly continuous time,

rich control/DSP libraries

Tossim

Emsim / Emstar

Ptolemy II / VisualSense

Simulink

DyMNDSim

Pro

cess

or

fid

elit

yG

ener

ic m

od

elin

g

DyMNDEE


Ptolemy II Features in DyMND-EE

Actor-oriented development environment– Actors implement operations on data – Directors specify model of computation (timing and order of

communication between actors)

Multiple models of computation in a designed system– Example: discrete-event sensor model to track evader with

continuous control law

Extensions to WirelessDirector (implicit connectivity)– ComposableChannel allows modification of each token– Channel modifiers for propagation delay, deterministic /

probabilistic message loss, bit errors

Extension to create distributed director


EmStar Features in DyMND-EE

Emstar is a flexible software framework for heterogenous NesC Code simulation using the FUSD kernel module. (http://cvs.cens.ucla.edu/emstar)

Features– Executes TinyOS/NesC code via EmTOS– HIL and simulated execution– Heterogeneous node simulations– Can use real or simulated RF channel– FUSD user space devices– Non-TinyOS dependent as any device can be

registered under FUSD – Provides a clean interface to abstract hardware.


DyMND-EE Concept of Operations

Graphical representation of a configuration of the DyMND-EE simulation environment

EmStar runs NesC code

Data Logging to Files for Analysis

Ptolemy Executes aggregate sensingand comm. models

DyMNDES generates simulation configuration and generates PtolemyModels.


DyMND-EE Architecture

Decomposes simulation into sensing/actuation “universes”

Implemented as composite actors with directors– Radio and sensors as actors– Services as Emstar processes

with proxy actors Software services execute in

Emstar (native code) or Ptolemy (prototype code)

Communicates with external processes via XML/object streams

Configurations generated from DES script using XSLT and predefined Ptolemy II actors

Ptolemy II

Emstar (Linux user space)

Emlink

MagnetometerUniverse

AcousticUniverse

Processors Radio Universe

Node 1Microphone

Node nMicrophone

Node 1EmstarProxy

Node 2EmstarProxy

Node nEmstarProxy

Node 1Radio

Node 2Radio

Node nRadio

EmstarGateway

Radio ChannelAbstraction


Sensing ChannelAbstractions

(EmSensorServer)

Radio Channel

AbstractionNode 1

Processes

Radio Channel

AbstractionNode 2

Processes

Radio Channel

AbstractionNode n

Processes

FUSD


DyMND-EE Component Collaboration

1. Target emits chirp2. WirelessDirector transmits chirp

to node k mic3. Mic processes chirp4. Ptolemy forwards ActiveMessage

(AM) to node k EmstarProxy5. Proxy routes AM to Emstar

gateway6. Gateway routes AM to

EmSensorServer7. EmSensorServer forwards AM to

node k processes via FUSD8. Node k process generates

detection message9. Node k process forwards radio

AM to radio abstraction via FUSD10. Radio abstraction forwards AM to

Emstar gateway11. Node k EmstarProxy forwards

AM to node k radio12. Node k radio broadcasts alert

Ptolemy II

Emstar (Linux user space)

Emlink

Acoustic Universe Processors Radio Universe

Node 1 Mic

Noden Mic

Node 1EmstarProxy

Node kEmstarProxy

Node nEmstarProxy

Node 1Radio

Node kRadio

Node nRadio

EmstarGateway



Sensing ChannelAbstractions

(EmSensorServer)

Radio Channel

AbstractionNode 1

Processes

Radio Channel

AbstractionNode k

Processes

Radio Channel

AbstractionNode n

Processes

Node k Mic

Target Chirp 1

2

3 4

5

6

7

8

9

10

11

12


Outline

Robust DyMND Engineering– DyMND-Sim and DyMNDES: The first generation– Lessons learnedDyMND-EE: The next generation

• Architecture• Implementation• Staged Development Configuration• Experiments




Implementing DyMND-EE: Extensions To Emstar

Primary Modifications– Interfaces into TinyOS scheduler for fine grained control of timing

and interrupt execution

– New sim-component EmLink implements radio and sensing link layer driver registration for interface into Ptolemy

– Sensing components in Emstar component library are rewritten to use new ADC.nc components that register link users. No changes are required to get NesC code working as expected in DyMND-EE.

– Additional components, including hypothetical components, are emulated to examine effects of sensor networks with actuation capabilities via Ptolemy environment

– Extensions for non-TinyOS-based node synchronization with external simulations


Implementing DyMND-EE: Emstar-Ptolemy Link

EmLink– Encapsulates TinyOS ActiveMessages as Ptolemy tokens

– EmSensorServer implements Linux user-space proxy for Ptolemy sensor/radio models

– P-Star protocol defines send/receive, device read/write, synchronization, sense/actuate, and configuration messages

– Communication between Ptolemy-space and user space transparent to services and actors

P-Star Protocol– Protocol between Ptolemy and EmStar– Header format includes simulation time, device ids, group ids (for spatial or

logical partitioning of EmStar simulations), and various simulation options. EmStar Java Gateway

– Consumes P-Star messages and dispatches them to EmstarProxy actors– Actors can represent entire nodes, components of a node, node tasks and

services.


DyMND-EE Configurations

Configuration Options Modeling the physical environment

– Radio Channel models– Sensor channel models

Modeling component interactions– Service coordination– Timing & interrupts– Sensing & actuation


Staged Development in DyMND-EE

Matlab – processing

Ptolemy – distributed radio and sensing model

Matlab

Minimal


Staged Development in DyMND-EE (Contd.)

Introducing NesC code into the simulations



Replacing Ptolemy models with actual environment

Testing performance in real world



Transfer code to actual hardware and test like you fly


Outline






DyMND-EE Experiment 1: Target Detection

Simple translation of wireless sound detection Ptolemy demo (UCB) to nesC

Simulation setup models radio and sensing channels

NesC code performs triangulation


Target Detection Scenarios

ScenarioRobustness

ConcernsControl Metrics Expt. Setup

Moving Target(s)

- latency

- Sampling rate less than target speed

- sleep/wake cycle- Sample rate- Time windows

- false positives or negatives

- deviation of track

- insert a moving target

- set params

Distribution of Aggregation nodes and detection nodes

- Flooding- Holes

- Distribution of Aggregators- Distribution of Detectors- Detection weights-Time windows

-false positives or negatives-collisions

- vary # of aggregator and detection nodes

- set params for weights and time windows

Environment Effects

- Increasing sensor drift

- Calibration frequency

- threshold

- false positives or negatives

- add sensor error as a function of env. params.


Experiment Configurations

Experiment parameters are easily changed in Ptolemy. – Experiment parameters

can be set either by the DyMNDES or by the Vergil GUI in Ptolemy

– Experiment runs can be saved in Ptolemy or as a script by the DyMNDES.


Source Localization Results

Limited radio range More aggregators; modified density of detectors

Increased target speed Increased target speed; limited radio ranges


Issues

Insight from these simple experiments demonstrate that its easy to find the optimal simulation configuration to give data that “fits” your algorithm.

Results from the perfect simulation configuration


DyMND-EE Experiment 2: Localization

Kestrel localization code

No changes necessary in NesC code

Simulation setup models radio and sensing channels.

NesC code performs localization


Localization Scenarios

Scenario Robustness Concerns

Control Metrics Experiment Setup

Irregular

Distribution

- detection bias- congestion

- power mgmt. schedule - clustering

- Probability of detection given node density & distribution

- throughput

- vary layout of nodes- set power on/off schedule

Critical Node Death

-Failed processes (e.g. sentry) - Failed Links (e.g. node is gateway)

- routing- role reconfiguration

- data loss - retransmission- recovery time- delay

-kill key nodes along critical paths

Temporary Node Failure and Revitalization

-Self-stabilization- Failed processes - Failed links

- retransmission rate- receive/transmit gain

- data loss

- retransmission

- recovery time

- delay

- temporarily disable nodes

- set radio

- set transmission params.


Comparing Fidelity of Simulation

DyMND-EE results match Kestrel’s results (TOSSIM w/custom sensor models)

Confirmed that localization produced standard deviation ~0.7-1.0% of total distance for accurate ranging


Advantages of the DyMND-EE Approach

Software reuse and rapid prototyping– Reuse/test embedded code – Implement software prototypes as Ptolemy actors or Linux

user-space processes (managed memory, out-of-band communications)

– Implement new sensing concepts as Ptolemy actors Extensibility

– Simulation container not restricted to single hardware model Scalability

– Functional decomposition allows tradeoffs between spatial locality (neighboring nodes) and functional locality (sensors vs. radios vs. processors) for distributed implementation

Synchronization– Ptolemy messaging bus synchronizes events and

communication between services


Limitations

Limitations– Task-level simulator executes in a single thread per node.

Timing and interrupts may not be handled as expected if using interrupts directly

– NesC component libraries only partially ported – UART, ADC, and actuation not yet completed.

– Radio and sensor models still under development and require compilation for new models

– Each node runs C or NesC code


Outline






Optimal control: The DyMND Approach

Management of sensor nets can be framed as an optimal control problem– Optimal policy maximizes value function:

Expected (discounted) sum of future rewards– Engineer specifies rewards/penalties for good/bad events– Actions rewarded/penalized per node (no global oracle)

Objective function depends on …– Quality of sensor data (higher is better)– Power consumption (lower is better)– Detection performance (precision & recall)

… over the life of the network Large sensor nets require global optimization over

(large numbers of) local state variables– Low communications overhead (packet size, power management)– Low memory overhead (!)


Optimal control: An example

Intruder detection with wireless sensors State variables (per mote)

– Internal state: e.g., mote battery power, current detection assignment

– Observed state:e.g., amplitude and frequency of acoustic sample

Actions (per mote)– Changing sensor sampling rates– Sending a detection message– Switching between sensing “policies” (e.g. “vigilant” / “slack”)

Rewards/penalties (per mote)– Penalty proportional to power consumption:

Encourages long network life– Reward proportional to “quality” of sensor coverage

Variance of observations over mote and neighbors


Junction Tree RL Coordination Service

Initializing a junction tree Identify neighbors and shared state Build routing tree and propagate shared state

Making decisions (policy improvement) Pass messages to ascertain neighbors’

policies Determine the optimal policy conditioned on

– Neighbors’ policies– Current local value function estimate

Pass state observations in toward root, then out toward leaves

Updating value estimates (policy evaluation) New estimate is convex combination of:

– Old estimate– New observation of (rewards + successor-state

expected value)

Sampling rateLo Hi


Loopy value propagation

Junction-tree control imposes routing tree on communications mesh

– Worst-case path length radius of network

– Costly to maintain running intersection property (storage exponential in path length) for large networks

– Expensive recovery from link/node loss to maintain tree Analogy to probabilistic inference

– Junction-tree RL relies on optimal-control form of Generalized Distributive Law (GDL)

– Statistical-mechanics approximation to GDL assumes interaction graphs have cycles (atoms in a crystal lattice)

Junction graph relaxes global tree structure– Tree-structured coordination graph per variable

– Natural coordination structure for sensor nets (immediate neighborhoods)

Introduces circular dependencies to decision-making process

– Decisions are globally suboptimal, but locally optimal

(and approach optimality after many iterations)

Standard JTRL:- 5 hops

2 extra nodes- 3 hops

1 extra node

Loopy JTRL:- 2 hops- 2 hops

0 extra nodes!

Junction-tree RL makes MDPs feasible for small sensor nets;Loopy junction-tree RL makes approximate MDPs feasible for large sensor nets!


RL in TinyOS: Design Constraints

RL implementation requires three components– Exploration strategy to balance “exploration” (visiting new states)

and “exploitation” (returning to old states)• Exploration can occur in simulation/lab• Deployed system will favor exploitation

– Value function approximator to identify best actions for states• Initial values biased to incorporate domain knowledge• Exact (tabular) value functions are storage-intensive• Nonlinear approximators (neural nets) are compute-intensive• Linear combinations of features are efficient

– Update rule to apply changes to value function/policy

Minimal computational overhead RL service requires access to service state Messages between nodes should be small and infrequent

– Loopy value propagation avoids long message paths

– Coarse-grained policy learning reduces complexity of interaction


RL in TinyOS: Implementation Options

1. Runtime configuration: RL-aware services register callback with RL service Registered services provide accessors for relevant

state/action or Registered services fire events for state changes, and RL

service fires events for policy changes Similar to ServiceC/ServiceM approach Requires retrofit of existing services

2. Compile-time configuration: RL service code generated with static references to ordinary nesC services Exploits static allocation: RL service directly reads state

variables, writes configuration parameters Reuses services without modification Violates isolation of nesC services “Shared memory” hack risks race conditions


Summary

Developed DyMND-EE for – End-to-end validation of NEST systems– Staged development of NEST systems– Robust analysis via controlled experiments within an

accurate validation environment

Experimented with DyMND-EE– Replicated experiments to validate claims (localization)

Design of adaptive mechanisms for optimal performance in the context of change– Developed algorithms– API design for implementing within DyMND-EE


Future Plan

Extensions to DyMND-EE– Include actuation processes– Interruptable tasks– Higher fidelity sensor, radio models

Experimental approach to design/evaluation of robust sensing/actuator networks – Modeling (sensor, radio, actuation) in DyMND-EE – Experiment driven design of services (parameters)

for specific application contexts and dynamics

Adaptive coordination implementation

1 dymnd: robust adaptive coordination in dynamic meshes of networked devices nest pi meeting june...

Documents

aggregates dymnd

dymndee goalsbased

dymndee summary

continuous time

validation strategy

aggregate simulation

model of computation

discreteevent sensor