1

Adaptive Control of Network Centric Dynamic Systems

Jagannathan (Jag) SarangapaniProfessor of Department of Electrical and Computer Engineering

& Computer Science

University of Missouri-Rolla

Rolla, Missouri 65409.

Tel: 573-341-6775; Email:sarangap@umr.edu

&

Site Director, NSF I/UCRC on Intelligent Maintenance Systems

2

Outline

• Overview of Research at my Lab/Center

• Energy Aware Protocols

– Adaptive Congestion Control– Routing Protocol

• Conclusions

3

OBJECTIVE

COST AND SCHEDULE APPROACH

• Cooperative decision making and control of multiple unattended formation of robots (mobile adhoc networks) with continuous RF communication (UMR mote)

Total Cost$250K

MILESTONES Qtr 1 Qtr 2 Qtr 3 Qtr 4

75 100 50 25

Milestone 1

($K)

Milestone 2

Milestone 3

Milestone 4

Milestone 5

BACKGROUND• Network communication among a formation of robots is necessary in order to perform any task. This communication must be present at all times. This communication must be reliable even when the robots are in motion and when energy is limited. This requires a novel distributed hybrid system architecture, reliable communication hardware and energy efficient protocols for communication that guarantees quality of service requirements.

• Milestone 1: Develop distributed embedded hybrid architecture

• Milestone 2: Develop wide band communication-based UMR Mote

• Milestone 3: Integrate sensors and RF hardware on the robot

• Milestone 4: Implement the energy efficient communication protocols using energy delay metric

• Milestone 5: Implement control strategy and demonstrate

DELIVERABLES

• UMR RF Mote

• Energy Efficient Protocols

• Demonstration and Reports

POINT OF CONTACT Dr. J. Sarangapani, Electrical and Computer Eng Dept., UMREmail: sarangap@umr.edu; Phone: 573-341-6775;URL: www.umr.edu/~sarangap

Sensor nodesCluster

Heads

Mission HeadQuarters

SENSOR “ARRAY”

WirelessPoint-to-point

Network of Autonomous RobotsRequirement: Continuous Communication among 10 robots/unattended sensors

3

UMR Mote

4

Manipulation of Microscale/Nano Scale Objects using Micro/Nano Robots

• Manipulation of a micro-sphere– Pickup a micro-sphere from a

planar substrate• Sequence of operations

– Lower a probe– Pickup the micro-sphere– Retract the probe

• Novel intelligent controllers designed outperform available ones

• AFM is used as a feedback for manipulation and drift compensation of nano particles

Fig. 1. Image sequences taken at 256 sec intervals without drift compensation (first row) and with drift compensation (second row). The scanned area is 512×512nm2.

5

Prognostics on a Chip

Database

RemainingUseful Life

Confidence

Severity

Distributed Sensors

Wir

eles

s MultivariableAnalysis

with Learning

TrendingConfidence

Degradation

U M R M o t e Service History

Wireless

U M R M o t e ReliabilityInformation

Wireless

6

Prognostic Agent Methodology

7

Monitoring Using Wireless Sensor Networks

Cluster Head

Sensor Node

Base Station

Self Organizing NetworkDetect Damage ProgressionDiagnosis/PrognosisEmission Control

8

Auto-ID Solutions for Network-Centric Manufacturing Environments

Objective: Develop concepts, models, and prototypes for effective integration of Auto-ID technology in a Network-Centric Manufacturing Environment*, aimed at reducing delays and eliminating non-value added production activities and responding rapidly to unexpected events on the shop floor.

Application Potential: 1) Inventory control of time and temperature sensitive materials 2) Receiving & shipping operations3) Aircraft assembly line flow

ROADMAP

Re-EngineerRe-Engineer

Integrated Systems Facility (Emgt)Integrated Systems Facility (Emgt)Embedded Systems and Networking Laboratory (EE)Manufacturing R&D Assembly Lab.

Auto-ID Solutions Development

Auto-ID SolutionsAuto-ID

Solutions

System Simulation (after)

Analysis: amount of wasteno of man-hours PrototypesPrototypes

Tag Simulation

Simulation Simulation withhardware-in-the-loop

Hardware

SYSTEMS

All aircraft assembled at Boeing

Process Modeling

Process Map

Analysis: amount of wasteno of man-hours

System Simulation

(before)

SynthesisSynthesis

Research Highlight:

• Develop techniques for effective use of real-time data provided by Auto-ID technology.

• Demonstrate integration of Auto-ID technology with the aircraft manufacturing practice.

* Network-centric Manufacturing Environment incorporates a dynamic network of self-organizing, autonomous units that operate, collaborate, cooperate, and compete upon basic principles of decentralization, participation, and coordination in order to accomplish the goals set at system level.

9

Networking Infrastructure forAdaptive Inventory Management

Objective: To demonstrate integration of decision making in a multi-technology environment.

Conclusions: (1)Modularity of the architecture facilitates expansion of the application domain and experimentation with complex models, and (2) the infrastructure allows for investigating different networking topologies, protocols, and alternative hardware.

Expeditor

Decision Maker – C++

(ES&NL)

Internet

2-BufferArena Model

(ISF)

F1 F2

Sockets

Wireless Access Port

Database -MySQL

CommandsResponse

PDA - Java

Buffers(Shop Floor)

Reader

XML

Antenna

Reader

XML

Antenna

10

Neural Emission Control

• Lean operation and with high EGR levels in certain SI Engine can reduce emissions (HC, CO & NOx) by as much as 30% and also it improves fuel efficiency by as much as 5 ~ 10% (Inoue, 1993).

Heywood, 1988 Cyclic dispersion without control

Objective: Minimize the cyclic dispersion at lean engine operation by applying the NN controller

11

Nonlinear Discrete-time System

Nonlinear

1 1 1 2 1 1 2 2 1

2 2 1 2 2 1 2 2

3 1 2

1 , ,

1 , ,

1 ,

x k f x k x k g x k x k x k d k

x k f x k x k g x k x k u k d k

y k f x k x k

Back Stepping

NonStrict

Back Stepping

12

Comparison of Experimental and Simulation Data

Experimental data (Sutton, 2000) Simulation result

13

NN Output Feedback Lean Emission Control

0 2 4 6 8 10 120

2

4

6

8

10

12

Heat Release (i)

Hea

t R

elea

se (

i+1)

Heat release return map without control, = 0.705

Heat releaseFiexed point

1x

2x

nx

1y

2y

my

TV TW

(.)

(.)

(.)

(.)

(.)

(.)

(.)

1

2

3

LInputsHidden Layer

outputs

Neural Network (NN) Controller

Measurements

ControlInputs

Engine

NN Observer

0 1 2 3 4 5 6 7 8 90

1

2

3

4

5

6

7

8

9

Heat Release (i)

Hea

t R

elea

se (

i+1)

Heat release return map with output control, = 0.705

Heat releaseFiexed point

14

Contributions

• Nonstrict Feedback Nonlinear Discrete-time Systems introduced and optimal controllers designed

• Noncausal control problem is overcome• Separation Principle, Certainty Equivalence,

Persistency of Excitation Condition, and Linear in the Unknown Parameters are relaxed

• Fuel efficiency improvement by 5 to 10%, NOx by 98% and uHC by 30% was noted

15

NN Control Book in 2006

16

Outline

• Overview of Research at ESNL

• Energy Aware Protocols

– Congestion Control

– Routing

• Conclusions

17

Performance Requirements

• Congestion causes– Reduced channel capacity – Energy wastage

• QoS suffers– Throughput, network efficiency– Delay , jitter– Fairness– Energy-efficiency

• Proposed scheme consists of– Congestion prediction and control

mechanisms to prevents buffer overflows– Fairness mechanism

18

Congestion in Wireless Sensor Networks (WSN)

• Congestion can be a result of:– Channel fading– Traffic exceeding channel capacity

• Note: Typical WSN has only one “sink”

19

Ways of Alleviating Congestion

• Energy aware congestion control• Adaptive Back off Interval scheme• Adaptive energy delay routing

* Cross Layer Design is needed

20

Background

• Sensor versus ad hoc– Processing, memory communication and energy

constrains in WSN in contrast to ad hoc networks

• Previous works– End-to-end protocols (e.g. TCP)

• Drawback of large feedback latency (round-trip)

– CODA, Fusion• Small processing overhead• Congestion message is broadcasted to throttle

traffic, with sources reducing introduced traffic• Use fixed thresholds to initiate flow control• Dropping packets when the congestion occurs

21

Objectives of Congestion Control

• Detection of the congestion and an onset of one– Channel estimation from Distributed Power Control

(DPC - our previous work) predicts severe fading and temporarily suspend outgoing and incoming flows

– Buffer occupancy used to control incoming/outgoing flows

• Quality of Service (QoS)– Weighted Fairness expressed as:

where W – throughput, φ – weight• Congestion control using backpressure – a set of three

proposed schemes (described later)

0),(),( 2121

m

m

f

f ttWttW

(1)

22

Proposed Scheme

• Rate-based congestion control with weights changed adaptively– Rate selection to prevent buffer overflow– Rate allocation to flows according to weights (using

fair scheduling)– Selection of back-off interval to achieve the selected

rate (predictive, distributed, mathematically guaranteed)

• (OPTIONAL) Adaptive weights allocated to each packet to improve weighted fairness– Adaptive re-calculation of weights for each packet– Fair scheduling and rate allocation to flows based on

adopted weights

23

Buffer occupancy and Error Dynamics

• Consider buffer occupancy at a particular node

– where T is the measurement interval, qi(k) is the buffer occupancy of node ‘i’ at time instant k, ui(k) is a regulated (incoming) traffic rate, and fi(k) is an outgoing traffic rate.

• Consider the desired buffer occupancy at node i to be qid. Then, buffer occupancy error defined as ei(k)=qi(k)-qid can be expressed as

(2)

(3)

)()(1 1 kdkufkuT+kqSat=+kq iiiipi

idiiiibi qkdkufkuTkqke )())(()()()1( 1

24

Rate Selection

• Define the traffic rate input, ui(k) as

where kv is a gain parameter.• Unknown outgoing traffic is estimated

using adaptive scheme

where the parameter vector is updated

• Selected incoming rate is divided fairly (φj) among incoming flows

)()()()( 1 kekkfTkqqTku iviiidi

(5)

(4)

)1()(ˆ))((ˆ1 kfkkuf iii

25

Given the incoming rate selection scheme above with variable i estimated

accurately (no estimation error), and if the incoming traffic is updated as (4),

then the mean estimation error of the variable i along with the mean error in

queue utilization converges to zero asymptotically, if the parameter i is

updated as

provided: (a) 1

2<kui and (b) δ<K fvmax 1 , where 2

11 ku=δ i , fvmaxK is

the maximum singular value of fvK , is the adaptation gain, and

)(ˆ)()( kfkfke iifi is the error between the estimated value and the actual one.

)1()()(ˆ)1(ˆ kekukk fiiii

Theorem: Ideal Case

26

0 5 10 15 20 250

5

10

15

20

Time [iteration]

Que

ue le

vel [

pack

ets]

Thr

ough

put

[pac

kets

/ it

erat

ion]

qi

estimated fout

fout

0 5 10 15 20 25-10

-5

0

5

10

Time [iteration]

Buf

fer

occu

panc

y er

ror

[pac

kets

]T

hrou

ghpu

t es

timat

ion

erro

r[p

acke

ts/it

erat

ion]

ebi

ef

Queue utilization and estimation of the outgoing flow.

Queue utilization error and outgoing traffic

estimation error.

Simulation Results for Rate-based Buffer Control

27

Adaptive Back-off Selection Algorithm

• GoalSelect back-off interval BOi at i-th transmitting node such

that the actual throughput meets the desired outgoing rate fi(k).

• Consider inverse of the back-off interval, and call it a virtual rate VRi

where VRi is the virtual rate at i-th node, and BOi is the corresponding back-off interval.

• NOTE: the virtual rate is not equal to the actual rate; instead, the virtual rate is proportional to the actual rate.

ii BO=VR /1 (6)

28

Outgoing Rate Selection

• The actual rate of an i-th node is a fraction of the channel bandwidth B(t) defined as

where TVRi is the sum of all virtual rates for all neighbor nodes.

• Differentiating and transforming into discreet time domain the outgoing rate is equal

where kvkβ+kαkR=+kR iiiii 1

tTVR

tVRtB=

tVR

tVRtB=tR

i

i

iSll

ii

(8)

(7)

kTVR+kTVR=kα iii 11 kVRkR=kβ iii

111 +kBO=+kVR=kv iii

(9)

29

Closed-loop Control of Backoff Selection

• Now consider feedback equation for closed-loop controller

where âi(k) is estimate of ai(k), and ei(k)=qi(k)-qid is the throughput error

• Parameters updates taken as

• The closed loop throughput error system with estimation error, ε(k), as

keKkkRkfkkv Riviijiii )()()()()( 1 (10)

(11) 1ˆ1ˆ kekR+kα=+ka Riiii

(12) )()()(1 kkRkkeKke iiivi

30

Convergence and Stability

• Theorem (General case):– Given the back-off selection scheme above with an

interval updated as (10), and with uncertainties estimated by (11), with ε(k) as the error in estimation which is considered bounded above , with εN a known constant.

– Then the mean error in throughput and the estimated parameters are bounded provided

and hold

– where , Kvmax is the maximum singular values of Kv, and σ is the adaptation gain.

12

<kRσ iδ<Kvmax 1

Nεkε

211 kRσ=δ i

31

Simulation Results

1 2 3 4 5

0

0.2

0.4

0.6

0.8

1

1.2

1.4

Proposed scheme w/o weight adap-

Proposed scheme

DPC

CODA

Flow ID

Wei

ght *

Del

ay

Performance for unbalanced tree topology.

Weighted delay with equal flow weights (const=0.2).

1 2 3 4 5

0

100

200

300

400

500

600

700 Proposed scheme w/o weight adap -

Proposed scheme

DPC

IEEE802.11

CODA

flow ID

Thr

ough

put /

Wei

ght [

kbps

]

32

Impact on Routing Protocol

• Hardware limitation explored– Memory limits queue size– Processing limits number of connections that can

be handled (also for information aggregation)

• Localized congestion – For example due to electro–magnetic interferences– Can be mitigated by selecting a alternative route

around the congested area

33

OEDSR

• Optimal Energy Delay Sub-Network Routing (OEDSR) protocol– Maximizes Link Cost Factor (LCF) to perform optimal routing

based on network parameters

– LCF is given by

– Where Ei is the energyenergy in the next node, Di is the delaydelay, and xi is the distancedistance from the next node to the base station

• Cluster heads (CH) and relay nodes (RN) are used to route data from a data source to the base station (BS)

• Implemented on UMR hardwareImplemented on UMR hardware

ii

ii xD

ELCF

34

OEDSR: Optimal Relay Node

9 10 13

8

14

11

12

BS

CH 1

1234

15

567

67

CH 2 CH 3

67

7

1

5

2

4

3

“HELLO_CH” packet

“HELLO” packet

“RESPONSE” packet Node ID End-to-End Delay Energy Available Distance from BS

Node IDDistance from

BS

List of all nodes in Range

CH 1 CH 2

567

1234

67

“RELAY_SELECT” packet

2

3

1

35

OEDSR: Optimal Relay Node

9 10 13 14

11

BS

CH 1

1234

567

67

CH 2 CH 3

67

7

1

5

2

3

CH 1 CH 2

567

1234

6

2

3

1

DistDelay

ErfactortLink n

_cos_

Ern - energy available in the given node

Delay - average end-to-end delay between any two CHs

Dist - distance from the node to the BS

7

8

12

15

4

7

412158

36

Modification to OEDSR

• Link Preference Factor (LPF)

– Load factor i is for balancing load by distributing traffic between several nodes

– where, Fi is the maximum designed capacity of a node i, and fi is the current load at node i (measured in number of flows routed at the node)

iii

ii xD

ELPF

iii fF 1

37

Experimental Results for Routing Protocol

• Simple topology used to verify claim of balancing load between two relay nodes versus sending the traffic through one node only

BS

SourcesRelaynodes

3 sources Total Throughput

Average per source

Average per relay node

Through 1 relay

6.88 kbps 2.29 kbps 6.88 kbps (1 relay)

Through 2 relays

8.01 kbps 2.68 kbps 2.43 kbps (2 flows)5.69 kbps (1 flow)

38

Demonstration

<Play video>

39

Contributions

• Novel adaptive schemes based on control theory developed

• Analytically Guaranteed• Many of these are demonstrated on

Mote Hardware• Deployed in various industrial

environments

40

New Book to appear in 2007

• Wireless Ad hoc and Sensor Networks: Protocols, Performance, and Control

• CRC Press, 471 pages

41

Conclusions

• Analytical and simulation results show that the proposed scheme– Increases throughput– Guarantee desired QoS and weighted fairness – Also during congestion and fading channels.

• The proposed scheme mitigates congestion using a hop by hop mechanism for throttling packet flow rate

• The convergence analysis is demonstrated by using a Lyapunov-based analysis

• Experimental results show that Congestion-aware routing protocol improves performance of the resource constrained network

42

Questions?