DSPC - 1

Berkeley Wireless Foundations Workshop, Feb. 11 2005

D i s t r i b u t e d S i g n a l P r o c e s s i n g a n d

C o m m u n i c a t i o n i n S e n s o r N e t w o r k s

M a r t i n V e t t e r l i , E P F L a n d U C B e r k e l e y

joint work with:T.Ajdler, R.Cristescu, R.Konsbruck (EPFL),

B.Beferull-Lozano, L.Sbaiz (EPFL)P.L.Dragotti ( Imperial) , M.Gastpar (UCBerkeley)

Work done under the auspice of the NSF Swiss National Center onMobile Information and Communication Systems

http:/ /www.mics.org

DSPC - 2

Outl ine

1. Introduction

• The Center on “Mobi le Informat ion and Communicat ion Systems”:• Wireless sensor networks: f rom “one to one” to “many to many”

2. Distributed Signals and Sampling

• Sensor networks as sampl ing devices• Distr ibuted image processing: The plenopt ic funct ion• Spat ia l sound processing: The plenacoust ic funct ion• The heat equat ion and i ts kernel

3. On the interaction of source and channel coding

• Correlated source coding and transmission: NSW, DKLT• The wor ld is analog, why go digi ta l? JSCC• The wor ld is physical : Interact ion of physics wi th Sig.Pro. and Comm.

4. Environmental monitoring

• monitor ing for scient ific purposes- wind tomography

• monitor ing for intel l igent bui ld ings- SensorScope

5. Conclusions

DSPC - 3

Acknowledgements

• Swiss and US NSF

• The Nat ional Competence Center on Research‘ ’Mobi le Informat ion and Communicat ion Systems’’ (MICS)

• K.Ramchandran and his group at UC Berkeley,for shar ing pioneer ing work on distr ibuted source coding

• Col leagues at EPFL and ETHZ involved in MICS- J.P.Hubaux,for pushing ad hoc ntws- M.Grossglauser, for making things move- E.Telatar, for wisdom and figures!- H.Dubois-Ferr iere, for anyt ime help- J.Bovay , for NCCR matters

DSPC - 4

1. Introduction

The Swiss National Competence Center on Research‘’Mobile Information and Communication Systems’’

http:/ /www.mics.org

Goal:

study fundamental and appl ied quest ions raised by new generat ion mobi le communicat ion and informat ion services, based on sel f -organisa-t ion.

Cross-layer investigation:

mathematical issues (stat ist ical physics based analysis, informat ion and communicat ion theory) to networking, s ignal processing, secur i ty, d istr ibuted systems, sof tware arch. and economics.

Examples:

ad-hoc and sensor networks, peer- to-peer systems

Network of researchers:

• EPFL, ETHZ, CSEM, UNI-BE,L,SG,ZH• 30 professors, 70 PhD students• 11 indiv idual projects

Budget:

• 8 MSfr/Year (6 M$/Y-> 7 M$/Y)• 4-12 years hor izon• Note: s imi lar to a US-NSF ERC or STC

DSPC - 5

The NCCR MICS network

See the website http:/ /www.mics.org for detai ls, publications etc

EPFL: Schools of Computer and Communication Sciences(Leading House ) and Engineering

University LausanneEcole des Hautes Etudes Commerciales

University Berne : Institute ofComputer Science and Applied Mathematics

University St Gallenmcm Institute

University ZurichInstitute of Computer Science

CSEM, Swiss Center forElectronics and Microtechnology

ETHZ: Departments of ElectricalEngineering and Computer Science

EPFL: Schools of Computer and Communication Sciences(Leading House ) and Engineering

University LausanneEcole des Hautes Etudes Commerciales

University Berne : Institute ofComputer Science and Applied Mathematics

University St Gallenmcm Institute

University ZurichInstitute of Computer Science

CSEM, Swiss Center forElectronics and Microtechnology

ETHZ: Departments of ElectricalEngineering and Computer Science

A nation-wide network of research groupsfocusing and interacting on distributed,mobile information andcommunicationsystems

One of the 14 Swiss NCCRs - started in 2001, horizon 10-12 yearsBudget: CHF 32M (US$ 20M) over 4 years - ∫ SNF, ∫ matchingSize: about 30 faculty members and 60-70 PhD students, 11 projects

DSPC - 6

From central ized to ‘ ’self-organized: Possible solutions

Classic solution (e.g. GSM, UMTS):characterized by heavy fixed infrastructures

Hybr id solut ion:mult ihop access to backbone

Fully multihop solution(e.g. sensor network),« Terminodes »

DSPC - 7

Current Status, Research (1/3)

Mathematical Foundations and Physical Layer

• Connect iv i ty models (e.g. hybr id networks)• Interference models (physical model and percolat ion)• Capaci ty of ad-hoc networks• Power-distort ion t radeoffs in sensor networks (see below)

Dousse/Thiran IP1

Enz IP11

DSPC - 8

An Example: Percolation Theory as a Fundamental Concept

it “percolates” through connectivity, capacity, P2P, gossip, etc

E.Telatar

DSPC - 9

Current Status, Research (2/3)

Networking, Security and Signal Processing

• Rout ing (var ious approaches)• Distr ibuted trust , Secur i ty• Distr ibuted sampl ing, d istr ibuted compression (DKLT)

LeBoudec IP4PGP Trust Graph

Fresh RoutingDubois/Grossglauser/V IP1/7

Hubaux IP4

DSPC - 10

Current Status, Research (3/3)

Information systems, distributed systems, software architecture

• Peer- to-peer systems• Environment-aware software• Bluetooth smart nodes (BTNodes)

Aberer IP5

DSPC - 11

1.2 The view of the world: Wireless sensor networks!

Signals exist everywhere.. . they just need to be sensed!

• d istr ibuted signal aquis i t ion (many cameras, microphones etc)• these signals are not independent

- the more sensors, the more correlat ion• there can be some substant ia l structure in the data,

due to the physics of the processes involved

Computation is cheap

• local computat ion• complex algor i thms to retr ieve data are possible

Communication is everywhere

• th is is the archetypical mult i terminal chal lenge• mobi le ad hoc networks• dense, sel f -organized sensor networks are bui l t• the cost of mobi le communicat ions is st i l l the main constraint

Cross-disciplinarity

• fundamental bounds (what can be sensed?)• algor i thms (what is feasible?)• systems (what and how to bui ld)

DSPC - 12

The Change of Paradigm

Old view: one source, one channel, one receiver (Shannon 1948.. . )

New view: distributed sources, many sensors/sources, distributed communication medium, many receivers (now.. . )

Note: most questions are open!

source receiv.channelcoding deco.

communicationsmedium

sources receivers

DSPC - 13

The swiss version of homeland security ;)

Distributed sensor network for avalanche monitoring:

Method: drop sensors, self-organized tr iangulation, monitoringof location/distance changes, download when crit ical situation

Challenges: extreme low power, high precision, asleep most of the t ime, when waking up, quick download

.. . and al l self-organized!

Legacy technology: build a chalet, see if i t stands after 50 years!

DSPC - 14

Technological Change of Paradigm

Orders of magnitude less cost for sensing:

Orders of magnitude of difference in price, size and power!

We expect this wil l have a t idal effect on

• what is monitored• how i t is monitored

and there are Berkeley motes to save the world!

100K$ 10-100$

DSPC - 15

Outl ine

1. Introduction

2. Distributed Signals and Sampling

• Sensor networks as sampl ing devices• Distr ibuted image processing: The plenopt ic funct ion• Spat ia l sound processing: The plenacoust ic funct ion• The heat equat ion

3. On the interaction of source and channel coding

4. Environmental monitoring

5. Conclusions

DSPC - 16

3. Distributed Signals and Sampling

A sensor network is a distributed sampling device

Physical phenomena

• d istr ibuted signals are governed by laws of physics• PDE at work• spat io- temporal d istr ibut ion

Sampling

• regular/ i r regular, densi ty• in t ime: easy• in space: no fi l ter ing before sampl ing• spat ia l a l iasing is key phenomena

Note: here we assume that we are interested by the ‘ ’ true’’ phenomena(decision/control: can be different)

X

XX X

X

X

X

X

X

X X

X X

X

X

X

X

X

X

DSPC - 17

3.1 Sampling the real world

We consider 3 ‘ ’real’’ cases, and fol low:

• what is the physical phenomena• what can be said on the ‘ ’d iscret izat ion” in t ime and space• is there a sampl ing theorem• what is the structure of the sampled signal

Light fields

• wave equat ion for l ight or ray t racing• plenopt ic funct ion and i ts sampl ing

Sound fields

• wave equat ion for sounds• plenacoust ic funct ion and i ts sampl ing

Temperature distributions

• heat equat ion, d i f fusion

DSPC - 18

The Plenoptic Function [Adelson]

Multiple camera systems

• physical wor ld (e.g. landscape, room)• distr ibuted signal aquis i t ion• possible images: plenopt ic funct ion, 7-dim!

Background:

• p inhole camera & epipolar geometry• mult id imensional sampl ing

Implications on communications

• camera sources are correlated in a par t icular way• l imi ts on number on ‘ ’ independent ’’ cameras• di f ferent BW requirements at d i f ferent locat ions

f(x,y,z,

α

,

θ

,

λ

,i)

DSPC - 19

Example:

Thus:

• h idden, uncovered objects• depth of field (object move faster i f they are c loser)• project ive geometry

DSPC - 20

On Plenoptic Sampling [Shum et al]

Model

Epipolar geometry

• points become l ines• s lope depends on depth of field

Plenoptic function

• a col lect ion of l ines (modulo cover ing/uncover ing)• s lopes banded by (min, max) depth• Four ier t ransform.. .a pie s l ice. . . can be sampled

z

t’t

v v’

t

vf t

v

DSPC - 21

Fourier transform (approx.) and after sampling in space

Examples of recent results

1. Bandlimited walls/fcts

[DoMMV:04] Plenoptic function not BL unless l inear wall .

2. Plenoptic function of finite complexity objects

[Maravic et al] For cer tain ‘ ’simple scenes’’ (collection of Diracs),the plenoptic function can be sampled exactly (Radon xform)

ωt

ωs

ωt

ωs

DSPC - 22

The Plenacoustic Function [AjdlerV:02/03]

Multiple microphones/loudspeakers• physical wor ld (e.g. f ree field, room)• distr ibuted signal aquis i t ion of sound with “many” microphones• sound render ing with many loudspeakers (wavefield synthesis)

This is for real!• sound recording• special ef fects• movie theaters (wavefield synthesis)• MP3 surround etc

254 LS!

DSPC - 23

Plenacoustic function and its sampling

Set up:

Questions:• Sample wi th ‘ ’ few’’ microphones and hear any locat ion?• Solve the wave equat ion? in general , i t is much simpler to sample the

plenacoust ic fct• Dual quest ion also of interest for synthesis (moving sources)• Impl icat ion on acoust ic local izat ion problems• appl icat ion for acoust ic echo cancel lat ion

. . .

DSPC - 24

Examples: PAF in free field and in a room for a point source

We plot: , that is, the spatio-temporal impulse response

The key question for sampling is: , that is, the Fourier transform

A precise characterization of for large φ and ω will al low

sampling and reconstruction error analysis

p x t,( )

P φ ω,( )

P φ ω,( )

DSPC - 25

Plenacoustic function in Fourier domain (approx.):

Sampled version:

ω

φ

slope: ω = c φ

ω

φ

DSPC - 26

Plenacoustic function in Fourier domain (approx.):

Sampled version:

slope: ω = c φ

²

¯

0

²

¯

²

¯s

DSPC - 27

Computed Plenacoustic Functions

DSPC - 28

Example of a plenacoustic function in a room

computed measured

nice and (almost) bandlimited!

DSPC - 29

A sampling theorem for the plenacoustic function

Assume a max temporal frequency ω0

Pick a spatial sampling frequency φN > ω0/c

Then, the spatio-temporal signal can be interpolated

from samples taken at (2ω0,2φN).

Argument:• take a cut through PAF• use exponent ia l decay away from central t r iangle• bound al iasing

N

A

Y¯ 3¯¯SN N ¯¯¯

A

DSPC - 30

Some generalizations

Sampling patterns

Other geometries

y

²

¯

¯

x

ĭ30ĭ20

ĭ100

1020

30

ĭ30ĭ20

ĭ100

1020

300

100200

¯x

²=1500 [rad/s]

¯y

Am

plitu

de

ĭ30ĭ20

ĭ100

1020

3

ĭ30ĭ20

ĭ100

1020

300

50100

¯x

²=3000 [rad/s]

¯y

Am

plitu

de

DSPC - 31

The Heat Equation

Assume you want to put temperature sensing• what is the spat io- temporal d istr ibut ion• can this be sampled

I t is al l in the heat kernel!

• kernel : non-bandl imted (but very smooth)

10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1x 10

−8

Rate (nats/s)

Dis

tort

ion

(MS

E/s

)

centralizedspatially and temporally independenthigh−rate Wyner−Ziv codingspatially independenthigh−rate DPCM with local predictionhigh−rate DPCM with feedback

x2

2

∂

∂ T x t,( ) g x t,( )k

---------------- µT x t,( )–+t∂∂ T x t,( )

α----------------=

DSPC - 32

On Sampling and Representation

We saw a few examples:• Plenopt ic funct ion and l ight fields• Plenacoust ic funct ion and sound fields• Heat equat ion

I t is a general phenomena• e lectromagnet ic field is another good example

This has implications on• Sampl ing: where, how many sensors• How much informat ion is there to be sensed• Gap between simple (separate) and jo int coding• Spat io- temporal waterpour ing

DSPC - 33

Outline

1. Introduction

2. Distributed Signals and Sampling

3. On the interaction of source and channel coding• Introduct ion• Correlated source coding and transmission: NSW, DKLT• The wor ld is analog, why go digi ta l? JSCC• The wor ld is physical : Interact ion of physics wi th Sig.Pro. and Comm.

4. Environmental monitoring

5. Conclusions

DSPC - 34

3.2 Correlated source coding and transmission

Dense sources = correlated sources• physical wor ld (e.g. landscape, room)• degrees of f reedom ‘ ’ l imi ted’’• denser sampl ing: more correlated sources

Background: • Slepian- Wolf ( lossless correlated source coding with binning)• Wyner-Ziv (source coding with s ide informat ion)

Implications on communications• such resul ts are rarely used.. .• many open problems ( lossy Wyner-Ziv is st i l l an open problem.. . )• separat ion might not be the way.. .• are there l imi t ing resul ts?

Below, two specific results:• Data gather ing t rees: Slepian-Wolf and NP completeness• Distr ibuted compression: a distr ibuted Karhunen-Loeve transform

DSPC - 35

S

X

X1X2

X3

N

The Correlated Data Gathering Problem[Cristescu-Beferull-V:03]

Given: • a graph with a set of correlated sources X1, X2, . . . XN at the nodes• a s ink node S• a weight d( i , j ) on each l ink

Goal:• minimize total cost for t ranspor t of X1, X2, . . . XN data to node S:• find rate al locat ion R1, R2, . . . RN AND spanning tree ST:•

Two approaches:• use Slepian-Wolf g lobal ly -> convex

programming (SPT opt imal t ree)• use local correlat ion only -> NP

completeness (approx. t rees)

min RidST i S,( )i V∈∑

DSPC - 36

Slepian-Wolf (1973.. . )

Given• X, Y i . i .d wi th p(x,y)

Then: code separately, decode jointly, without coders communicating

Achievable rate region• • •

• for many sources.. . rather complex! (b inning)

R1 H X Y⁄( )≥R2 H Y X⁄( )≥R1 R2+ H X Y,( )≥

R1

R2

H(X)

H(Y)

H(X/Y)

H(Y/X)

DSPC - 37

Slepian-Wolf based rate al location [Cristescu-Beferull-V:03]

Convex programming problem:

• under constraints, for a l l Y subsets of V (Slepian-Wolf) :•

LP Solution (closed-form)• order nodes X from 1.. .N as distance from S increases,

• intersect ion of the cost plane with the last corner of the SW region

min RidSPTi S,( )

i V∈∑

Rj H Y YC

( )≥j Y∈∑

Ri H Xi Xi 1– …X1,( )=

H(X |X )

H(X |X ) H(X )

R

R

H(X )

2

2

2 1

1 2 11

1 R d (X ,S) + R d (X ,S) SPTSPT1 22

DSPC - 38

Approximation• For distr ibuted, local info algor i thm, condi t ion node entropy only on

nodes closer to S than i tsel f.• Correlat ion dependent on distance -> very c lose to opt imal!

Extension• general t raffic matr ices

Neighborhood Approximation for Slepian-Wolf coding (left); Steiner trees for a general traffic matrix (right)

i+1i

1

2

N−1 N

S

S

S

VV1

2

1

2

ij

k

DSPC - 39

Explicit communication

Assume: side information is available only explicitely (when node is used as relay by other nodes).

Model: (simplification)• i f node has data alone: R bi ts need to be transmit ted• i f node has already some other data: r<R bi ts

I f r = R, simply shortest path tree, easy

If r = 0, (mult iple) traveling salesman.. .hard

Results:• Problem is in NP for r<R• Good distr ibuted approximat ion algor i thms• Can make a large di f ference in energy consumption

Main point:• source structure influences communicat ion structure

(unl ike Slepian-Wolf, no separat ion!)

DSPC - 40

Simple network examples

if R>2r, SPT is better than TSP

pTSPpSPT---------------

N ∞→lim 1 ρ–( ) 1

2D------- 1+ =

(a) SPT (b) TSP

R R

R R

r

2R+r

4R+r 3R+3r

r

R+r

rR

R+2r

S S

R

21

...........

rr

rr

D

1

S

R

N

33N−1

TSPSPT

dd

D

1

S

RR

RR

R

N−1N

21

DSPC - 41

SPT/TSP approximation algorithm - Example

SPT (right) , SPT/TSP balanced tree ( left)

up to 40 % improvement over SPT/TSP

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

80

90

100

DSPC - 42

Scaling laws ( init ial results)

1D model: • compare Slepian-Wolf wi th expl ic i t communicat ion

• dense versus expanding networks• regular versus singular processes• condi t ional entropy laws

d dd SX X X X

N N−1

NH(X )

H(X )

N−1

H(X | X ,X ,...,X )

H(X | X ,X ,..., X )

H(X |X )

H(X |X )N

N−2 1 2 1 1

N321

Explicit Communication

Slepian Wolf

. . . .

. . . .N N−1 2 1

DSPC - 43

One-dimensional example

2 Sink1N−1N

1

H(X | X )2 1

N−1 1N−2

N N−1H(X |X ,..., X )

1

. . .

H(X | X ,..., X )

. . .

H(X )H(X | X ,..., X )

2 Sink1N−1N

N−1 NH(X | X )

2 3 N

H(X )N

. . .

. . .

H(X | X ,..., X )1 2 N

tSWcos N( ) idaii 1=

N

∑= tECcos N( ) N i– 1+( )daii 1=

N

∑=

tSW N( )cos

tEC N( )cos----------------------------- 0 ? yes, if i a⋅ i( ) 0→→

DSPC - 44

Performance gain depends on condit ional entropy law

0 100 200 300 400 500 600 700 800 900 10000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Number of nodes N

cos

t SW

/cos

t EC

H(X1|X

2,...,X

i) ≈ 1/i + C

H(X1|X

2,...,X

i) ≈ 1/i0.5

H(X1|X

2,...,X

i) ≈ 1/i

H(X1|X

2,...,X

i) ≈ 1/i1.5

DSPC - 45

Random fields (refinement network)

[Cond. entropy] * [# of cond. nodes], and ratio of costs,

for Gaussian processes (up) and bandlimited processes (down).

0 5 10 15 20 25 30 35 400

5

10

15

20

25

30

Number of nodes on which conditioning is done

i * H

(Xi|X

i−1,..

,X1)

e−c |τ|2

correlation

e−c |τ| correlation

0 5 10 15 20 25 30 35 400

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Network size N

cost

(N)

SW

/ co

st(N

)E

C

e−c |τ|2

correlation

e−c |τ| correlation

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

20

40

60

80

100

120

140

160

180

200

220

Number of nodes on which conditioning is done

i * H

(Xi|X

i−1,..

.,X1)

B=0.7/dL

B=0.85/dL

B=1/dL

dL=1/8

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 150

0.2

0.4

0.6

0.8

1

Network size N

cost

SW

(N)/

cost

EC

(N)

B=0.7/dL

B=0.85/dL

B=1/dL

dL=1/8

DSPC - 46

The Distributed Karhunen-Loeve Transform (DKLT)[Gastpar-Dragotti-V:02/03]

The Karhunen-Loeve transform (KLT) is a key par t of source compression.

• For example JPEG, MPEG, MP3 al l use some version of KLT(DCT, wavelets, fi l ter banks)

• th is is the workhorse of source compression!

Assume a correlated vector source [X1, X2, X3, . . . XN], zero mean, autocorrelation RX

• best M < N subspace approximat ion is given by the project iononto the M eigenvectors of RX with largest eigenvalues

• best compression (Gaussian case) is waterpour ing overthe eigenvalues in the eigenspace

KLT

X1

X2

XN

X3 Q

Y1

Y2

Y3

Y4

Approximation

Compression

DSPC - 47

What about a distributed scenario?

Distributed sensors measure correlated data

What is the best way to separately compress this sourceby L local compressors, for a joint decoder?

This answers ( in par t) a distributed source coding problem

The problem of the distributed KLT addresses:• non- l inear approximat ion (NLA)• rate-distor t ion behavior

...

...

X1 hX1

Enc1

Enc2

R1

R2

XNRL

hXNEncL

Dec

DSPC - 48

The par tial KLT

Assume only a par t of the sources are observed, [X1, X2, X3, . . . XM]but the entire vector N > M needs to be reconstructed.

Model: Xuo = A Xo + V (e.g. jo int ly Gaussian)

Results:• NLA: k dim. approx. wi th largest modified eigenvalues• Compression: R(D) s imi lar to Gaussian, wi th modified eigenvalues

Note: this corresponds to a par tial ly observed sensor field

.......

... ...partial

KLT

X1

XM

XN

Y1 R1

RM

Dec

Enc1

EncM

hX1

hXM

hXM1

hXN

YM

XM1

DSPC - 49

The condit ional KLT

Assume that a par t of the sources are available as side information, the others are observed and coded.

The entire vector needs to be reconstructed.

Cond. KLT: C Σs/s CT = diag(λ i ) , that is, Y is condi t ional ly uncorrelated

Results:• NLA: k dim. approx = k cond. e.vectors wi th largest e.value• Compression: (Gaussian case) separate WZ compression af ter C

.....

... ...KLT

X1

XM

XN

Y1 R1

RM

Dec

Enc1

EncM

hX1

hXM

cond

XM1

YM

DSPC - 50

The combination.. .

Assume that some sources are available as side information, some sources are observed and coded, and some are hidden.The entire vector needs to be reconstructed.

Result:• NLA: use condi t ional and par t ia l KLT in turn

• Compression: improves non-distr ibuted solut ion

...... U

X1

Enc

YScp

YScppXN

Approx

Dec

hXN

hX1

XM1

XM

DSPC - 51

Example of cost surface

depends obviously on the structure of correlation matrix• length 4 sensor vector, in two bloks of 2• there can be local minimas ( lef t )• Markov-1: n ice (r ight)

−2

−1

0

1

2

−2

−1

0

1

23.6

3.7

3.8

3.9

4

4.1

4.2

4.3

4.4

4.5

α

MSE with two minima

β

MS

E

−4

−2

0

2

4

−4

−2

0

2

40

0.5

1

1.5

2

2.5

3

3.5

4

α

Gauss−Markov process (ρ=0.9)

β

MS

E

DSPC - 52

Example of rate-distor tion function

Size 4 vector, coded in 2 block

Correlation matrix

Compression:

D(R) based on observing first 2 D(R) using the last 2 as side info

Σx

0.11 0 0.1 0.10 0.1 0.25 0

0.1 0.25 1 0.250.1 0 0.25 1

=

0 0.02 0.04 0.06 0.08 0.1 0.12 0.140

1

2

3

4

5

6

7

8

D

R

0 1 2 3 4 5 61.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

2

2.1

2.2

R

D

DSPC - 53

On distributed source coding.. . .

Two cases seen:• Data gather ing wi th Slepian-Wolf and energy opt imized trees• Distr ibuted versions of the KLT

These are difficult problems.. . .• lossy distr ibuted compression most ly open

In many case• Strong interact ion of “source” and ‘ ’channel ’’• Large gains possible

but we are only seeing the beginning of ful ly taking advantageof the sources structures and the communication medium.. .

DSPC - 54

3.3 The world is analog: why go digital?[Gastpar et al]

Going digital is t ightly l inked to the separation principle:• in the point to point case, separat ion al lows to use

“bi ts” as a universal currency

• but th is is a miracle! (or a lucky coincidence)

There is no reason that in multipoint source-channel transmissionthe same currency wil l hold

So, as a star t:• uncoded transmission can be opt imal• capaci ty of the relay channel wi th many relays• source-channel coding for sensor networks

analog

01010 01010SC CC CD SDX Y

analogworld

analogworld

channel

DSPC - 55

Uncoded transmission for lossy source-channel communication

[To code, or not to code: GastparRV:02/03]

It is well known that a Gaussian source over a AWGN channelcan be ‘ ’sent as is’’ , achieving optimal performance

• easy way to achieve best performance (no delay. . . )

The parameters of source-channel coding are:• source distr ibut ion: PS(s)• source distor t ion or error measure: D(s,s)• channel condi t ional d istr ibut ion: PY/X(y/x)• channel input cost funct ion: ρ (x)

The ar t is measure matching!• D(R): channel has to look l ike the test channel to the source• C(P): source has to look l ike a capaci ty achiev. d istr ib. to the channel

• in the Gaussian case, i t a l l matches up! (MSE, power, densi t ies)

X Y SSchannel recv.sourc. F G

DSPC - 56

Sensor networks and source-channel coding[GastparV:03/04]

Consider the problem of sensing• one sourceof analog informat ion• many sensors• reconstruct an est imate at the base stat ion

Model: The CEO problem [Berger et al]

Question: distr ibuted source compression and mult iantenna transmission or uncoded transmission?

Y2

Y1

UM

g

hX

W1

W2

WMYM

U2

U1

fM

f2X

f1

DSPC - 57

The Gaussian Example

• Gaussian mult iaccess channel• assume synchronizat ion for now

Source

W1

W 2

W M

U1

U2

UM

X 1

X2

XM

F1

F2

FM

GS SY

Z

DSPC - 58

Example: Gaussian source σσσσs2, Gaussian noise σσσσw

2

Performance (cst or poly. growing power shared among sensors):

• wi th uncoded transmission:

• wi th separat ion: !

Exponent ia l subopt imal i ty!

Condition for optimality: measure matching!• • I (S,S) = I (X; Y1, Y2, . . . , YM)

Can be generalized to many sources X1, X2, . . . , XN

W1

W2

WM

Y

XM

X1

W

hX

f1

X2

YM

f2

fM

gX

Y2

Y1

MSE O 1M----- =

MSE O 1Mlog

------------- =

d X X,( ) p X X⁄( )log–=

DSPC - 59

I t is the best one can do:

Communication between sensors does not help as M grows!

Intriguing remark:• by going to ‘ ’b i ts ’’ , MSE went f rom 1/M to 1/Log(M)• ‘ ’b i ts ’’ might not be a good idea for distr ibuted sensing and

communicat ions

I f not ‘ ’bits’’ , what is information in networks? [Gastpar:02]

W1

W2

WM

Y

XM

X2

YM

X1

Y2X

g

Y1

W

hX

f1

f2

fM

DSPC - 60

On going work on “analog” sensor networks

Multiple sources• L sources, L channel usages• sources to sensors matr ix A

Multiple sources and sinks• sensors to s inks matr ix B

‘ ’Physical’’ process: degrees of freedom, sampling

Robustness to fading, synch loss (Rician fading)

src

S1

S2...

SL

A

W1

U1

W2

U2

WM

UM

F1

X1

F2

X2

...

FM

XM

B

Z1

Y1

...

ZN

YN

G

S1

S2...

SL

des

DSPC - 61

An intermezzo.. . .

How about computing the DKLT with uncoded transmission?• g iven is R, the autocorrelat ion of the vector process• the KLT is given by a matr ix T i j• the eigenvalues at λ j j=0. . .N-1• the channel is the mult iaccess channel ( for s impl ic i ty: uni t gain

from al l sensors to BS)

Simple scheme:• waterfi l l ing indicates one should keep λ0 to λk-1 only• use K t ime slots where at t ime i sensor j sends i ts value

weighted by Ti j comput ing impl ic i te ly the i - th KLT coefficient• BS gets

This can be shown• to be opt imal in cer tain cases and in the scal ing sense

yi Tij Uj Wj+( )⋅j 0=

K 1–

∑ Wbs+=

DSPC - 62

3.4 The world is physical: Interaction of physics with signal processing and communications

[Beferull-Lozano, Konsbruck, V04]

Consider a real physical phenomena:• temperature distr ibut ion on a c i rcular rod (Four ier ’s c lassic problem)• heat equat ion• N wireless temperature sensors• reconstruct ion of temperature field at basestat ion

BS

DSPC - 63

Physics• Heat source: BL Gaussian g(x, t )• loss through convect ion µ• d i f fusion equat ion

• boundary condi t ion (per iodic) and in i t ia l condi t ions (0 at )

x2

2

∂

∂ T x t,( ) g x t,( )k

---------------- µT x t,( )–+t∂∂ T x t,( )

α----------------=

∞–

BS

x

2L02L

2M 1+-----------------

DSPC - 64

i t is al l in the Green function!

2D LTI system• c i rcular convolut ion in space• regular convolut ion in t ime• heat kernel H(Ωx,Ω t)• dr iv ing funct ion wi th power spectrum Sg(Ωx,Ω t)• resul t ing power spectra of temperature ST(Ωx,Ω t)

G0

Sg Ωx Ωt,( )

Ωt

πΩx

G1G3

G2

b0

H Ωx Ωt,( )2

Ωt

πΩx

b1b2b3b4

A0ST Ωx Ωt,( )

Ωt

πΩx

A1A2 A3

DSPC - 65

Example

Diracs of heat on the circular rod, random points and t imes

0

0.05

0.1

0.15

0.2

0

1

2

3

4

0

50

100

150

200

250

x (spatial position)

time

T(x,t)

DSPC - 66

Computing Rate-Distor tion:

it is al l about waterfi l l ing• in space, discrete-t ime Four ier ser ies• in t ime, discrete-t ime Four ier t ransform• pick a water level Θ . . . .

•

A0

ST ωx ωt,( )

ωt

π

ωx

A1A2

A3

Sβ0ωt( )

θ

D0 θ( )

DSPC - 67

Central ized rate-distor tion Rc(D)

Compute the “joint” rate-distor tion of the sensor vector process• ideal or best case• lower bound for any distr ibuted scenar io• method: space-t ime waterfi l l ing (equal s lopes)• Four ier domain solut ion

0 10 20 30 40 500

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2x 10

−8

Rate (nats/s)

Dis

tort

ion

(MS

E/s

)

SilverAluminiumIron

DSPC - 68

Distributed coding approaches

1. Independent coding, Ri(D)• ignore spat ia l and temporal correlat ion, worst case

2. Sensors ignore spatial correlation, Ris(D)• rate-distor t ion of temperature process

3. The basestation runs the heat equation, and sends back its prediction, Rfb(D)

• rate-distor t ion wi th feedback

4. Wyner-Ziv at high rates, RWZ(D)• s imi lar to predict ion, but wi thout need for feedback

DSPC - 69

Comparison of various scenarios

Distor tion versus rate

10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1x 10

−8

Rate (nats/s)

Dis

tort

ion

(MS

E/s

)centralizedspatially and temporally independenthigh−rate Wyner−Ziv codingspatially independenthigh−rate DPCM with local predictionhigh−rate DPCM with feedback

DSPC - 70

On-going work on “physics” based distributed sensing

Heat problem• gener ic case based on PDE• can be analyzed in great detai l• c lear ly maps out potent ia ls and l imi tat ions of d istr ibuted

approaches to sensing of physical phenomenasOther cases of interest

• Plenopt ic funct ion and distr ibuted camera systems(geometry at work)

• Plenacoust ic funct ion and microphone arrays(wave equat ion)

Related work• d istr ibuted compression appl ied to v ideo coding,

PRISM, Ramchandran, UCB

DSPC - 71

Outline

1. Introduction

2. Distributed Signals and Sampling

3. On the interaction of source and channel coding

4. Environmental monitoring• monitor ing for scient ific purposes

- wind tomography• monitor ing for intel l igent bui ld ings

- SensorScope

5. Conclusions

DSPC - 72

4. The case for environmental monitoring (MICS applications)

4.1 Monitoring for scientific purposes• ‘ ’create’’ a new instrument for cr i t ical data• most current aquis i t ions are undersampled• ver ificat ion of theory, s imulat ions

Environmental data• unstable terrain, g laciers• watershed monitor ing, Common Sense Project ( IP4 & I ISc)• pol lutant monitor ing• forest CO2 monitor ing

• Example: UCLA CENS. environmental monitor ing

University of Baselcanopy sensing andactuating

DSPC - 73

4.2 Environmental monitoring with societal impact• pol lut ion a major societal concern• communit ies and NGO’s are interested• potent ia l of h igh v is ib i l i ty and impact• legal requirements, regulat ions. . . .

Examples:• NO2• Ozone

How about a real-t ime map on the web?

Lausanne400 sensorsNO28 weeks 2x/Ymanual....

DSPC - 74

4.3 Monitoring for intel l igent buildings, commercial applications• energy management• comfor t issues• secur i ty• costs of wir ing, energy sources

Note: modern buildings have already several 1000’s sensors/actuators

MICS Project: CAAD Lab and IP8

Industrial interest• b ig players ( Intel , Siemens, etc)• star tups (e.g. local ly: IP01, Adhoco, Shockfish.. . )

DSPC - 75

An Example: Wind Tomography (L.Sbaiz, IP7)

The sensor network in a distributed measurement device• N sensors lead to a sampl ing of the f ie ld of interest in N points• Is there a way to get more informat ion out of N measurement points?

Idea: tomography, with scaling law:• N measurement points• reconstructed values

• acoust ic anemometer measurement f ie ld

O N2( )

Γ

DSPC - 76

Tradit ional Tomography and Flow Tomography

receivers: measurement is integral along straight line

=> Radon transform

receivers: measurement influenced by unknown path

Acoustic measurement

• delay ( t ime of f l ight)• d i rect ion of arr ival

Source(e.g. X-ray)

Source

Wind tomography:The path integral is part of the unknowns

DSPC - 77

Results

Lagrange tr iangle tesselation. 16 stations, maximum wind speed 380 Km/h

«green»= true field/trajectories, «blue»= estimate field/trajectories

Convergence in 4 iterations

−10 −5 0 5 10

−10

−5

0

5

10

−2 −1 0 1 2 3 4 5

−3

−2

−1

0

1

2

3

DSPC - 78

5. The SensorScope Project (H.Dubois-Ferriere, T.Schmid)http:/ /sensorscope.epfl .ch

What are we trying to accomplish?

Goal: build a new scientific instrument to see the world by leveraging• advances in sensor network technology• advances in s ignal processing and communicat ions• advances in in networking and database technologies

with real impact on• scient ific quest ions• appl icat ions

The conceptual tool:

SensorScope: • d istr ibuted sensing instrument • re levant datasets wi th c lear documentat ion• al l data on- l ine• anybody can compute/analyze with

DSPC - 79

What can be used?

Sensor nodes:• many possible plat forms inc. low power• many types of sensing (e.g. cyclops)

Signal processing• advances in sampl ing, d istr ibuted signal processing

Communication• capaci ty, MAC and MAC mechanisms

Networking• robust rout ing

Distributed Databases

Data analysis• large scale datasets

Software and Web• open source, t inyOS, on- l ine and open format

All stuff from MICS!

DSPC - 80

Welcome to the new building!

DSPC - 81

The network today

DSPC - 82

• on- l ine, growing

DSPC - 83

Reliabil i ty: working on it!

• th is is standard best-ef for t rout ing. • current ly higher rel iabi l i ty, thanks to:

- hop-by-hop retransmissions and packet combining • no more big blackouts, the system is now qui te stable.

DSPC - 84

SensorscopeToday

Motes:• 6 Berkeley mica2• 3 sensorboards for mica2 with l ight / temp/sound/accel /magnet ic sen-

sors• 15 Berkeley mica2dot• 6 sensorboards for mica2dot wi th l ight / temp/sound/accel sensors

Server:• Java Middleware which stores incoming data to postgresql database• Apache webserver wi th PHP extension to present data• Python scr ipts to generate graphs and send out SMS on di fferent

events (e.g. i f motes have low battery vol tage)

Status• smal l but working• on l ine!

DSPC - 85

5. Conclusions

There are some good questions on the interaction of• physics of the process: space of possible values• sensing: analog/digi ta l• representat ion & compression: local /g lobal• t ransmission: separate/ jo int• decoding & reconstruct ion: appl icat ions

From joint source-channel coding to source-channel communication• This goes back to Shannon’s or ig inal quest ion,

but mult i -source mult i -point communicat ion is hard. . .• and we did not even throw networking into the picture!

On-going basic questions:• are there some fundamental bounds on cer tain data sets?• are there pract ical schemes to approach the bounds?

Many interesting and open problems!

but remember: you might have to forget about ‘ ’bits’’ . . .

DSPC - 86

Is communications engineering on the way to becoming the next "power engineering",

• i .e. , most ly a commodity wi th l i t t le need for fur ther research?

Where and how does the performance of communication systems ben-efit from general theories,

• as opposed to the isolated analysis of very specific and l imi ted mod-els or exper imental observat ions? ( i f at a l l )

Is data rate performance the only thing that matters?

What is relevant, fixed _area_ or fixed _density_ scaling?

On the one hand, we view Moore's law as a l icense to study the high-complexity l imits, while the hardware community often seems to sees the excit ing future in the low-complexity but small-size or low-cost l imits.

• Where do you see the fundamental contr ibut ions ofcommunicat ions theory coming from in the low complexi ty regime?

What is the broader impact of information and communication theory research beyond the area of communications engineering?

DSPC - 87

Thank you for your attention!

the plenoptic function from my window.. . .

Questions?

DSPC - 88

References

On the NCCR-MICS: http:/ /www.mics.org: al l papers on l ine• J.-P. Hubaux, T. Gross, J.-Y. Le Boudec, M. Vetter l i . Toward sel f -orga-

nized mobi le ad hoc networks: the terminodes project , IEEE Commu-nicat ions Magazine, Jan. 2001.

On sampling• M. Vetter l i , P. Marzi l iano, T. Blu. Sampl ing s ignals wi th finite rate of

innovat ion. IEEE Tr. on SP, Jun. 2002.• I . Maravic, M. Vetter l i , "A Sampl ing Theorem for the Radon Transform

of Fini te Complexi ty Objects", in Proc. ICASSP, May 2002.• A.Chebira, P.L.Dragott i , L.Sbaiz, and M.Vetter l i , Sampl ing the ple-

nopt ic funct ion, IEEE ICIP 2003.• T. Ajdler, L.Sbaiz and M. Vetter l i , The plenacoust ic funct ion and i ts

sampl ing, IEEE Tr on SP, Submit ted

Correlated source coding• R.Cr istescu, B.Beferul l and M.Vetter l i , Correlated data gather ing,

Infocom2004.• M. Gastpar, P. L. Dragott i , and M. Vetter l i . The distr ibuted Karhunen-

Loeve transform. IEEE Tr. on IT, submit ted.

DSPC - 89

Uncoded transmission, relays, and sensor networks

• M. Gastpar, B. Rimoldi , M. Vetter l i . To code or not to code: lossy source-channel communicat ion revis i ted, IEEE Tr. on IT, to appear.

• M. Gastpar and M. Vetter l i . On the capaci ty of wireless networks: The relay case. In Proc IEEE Infocom 2002.

• M. Gastpar and M. Vetter l i , Source-channel communicat ion in sensor networks, IPSN 2003.

• M. Gastpar, To Code or not to Code, PhD Thesis, Communicat ion Systems, EPFL, Fal l 2002.

Physics based distributed sensing• B. Beferul l -Lozano, R.Konsbruck and M. Vetter l i , Rate-Distor t ion

Problem for Physics Based Distr ibuted Sensing, IEEE ICASSP, Spe-cial Session on Distr ibuted Signal Processing for Sensor Networks, May 2004, Montreal , Canada.

• B. Beferul l -Lozano, R. Konsbruck, M. Vetter l i , Rate-Distor t ion Prob-lem for Physics Based Distr ibuted Sensing, Internat ional Symposium on Informat ion Processing in Sensor Networks, ( IPSN '04), Berkeley, 26-27 Apr i l 2004.

Flow Tomography• L.Sbaiz and M.Vetter l i , Acoust ic Flow Tomography, TR, 2004