Rice Universitydsp.rice.edu/cs
DistributedCompressiveSamplingA Framework forIntegrated Sensing and Processingfor Signal Ensembles
MarcoDuarte
ShriramSarvotham
MichaelWakin
DrorBaron
RichardBaraniuk
DSP Sensing• The typical sensing/compression setup
– compress = transform, sort coefficients, encode– most computation at sensor (asymmetrical)– lots of work to throw away >80% of the coefficients
compress transmit
receive decompress
sample
• Measure projections onto incoherent basis/frame– random “white noise” is universally incoherent
• Reconstruct via nonlinear techniques• Mild oversampling:• Highly asymmetrical (most computation at receiver)
project transmit
receive reconstruct
Compressive Sampling (CS)
CS Reconstruction• Underdetermined• Possible approaches:
• Also: efficient greedy algorithms for sparseapproximation
wrong solution(not sparse)
right solution,but not tractable
right solution and tractableif M > cK (c ~ 3 or 4)
Compressive Sampling• CS changes the rules of the data acquisition game
– changes what we mean by “sampling”– exploits a priori signal/image sparsity information
(that the signal is compressible in some representation)– Related to multiplex sampling (D. Brady - DISP)
• Potential next-generation data acquisition– new distributed source coding algorithms for
multi-sensor applications– new A/D converters (sub Nyquist) [Darpa A2I]
– new mixed-signal analog/digital processing systems– new imagers, imaging, and image processing algorithms– …
• Longer signals via “random” transforms• Non-Gaussian measurement scheme
• Low complexity measurement• (approx O(N) versus O(MN))
– universally incoherent• Low complexity reconstruction
– e.g., Matching Pursuit
– compute using transforms(approx O(N2) versus O(MN2))
Permuted FFT (PFFT)
FastTransform(FFT, DCT,
etc.)
Truncation(keep Mout of N)
PseudorandomPermutation
Reconstruction from PFFTCoefficients
Original65536 pixels
Wavelet Thresholding6500 coefficients
CS Reconstruction26000 measurements
•4x oversampling enables good approximation•Wavelet encoding requires
–extra location encoding + fancy quantization strategy
•Random projection encoding requires –no location encoding + only uniform quantization
Random Filtering[with J. Tropp]
• Hardware/software implementation
• Structure of– convolution Toeplitz/circulant– downsampling keep certain rows– if filter has few taps, is sparse– potential for fast reconstruction
• Can be generalized to analog input
Downsample(keep M out
of N)
“Random” FIR Filter
Time-sparse signals N = 128, K = 10
Fourier-sparse signalsN = 128, K = 10
Rice CS Camera
single photon detector
randompattern onDMD array
(see also Coifman et al.)
imagereconstruction
• Sensor networks:intra-sensor andinter-sensor correlation
dictated by physical phenomena• Can we exploit these to jointly compress?• Popular approach: collaboration
– inter-sensor communication overhead– complexity at sensors
• Ongoing challenge in information theorycommunity
Correlationin SignalEnsembles
Benefits:• Distributed Source Coding:
– exploit intra- and inter-sensorcorrelations
⇒ fewer measurements necessary– zero inter-sensor
communication overhead
DistributedCompressive Sampling (DCS) compressed
data
destination(reconstruct jointly)
•Compressive Sampling:–universality (random projections)–“future-proof”–encryption–robustness to noise, packet loss–scalability–low complexity at sensors
Joint sparsity modelsand algorithms fordifferent physicalsettings
JSM-1: Common + Innovations Model
• Motivation: sampling signals in a smooth field
• Joint sparsity model:– length- sequences and
– is length- common component
– , length- innovation components– has sparsity
– , have sparsity ,
• Measurements
• Intuition: Sensors should be able to “share theburden” of measuring
JSM-2: Common Sparse Supports
• measure J signals, each K-sparse• signals share sparse components,
different coefficients
…
• Joint sparsity model #3 (JSM-3):– generalization of JSM-1,2: length- sequences⇒ each signal is incompressible– signals may (DCS-2) or may not (DCS-1) share sparse supports
• Intuition: each measurement vector contains clues about thecommon component
JSM-3: Non-Sparse Common Model
…
DCS Reconstruction• Measure each xj independently with Mj random
projections
• Reconstruct jointly at central receiver
“What is the sparsest joint representation thatcould yield all measurements yj?”
linear programming: use concatenation ofmeasurements yj
greedy pursuit: iteratively select elements ofsupport set
similar to single-sensor case, but more cluesavailable
Theoretical Results
•Mj << (standard CS results); furtherreductions from joint reconstruction
• JSM-1: Slepian-Wolf like bounds for linearprogramming
• JSM-2: c = 1 with greedy algorithm as Jincreases. Can recover with Mj = 1!
• JSM-3: Can measure at Mj = cKj,essentially neglecting z; use iterativeestimation of z and zj. Would otherwiserequire Mj = N!
JSM-1: Recovery via Linear Programming
JSM-2 SOMP ResultsK=5N=50
SeparateJoint
One Signal from Ensemble
Experiment: Sensor Network Light data with JSM-2
JSM-3 ACIE/SOMP Results(same supports)
K=5N=50
Impact of vanishes as