spie proceedings [spie spie defense, security, and sensing - baltimore, maryland, usa (monday 29...

Tree-Based Adaptive Measurement Design for CompressiveImaging Under Device Constraints

David Bottisti and Robert Muise

Lockheed Martin - Missiles and Fire Control, 5600 Sand Lake Rd., MP 450, Orlando, FL 32819

ABSTRACT

We look at the design of projective measurements for compressive imaging based upon image priors and deviceconstraints. If one assumes that image patches from natural imagery can be modeled as a low rank manifold,we develop an optimality criterion for a measurement matrix based upon separating the canonical elementsof the manifold prior. We then characterize this manifold based upon prior training imagery under a tree-based framework which can be implemented adaptively. We also illustrate how these adaptive measurementscan incorporate prior knowledge regarding the constrains of the device being used to collect the measurements.Simulated performance results are presented and compared against a standard imaging paradigm as well as moreconventional compressive imaging techniques.

Keywords: Compressive Imaging, Compressive Measurements, Sparse Representation, Device Constraints

1. INTRODUCTION

Much of the compressive sensing literature1–3 that has been published to date focuses on the task of determiningan optimal measurement set under a given set of constraints. These measurements are usually assumed to betaken simultaneously. However, in practice, to take m simultaneous measurements of a scene, the optics pathwould need to be split into m separate streams. This leads to a degradation of the optics path and hence isinfeasible for any application requiring more than a handful of measurements. For a tutorial on compressivesensing, the reader is directed to Ref. 4.

To alleviate this shortcoming, many authors have proposed that measurements are to be taken sequentiallyover the scene.5,6 Providing these measurements can be taken in fast enough succession, this appears to be anacceptable workaround. Often times these measurements are simply queued up in a processing stream until allthe measurements for a particular scene have been sensed. In this paper we propose a sensing paradigm thatdoes not ignore the value of each measurement during the sensing process, but rather uses the measured valuesto adaptively control successive measurements, leading to a greater signal-to-noise ratio.

The rest of this paper is organized as follows. In Sec. 2 we introduce notation to be used in the rest of thedocument, as well as present some useful mathematical background of the compressive sensing problem. Section3 introduces our measurement scheme and describes how this measurement scheme can be used to sense andsubsequently reconstruct a set of image patches. We then present our simulated results in Sec. 4. In Sec. 5 wediscuss some possible extensions to our framework and future work planned. Lastly we conclude in Sec. 6.

2. MATHEMATICAL BACKGROUND

Let us assume that a given image X can be represented by a collection of non-overlapping image patches xi ofsize

√n ×

√n. The sizing of these patches is arbitrary and need not be square, but in practice we have found

8×8, 16×16 and 32×32 image patches to be convenient. Without loss of generality we assume that each imagepatch xi has been ordered lexicographically, and will drop the subscript when no ambiguity is possible.

This work is based upon work supported by DARPA and the SPAWAR System Center Pacific under Contract No.N66001-11-C-4092. The views expressed are those of the author and do not reflect the official policy or position of theDepartment of Defense or the U.S. Government.

Invited Paper

Optical Pattern Recognition XXIV, edited by David Casasent, Tien-Hsin Chao, Proc. of SPIE Vol. 8748, 874802 · © 2013 SPIE · CCC code: 0277-786X/13/$18 · doi: 10.1117/12.2015453

Proc. of SPIE Vol. 8748 874802-1

Downloaded From: http://proceedings.spiedigitallibrary.org/ on 10/02/2013 Terms of Use: http://spiedl.org/terms

We say that an image is compressible if, for some basis Ψ, the image can be represented as x ≈ Ψc, wherethe coefficient vector c is sparse. Assuming that Ψ is invertible, one can measure the coefficients c as c = Ψ−1x.To approximate c we set all but the k largest magnitude of c to 0. In other words,

ci =

{ci if |ci| >= t

0 otherwise(1)

for 1 ≤ i ≤ n where t is the mth largest magnitude coefficient of c. Here m controls the compression factor andhence the quality of the resulting reconstruction. Having found c, one can reconstruct the approximation to theoriginal image x = Ψc.

The task of compressive sensing aims to find the coefficient vector c without knowing the vector x a priori.Since calculation of c via (1) requires knowledge of c and hence x, this method cannot be used directly withincompressive sensing. This is the fundamental task of compressive sensing: careful design of the measurementand reconstruction operators Ψ∗ and Ψ, respectively, such that

c = Ψ∗x (2)

andx = Ψc. (3)

One technique for the design of Ψ∗ and Ψ is to set Ψ to be a subset of the columns of the discrete cosinetransform (DCT) matrix. Typically the m columns chosen correspond to the low-frequency components of theDCT, since this is where most of the information in an image is usually contained. Then, since the columns ofthe DCT matrix are orthonormal, we can set Ψ∗ = ΨT . Hence, the coefficients found by c = ΨT x are a subset ofthe DCT coefficients of the image x, and the reconstruction can be approximated by x = Ψc. Other techniquesfor the choice of Ψ∗ and Ψ involve finding an ideal basis given a set of data using a method such as K-SVD,7 andsolving the optimization using Orthogonal Matching Pursuit (OMP)8 or other L1 minimization techniques.9

3. TREE-BASED ADAPTIVE MEASUREMENTS

As described above, the measurement matrix Ψ∗ is typically designed such that each row is applied to each imagepatch without any regard to measurements previously taken. However, we hypothesize that each measurementtaken gives us a glimpse into the structure of the underlying image patch, and can therefore be used to helpsteer future measurements in a way that makes them more meaningful. To this end, we design a tree-basedmeasurement scheme where the decision of which measurement to take next sequentially is dictated by themeasurements taken thus far.

3.1 Building the Tree

Given a set of√

n×√

n training image chips, ordered lexicographically into n-dimensional vectors and concate-nated into an n×t matrix T , we can compute the first principal component v of the matrix T , normalized to unitlength. By its nature, this vector points in the direction of highest variance of the data, provided the columnsof T have zero-mean. However, we intentionally do not zero-mean the columns of T , so v will point towardsthe average of all of the data. The vector v is also the axis which provides the most ”information” about theunderlying data and thus we use it as our first measurement, i.e., vT becomes the first row of Ψ∗.

Having chosen our first measurement vector, we proceed by finding the second and subsequent vectors. Thevector v not only gives us the dimension of highest variability of the data, the coefficients c given by the projectionc = vT T give an implicit ordering of the data. This ordering can be used to categorize the data into subsets Ts

where s is the index of the subset. This 1 − dimensional categorization can be performed using any clusteringtechnique such as k −means or by simply comparing the coefficients against a threshold. In practice, we havefound the median of the coefficients c gives an easy to calculate method to bi-partition the data set.

After clustering the data into 2 or more subsets, we proceed recursively by calculating the measurementvector (and corresponding coefficient threshold) of each of the subsets. However, if left unaltered, these data sets



Figure 1. An example of several levels of the measurement tree created by our training process. Here each measurementvector has been reshaped into a square image for illustrative purposes.

would continue to produce measurement vectors projecting in directions overlapping with previously computedmeasurement vectors. We wish to enforce orthogonality among the measurement vectors, so the measurementvector v is projected off of each subset Ts before proceeding to the next level of recursion. Let ρ be an indexvector whose ith entry ρi indicates which cluster the ith data vector Ti belongs to. Then each subset is computedby

Ts = {Tj − vT Tjv such that ρj = s} (4)

for each cluster s.

We represent the measurement vectors and corresponding threshold values in a tree-based structure. Providedthat the data at each recursive level is bi-partitioned into equal sub-sets, this tree is a balanced binary tree. Ateach node, we store the measurement vector v as well as the threshold value τ . The node also includes references(either in the form of a pointer or an index) to the two child nodes, if they exist. Without loss of generality, weassume that the left child represents those data points whose response to the measurement vector compares lessthan or equal to the threshold, i.e., vT x ≤ τ . Likewise, the right child are those data points comparing greaterthan τ . See Fig. 1 for an example of the measurement vectors for several layers of a built tree.

3.2 Measuring Using the Tree

Prior to discussing the specific manner in which our tree can be used to collect data measurements, we firstdiscuss one possible technique whereby multiple compressive measurements of a scene may be measured. InFig. 2 we see a graphical representation of how such a technique can be used. First, the scene is mapped tothe elements of a 4096× 2048 pixel spatial light modulator (SLM) such as a digital micro-mirror device (DMD)of the same size. The number of element within the SLM directly relates to the number of pixels in the finalreconstructed image. In addition, the elements of the SLM are grouped into blocks of the same size, in this case32×32. Onto each of these blocks is encoded the mask that is currently being sensed. The scene is projected ontothe SLM (step 1) which then will reflect some of the light (depending upon the current mask) onto a detectorarray (step 2). Each block of the SLM is aligned with one single detector element where the light reflected offthe SLM is integrated into a single readout (step 3). The final result is a set of 128×64 measurements, the valueof which is the dot product of the scene intensities and the mask for each 32 × 32 region in the original scene.Collecting multiple measurements temporally is used to build up a measurement set to be later decoded.

Once our tree has been built, it is a simple matter to collect the measurements (coefficients) of imagery forlater reconstruction. We start at the root node and project the image patch x onto the node’s measurementvector to determine the first coefficient, c1 = vT x. This coefficient is stored for later transmission as our encodeddata stream. The value of this coefficient is compared against the node’s threshold value τ . The result of this



1)Aperture images scene on to SLM

32 x 32region

2) Re- imaging opticsrelays coded imagefrom SLM to detectorarray

A32x 32 mask is repeated across the SLM

iiIiiis*ilii41444ii4ï4iï........... .............................iïiï........... .....4444..ï.44iïiï........... ...........

........... ...........

4i4iï4iï4ï........... ...................A 128 x 64 detector array is read outfor every mask. Several masks neededfor each frame.

3) Relatively largedetectors collectlight from a groupof SLM elements

Det,rtU i

Figure 2. A typical mechanism whereby compressive measurements of a scene can be collected.

comparison determines if the measurement path should proceed down the left or right child. (This descriptionassumes the tree is binary. Tertiary or even variable partitioning schemes could easily be incorporated into ourframework with the appropriate modifications to the thresholds τ .)

A more mathematical analysis of the measurement process is in order. Assume that the entire path down themeasurement tree is known a priori. Then we can construct (Ψ∗)T as the collection of measurement vectors valong that path, and then use Ψ∗ to compute our coefficient vector c = Ψ∗x. Hence, our tree-based measurementprocess is analogous to the traditional measurement scheme described in (2), albeit with a measurement matrixthat is adaptively tuned to the data already sensed.

It is important to note that only the values of the measurements are transmitted as part of the data stream.Since the path down the measurement tree is determined entirely by the coefficient values, this information caneasily be inferred during reconstruction without the need to communicate additional path information as partof the data stream.

3.3 ReconstructionHaving received the stream of coefficients for a particular image patch, we can now reconstruct an approximationto the original data set. As described above, our measurement process is analogous to the traditional measurementscheme, and similarly our reconstruction scheme can be computed using (3), providing the path down themeasurement tree is known. Using the fact that the measurement vector Ψ∗ for each possible path is orthonormal,we have Ψ = (Ψ∗)T . Thus our reconstruction operator is simply the transpose of the measurement operator.

While we can easily determine the path down the tree taken during measurement from the set of coefficientsreceived, and build the matrix Ψ column-by-column, we instead choose to combine the tree traversal and recon-struction together. Starting at the root node, we initialize our reconstructed x = v ∗ c0 where v is the current



node’s measurement vector. Comparing the coefficient c0 with the current node’s threshold τ we determinewhich child node to traverse next. Recursively, x is accumulated as

xi = xi−1 + v ∗ ci

x0 =−→0

where xi represents the value x at the ith level of the tree recursion and ci is the ith coefficient transmitted.

3.4 Device ConstraintsThe hierarchical design of our measurement scheme was designed specifically with the understanding that realcompressive sensing devices will not be able to take multiple measurements simultaneously. However, mostdevices will in fact have additional constraints including, but not limited to, measurements being limited toseveral quantization levels (i.e., 12-, 8- or even 1-bit), measurement noise, and the inability to capture bothpositive and negative measurements at the same time.

Many of these device constraints can easily be incorporated into our framework due to the hierarchical natureof its training. One method of accomplishing this is to modify the measurement vectors at the end of the trainingso as to fit the desired device constraints. However, this violates the assumption held during tree construction:specifically that the subsets created when training a particular node are clustered based upon their response tothe measurement vector. By changing the measurement vector after training, this fact no longer holds true.

An alternative approach is to adapt the measurement vectors at the time of training. By doing so at eachlevel of the build (before the next subsets have been determined), one can then use these modified measurementvectors for classification, thus not violating the assumption described above. Training continues as normal withthis one minor adjustment.

We have so far not addressed the question of how to modify the measurement vectors. Since the originalvectors were identified by principal component analysis (PCA), any modification of these vectors will violate theproperty of maximum variance. While this is unavoidable, the side-effects of such a change can be minimized.By performing a search within the feasible region surrounding the ideal measurement vector, the feasible vectorwith the most ideal properties can be chosen. This search can be stochastic, gradient-descent based, or evenincorporated into the PCA directly (using a modified version of the power-method). We present examples ofsome of our work incorporating device constraints into the measurement process in Sec. 4.

4. RESULTS

To test our approach, we conduct several simulations using various test images and device constraints. For eachtest, our algorithm was trained on overlapping chips taken from the same set of training images∗ shown in Fig.3. For each trial conducted, the size of the training chips were 8 × 8 pixels. Samples were collected from, andcompared against, and 50-frame test sequence, several frames of which are shown in Fig. 4.

In our first set of experiments, training was performed without restricting the measurement vectors. Since themeasurement vectors are the results of PCA, this means that each one has unit length and thus for each elementvi of the measurement vector, vi ∈ [−1, 1]. Experiments were performed varying the number of measurements Mcollected for each frame. We set our compression ratio N

M by varying M while N = 64 for all of our experiments.

As a control, we compare our results against a simple DCT compression scheme with the same compressionratio. This scheme involves computing the top M DCT coefficients for each chip and reconstructing the imageas if only these coefficients had been measured. Results of this experiment can be seen in Table 1. As can beseen, our results come very close to the control DCT compression without having the advantage of knowing thestructure of the sparse vector a priori. The same results are presented in graphical form in Fig. 5. Notice theasymptotic behavior of our algorithm after about 16 measurements indicating that our algorithm performs bestrelative to the DCT for high compression ratios. A sample reconstructed frame using our algorithm with thesemeasurements can be seen in Fig. 6.

∗Imagery presented in this paper are from the CLIF2007 dataset; DISTRIBUTION STATEMENT A. Approved forpublic release; distribution is unlimited.



"VI rJraim, "WM/r 11» 111 7J

11r 01/t01 1 ,. . 110,110/ 10 r0t Ñ0 =

Figure 3. The source images of all training chips used for our experiments.

Figure 4. A subset of frames from the 50-frame test video.

M Compression Ratio (N/M) Average SNR of our Algorithm Average SNR using DCT1 64 17.59 17.592 32 19.22 19.984 16 22.42 22.585 12.8 23.29 23.578 8 25.48 25.5510 6.4 26.67 26.7216 4 28.02 29.8920 3.2 28.05 31.83

Table 1. Comparative results of our algorithm using ideal (unrestricted) measurements verses the DCT compression forvarious compression ratios.



34

32

30

28

26

24

22

20

18

16e

Experienental Results with Ideal Measurements

5 10 15

Nunber of Neasurenents (N)

20

Figure 5. Graphical representation of the results presented in Table 1.

M Compression Ratio (N/M) Average SNR of our Algorithm Average SNR using DCT1 64 17.59 17.592 32 18.92 19.984 16 20.96 22.585 12.8 21.28 23.578 8 21.29 25.5510 6.4 21.29 26.7216 4 21.35 29.8920 3.2 21.41 31.83

Table 2. Comparative results of our algorithm using binary (restricted) measurements verses the DCT compression forvarious compression ratios.

We repeated this experiment using a tree designed for binary measurements (i.e., vi ∈ {−1, 1}). A hierarchicalrepresentation of some of the measurements taken is shown in Fig 7. These results are shown in Table 2 andFig. 8. We compare the results with the same DCT compression scene described above. As expected, usingbinary measurements results in a lower SNR then the unrestricted measurements of our previous experiment.This can also be seen by comparing figures 6 and 9. Notice also that the asymptote of this experiment occurs atfewer measurements. This can be explained by the fact that our measurements (as seen in Fig. 7) do not varysignificantly after about the fourth measurement. We believe that this can be improved by employing a morecarefully designed feasibility search within our algorithm; an area for future study.



Sr'

Figure 6. A frame reconstructed by our algorithm using ideal (unrestricted) measurements. The corresponding truth forthis frame can be seen as the last frame in Fig. 4.



34

32

30

28

26

24

22

20

18

16e

Experienental Results with Binary Measurenents

5 10 15

Nunber of Measurenents (M)

20

Figure 7. A portion of the measurement tree created by restricting measurements to a feasible set.

Figure 8. Graphical representation of the results presented in Table 2.



Figure 9. A frame reconstructed by our algorithm using binary (restricted) measurements. The corresponding truth forthis frame can be seen as the last frame in Fig. 4.

Proc. of SPIE Vol. 8748 874802-10


5. FUTURE WORK

Our approach contains many possible avenues for extension. As mentioned earlier, we are currently using asimple median computation to find the data subsets at each level of the tree. We plan to explore alternativeclustering approaches such k-means or a dynamic programming approach to finding the maximum separationbetween clusters. Additionally, the tree-based structure does not need to be strictly binary. In fact, the numberof child nodes for each parent node need not be constant. This opens up other possible clustering techniques forfuture exploration.

Another area of interest is to study the effects of restricting the measurement vectors to a fixed dictionary.This dictionary could be a basis set such as the DCT or one of the wavelet basis, or it could be an over-completedictionary such as the over-complete DCT or the dictionary created by the K-SVD7 algorithm.

6. CONCLUSION

In this paper we presented a novel approach to finding a measurement paradigm for compressive sensing that isnot only adaptive but requires very little overhead during sensing. We have demonstrated how the reconstructionfound by using our technique is competitive in quality to DCT compression having the entire data set known apriori.

REFERENCES[1] Candes, E., Romberg, J., and Tao, T., “Robust uncertainty principles: exact signal reconstruction from

highly incomplete frequency information,” IEEE Transactions on Information Theory 52, 489–509 (Feb.2006).

[2] Candes, E. J. and Tao, T., “Near-Optimal Signal Recovery From Random Projections: Universal EncodingStrategies?,” Information Theory, IEEE Transactions on 52, 5406–5425 (Dec. 2006).

[3] Donoho, D. L., “Compressed sensing,” IEEE Trans. Inform. Theory 52, 1289–1306 (2006).[4] Baraniuk, R., “Compressive sensing,” IEEE Signal Processing Mag , 118–120 (2007).[5] Bottisti, D. and Muise, R., “Image exploitation from encoded measurements,” Proceedings of SPIE 8165,

816518–816518–9 (Sept. 2011).[6] Muise, R. and Bottisti, D., “Compressive imaging measurement design from an image patch manifold prior,”

Proceedings of SPIE 8399, 839905–839905–8 (May 2012).[7] Aharon, M., Elad, M., and Bruckstein, A., “K-SVD: An Algorithm for Designing Overcomplete Dictionaries

for Sparse Representation,” Signal Processing, IEEE Transactions on 54, 4311–4322 (Nov. 2006).[8] Tropp, J. A., Anna, and Gilbert, C., “Signal recovery from random measurements via orthogonal matching

pursuit,” IEEE Trans. Inform. Theory 53, 4655–4666 (2007).[9] Candes, E. J. and Tao, T., “Decoding by linear programming,” Information Theory, IEEE Transactions

on 51, 4203–4215 (Dec. 2005).

Proc. of SPIE Vol. 8748 874802-11


spie proceedings [spie spie defense, security, and sensing - baltimore, maryland, usa (monday 29...

Documents