1. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI) Compressive Spectral Imaging Gonzalo R. ArceDepartment of Electrical and Computer Engineering University of Delaware Email:arce@ece.udel.edu Distinguished Lecture SeriesAristotle University of ThessalonikiOctober 19th - 2010 Gonzalo R. ArceCompressive Spectral Imaging -1

2. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)Outline Introduction to Compressive Sensing Sparsity and 1 Norm Incoherent Sampling Sparse Signal Recovery Compressive Spectral Imaging Single Shot CASSI System Spectral Selectivity in (CASSI) Random Convolution SSI (RCSSI) Low-rank Anomaly Recovery in (CASSI) Gonzalo R. ArceCompressive Spectral Imaging -2 3. Introduction to Compressive Sensing Sparsity and 1 norm Compressive Spectral ImagingIncoherent Sampling Low-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryTraditional signal sampling and signal compression.Nyquist sampling rate gives exact reconstruction. Pessimistic for some types of signals!Gonzalo R. ArceCompressive Spectral Imaging -3 4. Introduction to Compressive Sensing Sparsity and 1 normCompressive Spectral ImagingIncoherent SamplingLow-rank Anomaly Recovery in (CASSI)Sparse Signal RecoverySampling and Compression Transform data and keep important coefcients. Lots of work to then throw away majority of data!. e.g. JPEG 2000 Lossy Compression: A digital camera can take millions of pixels but the picture is encoded on a few hundred of kilobytes. Gonzalo R. ArceCompressive Spectral Imaging -4 5. Introduction to Compressive Sensing Sparsity and 1 norm Compressive Spectral ImagingIncoherent Sampling Low-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryProblem: Recent applications require a very large number ofsamples:Higher resolution in medical imaging devices, cameras,etc.Spectral imaging, confocal microscopy, radar arrays, etc. y x Spectral Imaging Medical ImagingGonzalo R. ArceCompressive Spectral Imaging -5 6. Introduction to Compressive SensingSparsity and 1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryFundamentals of Compressive Sensing Donoho , Cands , Romberg and Tao, discovered important results on the minimum number of data needed to reconstruct a signal Compressive Sensing (CS) unies sensing and compression into a single task Minimum number of samples to reconstruct a signal depends on its sparsity rather than its bandwidth. D. Donoho. "Compressive Sensing". IEEE Trans. on Information Theory. Vol.52(2), pp.5406-5425, Dec.2006. E. Cands, J. Romberg and T. Tao. "Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information". IEEE Trans. on Information Theory. Vol.52(4), pp.1289-1306, Apr.2006.Gonzalo R. ArceCompressive Spectral Imaging -6 7. Introduction to Compressive Sensing Sparsity and 1 norm Compressive Spectral ImagingIncoherent Sampling Low-rank Anomaly Recovery in (CASSI)Sparse Signal RecoverySparsity Signal sparsity critical to CS Plays roughly the same role in CS that bandwidth plays in Shannon-Nyquist theory A signal x RN is S-sparse on the basis if x can be represented by a linear combination of S vectors of as x = with S NAt most S non-zero components x Gonzalo R. Arce Compressive Spectral Imaging -7 8. Introduction to Compressive Sensing Sparsity and 1 normCompressive Spectral ImagingIncoherent SamplingLow-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryThe 1 Norm and Sparsity Sparsity of x is measured by its number of non-zero elements, the 0 normx0 = #{i : x(i) = 0} The 1 norm can be used to measure sparsity of x x1 = |x(i)|i The 2 norm is not effective in measuring sparsity of x x 2 =( |x(i)|2 )1/2i The 0 and 1 norms promote sparsity Gonzalo R. ArceCompressive Spectral Imaging -8 9. Introduction to Compressive Sensing Sparsity and 1 norm Compressive Spectral ImagingIncoherent Sampling Low-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryWhy 1 Norm Promotes Sparsity?Given two N -dimensional signals:x1 = (1, 0, ..., 0) "Spike" signal x2 = (1/ N , 1/ N , ..., 1/ N ) "Comb" signal x 2 x1 and x2 have the same 2 norm:x1 2 = 1 and x2 2 = 1. x 1 However, x1 1 = 1 andx2 1 = N .Gonzalo R. ArceCompressive Spectral Imaging -9 10. Introduction to Compressive SensingSparsity and 1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryCompressive MeasurementsMeasurements in CS are different than samples taken intraditional A/D converters.The signal x is acquired as a series of non-adaptive innerproducts of different waveforms {1 , 2 , ..., M } yk =< k , x >; k = 1, ..., M ; with M N y xMx1 MxN MeasurementsSampling Operator Nx1Sparse SignalGonzalo R. Arce Compressive Spectral Imaging -10 11. Introduction to Compressive Sensing Sparsity and 1 normCompressive Spectral ImagingIncoherent SamplingLow-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryRecoverabilityyk =< k , x >; k = 1, ..., M ; with M NRecovering x from yk is an inverse problem.Need to solve an under determined system of equationsy = x.Innitely solutions for the system since M N .Amplitude Amplitude Original sparse signal Compressed measurementsReconstructed signal using least-squares.Solution not sparseGonzalo R. ArceCompressive Spectral Imaging -11 12. Introduction to Compressive Sensing Sparsity and 1 norm Compressive Spectral ImagingIncoherent Sampling Low-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryRecoverability: Incoherent SamplingThe number of samples required to recover x from M samplesdepends on the mutual coherence between and Mutual Coherence(, ) =N max{| < k , j > | : k Rows(), j Columns()};where,j 2 = k2 =1The coherence (, ) satises: 1 (, ) NGonzalo R. ArceCompressive Spectral Imaging -12 13. Introduction to Compressive SensingSparsity and 1 norm Compressive Spectral Imaging Incoherent Sampling Low-rank Anomaly Recovery in (CASSI) Sparse Signal RecoveryRecoverability: Incoherent Sampling The random measurement matrix has to be incoherent to the dictionary and x can be recovered from M samples exactly when M satises:M C 2 S log(N ), C 1 (a)(b) (a) Very sparse vector. (b) Examples of pseudorandom, incoherent test vectors k .J. Romberg. "Imaging Via Compressive Sampling". IEEE Signal Processing Magazine. March,2008.Gonzalo R. Arce Compressive Spectral Imaging -13 14. Introduction to Compressive Sensing Sparsity and 1 normCompressive Spectral ImagingIncoherent SamplingLow-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryCompressive Sensing Signal Reconstruction Goal: Recover signal x from measurements y Problem: Random projection not full rank (ill-posed inverse problem) Solution: Exploit the sparse/compressible geometry of acquired signal xy x Gonzalo R. ArceCompressive Spectral Imaging -14 15. Introduction to Compressive Sensing Sparsity and 1 normCompressive Spectral ImagingIncoherent SamplingLow-rank Anomaly Recovery in (CASSI)Sparse Signal RecoveryReconstruction Algorithms Different formulations and implementations have been proposed to nd the sparsest x subject to y = x Those are broadly classied in: Regularization formulations (Replace combinatorial problem with convex optimization) Greedy algorithms (Iterative renement of a sparse solution) Bayesian framework (Assume prior distribution of sparse coefcients) Gonzalo R. ArceCompressive Spectral Imaging -15 16. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Compressive Spectral Imaging Collects spatial information from across the electromagnetic spectrum. Applications, include wide-area airborne surveillance, remote sensing, and tissue spectroscopy in medicine. Gonzalo R. ArceCompressive Spectral Imaging -16 17. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Compressive Spectral ImagingSpectral Imaging System - Duke UniversityA. Wagadarikar, R. John, R. Willett, D. Brady. "Single Disperser Design for Coded Aperture Snapshot Spectral Imaging."Applied Optics, vol.47, No.10, 2008.A. Wagadarikar and N. P. Pitsianis and X. Sun and D. J. Brady. "Video rate spectral imaging using a coded aperturesnapshot spectral imager." Opt. Express, 2009.Gonzalo R. ArceCompressive Spectral Imaging -17 18. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot Compressive Spectral ImagingSystem design With linear dispersion: f1 (x, y; )= f0 (x, y; )T (x, y) f2 (x, y; )= (x [x + ( c )](y y)f1 (x , y ; ))dx dy = (x [x + ( c )](y y)f0 (x , y ; )T (x, y))dx dy = f0 (x + ( c ), y; )T (x + ( c ), y)Gonzalo R. Arce Compressive Spectral Imaging -18 19. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot Compressive Spectral ImagingExperimental results from Duke University Original Image Reconstructed image cube of size:128x128x128. MeasurementsSpatial content of the scene in each of 28 spectral channels between 540 and 640nm. A. Wagadarikar, R. John, R. Willett, D. Brady. "Single Disperser Design for Coded Aperture Snapshot Spectral Imaging."Applied Optics, vol.47, No.10, 2008. Gonzalo R. Arce Compressive Spectral Imaging -19 20. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot Compressive Spectral ImagingSimulation results in RGBOriginal Image Measurements R ReconstructedImageGonzalo R. ArceCompressive Spectral Imaging -20 21. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI System Object with spectral information only in (xo , yo ) Only two spectral component are present in the object Gonzalo R. ArceCompressive Spectral Imaging -21 22. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI SystemObject with spectral information only in (xo , yo ) Gonzalo R. ArceCompressive Spectral Imaging -22 23. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI SystemOne pixel in the detector has information from different spectralbands and different spatial locations Gonzalo R. ArceCompressive Spectral Imaging -23 24. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI SystemEach pixel in the detector has different amount of spectralinformation. The more compressed information, the moredifcult it is to reconstruct the original data cube. Gonzalo R. ArceCompressive Spectral Imaging -24 25. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI SystemEach row in the data cube produces a compressedmeasurement totally independent in the detector.Gonzalo R. ArceCompressive Spectral Imaging -25 26. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI SystemUndetermined equation system:Unknowns = N N M and Equations: N (N + M 1) Gonzalo R. ArceCompressive Spectral Imaging -26 27. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI System Complete data cube 6 bands The dispersive element shifts each spectral band in one spatial unit In the detector appear the compressed and modulated spectral component of the object At most each pixel detector has information of six spectral components Gonzalo R. ArceCompressive Spectral Imaging -27 28. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Single Shot CASSI SystemWe used the 1 s reconstruction algorithm .S. J. Kim, K. Koh, M. Lustig, S. Boyd and D. Gorinevsky. "An interior-point method for large scale L1 regularized least squares." IEEE Journal of Selected Topics in Signal Processing, vol.1, pp. 606-617, 2007. Gonzalo R. Arce Compressive Spectral Imaging -28 29. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Coded Aperture Snapshot Spectral Image System(CASSI)(a) Advantages: Enables compressive spectral imag- ing Simple Low cost and complexity Limitations: Excessive compression Does not permit a controllable SNR May suffer low SNR gmn =f(m+k)nk P(m+k)n + wnm Does not permit to extract a specick subset of spectral bands= (Hf )nm + wnm = (HW )nm + wnmA. Wagadarikar, R. John, R. Willett, and D. Brady. "Single disperser design for coded aperture snapshot spectral imaging."Appl. Opt., Vol.47, No.10, 2008.Gonzalo R. Arce Compressive Spectral Imaging -29 30. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Bands RecoveryTypical example of a measurement of CASSI system. A set of bandsconstant spaced between them are summed to form a measurement Gonzalo R. ArceCompressive Spectral Imaging -30 31. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-Shot CASSI SystemMulti-shot compressive spectral imaging systemAdvantages:Multi-Shot CASSI allowscontrollable SNRPermits to extract a hand-picked subset of bandsExtend Compressive Sens-ing spectral imaging capabil-ities Lgmni = fk (m, n + k 1)Pi (m, n + k 1) k=1 Li = fk (m, n + k 1)Pr (m, n + k 1)Pg (m, n + k 1) k=1 Ye, P. et al. "Spectral Aperture Code Design for Multi-Shot Compressive Spectral Imaging". Dig. Holography andThree-Dimensional Imaging, OSA. Apr.2010. Gonzalo R. ArceCompressive Spectral Imaging -31 32. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Mathematical Model of CASSI SystemLgmni =fk (m, n + k 1)Pi (m, n + k 1)k=1 L i =fk (m, n + k 1)Pr (m, n + k 1)Pg (m, n + k 1)k=1where i expresses ith shot Each pattern Pi is given by,i Pi (m, n) = Pg (m, n)xPr (m, n) i1 mod(n, R) = mod(i, R)Pg (m, n) =0 otherwiseOne different code aperture is used for each shot of CASSI system Gonzalo R. ArceCompressive Spectral Imaging -32 33. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Code Apertures Code patterns used in multishot CASSI systemCode patterns used in multishot CASSI systemGonzalo R. Arce Compressive Spectral Imaging -33 34. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Cube Information and Subsets of Spectral BandsSpectral axis,SpatialL bands axis, NSpectral data cube L bandspixelsR subsets of M bands each one Complete Spectral (L = RM ) Each component Data Cubeof the subset is spaced by R Spatialbands of each other axis, N pixelsSubset 1M bandsR R Subset 1Subset 2Subset 3 ... Subset R M=bands M=bands M=bandsM bandsGonzalo R. Arce Compressive Spectral Imaging -34 35. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Cube Information and Subsets of Spectral Bands Spectral axis, Spatial L bandsaxis, NSpectral data cube L bandspixels R subsets of M bands each one Complete(L = RM ) Each component Spectralof the subset is spaced by R Data Cube Spatial bands of each other axis, N R R pixelsSubset 2 M bands Subset 1 Subset Subset ...Subset R M=bandsM=bands M=bandsM=bandsGonzalo R. ArceCompressive Spectral Imaging -35 36. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-Shot CASSI System First shot and Second shot and R shot and measurementmeasurement measurementGonzalo R. ArceCompressive Spectral Imaging -36 37. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Single Shot Multi-Shot One shot of CASSIInformation of all band exists in all shots system. One high compressing measurement. First shot Second shot Third shotReconstructionAlgorithmRe-organization algorithm Reconstructed spectral data cube.Bands 1,4,7 Bands 2,5,8 Bands 3,6,9Gonzalo R. Arce Compressive Spectral Imaging -37 38. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI) Multi-Shot Reorder ProcessRR R Lgmnk = j=1 fj (m, n + j 1)Pi (m, n + j 1) Li First shot Second shot Third shot=j=1 fj (m, n + j 1)Pr (m, n + j 1)Pg (m, n + j 1) Re-organizationalgorithm=mod(n+j1,R)=mod(i,R) fk (m, n+ k 1)Pr (m, n + j 1)= (Hk Fk )mnBands 1,4,7 Bands 2,5,8 Bands 3,6,9 Gonzalo R. ArceCompressive Spectral Imaging -38 39. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-Shot Reorder ProcessRR R Lgmnk =j=1 fj (m, n + j 1)Pi (m, n + j 1)LiFirst shot Second shotThird shot= j=1 fj (m, n + j 1)Pr (m, n + j 1)Pg (m, n + j 1)Re-organizationalgorithm= mod(n+j1,R)=mod(i,R) fk (m, n + k 1)Pr (m, n + j 1)= (Hk Fk )mnBands 1,4,7Bands 2,5,8 Bands 3,6,9Gonzalo R. ArceCompressive Spectral Imaging -39 40. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-ShotRecover any of the subsetsindependentlyRecover of complete spec-tral data cube is not neces-saryGonzalo R. ArceCompressive Spectral Imaging -40 41. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-ShotHigh SNR in each re-constructionEnable to use paral-lel processingTo use one proces-sor for each indepen-dent reconstructionGonzalo R. ArceCompressive Spectral Imaging -41 42. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-ShotSingle ShotOne shot of CASSIsystem. One highcompressingmeasurement. Reconstruction AlgorithmReconstructedspectral datacube.Gonzalo R. Arce Compressive Spectral Imaging -42 43. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-Shot Reconstruction Reconstructed image of one spec- tral channel in 256x256x24 data cube from multiple shot measure- ments.(a) One shot result,PSNR (a) One shot (b) 2 shotsP SN R = 17.6dB(b) Two shots result,PSNRP SN R = 25.7dB(c) Eight shots result,PSNRP SN R = 29.4(d) Original image(c) 8 shots (d) OriginalGonzalo R. ArceCompressive Spectral Imaging -43 44. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Multi-Shot ReconstructionReconstructed image for dif-ferent spectral channels in the256x256x24 data cube fromsix shot measurements. (a) Band 1 (b) Band 13 (c) Band 8 (d) Band 20 (a) and (b) are recon- structed from the rst group of measurements (c) and (d) are recon- structed from the second group of measurements Gonzalo R. ArceCompressive Spectral Imaging -44 45. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Random Convolution Spectral ImagingGonzalo R. ArceCompressive Spectral Imaging -45 46. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution ImagingJ. Romberg. "Compressive Sensing by Random Convolution." SIAM Journal on Imaging Science, July,2008. Gonzalo R. Arce Compressive Spectral Imaging -46 47. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Random Convolution ImagingRandom ConvolutionCircularly convolve signal x Rn with a pulse h Rn , thensubsample.The pulse is random, global, and broadband in that its energy isdistributed uniformly across the discrete spectrum.x h = Hxwhere H = n1/2 F F Ft, = ej2(t1)(1)/n , 1 t, n as a diagonal matrix whose non-zero entries are the Fouriertransform of h. Gonzalo R. ArceCompressive Spectral Imaging -47 48. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI)Compressive Spectral Imaging Spectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Random Convolution 1 0 0 2 =.... . . n=1: 1 1 with equal probability,2 < n/2 + 1: = ej , where Uniform([0, 2]), = n/2 + 1: n/2+1 1 with equal probability,n/2 + 2 n: = n+2 , the conjugate of n+2 . Gonzalo R. Arce Compressive Spectral Imaging -48 49. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Random ConvolutionHThe effect of H on a signal x can be broken down into adiscrete Fourier transform, followed by a randomization ofthe phase (with constraints that keep the entries of H real),followed by an inverse discrete Fourier transform.Since F F = F F = nI and = I,H H = n1 F F F F = nISo convolution with h as a transformation into a randomorthobasis.Gonzalo R. ArceCompressive Spectral Imaging -49 50. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Randomly Pre-Modulated Summationym1 = mn xn1 = Pmn nn Hnn xn1where ones(n/m, 1)0000 ones(n/m, 1) 0 0 Pmn = ..00.0 000 ones(n/m, 1) mn 1 00 0 0 1 00 nn= .. 00. 0 000 1nn Gonzalo R. ArceCompressive Spectral Imaging -50 51. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Main Result H will not change the magnitude of the Fourier transform, so signals which are concentrated in frequency will remain concentrated and signals which are spread out will stay spread out. The randomness of will make it highly probable that a signal which is concentrated in time will not remain so after H is applied. Gonzalo R. ArceCompressive Spectral Imaging -51 52. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Main Result (a) A signal x consisting of a single Daubechies-8 wavelet. (b) Magnitude of the Fourier transform F x. (c) Inverse Fourier transform after the phase has been randomized. Although the magnitude of the Fourier transform is the same as in (b), the signal is now evenly spread out in time. J. Romberg. "Compressive Sensing by Random Convolution." SIAM Journal on Imaging Science, July,2008.Gonzalo R. Arce Compressive Spectral Imaging -52 53. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI)Compressive Spectral ImagingSpectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Fourier OpticsFourier optics imaging experiment.(a) The 256 256 image x.(b) The 256 256 image Hx.(c) The 64 64 image P Hx. Gonzalo R. ArceCompressive Spectral Imaging -53 54. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)(a) The 256 256 image we wish to acquire.(b) High-resolution image pixellated by averaging over 4 4 blocks.(c) The image restored from the pixellated version in (b), plus a set ofincoherent measurements. The incoherent measurements allow us toeffectively super-resolve the image in (b).Gonzalo R. ArceCompressive Spectral Imaging -54 55. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI) Compressive Spectral Imaging Spectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)Fourier Optics a) b)c) d)e)f)Pixellated images: (a) 2 2. (b) 4 4. (c) 8 8. Restored from: (d) 2 2 pixellatedversion. (e) 4 4 pixellated version. (f) 8 8 pixellated version.Gonzalo R. Arce Compressive Spectral Imaging -55 56. Introduction to Compressive Sensing Single Shot Coded Aperture System (CASSI) Compressive Spectral ImagingSpectral Selectivity in (CASSI) Low-rank Anomaly Recovery in (CASSI)Random Convolution SSI (RCSSI)Random Convolution Spectral ImagingGonzalo R. ArceCompressive Spectral Imaging -56 57. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI)Compressive Spectral Imaging Spectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)20406080 100 120 20 4060 80 100 120 Gonzalo R. Arce Compressive Spectral Imaging -57 58. Introduction to Compressive SensingSingle Shot Coded Aperture System (CASSI)Compressive Spectral Imaging Spectral Selectivity in (CASSI)Low-rank Anomaly Recovery in (CASSI) Random Convolution SSI (RCSSI)20406080 100 120 20 4060 80 100 120 Gonzalo R. Arce Compressive Spectral Imaging -58 59. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)Low-rank Anomaly Recovery in (CASSI) Spectral video analysis Video surveillance: Anomaly detection Stationary background corresponds to low-rank contribution and the moving objects corresponds to sparse data. Gonzalo R. ArceCompressive Spectral Imaging -59 60. Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Connection Between Low-Rank Matrix Recovery andCompressed SensingLow-rankRank miniz.Convex Relax.Recoverymin rank(X) L min L s.t. M=S+L s.t. M=S+L Compressed Rank miniz.Convex Relax. Sensing B. Recht, M. Fazel and P. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear NormMinimization," SIAM Review, Aug. 2010.Gonzalo R. ArceCompressive Spectral Imaging -60 61. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI)Problem Description (i)Consider the video surveillance of Fk,n1,n2 RN1 N2 K ,i = 1, ..., N frames.The ith scene is assumed to be composed by a stationarybackground L(i) and an event changing in time S(i) , (i)(i)(i)Fk,n1,n2 = Lk,n1,n2 + Sk,n1 ,n2CASSI encodes both 2D spatial information and spectralinformation in a 2Dsingle measurement G(i) fori = 1, ..., N .GOAL: recover anomalies occurring in both time and spectrafrom a sequence of spectrally compressed video framesG(1) , G(2) , ..., G(N ) . Gonzalo R. ArceCompressive Spectral Imaging -61 62. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI)Recovering anomalies:Form G as the large data matrix G = [g(1) , g(2) , . . . , g(N ) ],where g(i) is the column representation of G(i) .G = L + S where L is the stationary background and S issparse capturing the anomalies in the foreground Gonzalo R. ArceCompressive Spectral Imaging -62 63. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)Principal Component Pursuit The matrix G is decomposed into a low-rank matrix L and a sparse matrix S, such that G =L+S (1) Using Principal Component Pursuit. Principal Component Pursuitmin L + S1n L = i=1 i (L), is the nuclear norm of L. S1 = ij Sij is the 1 -norm of the matrix S E. J. Cands, X. Li, Y. Ma, and J. Wright. "Robust Principal Component Analysis?," Submitted to Journal of the ACM.2009. Gonzalo R. ArceCompressive Spectral Imaging -63 64. Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI)Spectral recovery of anomalies. Coded measurements in S have been biased by the background reconstruction Identify spatial location of the anomalies in S by:Filter |S| with a Weighted Median (WM) lter as (i) (i) Mn1 ,n2 = MEDIAN{Tv,w |Sn1 +v,n2 +w | : (v, w) [3, 3]}where T is a WM lter of size (L L) with centered weight(L + 1)/2, and linearly decreasing weightsSpectrally coded measurements of anomalies denoted byG(i) are estimated asG(i) = G(i) U(M(i) Th )Th is a thresholding parameter that extracts the pixels thatare most likely to be in the region of interestGonzalo R. ArceCompressive Spectral Imaging -64 65. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)Low-Rank Anomaly Recovery in (CASSI) Recover S(i) from G(i) by (i) = min( g(i) H (i) s 2 2 + (i) 1 ) (2) s where (i) and g(i) are the column representation of S(i) and G (i) , respectively. Gonzalo R. ArceCompressive Spectral Imaging -65 66. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)(video) (video) Gonzalo R. ArceCompressive Spectral Imaging -66 67. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)(video) (video) Gonzalo R. ArceCompressive Spectral Imaging -67 68. Introduction to Compressive SensingCompressive Spectral ImagingLow-rank Anomaly Recovery in (CASSI)(video) (video) Gonzalo R. ArceCompressive Spectral Imaging -68 69. Introduction to Compressive Sensing Compressive Spectral Imaging Low-rank Anomaly Recovery in (CASSI)SummaryCompressive SensingSpectral ImagingLow-Rank Recovery Eu !Gonzalo R. ArceCompressive Spectral Imaging -69