![Page 1: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/1.jpg)
Entropy, Information and Compressive Sensing in the
Quantum Domain.
John Howell
Greg Howland
James Schneeloch
Daniel Lum
Sam Knarr
Clemente Cuevas(REU)
Matt Ware (REU)
Robert Boyd
Cliff Chan
Petros Zerom
![Page 2: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/2.jpg)
Outline• Introduction to compressive sensing
– Shannon entropy– Nyquist sampling– Lossy compression– K-sparse sensing– L1 norm reconstruction– Advantages
• Applications– Ghost imaging with entangled
photons– Photon counting Lidar– Depth Maps– Object tracking– High dimensional entanglement
characterization
![Page 3: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/3.jpg)
Shannon Entropy
• Entropy– Measure uncertainty of
random variable X with distribution p(x).
– Find number of symbols and bits per symbol (e.g., 0 or 1 binary)
– Compression removes intersymbol correlations
• Example (alphanumeric)• _ _ _ _ _ _ _ _ _ _ _ _ _
• _ _ _ _ R _ _ E _ _ _ _ _
• Q _ _ _ R _ _ E _ T _ _ _
• Q S T A R T M E E T _ N G
• Shannon showed there are approximately 1 bit per symbol in English language
)(log)()( 2 xpxpXH
![Page 4: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/4.jpg)
Information Theorist Marriage Therapist
“I see that your channel capacity is quite low. You need to spend more time maximizing your mutual information. Please increase the entropy of your communication while minimizing the noise in your
classical channels.”
![Page 5: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/5.jpg)
Shannon Entropy for Images: Compression After Sensing
• Compression removes interpixel correlations
• Decompose in decorrelated transform basis– k-sparse representation– DCT, DFT, wavelets etc.
• Significant reduction in memory requirements, reduced uncertainty
![Page 6: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/6.jpg)
• Standard Sensing Paradigm (e.g., jpeg)– Sample (at least two
times above Nyquist)
– Transform to sparse basis
– Preferentially attenuate high frequency components
– Round coefficients
– Inverse transform
![Page 7: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/7.jpg)
Sensing Paradigms
• Typical Sensing: Compress after you sense
• Compressed sensing: Compress while you sense
![Page 8: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/8.jpg)
CS Literature of Interest
• Tutorials on CS– R.G. Baraniuk, IEEE SIGNAL PROCESSING
MAGAZINE [118] JULY 2007– E.J. Candes and M.J. Wakin, IEEE SIGNAL
PROCESSING MAGAZINE [21] MARCH 2008
• Single Pixel Camera– Duarte et al, IEEE SIGNAL PROCESSING
MAGAZINE [83] MARCH 2008
![Page 9: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/9.jpg)
Introduction to CS
sx
sxy
Consider a 1 dimensional signal x of length N and a transform basis s.
Transform matrix of dimension NxN
We require a sensing matrix which is not sparse when transformed (incoherence or restricted isometry property).
Random, length N, sensing matrices satisfy this requirement.
![Page 10: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/10.jpg)
Introduction to CS
Nk
NkM 2log
N
iis
1
min
M random measurements needed to recover signal
Reconstruct image using l1 norm minimization
sy
We use Gradient Projection for Sparse Reconstruction algorithm (noise tolerant BPDN)
Figueiredo et al, IEEE Selected Topics in Signal Processing, 2007
![Page 11: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/11.jpg)
Compressive sensingScene
Reflected light
DMD
Photodiode
![Page 12: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/12.jpg)
We take M << N measurements with different random DMD patterns and then reconstruct x
Outside information is needed to solve our underdetermined linear system to reconstruct our image!
Compressive sensing
![Page 13: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/13.jpg)
Why CS?
• Sampling rate – Above information rate – NOT above Nyquist rate.
• Resource efficient – Single pixel measurements– Fewer measurements – Automatically finds large k-sparse coefficients
![Page 14: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/14.jpg)
Cool stuff recently done
• “Compressive Sensing Hyperspectral Image” T. Sun and K. Kelly (COSI) (2009)
• “Sparsity-based single-shot sub-wavelength coherent diffractive imagin” A. Szameit et al (M. Segev), Nature Materials 11, 455 (2012)
• “Compressive Depth Map Acquisition Using a Single Photon-Counting Detector” A. Colaco et al Proc. IEEE Conf. Computer Vision and Pattern Recognition (2012)
• “Compressive Sensing for Background Subtraction” Volkan Cevher, (Baraniuk)
• 3D COMPUTATIONAL IMAGING WITH SINGLE-PIXEL DETECTORS (Padgett) Science 340, 844 (2013)
![Page 15: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/15.jpg)
Some of our applications
• Ghost imaging with entangled photons• Photon counting Lidar• Depth Maps• Object tracking• High dimensional entanglement
characterization
![Page 16: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/16.jpg)
Entangled Photon Compressive Imaging
![Page 17: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/17.jpg)
Comparison to Raster Scanning
• Compressive Sensing– 4500 measurements– N=128x128 pixels– 9 seconds/measurement– SNR 8– Acquisition time
• 4500 x 9s~ ½ day• Raster Scanning
– For same SNR and resolution it would take almost 3 years to acquire image with same flux
![Page 18: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/18.jpg)
LIDAR with JIGSAW Pros•32x32 APD detectors•Time of Flight Measurements•High Signal to Noise (no amplification noise)•Foliage Penetrating, Aerosol •Low Light Level DetectionCons•Low Fill Factor•Difficult to Scale•Expensive•Resource Heavy•Visible Wavelengths•Large Payload
MIT LL
![Page 19: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/19.jpg)
JIGSAW
![Page 20: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/20.jpg)
Experimental Setup
G. Howland, P.B. Dixon and J.C. Howell, Appl. Optics 50, 5917 (2011)
![Page 21: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/21.jpg)
Results: 3D Imaging
U R
Wall
![Page 22: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/22.jpg)
Imaging Through Obscurants
![Page 23: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/23.jpg)
Low Flux 3D Object Tracking
![Page 24: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/24.jpg)
Swinging Ball Trajectory
![Page 25: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/25.jpg)
Frame by Frame
![Page 26: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/26.jpg)
Depth Map of Natural Scene
![Page 27: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/27.jpg)
2nd-Order Correlations
• >7 bits mutual information in X and P• P. Ben Dixon et al PRL 108, 143603 (2012)
Slow Method
![Page 28: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/28.jpg)
With CS
• Replace Raster with CS• N log N scaling rather
than N3 to N4. • 8 hours instead of a year
Efficient High-Dimensional Entanglement Imaging with a Compressive-Sensing Double-Pixel CameraGregory A. Howland and John C. HowellPhys. Rev. X 3, 011013 (2013).
![Page 29: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/29.jpg)
32x32 Position Position Correlations (3 raster)
![Page 30: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/30.jpg)
Mutual Information in X and P
Violation of Continuous-Variable Einstein-Podolsky-Rosen Steering with Discrete Measurements J. Schneeloch, P. Ben Dixon, G. A. Howland, C. J. Broadbent, and J. C. Howell Phys. Rev. Lett. 110, 130407 (2013).
![Page 31: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/31.jpg)
Background Subtraction Object Tracking
Compressive object tracking using entangled photons Omar S. Magana-Loaiza, Gregory A. Howland, Mehul Malik, John C. Howell, and Robert W. Boyd Appl. Phys. Lett. 102 231104 (2013).
![Page 32: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/32.jpg)
Ghost Object Tracking
![Page 33: Entropy, Information and Compressive Sensing in the Quantum Domain. John Howell Greg Howland James Schneeloch Daniel Lum Sam Knarr Clemente Cuevas(REU)](https://reader038.vdocuments.us/reader038/viewer/2022102808/56649ddb5503460f94ad20a0/html5/thumbnails/33.jpg)
Novel Acquisition Paradigm
• Quantum imaging
• Entanglement mutual information
• Low flux LIDAR
• Precision measurements
• Real-time video