compressed sampling cmos imager based on asynchronous...
TRANSCRIPT
© ANAFOCUS 2007
Compressed Sampling CMOS Imager based
on Asynchronous Random Pixel
Contributions
Marco Trevisi1, H.C. Bandala-Hernandez2, Ricardo Carmona-Galán1, and Ángel Rodríguez-Vázquez1
1Institute of Microelectronics of Seville (IMSE-CNM),
CSIC-Universidad de Sevilla, Spain2National Institute of Astrophysics, Optics and Electronics (INAOE),
Puebla, Mexico
Email: [email protected]
2
Introduction
Bio inspired approach and Compressive Sensing
3
Introduction
Bio inspired approach and Compressive Sensing
4
Introduction
Bio inspired approach and Compressive Sensing
5
Introduction
Bio inspired approach and Compressive Sensing
© ANAFOCUS 2007
Outlines
�Compressive Sensing, an Overview
�State of the Art VLSI implementations: Limitations and Solutions
�Proposed CMOS Imager Architecture: Advantages and Improvements
�MATLAB Analysis of the Performance of our Solution
�Conclusions
Restricted Isometry Property
being:
Ill-posed: if � � � there are more variables than equations [Candès 2006]
Sensing Problem
7
Compressive Sensing, An Overview
ΦNℜ
Mℜ
NM <
Φ: can be seen as a projection that maps a
higher dimensional space into a lower
dimensional space preserving proportions
among Euclidean distances of the
represented elements.
� ∈ ��
Φ � �
� ∈ ��
Measurement Matrix
Random Matrices: Incoherent Orthobasis Matrices:
ΦThe measurement matrix for Image Sensing should be universal, physically realizable and
easily implementable. No algorithms to create ad hoc matrices. Universal Strategies:
( ) εδε
≤⇒≥k
kNCkM
2
/log ( ) ( ) εδε
ε ≤⇒⋅≥
−
k
NCkM
2
15 loglog
8
• Gaussian
• Sub-Gaussian
• Poisson
• Fourier
• Cosine
• Hadamard Ensemble
Compressive Sensing, An Overview
9
The Importance of Sparseness for the Reconstruction
Sparse Full
Implemented Reconstruction Method: NESTA [Becker 2009]
Compressive Sensing, An Overview
10
The Importance of Sparseness for the Reconstruction
Sparse Full
Implemented Reconstruction Method: NESTA [Becker 2009]
Compressive Sensing, An Overview
11
State of the Art VLSI implementations: Limitations and Solutions
In a streaming setting each measurement will act on a different snapshot however it can be
assumed that through M fast snapshots the image changes very slowly allowing the
reconstruction of an almost static image that takes the place of a frame in a video sequence.
[Duarte 2009]
Single Pixel Camera, Rice University
12
State of the Art VLSI implementations: Limitations and Solutions
Measurement Matrix must be transmitted
Single Pixel Camera, Rice University
13
State of the Art VLSI implementations: Limitations and Solutions
Measurement Matrix must be transmitted
1 bit per Micromirror per Compressed Sample
Single Pixel Camera, Rice University
14
State of the Art VLSI implementations: Limitations and Solutions
Each Compressed Sample is the sum of M Pixels
Single Pixel Camera, Rice University
15
State of the Art VLSI implementations: Limitations and Solutions
Each Compressed Sample is the sum of M Pixels
To maintain the Resolution the ADC needs:
� ��� 8 � log� �
Single Pixel Camera, Rice University
16
State of the Art VLSI implementations: Limitations and Solutions
Ecole Polytechnique Fédérale de Lausanne CMOS Imager
[Majidzadeh 2010]
17
State of the Art VLSI implementations: Limitations and Solutions
Ecole Polytechnique Fédérale de Lausanne CMOS Imager
[Majidzadeh 2010]
� ��� 8 � log� �
18
State of the Art VLSI implementations: Limitations and Solutions
Center for VLSI and Embedded Systems Technology of Hyderabad Block-
Based CMOS Imager
Segmentation of an image into a set of sub-images
[Kaliannan 2014]
19
State of the Art VLSI implementations: Limitations and Solutions
Center for VLSI and Embedded Systems Technology of Hyderabad Block-
Based CMOS Imager
[Kaliannan 2014]
Segmentation of an image into a set of sub-images
Diminishes the sparseness of each sub-image
20
Proposed CMOS Imager Architecture: Advantages and Improvements
Measurement Matrix
To achieve a bit transmission rate inferior to that of an uncompressed image one must remove the
necessity to send the Measurement Matrix along with the samples.
21
Proposed CMOS Imager Architecture: Advantages and Improvements
Measurement Matrix
To achieve a bit transmission rate inferior to that of an uncompressed image one must remove the
necessity to send the Measurement Matrix along with the samples.
Store the patterns in an on-chip memory Generate the patterns on chip
22
Proposed CMOS Imager Architecture: Advantages and Improvements
Measurement Matrix
To achieve a bit transmission rate inferior to that of an uncompressed image one must remove the
necessity to send the Measurement Matrix along with the samples.
Store the patterns in an on-chip memory Generate the patterns on chip
23
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton
• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.
24
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton
• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.
• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.
25
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton
• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.
• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.
• Class 3: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.
26
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton
• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.
• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.
• Class 3: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.
• Class 4: Nearly all initial patterns evolve into structures that interact in complex and interesting
ways
27
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton
• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.
• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.
• Class 3: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.
• Class 4: Nearly all initial patterns evolve into structures that interact in complex and interesting
ways
28
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton implementing Rule 30
Class 3 Cellular Automaton: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.
29
Proposed CMOS Imager Architecture: Advantages and Improvements
1D Cellular Automaton implementing Rule 30
Class 3 Cellular Automaton: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.
30
Proposed CMOS Imager Architecture: Advantages and Improvements
Bit Resolution
To achieve a suitable bit resolution.
31
Proposed CMOS Imager Architecture: Advantages and Improvements
Bit Resolution
To achieve a suitable bit resolution.
Classic AD conversion Digital representation of time
32
Proposed CMOS Imager Architecture: Advantages and Improvements
Bit Resolution
To achieve a suitable bit resolution.
Classic AD conversion Digital representation of time
33
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
34
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
35
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
36
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
1 Compressed sample of 20 bits
37
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
1 Compressed sample of 20 bits
30 fps
38
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
1 Compressed sample of 20 bits
30 fps T� 1
4096 820
30 !"
39
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
1 Compressed sample of 20 bits
30 fps
8 bits counter
T� 1
4096 820
30 !"
40
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
1 Compressed sample of 20 bits
30 fps
8 bits counter T#$%�& T�
2'
T� 1
4096 820
30 !"
41
Proposed CMOS Imager Architecture: Advantages and Improvements
Readout Logic
64 × 64 Pixels, 8 bits each
1 Compressed sample of 20 bits
30 fps
8 bits counter T#$%�& T�
2'
F)%* 1
2 T#$%�&+ 24�,-
T� 1
4096 820
30 !"
42
MATLAB Analysis of the Performance of our Solution
128 256 512 1024 2048 40960
10
20
30
40
50
Number of Samples
RM
SE
of
reco
nst
ruct
ion
(%
)
full frame
block based
Average RMSE of Reconstruction
We compare the results achieved with the RMSE of reconstruction of 10 64×64 pixels images1
compressed performing a block based compressive sampling strategy (red bars) and a full frame
compressive strategy (blue bars). We devised the block based strategy by dividing each image
in 64 sub-images of 8×8 pixels to be treated separately.
1 These images can be found at: http://www2.imse-cnm.csic.es/icaveats/64x64px_images/
43
MATLAB Analysis of the Performance of our Solution
128 256 512 1024 2048 40960
20
40
60
80
100
120
Number of Samples
Tim
e o
f re
con
stru
ctio
n (
s)
full frame
block based
Average Time of Reconstruction
The resources needed to retrieve the images on lower compression ratios favour the block based
sampling strategy. However, as we diminish the amount of samples, the time of reconstruction
reverses its tendency allowing the full frame compressive strategy to outperform its block based
counterpart.
44
Conclusions
٭ We have introduced a new architecture able to collect compressed samples using
pseudo-random distributions generated on-chip.
٭ The pattern generated by the cellular automaton does not need to be transmitted and
can be easily recovered by simply knowing the initial seed.
٭ We avoid splitting the sensor array in smaller portions worsening the quality of the
samples or introducing asymmetries in the design of the sensor array.
٭ The solution applied to digitize the compressed samples improves their dynamic
range thus optimizing reconstruction.
Final Remarks
© ANAFOCUS 2007
ReferencesE. Candès. “Compressive sampling”. Int. Congress of Mathematics, pp. 1433-1452.
Madrid, Spain, 2006.
S. Becker, J. Bobin, and E. J. Candès. “NESTA: A Fast and Accurate First-Order
Method for Sparse Recovery”. SIAM Journal on Imaging Sciences, Vol. 4, No. 1,
pp. 1 - 39, Jan. 2011.
M. F. Duarte, M. A. Davenport, D. Takbar, J. N. Laska, T. Sun, K. F. Kelly, and R.
G. Baraniuk. Single-pixel imaging via compressive sampling. IEEE signal
processing magazine, Vol. 25, No. 2, pp. 83 – 91, Mar. 2008.
V. Majidzadeh, L. Jacques, A. Schmid, P. Vandergheynst and Y. Leblebici. “A
(256x256) Pixel 76.7mW CMOS Imager/Compressor Based on Real-Time In-Pixel
Compressive Sensing”. Proceedings of 2010 IEEE International Symposium on
Circuits and Systems (ISCAS), pp. 2956 – 2959. Paris, France. May, 2010
B. Kaliannan, V S. Rao Pasupureddi. “A Low Power CMOS Imager Based on
Distributed Compressed Sensing”. 27th International Conference on VLSI Design
and 13th International Conference on Embedded Systems, pp. 534 – 538. Mumbai,
India. Jan. 2014
© ANAFOCUS 2007
Acknowledgements
This work has been funded by the Spanish Government through project TEC2015-
66878-C3-1-R MINECO (ERDF/FEDER), Junta de Andalucía through project TIC
2338-2013 CEICE, the Office of Naval Research (USA) through grant
N000141410355 and CONACYT (Mexico) through grant 2016-MZO-2017-291062.