compressed sampling cmos imager based on asynchronous...

© ANAFOCUS 2007

Compressed Sampling CMOS Imager based

on Asynchronous Random Pixel

Contributions

Marco Trevisi1, H.C. Bandala-Hernandez2, Ricardo Carmona-Galán1, and Ángel Rodríguez-Vázquez1

1Institute of Microelectronics of Seville (IMSE-CNM),

CSIC-Universidad de Sevilla, Spain2National Institute of Astrophysics, Optics and Electronics (INAOE),

Puebla, Mexico

Email: trevisi@imse-cnm.csic.es

bandala@inaoep.mx

2

Introduction

Bio inspired approach and Compressive Sensing

3

Introduction

Bio inspired approach and Compressive Sensing

4

Introduction

Bio inspired approach and Compressive Sensing

5

Introduction

Bio inspired approach and Compressive Sensing

© ANAFOCUS 2007

Outlines

�Compressive Sensing, an Overview

�State of the Art VLSI implementations: Limitations and Solutions

�Proposed CMOS Imager Architecture: Advantages and Improvements

�MATLAB Analysis of the Performance of our Solution

�Conclusions

Restricted Isometry Property

being:

Ill-posed: if � � � there are more variables than equations [Candès 2006]

Sensing Problem

7

Compressive Sensing, An Overview

ΦNℜ

Mℜ

NM <

Φ: can be seen as a projection that maps a

higher dimensional space into a lower

dimensional space preserving proportions

among Euclidean distances of the

represented elements.

� ∈ ��

Φ � �

� ∈ ��

Measurement Matrix

Random Matrices: Incoherent Orthobasis Matrices:

ΦThe measurement matrix for Image Sensing should be universal, physically realizable and

easily implementable. No algorithms to create ad hoc matrices. Universal Strategies:

( ) εδε

≤⇒≥k

kNCkM

2

/log ( ) ( ) εδε

ε ≤⇒⋅≥

−

k

NCkM

2

15 loglog

8

• Gaussian

• Sub-Gaussian

• Poisson

• Fourier

• Cosine

• Hadamard Ensemble

Compressive Sensing, An Overview

9

The Importance of Sparseness for the Reconstruction

Sparse Full

Implemented Reconstruction Method: NESTA [Becker 2009]

Compressive Sensing, An Overview

10

The Importance of Sparseness for the Reconstruction

Sparse Full

Implemented Reconstruction Method: NESTA [Becker 2009]

Compressive Sensing, An Overview

11

State of the Art VLSI implementations: Limitations and Solutions

In a streaming setting each measurement will act on a different snapshot however it can be

assumed that through M fast snapshots the image changes very slowly allowing the

reconstruction of an almost static image that takes the place of a frame in a video sequence.

[Duarte 2009]

Single Pixel Camera, Rice University

12

State of the Art VLSI implementations: Limitations and Solutions

Measurement Matrix must be transmitted

Single Pixel Camera, Rice University

13

State of the Art VLSI implementations: Limitations and Solutions

Measurement Matrix must be transmitted

1 bit per Micromirror per Compressed Sample

Single Pixel Camera, Rice University

14

State of the Art VLSI implementations: Limitations and Solutions

Each Compressed Sample is the sum of M Pixels

Single Pixel Camera, Rice University

15

State of the Art VLSI implementations: Limitations and Solutions

Each Compressed Sample is the sum of M Pixels

To maintain the Resolution the ADC needs:

� ��� 8 � log� �

Single Pixel Camera, Rice University

16

State of the Art VLSI implementations: Limitations and Solutions

Ecole Polytechnique Fédérale de Lausanne CMOS Imager

[Majidzadeh 2010]

17

State of the Art VLSI implementations: Limitations and Solutions

Ecole Polytechnique Fédérale de Lausanne CMOS Imager

[Majidzadeh 2010]

� ��� 8 � log� �

18

State of the Art VLSI implementations: Limitations and Solutions

Center for VLSI and Embedded Systems Technology of Hyderabad Block-

Based CMOS Imager

Segmentation of an image into a set of sub-images

[Kaliannan 2014]

19

State of the Art VLSI implementations: Limitations and Solutions

Center for VLSI and Embedded Systems Technology of Hyderabad Block-

Based CMOS Imager

[Kaliannan 2014]

Segmentation of an image into a set of sub-images

Diminishes the sparseness of each sub-image

20

Proposed CMOS Imager Architecture: Advantages and Improvements

Measurement Matrix

To achieve a bit transmission rate inferior to that of an uncompressed image one must remove the

necessity to send the Measurement Matrix along with the samples.

21

Proposed CMOS Imager Architecture: Advantages and Improvements

Measurement Matrix

To achieve a bit transmission rate inferior to that of an uncompressed image one must remove the

necessity to send the Measurement Matrix along with the samples.

Store the patterns in an on-chip memory Generate the patterns on chip

22

Proposed CMOS Imager Architecture: Advantages and Improvements

Measurement Matrix

To achieve a bit transmission rate inferior to that of an uncompressed image one must remove the

necessity to send the Measurement Matrix along with the samples.

Store the patterns in an on-chip memory Generate the patterns on chip

23

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton

• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.

24

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton

• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.

• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.

25

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton

• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.

• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.

• Class 3: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.

26

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton

• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.

• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.

• Class 3: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.

• Class 4: Nearly all initial patterns evolve into structures that interact in complex and interesting

ways

27

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton

• Class 1: Nearly all initial patterns evolve quickly into a stable, homogeneous state.

• Class 2: Nearly all initial patterns evolve quickly into stable or oscillating structures.

• Class 3: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.

• Class 4: Nearly all initial patterns evolve into structures that interact in complex and interesting

ways

28

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton implementing Rule 30

Class 3 Cellular Automaton: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.

29

Proposed CMOS Imager Architecture: Advantages and Improvements

1D Cellular Automaton implementing Rule 30

Class 3 Cellular Automaton: Nearly all initial patterns evolve in a pseudo-random or chaotic manner.

30

Proposed CMOS Imager Architecture: Advantages and Improvements

Bit Resolution

To achieve a suitable bit resolution.

31

Proposed CMOS Imager Architecture: Advantages and Improvements

Bit Resolution

To achieve a suitable bit resolution.

Classic AD conversion Digital representation of time

32

Proposed CMOS Imager Architecture: Advantages and Improvements

Bit Resolution

To achieve a suitable bit resolution.

Classic AD conversion Digital representation of time

33

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

34

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

35

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

36

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

1 Compressed sample of 20 bits

37

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

1 Compressed sample of 20 bits

30 fps

38

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

1 Compressed sample of 20 bits

30 fps T� 1

4096 820

30 !"

39

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

1 Compressed sample of 20 bits

30 fps

8 bits counter

T� 1

4096 820

30 !"

40

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

1 Compressed sample of 20 bits

30 fps

8 bits counter T#$%�& T�

2'

T� 1

4096 820

30 !"

41

Proposed CMOS Imager Architecture: Advantages and Improvements

Readout Logic

64 × 64 Pixels, 8 bits each

1 Compressed sample of 20 bits

30 fps

8 bits counter T#$%�& T�

2'

F)%* 1

2 T#$%�&+ 24�,-

T� 1

4096 820

30 !"

42

MATLAB Analysis of the Performance of our Solution

128 256 512 1024 2048 40960

10

20

30

40

50

Number of Samples

RM

SE

of

reco

nst

ruct

ion

(%

)

full frame

block based

Average RMSE of Reconstruction

We compare the results achieved with the RMSE of reconstruction of 10 64×64 pixels images1

compressed performing a block based compressive sampling strategy (red bars) and a full frame

compressive strategy (blue bars). We devised the block based strategy by dividing each image

in 64 sub-images of 8×8 pixels to be treated separately.

1 These images can be found at: http://www2.imse-cnm.csic.es/icaveats/64x64px_images/

43

MATLAB Analysis of the Performance of our Solution

128 256 512 1024 2048 40960

20

40

60

80

100

120

Number of Samples

Tim

e o

f re

con

stru

ctio

n (

s)

full frame

block based

Average Time of Reconstruction

The resources needed to retrieve the images on lower compression ratios favour the block based

sampling strategy. However, as we diminish the amount of samples, the time of reconstruction

reverses its tendency allowing the full frame compressive strategy to outperform its block based

counterpart.

44

Conclusions

٭ We have introduced a new architecture able to collect compressed samples using

pseudo-random distributions generated on-chip.

٭ The pattern generated by the cellular automaton does not need to be transmitted and

can be easily recovered by simply knowing the initial seed.

٭ We avoid splitting the sensor array in smaller portions worsening the quality of the

samples or introducing asymmetries in the design of the sensor array.

٭ The solution applied to digitize the compressed samples improves their dynamic

range thus optimizing reconstruction.

Final Remarks

© ANAFOCUS 2007

ReferencesE. Candès. “Compressive sampling”. Int. Congress of Mathematics, pp. 1433-1452.

Madrid, Spain, 2006.

S. Becker, J. Bobin, and E. J. Candès. “NESTA: A Fast and Accurate First-Order

Method for Sparse Recovery”. SIAM Journal on Imaging Sciences, Vol. 4, No. 1,

pp. 1 - 39, Jan. 2011.

M. F. Duarte, M. A. Davenport, D. Takbar, J. N. Laska, T. Sun, K. F. Kelly, and R.

G. Baraniuk. Single-pixel imaging via compressive sampling. IEEE signal

processing magazine, Vol. 25, No. 2, pp. 83 – 91, Mar. 2008.

V. Majidzadeh, L. Jacques, A. Schmid, P. Vandergheynst and Y. Leblebici. “A

(256x256) Pixel 76.7mW CMOS Imager/Compressor Based on Real-Time In-Pixel

Compressive Sensing”. Proceedings of 2010 IEEE International Symposium on

Circuits and Systems (ISCAS), pp. 2956 – 2959. Paris, France. May, 2010

B. Kaliannan, V S. Rao Pasupureddi. “A Low Power CMOS Imager Based on

Distributed Compressed Sensing”. 27th International Conference on VLSI Design

and 13th International Conference on Embedded Systems, pp. 534 – 538. Mumbai,

India. Jan. 2014

© ANAFOCUS 2007

Acknowledgements

This work has been funded by the Spanish Government through project TEC2015-

66878-C3-1-R MINECO (ERDF/FEDER), Junta de Andalucía through project TIC

2338-2013 CEICE, the Office of Naval Research (USA) through grant

N000141410355 and CONACYT (Mexico) through grant 2016-MZO-2017-291062.