Embedded Signal Approach to Image Texture Reproduction Analysis

Peter D. Burnsa and Donald Baxterb

aBurns Digital ImagingbSTMicroelectronics Ltd.

Full paper:P. D. Burns and D. Baxter, Embedded Signal Approach to Image Texture Reproduction Analysis, Proc. SPIE Vol. 9016, Image Quality and System Performance XI, 2014

2

Introduction

1. Evolution of the intelligence and performance of camera noise-reduction (NR) algorithms poses a significant challenge for texture evaluation

• Algorithms are ‘content aware’ and adapt to local image content• Deadleaves and sine-wave methods can yield results that are

inconsistent with visual impression2. Successful methods for measuring the capture of image texture,

• Test target should have texture fluctuations similar to natural materials such as grass, to ensure realistic NR algorithm behavior.

• Analysis should result in a reliable texture MTF in the presence of image noise such as that observed with current equipment and high ISO (low-light) settings.

3

Introduction

• ISO 25,600 grass and Siemens Star images, processed with the same non-local means filter strength

• grass is blurred but the low contrast Siemens Star < 0.4 cycles/pixel are not• Siemens Star image for the non-local means filter had a constant contrast

(9% amplitude) up to 0.4cycles/pixel

4

Embedded test signals

We investigate the use of two types of embedded test signals1. Grid multi-burst method

• Low amplitude version of that used in IEC-61146-1, Video cameras - Methods of measurement - Part 1: Non-broadcast single-sensor cameras

2. Stacked sine-wave method• Several frequencies combined, or stacked

Combine(embed)texture

Camerasimulation

Lens MTFnoise

Embedded signal

Imageprocessing

Texture MTFAnalysis

noisereduction

signal detectionestimation

Stacked sine-waveMulti-burst grid

5

Stacked sine-wave method

• One dimensional sine-wave signals• Several frequencies combined, or stacked

0 50 100 150 200 250 3000.6

0.8

1

1.2

1.4

1.6

1.8x 10

4

pixels

sign

al,

16-b

it va

lues

50 100 150 200 250 300

50

100

150

200

250

300

(higher contrast than used)

MjjfasN

nnnj ,...,1 ,2sin

1

N ,...,1 = random phase values

Six-frequency stacked sine-wave region and a cross-section

6

Stacked sine-wave analysis method

• Discrete Fourier transform

2/

12/

22/

12/

,,M

Mx

qypxiM

My

eyxsqpU

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.40

10

20

30

40

50

60

70

80

90

100

110

Resolution [cycles/pixel]

Mod

ulat

ion

[%]

• Zero-frequency cross-section 0,pU

• Peak detection• Example for 5 components

7

Embedded signal

Two types of texture natural• Computed deadleaves• Natural grass

Stacked sine-waves with deadleaves textureMulti-burst grid with grass texture

8

Camera simulationReference Texture

Image (Linear)Dead Leaves, Grass

Synthetic Embedded Signal

Synthetic Siemens Star

Gaussian Blursigma = 3 pixels

Image Sensor NoisePhoton shot and readout noise

Down Sample By 4

Non LocalMeans Filter

Buades

sRGB Gamma

3480 x 324014-bit linear sRGB

870x81014-bit linear sRGB

870x8108-bit sRGB

Texture MTF MetricsDFT and peak detection

2-D Sine Wave FitSiemens Star

Low Pass FilterCut-off in cycles/pixel

Filter Order

2048x2048linear sRGB

Additive noise model• 100 – 25600 ISO• 2 e- readout noise• 3.9 um pixel• APS-C sensor

Gaussian blur after down sampling

9

Stability of the measurement without texture

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

10

20

30

40

50

60

70

80

90

100

Resolution [cycles/pixel]

Mod

ulat

ion

[%]

ISO100

ISO400

ISO1600ISO6400

ISO25600

Stacked Sine-wave: without texture, for ISO 25600

Results for 6% amplitude sine-wave and various ISO (noise) settings

10

Stability of the measurement without texture

Results for 6% amplitude multi-burst and various ISO (noise) settings

Stacked sine-wave and multi-burst grid methods showed stable results for the range of noise-levels tested.

11

Stability of the measurement with texture

Stacked sine-wave with computed (dead-leaves) texture

Multi-burst with natural (grass) texture

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.50

10

20

30

40

50

60

70

80

90

100

Resolution [cycles/pixel]M

odul

atio

n [%

]

ISO100

ISO400

ISO1600ISO6400

ISO25600

12

Application to noise reduction

• Non-local means filtering: searches for a pattern around a pixel (similarity window within a larger area)

• Adapts to local structure• Example for grass texture alone

– Reduces higher frequency noise– Content >0.025 cy/pixel is filtered

• With embedded sine-wave signals– Stacked sine-waves results were visually closer than the grid

multi-burst to a visual match to the grass texture (only) after NL filtering

Buades, A., Coll, B., and Morel, J., “A non-local algorithm for image denoising,” Computer Vision and Pattern Recognition, 2, 60-65 (2005)

13

Application to noise reduction

ISO 25,600 after NL filter

Grass texture with embedded sine-waves, after NL filter

Multi-burst grid

Input grass texture

14

Application to noise reduction

Without texture• similar results except at high noise, low frequencies

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.450

10

20

30

40

50

60

70

80

90

100

110

Resolution [cycles/pixel]

Mod

ulat

ion

[%]

Reference

ISO100

ISO400ISO1600

ISO6400

ISO25600

15

Application to noise reduction

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4 0.450

10

20

30

40

50

60

70

80

90

100

Resolution [cycles/pixel]

Mod

ulat

ion

[%]

Reference

ISO100

ISO400ISO1600

ISO6400

ISO25600

With texture • Higher response for multi-burst method indicating contrast has been increased by

addition of grass texture• Stacked sine-wave results were better visual match to reference (no-texture)

results

16

Conclusions

• Two embedded-signal methods for texture measurement– Combining, or stacking several components with different frequencies– Test by embedding on both natural and computed texture– Low contrast so as no minimize interaction for image processing

• Camera simulation (MTF and noise) were used to generate test images• Both embedded methods yielded stable texture-reproduction measures for

the range of exposure levels• Stacked method can be used with a lower contrast than the grid method,

closer to visual detection• Appearance with and without texture is similar

• Non-local mean filtering was used to test embedded methods• Embedded methods were robust and extracted the reference (low-noise) MTF

profile• Blur generated by NL filter for embedded stacked texture method appeared

very similar to processed regions of normal texture – a positive result

17

Conclusions

• Stacked method can be used with a lower contrast than the grid method, closer to visual detection

• Appearance with and without texture is similar• Non-local mean filtering was used to test embedded methods• Embedded methods were robust and extracted the reference (low-

noise) MTF profile• Blur generated by NL filter for embedded stacked texture method

appeared very similar to processed regions of normal texture – a positive result