deep image compression with iterative non- uniform ... · october 9, 2018 international conference...

International Conference on Image Processing ICIP-2018October 9, 2018 1

Deep Image Compression with Iterative Non-uniform Quantization

Jianrui Cai and Lei Zhang

Dept. of Computing

The Hong Kong Polytechnic University


Introduction

Proposed Method

Experimental Results

Conclusions

Overview


Introduction: motivationFa

ceb

oo

kIn

stag

ram


Introduction : image compression

Image compression is a must to reduce the storage space and provide aneconomic solution to a wide range of image storage and transmission systems.

Image QualityHigh Low

Bit Per Pixel


Introduction : Pipeline

Image Compression Pipeline

ForwardTransform

Quantization Encoder

InverseTransform

De-QuantizationDecoder

0101011...110000Bitstream

Rate

Distortion


Introduction : formulation

Rate-Distortion curve

Loss = argmin [R(z) + λ D(X, Y)]

• X: the original image;

• Z: the latent image;

• Y: the reconstructed image;

• R(): the entropy of the latent image;

• D(): the distortion between two images.


Introduction: JPEG

DCT

QuantizationHuffmanEncoder

IDCT

De-Quantization

HuffmanDecoder

0101011...110000Bitstream

Pre-processing

Processing

Pipeline of JPEG

Pros & Cons: Simple but often with artifact (e.g., blocking)


Introduction: JPEG 2000

DWT

QuantizationEBCOT

Encoder

DWT

De-Quantization

EBCOTDecoder

0101011...110000Bitstream

Pre-processing

Processing

Pipeline of JPEG 2000

Pros & Cons: Much improved but still with visual artifact (e.g., rings)


Introduction: deep neural networks

0101011...110000Bitstream

Encoding Network Quantization Decoding Network

Entropy Network

CABAC

probability


Deep Neural Network

• Pros: end-to-end training; adaptively learn an effective encoder-decoder from a large amount of image data and in a larger context torepresent more complex image structures; reduce the visual artifactsin the decompressed image.

• Cons: time-consuming, uniform quantization

Introduction: deep neural networks


Proposed Method: formulation

Formulation

Reconstructed image

Decoder

Quantizer

Encoder

Original image

Network parameters


Inp

ut

25

6×

25

6×

3

Co

nv

64×3×3 (S2)

Co

nv

+ R

eLU

64×3×3 (S1)

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nca

t

Co

nv

128×3×3 (S2)

Co

nv

+ R

eLU

64×3×3 (S1)

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nca

t

3 Concat Blocks3 Concat Blocks

Co

nv

128×3×3 (S2)

Co

nv

+ R

eLU

128×3×3 (S1)

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nca

t

3 Concat Blocks

Conv+

Sigmoid32×3×3 (S1)

Quantizer

Ou

tpu

t

25

6×

25

6×

3

Co

nv

Co

nv

+ R

eLU

64×3×3 (S1)

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nca

t

64×3×3 (S1)

3 Concat Blocks 3 Concat Blocks

Sub

pix

els

128×3×3 (S1)

3 Concat Blocks

Sub

pix

els

Sub

pix

els

×2×2×2

… … …

………

Co

nv

12×3×3 (S1)

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nca

t

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nv

+ R

eLU

Co

nca

t

Proposed Method: network architecture


Proposed Method: loss function

Loss function


Proposed Method: alternative network and quantizer training

With the encoder, we have:

z: the latent image feature

The probability density function (PDF) of z, denoted by P(z):



The optimal quantizer:

quantized values zq:

decision boundaries bq:



Fine-tune the network with the updated quantizer:

Updated PDF of z:

Updated latent image feature z:


Proposed Method: entropy network

Current symbol

Context

Unknow-context

32

32

32 MC

on

v+

ReL

U

64×3×3 (S1)

MC

on

v+

ReL

U

MC

on

v+

ReL

U

MC

on

v+

ReL

U

SUM

Residual Block64×3×3 (S2)

FC1

FC2

1024 64 4

P( z=0 | context )

P( z=1 | context )

P( z=2 | context )

P( z=3 | context )


Experimental results: Kodak dataset


Experimental results: McMaster dataset


Experimental results: BSD300 dataset


Experimental results: visual examples


Experimental results: visual examples

(a) original (b) JPEG (c) JPEG 2000 (d) Uniform (e) Proposed


Conclusions

• We presented an iterative non-uniform quantizationscheme for deep image compression network.

• The quantizer and the encoder-decoder network areupdated alternatively.

• Compared with previous deep compressors, our methodexhibits better PSNR based rate-distortion curves, as wellas better visual quality.

deep image compression with iterative non- uniform ... · october 9, 2018 international conference...

Documents