lecture 4: data compression techniques
DESCRIPTION
TSBK01 Image Coding and Data Compression. Lecture 4: Data Compression Techniques. Jörgen Ahlberg Div. of Sensor Technology Swedish Defence Research Agency (FOI). Outline. Huffman coding Arithmetic coding Application: JBIG Universal coding LZ-coding LZ77, LZ78, LZW - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/1.jpg)
Lecture 4:Lecture 4:Data Compression TechniquesData Compression Techniques
TSBK01 Image Coding and Data CompressionTSBK01 Image Coding and Data Compression
Jörgen AhlbergDiv. of Sensor Technology
Swedish Defence Research Agency (FOI)
![Page 2: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/2.jpg)
OutlineOutline
Huffman codingHuffman coding Arithmetic codingArithmetic coding
Application: JBIGApplication: JBIG
Universal codingUniversal coding LZ-codingLZ-coding
LZ77, LZ78, LZWLZ77, LZ78, LZW
Applications: GIF and PNGApplications: GIF and PNG
![Page 3: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/3.jpg)
RepetitionRepetition
Coding:Coding: Assigning binary codewords to Assigning binary codewords to (blocks of) source symbols.(blocks of) source symbols.
Variable-lengthVariable-length codes (VLC) and codes (VLC) and fixed-fixed-lengthlength codes. codes.
Instantaneous codes Instantaneous codes ½½ Uniqely decodable Uniqely decodable codes codes ½½ Non-singular codes Non-singular codes ½½ All codes All codes
Tree codesTree codes are instantaneous. are instantaneous. Tree code Tree code ,, Kraft’s Inequality. Kraft’s Inequality.
![Page 4: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/4.jpg)
Creating a Code: The Data Creating a Code: The Data Compression ProblemCompression Problem
Assume a source with an alphabet Assume a source with an alphabet AA and and known symbol probabilities known symbol probabilities {{ppii}}..
GoalGoal: Chose the codeword lengths as to : Chose the codeword lengths as to minimize the bitrate, i.e., the average number minimize the bitrate, i.e., the average number of bits per symbol of bits per symbol llii ¢¢ ppii..
Trivial solution:Trivial solution: llii = 0 = 0 88 ii.. Restriction:Restriction: We want an instantaneous code, We want an instantaneous code,
so so 22-l-l
ii ·· 1 1 (KI) must be valid. (KI) must be valid. Solution Solution (at least in theory)(at least in theory):: llii = – = – loglog p pii
![Page 5: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/5.jpg)
In practice…In practice…
Use some nice algorithm to find the code Use some nice algorithm to find the code treetree– Huffman codingHuffman coding– Tunnstall codingTunnstall coding
![Page 6: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/6.jpg)
Huffman CodingHuffman Coding
Two-step algorithm:Two-step algorithm:1.1. Iterate:Iterate:
– Merge the least probable symbols.Merge the least probable symbols.– Sort.Sort.
2.2. Assign bits.Assign bits.
a
d
b
c
0.5
0.25
0.125
0.125
0.5
0.25
0.5
0.25
0.5
Merge
Sort
Assign
0
1
0
10
11
0
10
110
111Get code
![Page 7: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/7.jpg)
Coding of the BMSCoding of the BMS
Trick:Trick: Code blocks of symbols (extended source). Code blocks of symbols (extended source). Example:Example: pp11 = ¼ , = ¼ , pp22 = ¾. = ¾.
Applying the Huffman algorithm directly:Applying the Huffman algorithm directly:1 bit/symbol1 bit/symbol..
BlockBlock PP(block)(block) CodeCode
0000 9/169/16 00 ))
0101 3/163/16 1010 approx 0.85 approx 0.85
1010 3/163/16 110110 bits/symbolbits/symbol
1111 1/161/16 111111
![Page 8: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/8.jpg)
Huffman Coding: Pros and ConsHuffman Coding: Pros and Cons
++ Fast implementations.Fast implementations.++ Error resilient: resynchronizes in Error resilient: resynchronizes in ~ ~ ll2 2 steps.steps.- The code tree grows exponentially when the The code tree grows exponentially when the
source is extended.source is extended.- The symbol probabilities are built-in in the The symbol probabilities are built-in in the
code.code.Hard to use Huffman coding for extended Hard to use Huffman coding for extended sources / large alphabets or when the sources / large alphabets or when the symbol probabilities are varying by time.symbol probabilities are varying by time.
![Page 9: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/9.jpg)
Arithmetic CodingArithmetic Coding
Shannon-Fano-EliasShannon-Fano-Elias Basic idea:Basic idea: Split the interval [0,1] according Split the interval [0,1] according
to the symbol probabilities.to the symbol probabilities. Example:Example: AA = {a,b,c,d}, = {a,b,c,d}, PP = {½, ¼, 1/8, 1/8}. = {½, ¼, 1/8, 1/8}.
![Page 10: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/10.jpg)
b
c
a
0.6
0.5
0.5
0.20.8
0.2
0.2 Start in b.Code the sequence c c a.
cba 0.9
Code the sequence c c a.) Code the interval [0.9, 0.96]
Bit Interval Decoder
1 c0.5 - 1
1 0.75 - 1
1 0.875 - 1
0 0.875 - 0.9375
1 c a0.90624 - 0.9375
b c0.5 10
cba
0.9 10.96 0.98
![Page 11: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/11.jpg)
An Image Coding ApplicationAn Image Coding Application
Consider the image content in a local environment Consider the image content in a local environment of a pixel as of a pixel as a state in a Markov modela state in a Markov model..
Example (binary image):Example (binary image):
Such an environment is called a Such an environment is called a contextcontext.. A probability distribution for A probability distribution for XX can be estimated for can be estimated for
each state. Then arithmetic coding is used.each state. Then arithmetic coding is used. This is the basic idea behind the JBIG algorithm This is the basic idea behind the JBIG algorithm
for binary images and data.for binary images and data.
X
0 0 11 0
![Page 12: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/12.jpg)
Flushing the CoderFlushing the Coder
The coding process is ended (restarted) and The coding process is ended (restarted) and the coder flushedthe coder flushed– after a given number of symbols (FIVO)after a given number of symbols (FIVO)
oror– When the interval is too small for a fixed When the interval is too small for a fixed
number of output bits (VIFO).number of output bits (VIFO).
![Page 13: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/13.jpg)
Universal CodingUniversal Coding
A universal coder doesn’t need to know the A universal coder doesn’t need to know the statistics in advance. Instead, estimate from statistics in advance. Instead, estimate from data.data.
Forward estimation:Forward estimation: Estimate statistics in a Estimate statistics in a first pass and transmit to the decoder.first pass and transmit to the decoder.
Backward estimation:Backward estimation: Estimate from already Estimate from already transmitted (received) symbols.transmitted (received) symbols.
![Page 14: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/14.jpg)
Universal Coding: ExamplesUniversal Coding: Examples
1.1. An adaptive arithmetic coderAn adaptive arithmetic coder
2.2. An adaptive dictionary techniqueAn adaptive dictionary technique– The LZ coders The LZ coders [Sayood 5][Sayood 5]
3.3. An adaptive Huffman coder An adaptive Huffman coder [Sayood 3.4][Sayood 3.4]
Arithmeticcoder
Statisticsestimation
![Page 15: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/15.jpg)
Ziv-Lempel Coding (ZL or LZ)Ziv-Lempel Coding (ZL or LZ)
Named after J. Ziv and A. Lempel (1977).Named after J. Ziv and A. Lempel (1977). Adaptive dictionary technique.Adaptive dictionary technique.
– Store previously coded symbols in a buffer.Store previously coded symbols in a buffer.– Search for the current sequence of symbols to Search for the current sequence of symbols to
code.code.– If found, transmit buffer offset and length.If found, transmit buffer offset and length.
![Page 16: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/16.jpg)
LZ77LZ77
a b c a b d a c a b d e e e fc
Search buffer Look-ahead buffer
Output triplet <offset, length, next>
123456783
8 0 13 d 0 e 2 f
2
Transmitted to decoder:
If the size of the search buffer is N and the size of the alphabet is Mwe need
bits to code a triplet.
Variation:Variation: Use a VLC to code the triplets!PKZip, Zip, Lharc,PNG, gzip, ARJ
![Page 17: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/17.jpg)
Drawback with LZ77Drawback with LZ77
Repetetive patterns with a period longer Repetetive patterns with a period longer than the search buffer size are not found.than the search buffer size are not found.
If the search buffer size is 4, the sequenceIf the search buffer size is 4, the sequence a b c d e a b c d e a b c d e a b c d e …a b c d e a b c d e a b c d e a b c d e …will be expanded, not compressed.will be expanded, not compressed.
![Page 18: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/18.jpg)
LZ78LZ78
Store patterns in a dictionaryStore patterns in a dictionary Transmit a tuple <Transmit a tuple <dictionary indexdictionary index, , nextnext>>
![Page 19: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/19.jpg)
LZ78LZ78
Output tuple <dictionary index, next>
Dictionary:
1 a
2 b
3 c
4 a b
5 a b c
a b c a b a b c
0 aTransmitted to decoder: 0 b 0 c 1 b 4 c
Decoded: a b c a a bb c
Strategy needed for limiting dictionary size!Strategy needed for limiting dictionary size!
![Page 20: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/20.jpg)
LZWLZW
Modification to LZ78 by Terry Welch, 1984.Modification to LZ78 by Terry Welch, 1984. Applications: GIF, v42bisApplications: GIF, v42bis Patented by UniSys Corp.Patented by UniSys Corp. Transmit only the dictionary index.Transmit only the dictionary index. The alphabet is stored in the dictionary in The alphabet is stored in the dictionary in
advance.advance.
![Page 21: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/21.jpg)
LZWLZW
Output: dictionary index
Encoder dictionary:
1 a
2 b
3 c
4 d
5 a b
a b c a b a b c
1
Transmitted:
2 3 5 5
Decoded:
a b c a b a b
6 bc
7 ca
8 aba
9 abc
Decoder dictionary:
1 a
2 b
3 c
4 d
5 a b
6 bc
7 ca
8 aba
Input sequence:
![Page 22: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/22.jpg)
And now for some applications:GIF & PNG
![Page 23: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/23.jpg)
GIFGIF
CompuServe Graphics Interchange Format (1987, CompuServe Graphics Interchange Format (1987, 89).89).
Features:Features:– Designed for up/downloading images to/from BBSes via Designed for up/downloading images to/from BBSes via
PSTN.PSTN.– 1-, 4-, or 8-bit 1-, 4-, or 8-bit colour palettescolour palettes..– Interlace for Interlace for progressive decodingprogressive decoding (four passes, starts (four passes, starts
with every 8th row).with every 8th row).– Transparent colourTransparent colour for non-rectangular images. for non-rectangular images.– Supports multiple images in one file (”animated GIFs”).Supports multiple images in one file (”animated GIFs”).
![Page 24: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/24.jpg)
GIF: MethodGIF: Method
Compression by LZW.Compression by LZW. Dictionary size 2Dictionary size 2bb+1+1 8-bit symbols 8-bit symbols
– bb is the number of bits in the palette. is the number of bits in the palette.
Dictionary size doubled if filled (max 4096).Dictionary size doubled if filled (max 4096). Works well on computer generated images.Works well on computer generated images.
![Page 25: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/25.jpg)
GIF: ProblemsGIF: Problems
Unsuitable for natural images (photos):Unsuitable for natural images (photos):– Maximum 256 colors (Maximum 256 colors ()) bad quality). bad quality).– Repetetive patterns uncommon (Repetetive patterns uncommon ()) bad bad
compression).compression).
LZW patented by UniSys Corp.LZW patented by UniSys Corp. Alternative: PNGAlternative: PNG
![Page 26: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/26.jpg)
PNG: Portable Network GraphicsPNG: Portable Network Graphics
Designed to replace GIF.Designed to replace GIF. Some features:Some features:
– Indexed Indexed oror true-colour images true-colour images ((·· 16 bits per plane) 16 bits per plane)..– Alpha channel.Alpha channel.– Gamma information.Gamma information.– Error detection.Error detection.
No support for multiple images in one file.No support for multiple images in one file.– Use MNG for that.Use MNG for that.
Method:Method:– Compression by LZ77 using a 32KB search buffer.Compression by LZ77 using a 32KB search buffer.– The LZ77 triplets are Huffman coded.The LZ77 triplets are Huffman coded.
More information:More information: www.w3.org/TR/REC-png.html www.w3.org/TR/REC-png.html
![Page 27: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/27.jpg)
SummarySummary
Huffman codingHuffman coding– Simple, easy, fastSimple, easy, fast– Complexity grows exponentially with the block lengthComplexity grows exponentially with the block length– Statistics built-in in the codeStatistics built-in in the code
Arithmetic codingArithmetic coding– Complexity grows linearly with the block sizeComplexity grows linearly with the block size– Easily adapted to variable statistics Easily adapted to variable statistics )) used for coding of Markov used for coding of Markov
sourcessources Universal codingUniversal coding
– Adaptive Huffman or arithmetic coderAdaptive Huffman or arithmetic coder– LZ77: Buffer with previously sent sequences <offset,length,next>LZ77: Buffer with previously sent sequences <offset,length,next>– LZ78: Dictionary instead of buffer <index,next>LZ78: Dictionary instead of buffer <index,next>– LZW: Modification to LZ78 <index>LZW: Modification to LZ78 <index>
![Page 28: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/28.jpg)
Summary, contSummary, cont
Where are the algorithms used?Where are the algorithms used?– Huffman coding:Huffman coding: JPEG, MPEG, PNG, … JPEG, MPEG, PNG, …– Arithmetic coding: Arithmetic coding: JPEG, JBIG, MPEG-4, …JPEG, JBIG, MPEG-4, …– LZ77:LZ77: PNG, PKZip, Zip, gzip, … PNG, PKZip, Zip, gzip, …– LZW:LZW: compress, GIF, v42bis, … compress, GIF, v42bis, …
![Page 29: Lecture 4: Data Compression Techniques](https://reader036.vdocuments.us/reader036/viewer/2022070407/56814333550346895dafa619/html5/thumbnails/29.jpg)
FinallyFinally
These methods work best if the source These methods work best if the source alphabet is small and the distribution alphabet is small and the distribution skewed.skewed.– TextText– GraphicsGraphics
Analog sources (images, sound) require Analog sources (images, sound) require other methodsother methods– complex dependenciescomplex dependencies– accepted distortionaccepted distortion