Bits, Bytes, and Codes

Telecommunications 1Peter Mathys

Black and White Image

Suppose we want to draw a B&W image on a computer screen.We first subdivide the screen into small rectangles or squares called pixels or pels (from picture elements).A typical TV screen has about 300x400 = 120,000 pixels.A high resolution Computer monitor has about 1000x1300 = 1,300,000 pixels.

Screen with 9x13 = 117 Pixels

Black and White Image

To draw a B&W image, we paint some of the pixels black and leave the rest white.For instance, the letter “A” would look like this:

B&W House on 9x13 Grid

Binary Representation

To store such an image in a computer or transmit it from one computer to another, we need to represent it in binary, i.e., as a sequence of zeros and ones.Since we use only two colors, we can describe each pixel in binary as:

0: white1: black

B&W (numbered) House

Binary Representation

Internally, a computer stores this as:000000101000000000101100000000100010000000100000100000110111011000001010101000000101110100000010000010001111111111111

This requires 9x13 = 117 bits.

Adding the Color Red

Now, suppose that we want to make our picture of the house a little fancier and color the roof in red.Doing this by hand is a simple matter. We just erase the black pixels that make up the roof, grab a red pen, and then color the roof pixels with that pen.

3-Color House on 9x13 Grid

3-Color Binary Representation

Now we have 3 colors, which we can number 0, 1, 2, or, in binary, 00, 01, 10.Each pixel can now be described in binary as:

0 or 00: white1 or 01: black2 or 10: red

3-Color (numbered) House

33--Color Binary RepresentationColor Binary Representation

Internally, a computer stores this now as:000000000000100001000000000000000000100010010000000000000000100000001000000000000000100000000000100000000000100100010101000110000000000001000100010001000000000000010001010100010000000000000100000000000100000001010101010101010101010101

This requires 9x13x2 = 234 bits.

Do we Need this Many Bits?

For the B&W picture we used 0 for white and 1 for black.For the 3-color picture, why can’t we save some bits and use the following binary representation:

0: white1: black10: red

Do we Need this Many Bits?

To describe a single pixel, we could use the scheme. But, to describe a series of pixels, we must be able to parse unambiguously.Example: 101100011 could be parsed as

1,0,1,10,0,0,1,1 or10,1,1,0,0,0,1,1 or1,0,1,1,0,0,0,1,1

How about 4 Colors?

For the 3-color house we have used the binary strings 00, 01, 10.But 11, which corresponds to 3 in decimal, was not used.We can use 11 to make a green lawn. Then

0=00: white, 1=01: black,2=10: red, 3=11: green.

4-Color House on 9x13 Grid

4-Color (numbered) House

44--Color Binary RepresentationColor Binary Representation

Internally, a computer stores this now as:000000000000100001000000000000000000100010010000000000000000100000001000000000000000100000000000100000000000100100010101000110000000000001000100010001000000000000010001010100010000000000000100000000000100000011111111111111111111111111

It still requires 9x13x2 = 234 bits.

Using 5 Colors

What if we want to make the window of our house blue? Now we have 5 colors and we need a 3-bit binary representation:

0 or 000: white1 or 001: black2 or 010: red3 or 011: green4 or 100: blue

5-Color House on 9x13 Grid

5-Color (numbered) House

55--Color Binary RepresentationColor Binary Representation

A computer stores this now as:000000000000000000010000001000000000000000000000000000010000010001000000000000000000000000010000000000010000000000000000000000010000000000000000010000000000000000010001000100100100000001010000000000000000001000100000100000001000000000000000000001000100100100000001000000000000000000001000000000000000001000000000011011011011011011011011011011011011011

This requires 9x13x3 = 351 bits.

More Colors

We have not used the 3-bit strings 101 (decimal 5), 110 (decimal 6), and 111 (decimal 7).Thus, without increasing the number of bits used for each pixel, we could use 3 more colors.Or, we could try to find some clever scheme to keep the same number of colors, but compressthe binary representation.

Simple Lossless Compression

Huffman coding assigns short codewordsto frequent colors and long codewords to infrequent ones in such a way that parsing of codeword strings is unambiguous.Example for 5-color house:0: white, 100: green, 110: red

101: black, 111: blueThe resulting Huffman tree is shown next.

Huffman Code for 5 Colors

55--Color House, Compressed Color House, Compressed

A computer stores the 5-color house now as (using the Huffman code for compression):

00000011001010000000001100110101000000001100001100000000110000001100000011010101111111110101110000001010111011101010000001010111111111010100000010100000101000100100100100100100100100100100100100100

This requires only 197 bits (56% of 351).Compression ratio: x:1 where x=351/197=1.78

Compression of 32 x 16 “Logo”

There are 6 colors => 3 bits/colorUncompressed: 32 x 16 x 3 = 1536 bits

Completed Huffman Code

White (most frequent) has shortest codeBlack, red (least frequent) have longest codes

Completed Huffman Code

Code:

Bits used for image:

Compression ratio: x:1, x=1536/772=1.99 Bits per pixel: 772/512=1.51What is the best that can be done?=> Look at entropy of image (C.E. Shannon).

Entropy H(X) of Source

The notion of entropy (or uncertainty) was introduced by C.E. Shannon in 1950.Simplest case: Source output symbols, denoted by random variable X, are i.i.d. (independent and identically distributed) with probability mass function pX(x). If X can take on values 0,1,…,M-1, then entropy H(X) is

Binary Entropy FunctionH(X)=-p(0)*log2(p(0))-p(1)*log2(p(1))

Max entropy for p(0)=p(1)=1/2

Min entropy for p(0)=0 or p(0)=1

H(X) for “Fun” Picture

Probabilities:

Entropy H(X):

This is theoretical limit for lossless compression.Compare to 1.51 bits/pixel

What can m Bits Describe?

In general, if we have m bits, we can represent

different objects.Examples:

Counting in Binary

For m=2 we count:

00011011

corresponding to decimal 0,1,2,3.

For m=3 we count:000001010011100101110111

decimal: 0,1,2,…,6,7.

Powers of 2

Typical Powers of 2

The cases m=4,8,16,24,32,40,64 are quite common:2^4 = 162^8 = 2562^16 = 65,5362^24 = 16,777,2162^32 = 4,294,967,2962^40 = 1,099,511,627,776 = 1.1 x 10^122^64 = 18,446,744,073,709,551,616 = 1.8 x 10^19

Comparison: If youlive to be 100 years old,you have lived for3,155,760,000 seconds(approx. 3x10^9 sec).

Powers of 2

Each additional bit doubles the number of objects that can be described.Suppose now we have n objects. How many bits do we need to describe them?The answer is m bits, where m is equal to x rounded up to the nearest integer, and

x = log(n)/log(2)(both logs to the same base, e.g., log10 or ln)

The Alphabet in Binary

Suppose we have 26 letters. How many bits do we need to represent them?We first compute x as

x = log(26)/log(2) = 1.4150/0.3010 = 4.7004

The next highest integer is 5 and thus we need m=5 bits. Such a code was developed in 1874 by Emile Baudot and it is known as Baudot code. It is used in the Telex network.

The Alphabet in Binary

A more modern code is the ASCII (American Standard Code for Information Interchange) code which was developed in 1962.There are 26 small and 26 capital letters plus 10 numbers and 10 (or more) punc-tuation marks (,.!?:;()−/). Nowx = log(72)/log(2) = 1.8573/0.3010 = 6.1699

and thus we need m=7 bits.

ASCII Code

Here are a few samples from the ASCII code:

A = 1000001, F = 1000110, K = 1001011B = 1000010, G = 1000111, L = 1001100C = 1000011, H = 1001000, M = 1001101D = 1000100, I = 1001001, N = 1001110E = 1000101, J = 1001010, O = 1001111

The word “HELLO” would thus be1001000 1000101 1001100 1001100 1001111

ASCIICode

Bits, Bytes, Words

Computers normally use bits in blocks of 4, 8, 16, 32, and 64. The names for these are• 4 bits: Nibble• 8 bits: Byte• 16 bits: Word• 32 bits: Doubleword

Kilo Bits (kb) and Bytes (kB)Often we need more than a few bits or bytes, e.g., to describe the size of a text file or the speed of a modem.• 1 kilobit = 1 kb = 1000 bits.• 1 kilobyte = 1 kB = 1000 bytes = 8000 bits.

Memory size increases in powers of 2. In this case ‘kilo’ often means 2^10=1024.• 1 ‘kb’ = 1024 bits.• 1 ‘kB’ = 1024 bytes = 8192 bits.

Mega Bits (Mb) and Bytes (MB)

A big program (e.g., a Web browser) or a fast LAN (local area network) connection need even more bits/bytes for their description.• 1 megabit = 1 Mb = 1000 kb = 1,000,000 bits.• 1 megabyte = 1 MB = 1,000,000 bytes.

For memory size ‘mega’ often means 2^20=1,048,576 (approx 5% more than mega).• 1 ‘Mb’ = 1024 ‘kb’ = 1,048,576 bits.• 1 ‘MB’ = 1024 ‘kB’ = 1,048,576 bytes.

Giga Bits (Gb) and Bytes (GB)

To measure disk space or the speed of optical communications, yet a larger quantity, called giga, is used.• 1 gigabit = 1 Gb = 1000 Mb = 1,000,000,000 bits.• 1 gigabyte = 1 GB = 1000 MB = 1,000,000,000 bytes.

As before ‘giga’ means 2^30=1,073,741,824.• 1 ‘Gb’ = 1024 ‘Mb’ = 1,073,741,824 bits.• 1 ‘GB’ = 1024 ‘MB’ = 1,073,741,824 bytes.

Big, Bigger, Biggest

International System of Units:• (unit) 10^0 = 1• kilo k 10^3 = 1,000• mega M 10^6 = 1,000,000• giga G 10^9 = 1,000,000,000• tera T 10^12 = 1,000,000,000,000• peta P 10^15 = 1,000,000,000,000,000• exa E 10^18 = 1,000,000,000,000,000,000

Small, Smaller, Smallest

International System of Units:• (unit) 10^0 = 1• milli m 10^-3 = 0.001• micro µ 10^-6 = 0.000,001• nano n 10^-9 = 0.000,000,001• pico p 10^-12 = 0.000,000,000,001• femto f 10^-15 = 0.000,000,000,000,001• atto a 10^-18 = 0.000,000,000,000,000,001

Hexadecimal Notation

Long strings of bits are cumbersome to handle for humans. The hexadecimal (hex) notation uses a single symbol for 4 bits (a nibble) as follows:

0 = 0000, 4 = 0100, 8 = 1000, C = 11001 = 0001, 5 = 0101, 9 = 1001, D = 11012 = 0010, 6 = 0110, A= 1010, E = 11103 = 0011, 7 = 0111, B= 1011, F = 1111

ASCII Code Revisited

Representing ASCII codes in hex results in a much more compact representation:

A = 1000001 = 41, F = 1000110 = 46, K = 1001011 = 4BB = 1000010 = 42, G = 1000111 = 47, L = 1001100 = 4CC = 1000011 = 43, H = 1001000 = 48, M = 1001101 = 4DD = 1000100 = 44, I = 1001001 = 49, N = 1001110 = 4EE = 1000101 = 45, J = 1001010 = 4A, O = 1001111 = 4F

The word “HELLO” can now be written as

48 45 4C 4C 4F

Decimal Numbers

When we write a decimal number, e.g., 12059, then the rightmost digit represents the “ones”, the next digit represents the “tens”, the next one the “hundreds”, etc.Mathematically, a decimal number like 12059 is expressed as12059 = 9*10^0 + 5*10^1 + 0*10^2 + 2*10^3 + 1*10^4

Binary Numbers

Binary numbers use the same principle, except that now we talk about “ones”, “twos”, “fours”, “eights”, etc.Thus, the binary number 10111011 is expressed mathematically as (left to right)10111011 =

= 1*2^7 + 0*2^6 + 1*2^5 + 1*2^4 + 1*2^3 ++ 0*2^2 + 1*2^1 + 1*2^0

Binary to Decimal Conversion

Nothing prevents us now from evaluating the explicit binary representation in the familiar decimal system. Thus (from left to right)10111011 =

= 1*2^7 + 0*2^6 + 1*2^5 + 1*2^4 + 1*2^3 ++ 0*2^2 + 1*2^1 + 1*2^0

= 128 + 0 + 32 + 16 + 8 + 0 + 2 + 1= 187 (in decimal notation)

Decimal to Binary Conversion (LSB first)

Example (LSB first)

Decimal to Binary Conversion (MSB first)

Conversely, to convert a decimal number n to binary representation, we ask whether the quantity 2^m, for m=log(n)/log(2) (take next lowest integer for m) is contained in n. If so, we set n := n-2^m and repeat the process.Example: Convert 136 to binarylog(136)/log(2) = 7.087 => m = 7 and 2^m = 128log(136-128)/log(2) = 3 => m = 3 and 2^m = 8=> 136 = 1*2^7 + 1*2^3 = 10001000 (in binary)

Another Example (MSB first)

Convert n=205 to binary. We start fromlog(205)/log(2) = 7.68 => m=7

and then try all smaller m in sequence:205-2^7 = 205-128 = 77 => b7=1, n=7777-2^6 = 77-64 = 13 => b6=1, n=1313-2^5 = 13-32 = -19 => b5=0, n=1313-2^4 = 13-16 = -3 => b4=0, n=1313-2^3 = 13-8 = 5 => b3=1, n=55-2^2 = 5-4 = 1 => b2=1, n=11-2^1 = 1-2 = -1 => b1=0, n=11-2^0 = 1-1 = 0 => b0=1, n=0

=> 205 = 11001101 (= b7 b6 b5 b4 b3 b2 b1 b0)

Binary Bank of Boulder

Here’s another way to look at decimal to binary conversion.The Binary Bank of Boulder pays out money using bills of 1, 2, 4, 8, 16, 32, 64, and 128 MUs (monetary units).The bank teller can hand out any amount from 0 to 255 MUs but can never give two bills of the same denomination to a customer.The bills are shown on the next slide.

Binary Bank of Boulder Currency

Any amount between 0 and 255 can be paid using at most one bill of each value.

Example: Pay 50 MUs

A 64 MU bill is too much, so start from 32 MU.50-32=18 MU, so use a 16 MU bill next.18-16=2 MU, so add a 2 MU bill.Pay out:

Don’t use 8 MU, 4 MU, and 1 MU bills.Thus 50 is 110010 (2^5+2^4+2^1) in binary. Pad with zeros on the left to make byte: 00110010

Internet Addresses

Internet addresses use a “dotted decimal” notation to describe binary 32-bit addresses.Our server (ece.colorado.edu), for instance, has IP address

128.138.189.30This converts to the 32-bit address

10000000.10001010.10111101.00011110which every data packet to or from our server carries in its header.

ISBN-10 Code

Picture shows bar code with ISBN-10 number above and EAN number belowISBN: International Standard Book NumberEAN: European Article Number (extension of UPC, Universal Product Code)

ISBN-10 Number Format

ISBN-10 numbers are of the formd10,d9,d8,d7,d6,d5,d4,d3,d2,d1

The last digit (d1) is a check digitTo check integrity of number compute

ISBN-10 is ok if v is divisible by 11 (without remainder).

ISBN-10 Example

Check ISBN number: 0-07-249350-X

OK since 176 = 16*11

Generate Check digit for 0-471-00186-?

13*11 = 143 = 139+4 => d1 = 4Complete ISBN-10 is: 0-471-00186-4