Theory of Algorithms:Theory of Algorithms:Greedy TechniquesGreedy Techniques

Theory of Algorithms:Theory of Algorithms:Greedy TechniquesGreedy Techniques

James Gain and Edwin Blake{jgain | edwin} @cs.uct.ac.za

Department of Computer Science

University of Cape Town

August - October 2004

ObjectivesObjectives

To introduce the mind set of greedy techniques

To show some examples of greedy algorithms: Change Making

Huffman Coding

To discuss the strengths and weaknesses of greedy techniques

“Greed, for lack of a better word, is good! Greed is right! Greed works!” Gordon Grecko (Michael Douglas) in Wall Street

Greedy AlgorithmsGreedy Algorithms

Optimization problems solved through a sequence of choices that are:1. Feasible - satisfy problem constraints

2. Locally Optimal - best choice among all feasible options for that step

3. Irrevocable - no backing out

A Greedy grab of the best alternative, hoping that a sequence of locally optimal steps will lead to a globally optimal solution

Even if not optimal, sometimes an approximation is acceptable

Not all optimization problems can be approached in this manner!

Applications of the Greedy StrategyApplications of the Greedy Strategy

Optimal solutions: Change making Minimum Spanning Trees (MST) - Prim’s and Kruskal’s

Algorithms Single-source shortest paths - Dijkstra’s Algorithm Simple scheduling problems Huffman codes

Approximations: Traveling Salesman Problem (TSP) Knapsack problem Other combinatorial optimization problems

Change MakingChange Making

Problem: give change for a specific amount n, with the least number of coins of the denominations d1 > d2 > … > dm

Example: Smallest change for R2.54 using R5,

R2, R1, 50c, 20c, 10c, 5c, 2c, 1c R2 + 50c + 2c + 2c = R2.54

Algorithm: At any step choose the coin of the largest denomination

that doesn’t exceed the remaining total; Repeat

Note: Optimal for reasonable sets of coins

Minimum Spanning Tree ProblemMinimum Spanning Tree Problem

Problem: Given n points, connect them in the cheapest way so that there is a path between any pair of points

Spanning Tree (of a connected graph G ): A connected acyclic subgraph of G that includes all of G’s vertices.

Minimum Spanning Tree (of a weighted graph G ): A spanning tree of G with minimum total weight

Algorithms: Prim’s and Kruskal’s Algorithms Example: 3

42

14

26

1

3

3

42

1

2

1

3

Shortest Paths ProblemShortest Paths Problem

Single Source Shortest Paths Problem: Given a weighted graph G, find the shortest paths from

a source vertex s to each of the other vertices Solution:

Dijkstra’s Algorithm (a relative of Prim’s MST) for positive weights

3

42

14

26

1

3

2

1

2

42

1

2

3

31

4

Text CompressionText Compression

Variable length encoding: Compresses text by assigning codes of different length to characters

based on their probability of occurrence

Used by Samuel Morse in constructing telegraph codes

Prefix-free Codes: Codes are unique in that no code is a prefix for a code of another

character

Tree for Binary Codes: Leaves are characters

Left edge codes 0, right edge codes 1

The code of a character is a simple walk from root to leaf

Algorithm for Huffman CodingAlgorithm for Huffman Coding

Invented by David Huffman as part of a class assignment while he was an undergraduate

Algorithm: Construct a Huffman Tree that assigns shorter strings to higher

frequencies. This defines a Huffman Code

1. Initialize n one-node trees labeled with the characters of the alphabet. Record the frequency (weight) of each character in the root

2. REPEAT until a single tree is obtained:

Find two trees with the smallest weight

Make them the left and right sub-tree of a new tree and record the sum of their weights

in the root

Example: Huffman CodingExample: Huffman Coding

0.55C0.15

A0.4

B0.45

C0.15

A0.4

B0.45

0.55

C0.15

A0.4

B0.45

1.0

Char Code

A 01

B 1

C 00

Notes on Huffman CodingNotes on Huffman Coding

Compression Ratio: Standard measure of compression

CR = (x - y) / y * 100%, where x is compressed and y is uncompressed

Typically, 20-80% in the case of Huffman Coding

Yields optimal compression provided: The probabilities of character occurrences are

independent

Probabilities are known in advance

Strengths and Weaknesses of Strengths and Weaknesses of Greedy TechniquesGreedy Techniques

Strengths: Intuitively simple and appealing

Weaknesses: Only applicable to optimization problems

Doesn’t always produce an optimal solution

Summary To-DateSummary To-Date

Introduction 1.1-1.4

Fundamentals of the Analysis of Algorithm Efficiency

2.1-2.4, 2.6, 2.7

Brute Force 3.1-3.4

Divide-and-Conquer 4.1-4.3, 4.5, 4.6

Decrease-and-Conquer 5.1-5.6

Transform-and-Conquer 6.1, 6.5, 6.6

Space and Time Tradeoffs 7.1-7.2

Greedy Techniques 9.4