Lecture 19

Greedy Algorithms

Minimum Spanning Tree Problem

Example: Coin Changing Problem

• Give change using the minimum number of coins.• Greedy approach: always use larger coins first• Example when it doesn’t work: 16c due, coins of 12,

10, 5 and 1.• Greedy: 12, 1, 1, 1, 1• Optimal: 10, 5, 1• Moral: need to prove that greedy produces optimal

Greedy Algorithm

• Applies to Optimization Problems (find minimum or maximum such that…)

• Strategy: construct the solution by choosing elements one at a time in a greedy fashion (local optimum item), hoping that it will produce the global optimum structure.

Minimum Spanning Tree

• Given undirected connected weighted graph (“costs” assigned to edges).

• Spanning tree: minimum connected subgraph

• Minimum weight (or cost) spanning tree: sum over all edges is minimum over all possible spanning trees

Optimum Spanning Tree

Minimum Maximum

Naïve Solution

• Enumerate all trees, compute their costs• Find minimum (resp. maximum)

• Why bad?• Number of trees on n vertices is very large

(exponential): nn-2.

• Hence number of spanning trees can be very large => inefficient algorithm (exponential)

Greedy Algorithms for MST

• Prim’s algorithm: start with some vertex v. At next step, choose the minimum edge adjacent to already constructed tree, while maintaining the tree property

• Kruskal’s algorithm: sort edges in increasing order, add one at a time while maintaining forest property

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm for MST

Prim’s Algorithm: Invariant

•At each step maintain a tree

•Compute fringe: list of all edges out of the current tree

•Find minimum edge in the fringe

•Add it to the tree

Prim’s Algorithm: Analysis

•At each step add a vertex: n steps

•Compute fringe: list of all edges out of the current tree

•Find minimum edge in the fringe: O(m)

•Add it to the tree

•Total: O(nm) Naive

Prim’s Algorithm: Better Implementation and Analysis

•At each step add a vertex: n steps

•Compute fringe: list of all edges out of the current tree, keep as heap. Must look at all edges O(m)

•Find minimum edge in the fringe:

O(log m) = O(log n)

•Add it to the tree

•Total: O(m+nlog n)

Kruskal’s Algorithm for MST

• Sort all edges O(m2 log n)

• At each step, choose the next min weight edge from the list that does not create a cycle.

• Invariant: maintain a forest of the min weight edges encountered up to step k

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST

Kruskal’s Algorithm for MST: forest along the way

Kruskal’s Algorithm for MST:Analysis

• Sort all edges O(m2 log n)

• At each step, choose the next min weight edge from the list that does not create a cycle: O(log n). Total n-1 steps

• Total: O(m2 log n)

Correctness

• Must prove that the greedy strategy produces the optimum solution

• Prim: show that if the min edge is not chosen at some step k, then it is not a MST (there exists a tree of smaller cost)

• Kruskal: show that min edge must be in MST; then next one, and next one etc.

Correctness of Prim’s Algorithm

• Assume MST unique

• By contradiction. Show that if the min fringe edge is not chosen at some step k, then it is not a MST (there exists a tree of smaller cost)

Prim’s Algorithm: Correctness•Assume turquoise tree is MST but does not use red edge (min at this step)

•Red edge forms a cycle with tree edges, with at least one other edge in the fringe

•Removing this edge of the cycle and putting back the red edge creates a smaller cost tree

•Contradiction