Download - Minimum Spanning Tree Algorithms

Minimum Spanning Tree Algorithms

Prim’s Algorithm

Kruskal’s Algorithm

Runtime Analysis

Correctness Proofs

Copyright 1999 by Cutler/Head

Greedy/Prims 2

What is A Spanning Tree?

u

v

b

a

c

d

e

f

• A spanning tree for an undirected graph G=(V,E) is a subgraph of G that is a tree and contains all the vertices of G

• Can a graph have more than one spanning tree?

• Can an unconnected graph have a spanning tree?


Greedy/Prims 3

Minimal Spanning Tree.

4 4

3

2

9

15

8

1014

3

u

v

b

a

c

d

e

f

Mst T: w( T )= (u,v) T w(u,v ) is minimized

• The weight of a subgraph is the sum of the weights of it edges.

• A minimum spanning tree for a weighted graph is a spanning tree with minimum weight.

• Can a graph have more then one minimum spanning tree?


Greedy/Prims 4

Example of a Problem that Translates into a MST

The Problem• Several pins of an electronic circuit must be

connected using the least amount of wire.

Modeling the Problem • The graph is a complete, undirected graph

G = ( V, E ,W ), where V is the set of pins, E is the set of all possible interconnections between the pairs of pins and w(e) is the length of the wire needed to connect the pair of vertices.

• Find a minimum spanning tree.


Greedy/Prims 5

Greedy Choice

We will show two ways to build a minimum spanning tree.

• A MST can be grown from the current spanning tree by adding the nearest vertex and the edge connecting the nearest vertex to the MST. (Prim's algorithm)

• A MST can be grown from a forest of spanning trees by adding the smallest edge connecting two spanning trees. (Kruskal's algorithm)


Greedy/Prims 6

Notation

• Tree-vertices: in the tree constructed so far• Non-tree vertices: rest of vertices

Prim’s Selection rule

• Select the minimum weight edge between a tree-node and a non-tree node and add to the tree


Greedy/Prims 7

The Prim algorithm Main Idea

Select a vertex to be a tree-node

while (there are non-tree vertices) { if there is no edge connecting a tree node with a non-tree node

return “no spanning tree”

select an edge of minimum weight between a tree node and a non-tree node

add the selected edge and its new vertex to the tree}

return tree

6 4

5

2

158

1014

3

u

v

b

a

c

d

f


Greedy/Prims 8

Implementation Issues

• How is the graph implemented?

– Assume that we just added node u to the tree.

– The distance of the nodes adjacent to u to the tree may now be decreased.

– There must be fast access to all the adjacent vertices.

– So using adjacency lists seems better

• How should the set of non-tree vertices be represented?

– The operations are:

• build set

• delete node closest to tree

• decrease the distance of a non-tree node from the tree

• check whether a node is a non- tree node


Greedy/Prims 9


• How should the set of non-tree vertices be represented?

– A priority queue PQ may be used with the priority D[v] equal to the minimum distance of each non-tree vertex v to the tree.

– Each item in PQ contains: D[v], the vertex v, and the shortest distance edge (v, u) where u is a tree node

• This means:

– build a PQ of non-tree nodes with initial values -

• fast build heap O (V )

• building an unsorted list O(V)

• building a sorted list O(V) (special case)


Greedy/Prims 10


– delete node closest to tree (extractMin)• O(lg V ) if heap and • O(V) if unsorted list • O(1) sorted list

– decrease the distance of a non-tree node to the tree

– We need to find the location i of node v in the priority queue and then execute (decreasePriorityValue(i, p)) where p is the new priority

– decreasePriorityValue(i, p)

• O(lg V) for heap,

• O(1) for unsorted list

• O(V ) for sorted list (too slow)


Greedy/Prims 11


• What is the location i of node v in a priority queue?

– Find in Heaps, and sorted lists O(n)

– Unsorted – if the nodes are numbered 1 to n and we use an array where node v is the v item in the array O(1)

Extended heap

– We will use extended heaps that contain a “handle” to the location of each node in the heap.

– When a node is not in PQ the “handle” will indicate that this is the case

– This means that we can access a node in the extended heap in O(1), and check v PQ in O(1)

– Note that the “handle” must be updated whenever a heap operation is applied


Greedy/Prims 12


2. Unsorted list

– Array implementation where node v can be accesses as PQ[v] in O(1), and the value of PQ[v] indicates when the node is not in PQ.


Greedy/Prims 13

Lines 1-5 initialize the priority queue PQ to contain all Vertices. Ds for all vertices except r, are set to infinity.

r is the starting vertex of the TThe T so far is empty

Add closest vertex and edge to current T

Get all adjacent vertices v of u,update D of each non-tree vertex adjacent to u

Store the current minimum weight edge, and updated distance in the priority queue

Prim’s Algorithm

1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-heap(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u )

// execute relaxation 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v )) 14. return T // T is a mst.


Greedy/Prims 14

Prim’s AlgorithmInitialization

Prim (G )


4. PQ make-heap(D,V, {})//No edges 5. T


Greedy/Prims 15

Building the MST

// solution check

7. while PQ do//Selection and feasibility 8. (u,e ) PQ.extractMin()

// T contains the solution so far . 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v) ) 14. return T


Greedy/Prims 16

Using Extended Heap implementation

Lines 1 -6 run in O (V )

Max Size of PQ is | V |

Count7 =O (V )

Count7(8) = O (V ) O( lg V )

Count7(10) = O(deg(u ) ) = O( E )

Count7(10(11)) = O(1)O( E )

Count7(10(11(12))) = O(1) O( E )

Count7(10(13)) = O( lg V) O( E ) Decrease- Key operation on the extended heap can be implemented in O( lg V)

So total time for Prim's Algorithm is O ( V lg V + E lg V )

What is O(E ) ?Sparse Graph, E =O(V) , O (E lg V)=O(V lg V )

Dense Graph, E=O(V2), O (E lg V)=O(V2 lg V)

Time Analysis1. for each u V2. do D [u ] 3. D[ r ] 0 4. PQ make-PQ(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v)) 15. return T // T is a mst.Assume a node in PQ can be accessed in O(1)

** Decrease key for v requires O(lgV ) provided the node in heap with v’s data can be accessed in O(1)


Greedy/Prims 17

Using unsorted PQ

Lines 1 - 6 run in O (V )

Max Size of PQ is | V |Count7 = O (V )

Count7(8) = O (V ) O(V )

Count7(10) = O(deg(u ) ) = O( E )

Count7(10(11)) = O(1)O( E )

Count7(10(11(12))) = O(1) O( E )

Count7(10(13)) =O( 1) O( E )

So total time for Prim's Algorithm is O (V + V2 + E ) = O (V2 )

For Sparse/Dense graph : O( V2 )Note growth rate unchanged for adjacency

matrix graph representation

Time Analysis


4. PQ make-PQ(D,V, {})//No edges 5. T 6.7. while PQ do 8. (u,e ) PQ.extractMin() 9. add (u,e) to T10. for each v Adjacent (u ) 11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v))

15. return T // T is a mst.


Greedy/Prims 18

handle

ABC

123

Prim - extended HeapAfter Initialization

T PQ

0, (A, {})

, (B, {}) , (C, {})

1

2 3

A

B

C

2

56

G

Prim (G, r)1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-heap(D,V, { })5. T A

BC

G

A 2

A 6

B 2 C 6

C 5

B 5


Greedy/Prims 19

Prim - extended Heap Build tree - after PQ.extractMin

handle

ABC

Null21

T(A, {})

PQ

, (C, {})

, (B, {})

1

2

A

C

2

56

G

7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T

B


Greedy/Prims 20

Update B adjacent to A

handle

ABC

Null12

T(A, {})

PQ

2, (B, {A, B})

, (C, {})

1

2

A

C

2

56

G

10. for each v Adjacent (u ) 11. // relaxation operation

// relaxation11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v))

B


Greedy/Prims 21

Update C adjacent to A

handle

ABC

Null12

T(A, {})

PQ

2, (B, {A, B})

6, (C, {A, C})

1

2A

B

C

2

56

G


Greedy/Prims 22

Build tree - after PQ.extractMin

handle

ABC

NullNull1

T(A, {})(B, {A, B})

PQ

6, (C, {A, C})1

A

B

C

2

56

G



Greedy/Prims 23

Update C adjacent to B

handle

ABC

NullNull1

T(A, {})

PQT(A, {})(B, {A, B})

5, (C, {B, C})1


A

B2

56

GC


Greedy/Prims 24


handle

ABC

NullNullNull

T(A, {})

PQT(A, {})(B, {A, B}) (C, {B, C})


A

B2

56

GC


Greedy/Prims 25

Prim - unsorted listAfter Initialization

T PQ

A BA

B

C

12

54

G

0, (A, {}) , (B, {}) , (C, {})

C

Prim (G, r)1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-PQ(D,V, { })5. T

ABC

G

A 12

A 4

B 12 C 4

C 5

B 5


Greedy/Prims 26


T(A, {})

PQB

, (B, {}) , (C, {})

C

Null

A

B

C

12

54

G

A



Greedy/Prims 27

Update B, C adjacent to A

T(A, {})

PQB

12, (B, {A, B}) 4, (C, {A, C})

C

Null

A

B

C

12

54

G

A



Greedy/Prims 28


T(A, {})(C, {A, C})

PQ

B

Null12, (B, {A, B})

C

Null

AB

C

12

54

G

A



Greedy/Prims 29

Update B adjacent to C

T(A, {})

PQT(A, {})(C, {A, C})

B

Null5, (B, {C, B})

C

Null

A


B

C

12

54

G

A


Greedy/Prims 30


T(A, {})

PQT(A, {})(C, {A, C}) (B, {C, B})

B

Null Null

C

Null

A


B

C

12

54

G

A


Greedy/Prims 31

Prim (G)1. for each u V2. do D [u ] 3. D[ r ] 0

4. PQ make-heap(D,V, { })5. T

6 4

5

2

9

15

8

1014

3

e

f

b

a

c

d

g

h

r

PQ = {( 0,(a,)), (,(b,?)), ...(,(h,?))}

T = { } G =

D = [ 0, , …, ]


Greedy/Prims 32

6 4

5

2

9

15

8

1014

3

e

f

b

a

c

d

g

h

r

PQ = {

T = {

7. while PQ do 8. (u,e) PQ.extractMin()9. add (u,e) to T10. for each v Adjacent (u ) 11. // relaxation operation15. return T

G =

// relaxation11. do if v PQ && w( u, v ) < D [ v ] 12. then D [ v ] w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v))

D = [ 0,


Greedy/Prims 33

Lemma 1

Let G = ( V, E) be a connected, weighted undirected graph. Let T be a promising subset of E. Let Y be the set of vertices connected by the edges in T.

If e is a minimum weight edge that connects a vertex in Y to a vertex

in V - Y, then T { e } is promising.

Note: A feasible set is promising if it can be extended to produce not only a solution , but an optimal solution.

In this algorithm: A feasible set of edges is promising if it is a subset of a Minimum Spanning Tree for the connected graph.


Greedy/Prims 34

Outline of Proof of Correctness of Lemma 1

T is the promising subset, and e the minimum cost edge of Lemma 1

Let T ' be the MST such that T T '

We will show that if e T' then there must be another MST T" such that T {e} T".

Proof has 4 stages:

1. Adding e to T', closes a cycle in T' {e}.

2. Cycle contains another edge e' T' but e' T

3. T"=T' {e} - {e’ } is a spanning tree

4. T" is a MST


Greedy/Prims 35

The Promising Set of Edges Selected by Prim

MST T' but e T'

X

X

XX

X

X

X

TX

X

XX

X

X

XeX

Y

V -Y

Lemma 1


Greedy/Prims 36

Since T' is a spanning tree, it is connected. Adding the

edge, e, creates a cycle.

X

X

XX

X

X

XT' {e}

e

In T' there is a path from uY to v V -Y. Therefore the path must include another edge e' with one vertex in Y and the other in V-Y.

X

X

XX

X

X

X

X

Y

V -Y

e'

Lemma 1

u v Stage 1

Stage 2


Greedy/Prims 37

• If we remove e’ from T’ { e } the cycle disappears.• T”=T’ { e } - {e’} is connected. T’ is connected. Every pair of vertices

connected by a path that does not include e’ is still connected in T”. Every pair of vertices connected by a path that included e’ , is still connected in T” because there is a path in T" =T’ { e } - { e’ } connecting the vertices of e’.

X

Y

V -Y

Lemma 1

X

X

XX

X

X

Xe

T "

w( e ) w( e' )By the way Prim picks the next edge

Stage 3


Greedy/Prims 38

• w( e ) w( e' ) by the way Prim picks the next edge.• The weight of T", w(T") = w (T' ) + w( e ) - w( e' ) w(T').• But w (T' ) w(T") because T' is a MST.• So w (T' ) = w(T") and T" is a MST

X

Y

V -Y

Lemma 1

X

X

XX

X

X

Xe

T"

Conclusion T { e } is promising

Stage 4


Greedy/Prims 39

Theorem : Prim's Algorithm always produces a minimum spanning tree.

Proof by induction on the set T of promising edges.Base case: Initially, T = is promising.

Induction hypothesis: The current set of edges T selected by Prim is promising.

Induction step: After Prim adds the edge e, T { e } is promising.

Proof: T { e } is promising by Lemma 1.

Conclusion: When G is connected, Prim terminates with |T | = |V | -1, and T is a MST.

Kruskal’s Algorithm


Greedy/Prims 41

solution = { } while ( more edges in E) do // Selection

select minimum weight edgeremove edge from E // Feasibility

if (edge closes a cycle with solution so far)then reject edgeelse add edge to solution

// Solution check

if |solution| = |V | - 1 return solution

return null // when does this happen?

Kruskal's Algorithm: Main Idea

6 4

5

2

158

1014

3

u

v

b

a

c

d

f


Greedy/Prims 42

6 4

5

2

9

15

8

1014

3

e

f

b

a

c

d

g

h

C = { {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h} }

C is a forest of trees.

Kruskal's Algorithm:

1. Sort the edges E in non-decreasing weight2. T 3. For each v V create a set. 4. repeat 5. Select next {u,v} E, in order 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then8. add edge (u,v) to T 9. union ( ucomp,vcomp )10.until T contains |V | - 1 edges11. return tree T


Greedy/Prims 43

Kruskal - Disjoint setAfter Initialization

TA

B

C

2

56

G

1. Sort the edges E in non-decreasing weight2. T

3. For each v V create a set.

A B 2

Sorted edges

B C 5

A C 6

A B CDisjoint data set for G

D

B D 77

D


Greedy/Prims 44

Kruskal - add minimum weight edge if feasible

A

C

2

56

G

B

5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union( ucomp,vcomp )

TSorted edges

A B CDisjoint data set for G

Find(A) Find(B)

AB

C

After merge(A, B)

(A, B) A B 2

B C 5

A C 6

B D 7

D

D

7

D


Greedy/Prims 45


A

C

2

56

G

B

5. for each {u,v} in ordered E 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )

T

A

B

C

Find(B) Find(C)

AB C

After merge(A, C)

(A, B)

(B, C)

Sorted edges A B 2

B C 5

A C 6

B D 7

D

D

7

D


Greedy/Prims 46


A

C

2

56

G

B


T

A

B C

Find(A) Find(C)

A and C in same set

(A, B)

(B, C)

Sorted edges A B 2

B C 5

A C 6

B D 7

D

7

D


Greedy/Prims 47


A

C

2

56

G

B


T

A

B C

Find(B) Find(D)

(A, B)

(B, C)

(B, D)

Sorted edges A B 2

B C 5

A C 6

B D 7

D

A

B C D

After merge

7

D


Greedy/Prims 48

Kruskal ( G )1. Sort the edges E in non-decreasing weight2. T 3. For each v V create a set. 4. repeat 5. {u,v} E, in order 6. ucomp find (u) 7. vcomp find (v) 8. if ucomp vcomp then9. add edge (v,u) to T 10. union ( ucomp,vcomp )11.until T contains |V | - 1 edges12. return tree T

Count1 = ( E lg E )

Count2= (1)

Count3= ( V )

Count4 = O( E )

Using Disjoint set-height andpath compression

Count4(6+7+10)= O((E +V) (V))

Sorting dominates the runtime. We get T( E,V ) = ( E lg E), so for a sparse graph we get ( V lg V)for a dense graph we get( V2 lg V2) = ( V2 lg V)

Kruskal's Algorithm: Time Analysis

Download - Minimum Spanning Tree Algorithms

Top Related