minimum spanning trees cse 2320 – algorithms and data structures vassilis athitsos university of...
TRANSCRIPT
![Page 1: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/1.jpg)
1
Minimum Spanning Trees
CSE 2320 – Algorithms and Data StructuresVassilis Athitsos
University of Texas at Arlington
![Page 2: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/2.jpg)
2
Weighted Graphs• Each edge has a weight.• Example: a transportation
network (roads, railroads, subway). The weight of each road can be:– The length.– The expected time to traverse.– The expected cost to build.
• Example: in a computer network, the weight of each edge (direct link) can be:– Latency.– Expected cost to build.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 3: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/3.jpg)
3
Minimum-Cost Spanning Tree (MST)• Important problem in
weighted graphs: finding a minimum-cost spanning tree:
• A tree that:– Connects all vertices of
the graph.– Has the smallest possible
total weight of edges.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
0
1 7
2
5
3
4
6
10
20
3020
15
1015
![Page 4: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/4.jpg)
4
Minimum-Cost Spanning Tree (MST)• We will only consider
algorithms that compute the MST for undirected graphs.
• We will allow edges to have negative weights.
• Warning: later in the course (when we discuss Dijkstra's algorithm) we will need to make opposite assumptions:– Allow directed graphs.– Not allow negative weights.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
0
1 7
2
5
3
4
6
10
20
3020
15
1015
![Page 5: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/5.jpg)
5
Prim's Algorithm - Overview
• Prim's algorithm:– Start from an tree that contains a single vertex.– Keep growing that tree, by adding at each step the
shortest edge connecting a vertex in the tree to a vertex outside the tree.
• As you see, it is a very simple algorithm, when stated abstractly.
• However, we have several choices regarding how to implement this algorithm.
• We will see three implementations, with significantly different properties from each other.
![Page 6: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/6.jpg)
6
Prim's Algorithm - Simple Version
• Assume an adjacency matrix representation.– Each vertex is a number from 0 to V-1.– We have a V*V adjacency matrix ADJ, where:
ADJ[v][w] is the weight of the edge connecting v and w.– If v and w are not connected, ADJ[v][w] = infinity.
![Page 7: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/7.jpg)
7
Prim's Algorithm - Simple Version
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from the current tree
to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
![Page 8: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/8.jpg)
8
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 9: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/9.jpg)
9
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 10: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/10.jpg)
10
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 11: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/11.jpg)
11
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 12: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/12.jpg)
12
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 13: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/13.jpg)
13
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 14: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/14.jpg)
14
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 15: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/15.jpg)
15
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 16: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/16.jpg)
16
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 17: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/17.jpg)
17
Example
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from
the current tree to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
0
1 7
2
5
3
4
6
10
20
3020
15
1015
![Page 18: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/18.jpg)
18
Prim's Algorithm - Simple Version
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from the current tree
to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
• Running time?
![Page 19: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/19.jpg)
19
Prim's Algorithm - Simple Version
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from the current tree
to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
• Most naive implementation: time ???– Every time we add a new vertex and edge, go through all
edges again, to identify the next edge (and vertex) to add.
![Page 20: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/20.jpg)
20
Prim's Algorithm - Simple Version
1. Start by adding vertex 0 to the MST (minimum-cost spanning tree).
2. Repeat until all vertices have been added to the tree:3. From all edges connecting vertices from the current tree
to vertices outside the current tree, select the smallest edge.
4. Add that edge to the tree, and also add to the tree the non-tree vertex of that edge.
• Most naive implementation: time O(VE).– Every time we add a new vertex and edge, go through all
edges again, to identify the next edge (and vertex) to add.
![Page 21: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/21.jpg)
21
Prim's Algorithm - Dense Graphs
• A dense graph is nearly full, and thus has O(V2) edges.– For example, think of a graph where each vertex has at
least V/2 neighbors.
• Just reading the input (i.e., looking at each edge of the graph once) takes O(V2) time.
• Thus, we cannot possibly compute a minimum-cost spanning tree for a dense graph in less than O(V2) time.
• Prim's algorithm can be implemented so as to take O(V2) time, which is optimal for dense graphs.
![Page 22: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/22.jpg)
22
Prim's Algorithm - Dense Graphs• Again, assume an adjacency matrix representation.• Every time we add a vertex to the MST, we need to update,
for each vertex W not in the tree:– The smallest edge wt[W] connecting it to the tree.– If no edge connects W to the tree, wt[W] = infinity.– The tree vertex fr[W] associated with the edge whose weight is wt[W].
• These quantities can be updated in O(V) time when adding a new vertex to the tree.
• Then, the next vertex to add is the one with the smallest wt[W].
![Page 23: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/23.jpg)
23
Example• Every time we add a vertex
to the MST, we need to update, for each vertex W not in the tree:– The smallest edge wt[W]
connecting it to the tree.– If no edge connects W to the
tree, wt[W] = infinity.– The tree vertex fr[W]
associated with the edge whose weight is wt[W].
• When we add a vertex to the MST, we mark it as "processed" by setting: st[W] = fr[W].
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
![Page 24: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/24.jpg)
24
Prim's Algorithm: Dense Graphsvoid GRAPHmstV(Graph G, int st[], double wt[]) { int v, w, min; for (v = 0; v < G->V; v++) { st[v] = -1; fr[v] = v; wt[v] = maxWT; } st[0] = 0; wt[G->V] = maxWT; for (min = 0; min != G->V; ) { v = min; st[min] = fr[min]; for (w = 0, min = G->V; w < G->V; w++) if (st[w] == -1) { if (G->adj[v][w] < wt[w]) { wt[w] = G->adj[v][w]; fr[w] = v; } if (wt[w] < wt[min]) min = w; }}}
![Page 25: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/25.jpg)
25
Prim's Algorithm - Dense Graphs
• Running time: ???
![Page 26: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/26.jpg)
26
Prim's Algorithm - Dense Graphs
• Running time: O(V2)• Optimal for dense graphs.
![Page 27: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/27.jpg)
27
Prim's Algorithm for Sparse Graphs
• A sparse graph is one that is not dense.• This is somewhat vague.• If you want a specific example, think of a case where
the number of edges is linear to the number of vertices.– For example, if each vertex can only have between 1 and
10 neighbors, than the number of edges can be at most ???
![Page 28: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/28.jpg)
28
Prim's Algorithm for Sparse Graphs
• A sparse graph is "one that is not dense".• This is somewhat vague.• If you want a specific example, think of a case where
the number of edges is linear to the number of vertices.– For example, if each vertex can only have between 1 and
10 neighbors, than the number of edges can be at most 10*V.
![Page 29: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/29.jpg)
29
Prim's Algorithm for Sparse Graphs
• If we use an adjacency matrix representation, then we can never do better than O(V2) time.
• Why?
![Page 30: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/30.jpg)
30
Prim's Algorithm for Sparse Graphs
• If we use an adjacency matrix representation, then we can never do better than O(V2) time.
• Why? – Because just scanning the adjacency matrix to figure out
where the edges are takes O(V2) time.– The adjacency matrix itself has size V*V.
![Page 31: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/31.jpg)
31
Prim's Algorithm for Sparse Graphs
• If we use an adjacency matrix representation, then we can never do better than O(V2) time.
• Why? – Because just scanning the adjacency matrix to figure out
where the edges are takes O(V2) time.– The adjacency matrix itself has size V*V.
• We have already seen an implementation of Prim's algorithm, using adjacency matrices, which achieves O(V2) running time.
• For sparse graphs, if we want to achieve better running time than O(V2), we have to switch to an adjacency lists representation.
![Page 32: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/32.jpg)
32
Prim's Algorithm for Sparse Graphs
• Quick review: what exactly is an adjacency lists representation?
![Page 33: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/33.jpg)
33
Prim's Algorithm for Sparse Graphs
• Quick review: what exactly is an adjacency lists representation?– Each vertex is a number between 0 and V (same as for
adjacency matrices).– The adjacency information is stored in an array ADJ of lists.– ADJ[w] is a list containing all neighbors of vertex w.
• What is the sum of length of all lists in the ADJ array?
![Page 34: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/34.jpg)
34
Prim's Algorithm for Sparse Graphs
• Quick review: what exactly is an adjacency lists representation?– Each vertex is a number between 0 and V (same as for
adjacency matrices).– The adjacency information is stored in an array ADJ of lists.– ADJ[w] is a list containing all neighbors of vertex w.
• What is the sum of length of all lists in the ADJ array? – 2*E (each edge is included in two lists).
![Page 35: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/35.jpg)
35
Prim's Algorithm for Sparse Graphs
• For sparse graphs, we will use an implementation of Prim's algorithm based on:– A graph representation using adjacency lists.– A priority queue (heap) containing the set of edges on the
fringe.
• An edge F will be included in this priority queue if: for some vertex w NOT in the tree yet, F is the shortest edge connecting w to vertex in the tree.
![Page 36: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/36.jpg)
36
Prim's Algorithm - PQ Version
• Initialize a priority queue P.• v = vertex 0• While (true)
– Add v to the spanning tree.– Let S = set of edges of v:– If S is empty, exit.– For each F = (v, w) in S
• if w is already in the spanning tree, continue.• If another edge F' in P also connects to w, keep the smallest of F and F'.• Else insert F to P.
– F = remove_minimum(P)– v = vertex of F not yet in the tree.
![Page 37: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/37.jpg)
37
Prim's Algorithm - PQ Version
• Running time???
![Page 38: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/38.jpg)
38
Prim's Algorithm - PQ Version
• Running time? O(E lg V).• Why?
![Page 39: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/39.jpg)
39
Prim's Algorithm - PQ Version
• Running time? O(E lg V).• Why? Let's look at this piece of the pseudocode:
– For each F = (v, w) in S• if w is already in the spanning tree, continue.• If another edge F' in P also connects to w, keep the smallest of F and F'.• Else insert F to P.
• The number of iterations for this for loop will be 2*E (each edge is considered twice).
• At each iteration, we spend at most log(v) time.
![Page 40: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/40.jpg)
40
Kruskal's Algorithm: Overview
• Prim's algorithm works with a single tree, such that:– First, the tree contains a single vertex.– The tree keeps growing, until it spans the whole tree.
• Kruskal's algorithm works with a forest (a set of trees).– Initially, each tree in this forest is a single vertex.– Each vertex in the graph is its own tree.– We keep merging trees together, until we end up with a
single tree.
![Page 41: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/41.jpg)
41
Kruskal's Algorithm: Overview
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:3. Find the shortest edge F connecting two trees in the
forest.4. Connect those two trees into a single tree using edge F.
• As in Prim's algorithm, the abstract description is simple, but we need to think carefully about how exactly to implement these steps.
![Page 42: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/42.jpg)
42
Kruskal's Algorithm: An Example
0
1 7
2
5
3
4
6
10
20
3020
1530
10
20
15 25
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:
3. Find the shortest edge F connecting two trees in the forest.
4. Connect those two trees into a single tree using edge F.
![Page 43: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/43.jpg)
43
Kruskal's Algorithm: An Example
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:
3. Find the shortest edge F connecting two trees in the forest.
4. Connect those two trees into a single tree using edge F.
10
0
1 7
2
5
3
4
6
10
20
3020
1530
20
15 25
![Page 44: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/44.jpg)
44
Kruskal's Algorithm: An Example
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:
3. Find the shortest edge F connecting two trees in the forest.
4. Connect those two trees into a single tree using edge F.
10
0
1 7
2
5
3
4
6
10
20
3020
1530
20
15 25
![Page 45: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/45.jpg)
45
Kruskal's Algorithm: An Example
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:
3. Find the shortest edge F connecting two trees in the forest.
4. Connect those two trees into a single tree using edge F.
10
0
1 7
2
5
3
4
6
10
20
3020
1530
20
15 25
![Page 46: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/46.jpg)
46
Kruskal's Algorithm: An Example
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:
3. Find the shortest edge F connecting two trees in the forest.
4. Connect those two trees into a single tree using edge F.
10
0
1 7
2
5
3
4
6
10
20
3020
1530
20
15 25
![Page 47: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/47.jpg)
47
Kruskal's Algorithm: An Example
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:
3. Find the shortest edge F connecting two trees in the forest.
4. Connect those two trees into a single tree using edge F.
10
0
1 7
2
5
3
4
6
10
20
3020
15
15
![Page 48: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/48.jpg)
48
Kruskal's Algorithm: Simple Implementation
Assume graphs are represented usind adjacency lists.
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.– How? We will use the same representation for forests that
we used for union-find.– We will have an id array, where each vertex will point to its
parent.– The root of each tree will be the ID for that tree.
• Time it takes for this step ???
![Page 49: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/49.jpg)
49
Kruskal's Algorithm: Simple Implementation
Assume graphs are represented usind adjacency lists.
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.– How? We will use the same representation for forests that
we used for union-find.– We will have an id array, where each vertex will point to its
parent.– The root of each tree will be the ID for that tree.
• Time it takes for this step? O(V)
![Page 50: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/50.jpg)
50
Kruskal's Algorithm: Simple Implementation
2. Repeat until the forest contains a single tree:3. Find the shortest edge F connecting two trees in the
forest.– Initialize F to some edge with infinite weight.– For each edge F' connecting any two vertices (v, w):
• Determine if v and w belong to two different trees in the forest.• If so, update F to be the shortest of F and F'.
4. Connect those two trees into a single tree using edge F.
![Page 51: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/51.jpg)
51
Kruskal's Algorithm: Simple Implementation
2. Repeat until the forest contains a single tree:3. Find the shortest edge F connecting two trees in the
forest.– Initialize F to some edge with infinite weight.– For each edge F' connecting (v, w):
• Determine if v and w belong to two different trees in the forest.HOW?
• If so, update F to be the shortest of F and F'.
4. Connect those two trees into a single tree using edge F. HOW?
![Page 52: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/52.jpg)
52
Kruskal's Algorithm: Simple Implementation
2. Repeat until the forest contains a single tree:3. Find the shortest edge F connecting two trees in the
forest.– Initialize F to some edge with infinite weight.– For each edge F' connecting (v, w):
• Determine if v and w belong to two different trees in the forest.HOW? By comparing find(v) with find(w). Time:
• If so, update F to be the shortest of F and F'.
4. Connect those two trees into a single tree using edge F. HOW? By calling union(v, w). Time:
![Page 53: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/53.jpg)
53
Kruskal's Algorithm: Simple Implementation
2. Repeat until the forest contains a single tree:3. Find the shortest edge F connecting two trees in the
forest.– Initialize F to some edge with infinite weight.– For each edge F' connecting (v, w):
• Determine if v and w belong to the same tree in the forest. HOW? By comparing find(v) with find(w). Time: O(lg V)
• If so, update F to be the shortest of F and F'.
4. Connect those two trees into a single tree using edge F. HOW? By calling union(v, w). Time: O(1)
![Page 54: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/54.jpg)
54
Kruskal's Algorithm: Simple Implementation
2. Repeat until the forest contains a single tree:Total time for all iterations:
3. Find the shortest edge F connecting two trees in the forest. Time:
– Initialize F to some edge with infinite weight.– For each edge F' connecting (v, w):
• Determine if v and w belong to the same tree in the forest. HOW? By comparing find(v) with find(w). Time: O(lg V)
• If so, update F to be the shortest of F and F'.
4. Connect those two trees into a single tree using edge F. HOW? By calling union(v, w). Time: O(1)
![Page 55: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/55.jpg)
55
Kruskal's Algorithm: Simple Implementation
2. Repeat until the forest contains a single tree:Total time for all iterations: O(V*E*lg(V))
3. Find the shortest edge F connecting two trees in the forest. Time: O(E*lg(V))
– Initialize F to some edge with infinite weight.– For each edge F' connecting (v, w):
• Determine if v and w belong to the same tree in the forest. HOW? By comparing find(v) with find(w). Time: O(lg V)
• If so, update F to be the shortest of F and F'.
4. Connect those two trees into a single tree using edge F. HOW? By calling union(v, w). Time: O(1)
![Page 56: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/56.jpg)
56
Running Time for Simple Implementation
1. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
2. Repeat until the forest contains a single tree:3. Find the shortest edge F connecting two trees in the
forest.4. Connect those two trees into a single tree using edge F.
• Running time for simple implementation: O(V*E*lg(V)).
![Page 57: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/57.jpg)
57
Kruskal's Algorithm: Faster Version
1. Sort all edges, save result in array K.2. Initialize a forest (a collection of trees), by defining
each vertex to be its own separate tree.3. For each edge F in K (in ascending order).
4. If F is connecting two trees in the forest:5. Connect the two trees with F.6. If the forest is left with a single tree, break (we are done).
![Page 58: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/58.jpg)
58
Kruskal's Algorithm: Faster Version
1. Sort all edges, save result in array K. Time?2. Initialize a forest (a collection of trees), by defining
each vertex to be its own separate tree. Time?3. For each edge in K (in ascending order). Time?
4. If F is connecting two trees in the forest: Time?
5. Connect the two trees with F. Time?6. If the forest is left with a single tree, break (we are done).
![Page 59: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/59.jpg)
59
Kruskal's Algorithm: Faster Version
1. Sort all edges, save result in array K. Time: O(E lg E)2. Initialize a forest (a collection of trees), by defining
each vertex to be its own separate tree. Time: O(V)3. For each edge in K (in ascending order). Time: O(E lg V)
4. If F is connecting two trees in the forest:Time? O(lg V), two find operations
5. Connect the two trees with F. Time: O(1), union operation6. If the forest is left with a single tree, break (we are done).
• Overall running time???
![Page 60: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/60.jpg)
60
Kruskal's Algorithm: Faster Version
1. Sort all edges, save result in array K. Time: O(E lg E)2. Initialize a forest (a collection of trees), by defining
each vertex to be its own separate tree. Time: O(V)3. For each edge in K (in ascending order). Time: O(E lg V)
4. If F is connecting two trees in the forest:Time? O(lg V), two find operations
5. Connect the two trees with F. Time: O(1), union operation6. If the forest is left with a single tree, break (we are done).
• Overall running time: O(E lg E).
![Page 61: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/61.jpg)
61
Kruskal's Algorithm - PQ Version
• In the previous implementation, we sort edges at the beginning.– This takes O(E lg E) time, which dominates the running
time of the algorithm.– Thus, the entire algorithm takes O(E lg E) time.
• We can do better if, instead of sorting all edges at the beginning, we instead insert all edges into a priority queue.– How long does that take, if we use a heap?
![Page 62: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/62.jpg)
62
Kruskal's Algorithm - PQ Version• In the previous implementation, we sort edges at the
beginning.– This takes O(E lg E) time, which dominates the running time of the
algorithm.– Thus, the entire algorithm takes O(E lg E) time.
• We can do better if, instead of sorting all edges at the beginning, we instead insert all edges into a priority queue.– How long does that take, if we use a heap?– O(E) time.
• We can also do better if, for the find operation, we use the most efficient version discussed in the textbook.– That version flattens paths that it traverses.– Running time: O(lg* V).– lg*(V) is the number of times we need to apply lg to V to obtain 1.
![Page 63: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/63.jpg)
63
Detour: lg*
• lg*(2) = ?• lg*(4) = ?• lg*(16) = ?
![Page 64: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/64.jpg)
64
Detour: lg*
• lg*(2) = 1, because lg(2) = 1.• lg*(4) = 2, because lg(lg(4)) = 1.• lg*(16) = 3, because lg(lg(lg(16))) = 1.• lg*(???) = 4• lg*(???) = 5
![Page 65: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/65.jpg)
65
Detour: lg*
• lg*(2) = 1, because lg(2) = 1.• lg*(4) = 2, because lg(lg(4)) = 1.• lg*(16) = 3, because lg(lg(lg(16))) = 1.• lg*(65536) = 4, because lg(65536) = 16.• lg*(265536) = 5, because lg(265536) = 65536.
• I don't expect we will get to deal with data sizes larger than 265536 in our lifetime.
• Thus, lg* effectively has 5 as an upper bound, so for practical purposes we can treat it as a constant.
![Page 66: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/66.jpg)
66
Kruskal's Algorithm - PQ Version
1. Initialize a heap with the edges (using weight as key).
2. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree.
3. While (true). 4. F = remove_mininum(heap). 5. If F is connecting two trees in the forest:
6. Connect the two trees with F. 7. If the forest is left with a single tree, break (we are done).
![Page 67: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/67.jpg)
67
Kruskal's Algorithm - PQ Version
1. Initialize a heap with the edges (using weight as key). Time?
2. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Time?
3. While (true). Time?4. F = remove_mininum(heap). Time?5. If F is connecting two trees in the forest:
Time?6. Connect the two trees with F. Time?7. If the forest is left with a single tree, break (we are done).
• Overall running time?
![Page 68: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/68.jpg)
68
Kruskal's Algorithm - PQ Version
1. Initialize a heap with the edges (using weight as key). Time? O(E)
2. Initialize a forest (a collection of trees), by defining each vertex to be its own separate tree. Time? O(V)
3. While (true). Time? X lg V. X: number of iterations.4. F = remove_mininum(heap). Time? O(lg E)5. If F is connecting two trees in the forest:
Time? O(lg* V), find operation6. Connect the two trees with F. Time? O(1)7. If the forest is left with a single tree, break (we are done).
• Overall running time? E + X lg V.
![Page 69: Minimum Spanning Trees CSE 2320 – Algorithms and Data Structures Vassilis Athitsos University of Texas at Arlington 1](https://reader036.vdocuments.us/reader036/viewer/2022062422/56649ebb5503460f94bc42c6/html5/thumbnails/69.jpg)
69
Kruskal's Algorithm - PQ Version1. Initialize a heap with the edges (using weight as key). 2. Initialize a forest (a collection of trees), by defining
each vertex to be its own separate tree.3. While (true). Time? X lg V. X: number of iterations.
4. F = remove_mininum(heap). Time? O(lg E)5. If F is connecting two trees in the forest:
6. Connect the two trees with F. Time? O(1)7. If the forest is left with a single tree, break (we are done).
• Overall running time? E + X lg V.– X is the number of edges in the graph with weight <= the
maximum weight of an edge in the final MST.– E < V2, so lg E < 2 lg V, so O(lg E) = O(lg V).