design and analysis of computer algorithm september 10, 20151 design and analysis of computer...
TRANSCRIPT
Design and Analysis of Computer Algorithm April 19, 2023 1
Design and Analysis of Computer AlgorithmLecture 5-2
Pradondet Nilagupta
Department of Computer Engineering
This lecture note has been modified from lecture note for 23250 by Prof. Francis Chin , CS332 by David Luekbe
Design and Analysis of Computer Algorithm April 19, 2023 2
Greedy Method (Cont.)
Design and Analysis of Computer Algorithm April 19, 2023 3
TREE
● A tree is an undirected graph which is connected and acyclic. It is easy to show that if graph G ■ G (V,E) is connected.■ G (V,E) is acyclic.■ |E| =|V| -1
Design and Analysis of Computer Algorithm April 19, 2023 4
Spanning Trees
● A spanning tree in an undirected graph G(V,E) is a subset of edges that are acyclic and connect all the vertices in V.
● It follows from the above conditions that a spanning tree must consist of exactly n-1 edges.
● Now suppose that each edge has a weight associated with it: Say that the weight of a tree T is the sum of the weights of its edges;
ET
ZE:w
Te
)e(w)T(w
Design and Analysis of Computer Algorithm April 19, 2023 5
Example: Spanning Tree
2
1
3
4
67
5
28
16
12
14
10
24
22
18
connected graph
2
1
3
4
67
5
28
1614
10
24
18
2
1
3
4
67
5
28
1614
25 24
22
spanning treecost = 129
spanning treecost = 110
Design and Analysis of Computer Algorithm April 19, 2023 6
Definition
● The minimum spanning tree in a weighted graph
G(V,E) is one which has the smallest weight among all spanning trees in G(V,E).
Design and Analysis of Computer Algorithm April 19, 2023 7
A Greedy Strategy
Generic-MST(G, w)
A = {}
// Invariant: A is a subset of some MST
while A does not form a spanning tree
do find an edge (u, v) that is safe for A
A = A + {(u, v)}
// A remains a subset of some MST
return A
How to find a safe edge?
Design and Analysis of Computer Algorithm April 19, 2023 8
Example
● Example: spanning tree T {(a,b), (a,c), (b,d), (d,e), (d,f)}● w(T) = 4+8+8+7+3 = 30● T* = {(a,b), (b,e), (c,e), (c,d), (d,f)}● w(T*) = 4+7+1+2+3 = 17 (minimum ?, why ? see note 1)
Design and Analysis of Computer Algorithm April 19, 2023 9
Method 1: Brute force
● Enumerate all spanning trees, evaluate their costs and find the minimum.
● Drawback - the number of spanning trees must be very large, how large?
● Let Ns be the number of spanning trees for an n-node gr
aph.
● lower bound, (the least value of Ns): n!/2 for a chain, why?
● upper bound, (the largest value of Ns): Ce,n-1, where e = n
(n-1)/2, i.e. O(n (n-1) )
Design and Analysis of Computer Algorithm April 19, 2023 10
Method 2: Removing redundant edges● Edge of the largest weight is removed if the resultant gra
ph remains connected. Repeat the edge removal until not possible.
● Correctness: If the largest-weight edge was not removed, it can always be replaced by an edge of lower weight because of the redundancy.
● Time complexity:● sorting the edges - O(n2 logn) or O(elogn)● testing connectivity - O(n2 ) or O(e)● connectivity test for all edges - O(n4 ) or O(e2 )
Design and Analysis of Computer Algorithm April 19, 2023 11
Minimum Spanning Trees
● Theorem Given a spanning forest which is a subset of the MST, the minimum weight edge (u, v) emerging from a tree in the forest must be "safe", i.e., an edge in the MST.
Design and Analysis of Computer Algorithm April 19, 2023 12
Proof:
● Assume that the minimum weight edge (u, v) is not in MST. Let us add edge(u, v) to the MST to result a unique cycle which must contain another edge (u', v'), where u and u' belong to the some tree. As edge (u, v) is shorter than edge (u', v'), the weight of the new MST containing (u, v) and not (u', v') should be lower than the weight of the original MST. This contradicts our assumption that (u, v) is not in the MST.
Design and Analysis of Computer Algorithm April 19, 2023 13
Method 3: Ordered edge insertion (Krushal)● The edge with minimum weight is added to the "forest" a
s long as it is not redundant, i.e., it connects two trees in the forest. Repeat the edge insertion until not possible.
● Correctness: The proof follows directly from the theorem and can be proved by induction. Initially, the forest has n trees each of which contains a single node of G. The edge with minimum weight edge which is not redundant, i.e., emerging from some trees in the forest, must be a minimum weight edge in the MST.
Design and Analysis of Computer Algorithm April 19, 2023 14
Time Complexcity
● Time Complexity: sorting the edges - O(n 2 logn) or O(elogn)
Design and Analysis of Computer Algorithm April 19, 2023 15
Minimum Spanning Tree (MST)
Connect all the vertices Minimize the total edge weight // must be a tree
Problem Select edges in a connected graph to
f
d
a
b
c e
g
2
7
5
7
1
4
13
4
4
5
2
Design and Analysis of Computer Algorithm April 19, 2023 16
Cut
A cut of G = (V, E) is a partition (S, V-S) of V.
Ex. S = {a, b, c, f}, V-S = {e, d, g}
Edges {b, d}, {a, d}, {b, e}, {c, e} all cross the cut. The remaining edges do not cross the cut.
f
d
a
b
c e
g
2
7
5
7
1
4
13
4
4
5
2
Design and Analysis of Computer Algorithm April 19, 2023 17
Respected by a Cut
Ex. S = {a, b, c, d} A1 = {(a, b), (d, g), (f, b), (a, f)}A2 = A1 + {(b, d)}
The cut (S, V-S) respects A1 but not A2.
f
d
a
b
c e
g
2
7
5
7
1
4
13
4
4
5
2
Design and Analysis of Computer Algorithm April 19, 2023 18
Light Edge
Ex. S = {a, b, c, d}
Edge {b, e} has weight 3, “lighter” than the other edges {a, d}, {b, d}, and {b, e} that cross the cut.
The edge that crosses a cut with the minimum weight.
f
d
a
b
c e
g
2
7
5
7
1
4
1 3
4
4
5
2
light edge
Design and Analysis of Computer Algorithm April 19, 2023 19
Light Edge Is Safe!
Theorem Let (S, V-S) be any cut of G(V, E) that respects a subset A of E, which is included in some MST for G. Let (u, v) be a light edge crossing (S, V-S). Then (u, v) is safe for A; that is, A + {(u, v)} is also included in some MST.
S
V- S
4
2
6u
v
A includes all red edges.
Design and Analysis of Computer Algorithm April 19, 2023 20
Corollary Let A be a subset of E that is included in some MST of G, and C be a connected component in the forest G = <V, A>. If (u, v) is a light edge connecting C to some other component in G , then (u, v) is safe for A.
A
A
u
v
C
7
4
Design and Analysis of Computer Algorithm April 19, 2023 21
Minimum Spanning Tree
● Problem: given a connected, undirected, weighted graph:
1410
3
6 4
5
2
9
15
8
Design and Analysis of Computer Algorithm April 19, 2023 22
Minimum Spanning Tree
● Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight
1410
3
6 4
5
2
9
15
8
Design and Analysis of Computer Algorithm April 19, 2023 23
Minimum Spanning Tree
● Which edges form the minimum spanning tree (MST) of the below graph?
H B C
G E D
F
A
1410
3
6 4
5
2
9
15
8
Design and Analysis of Computer Algorithm April 19, 2023 24
Minimum Spanning Tree
● Answer:
H B C
G E D
F
A
1410
3
6 4
5
2
9
15
8
Design and Analysis of Computer Algorithm April 19, 2023 25
Minimum Spanning Tree
● MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees■ Let T be an MST of G with an edge (u,v) in the middle
■ Removing (u,v) partitions T into two trees T1 and T2
■ Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of G2 = (V2,E2) (Do V1 and V2 share vertices? Why?)
■ Proof: w(T) = w(u,v) + w(T1) + w(T2)(There can’t be a better tree than T1 or T2, or T would be suboptimal)
Design and Analysis of Computer Algorithm April 19, 2023 26
Minimum Spanning Tree
● Thm: ■ Let T be MST of G, and let A T be subtree of T■ Let (u,v) be min-weight edge connecting A to V-A■ Then (u,v) T
Design and Analysis of Computer Algorithm April 19, 2023 27
Minimum Spanning Tree
● Thm: ■ Let T be MST of G, and let A T be subtree of T■ Let (u,v) be min-weight edge connecting A to V-A■ Then (u,v) T
● Proof: in book (see Thm 24.1)
Design and Analysis of Computer Algorithm April 19, 2023 28
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
Design and Analysis of Computer Algorithm April 19, 2023 29
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
1410
3
6 45
2
9
15
8
Run on example graph
Design and Analysis of Computer Algorithm April 19, 2023 30
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
1410
3
6 45
2
9
15
8
Run on example graph
Design and Analysis of Computer Algorithm April 19, 2023 31
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
0
1410
3
6 45
2
9
15
8
Pick a start vertex r
r
Design and Analysis of Computer Algorithm April 19, 2023 32
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
0
1410
3
6 45
2
9
15
8
Red vertices have been removed from Q
u
Design and Analysis of Computer Algorithm April 19, 2023 33
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
0
3
1410
3
6 45
2
9
15
8
Red arrows indicate parent pointers
u
Design and Analysis of Computer Algorithm April 19, 2023 34
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
14
0
3
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 35
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
14
0
3
1410
3
6 45
2
9
15
8u
Design and Analysis of Computer Algorithm April 19, 2023 36
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
14
0 8
3
1410
3
6 45
2
9
15
8u
Design and Analysis of Computer Algorithm April 19, 2023 37
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10
0 8
3
1410
3
6 45
2
9
15
8u
Design and Analysis of Computer Algorithm April 19, 2023 38
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10
0 8
3
1410
3
6 45
2
9
15
8u
Design and Analysis of Computer Algorithm April 19, 2023 39
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10 2
0 8
3
1410
3
6 45
2
9
15
8u
Design and Analysis of Computer Algorithm April 19, 2023 40
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10 2
0 8 15
3
1410
3
6 45
2
9
15
8u
Design and Analysis of Computer Algorithm April 19, 2023 41
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10 2
0 8 15
3
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 42
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10 2 9
0 8 15
3
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 43
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
10 2 9
0 8 15
3
4
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 44
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
5 2 9
0 8 15
3
4
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 45
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
5 2 9
0 8 15
3
4
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 46
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
5 2 9
0 8 15
3
4
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 47
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
5 2 9
0 8 15
3
4
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 48
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
5 2 9
0 8 15
3
4
1410
3
6 45
2
9
15
8
u
Design and Analysis of Computer Algorithm April 19, 2023 49
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
What is the hidden cost in this code?
Design and Analysis of Computer Algorithm April 19, 2023 50
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
Design and Analysis of Computer Algorithm April 19, 2023 51
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
How often is ExtractMin() called?How often is DecreaseKey() called?
Design and Analysis of Computer Algorithm April 19, 2023 52
Review: Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v Adj[u]
if (v Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
What will be the running time?A: Depends on queue binary heap: O(E lg V) Fibonacci heap: O(V lg V + E)