5.3: disjoint sets - university of cambridge · disjoint sets data structure x h0=makeset(x) h0...

5.3: Disjoint SetsFrank Stajano Thomas Sauerwald

Lent 2015

Disjoint Sets (aka Union Find)

Handle makeSet(Item x)Precondition: none of the existing sets contains xBehaviour: create a new set {x} and return its handle

Handle findSet(Item x)Precondition: there exists a set that contains x (given pointer to x)Behaviour: return the handle of the set that contains x

Handle union(Handle h, Handle g)Precondition: h 6= gBehaviour: merge two disjoint sets and return handle of new set

Disjoint Sets Data Structure

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

5.3: Disjoint Sets T.S. 3

Outline

Disjoint Sets

Introduction to Graphs and Graph Searching

Handle MakeSet(Item x)Precondition: none of the existing sets contains xBehaviour: create a new set {x} and return its handle

Handle FindSet(Item x)Precondition: there exists a set that contains x (given pointer to x)Behaviour: return the handle of the set that contains x

Handle Union(Handle h, Handle g)Precondition: h 6= gBehaviour: merge two disjoint sets and return handle of new set

xh0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)h5=Union(h1,h2)

h2 h3 = h4

xh0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

xh0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

xh0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h2 h3 = h4

h0=makeSet(x)

h1=findSet(y)

h4=Union(h0,h3)

h5=Union(h1,h2)

h3 = h4

First Attempt: List Implementation

Add extra pointer to the lastelement in each list

UNION-Operation

Add backward pointer to the listhead from everywhere

⇒ FIND takes constant time

FIND-Operation

x1 x2 x3

Union(h1, h2) Need to findlast element!

FindSet(z3)

z1 z2 z3 z4

Need to update allbackward pointers!

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

Need to findlast element!

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

FindSet(z3)

z1 z2 z3 z4

⇒ UNION takes constant time

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

UNION-Operation

FIND-Operation

x1 x2 x3

Union(h1, h2)

FindSet(z3)

z1 z2 z3 z4

First Attempt: List Implementation (Analysis)

d = DisjointSet()

h0 = d .MakeSet(x0)

h1 = d .MakeSet(x1)h0 = d .Union(h1, h0)h2 = d .MakeSet(x2)h0 = d .Union(h2, h0)h3 = d .MakeSet(x3)h0 = d .Union(h3, h0)

h1h0h2

Cost for n UNION operations:

∑ni=1 i = Θ(n2)

better to append shorter list to longer Weighted-Union Heuristic

d = DisjointSet()h0 = d .MakeSet(x0)

h1h0h2

∑ni=1 i = Θ(n2)

h1 = d .MakeSet(x1)

h0 = d .Union(h1, h0)h2 = d .MakeSet(x2)h0 = d .Union(h2, h0)h3 = d .MakeSet(x3)h0 = d .Union(h3, h0)

∑ni=1 i = Θ(n2)

h1 = d .MakeSet(x1)h0 = d .Union(h1, h0)

h2 = d .MakeSet(x2)h0 = d .Union(h2, h0)h3 = d .MakeSet(x3)h0 = d .Union(h3, h0)

∑ni=1 i = Θ(n2)

h1 = d .MakeSet(x1)h0 = d .Union(h1, h0)h2 = d .MakeSet(x2)

h0 = d .Union(h2, h0)h3 = d .MakeSet(x3)h0 = d .Union(h3, h0)

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1 = d .MakeSet(x1)h0 = d .Union(h1, h0)h2 = d .MakeSet(x2)h0 = d .Union(h2, h0)h3 = d .MakeSet(x3)

h0 = d .Union(h3, h0)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

∑ni=1 i = Θ(n2)

h1h0h2

Cost for n UNION operations:∑n

i=1 i = Θ(n2)

h1h0h2

i=1 i = Θ(n2)

h1h0h2

i=1 i = Θ(n2)

Weighted-Union Heuristic

Keep track of the length of each list

Append shorter list to the longer list (breaking ties arbitrarily)

can be done easily without significant overhead

Using the Weighted-Union heuristic, any sequence of m operations, n ofwhich are MAKE-SET operations, takes O(m + n · log n) time.

Theorem 21.1

Amortized Analysis: Every operation has amortized costO(log n), but there may be operations with total cost Θ(n).

Theorem 21.1

Analysis of Weighted-Union Heuristic

Theorem 21.1

Can we improve on this further?Proof:

n MAKE-SET operations⇒ at most n − 1 UNION operations

Consider element x and the number of updates of the backward pointer

After each update of x , its set increases by a factor of at least 2

⇒ Backward pointer of x is updated at most log2 n times

Other updates for UNION, MAKE-SET & FIND-SET take O(1) time peroperation

Theorem 21.1

Can we improve on this further?

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Proof:

Theorem 21.1

Can we improve on this further?Proof:

How to Improve?

Basic Idea: Update BackwardPointers only during FIND

MAKE-SET: O(1)

FIND-SET: O(n)

UNION: O(1)

Doubly-Linked List

MAKE-SET: O(1)

FIND-SET: O(1)

UNION: O(log n) (amortized)

How to Improve?

MAKE-SET: O(1)

FIND-SET: O(n)

UNION: O(1)

Doubly-Linked List

MAKE-SET: O(1)

FIND-SET: O(1)

How to Improve?

MAKE-SET: O(1)

FIND-SET: O(n)

UNION: O(1)

Doubly-Linked List

MAKE-SET: O(1)

FIND-SET: O(1)

Disjoint Sets via Forests

{b, c, e, h}

rank = 2 rank = 3rank = 2 3f

{d , f , g} {b, c, d , e, f , g, h}

Rank may be just an upper bound on the height!

Set is represented by a rooted tree with root being the representative

Every node has pointer .p to its parent (for root x , x .p = x)

UNION: Merge the two trees

Forest Structure

Append tree of smaller height Union by Rank

{b, c, e, h}

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

rank = 2 rank = 3rank = 2 3

{d , f , g}

{b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

rank = 2 rank = 3rank = 2

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

rank = 2 rank = 3rank = 2

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

{b, c, e, h}

{d , f , g} {b, c, d , e, f , g, h}

Forest Structure

Path Compression during FIND-SET

b h c f

Maintaining the exact height would becostly, hence rank is only an upper bound!

0: FindSet(x)1: if x 6= x .p2: x .p =FindSet(x .p)3: return x .p

b h c f

Combining Union by Rank and Path Compression

Any sequence of m MAKE-SET, UNION, FIND-SET operations, n ofwhich are MAKE-SET operations, can be performed inO(m ·α(n)) time.

Theorem 21.14

In practice, α(n) is a small constant

Data Structure is essentially optimal! (for more details see CLRS)

α(n) =

0 for 0 ≤ n ≤ 2,1 for n = 3,2 for 4 ≤ n ≤ 7,3 for 8 ≤ n ≤ 2047,4 for 2048 ≤ n ≤ 1080

More than the number of atoms in the universe!log∗(n), the iterated logarithm, satifiesα(n) ≤ log∗(n), but still log∗(1080) = 5.

Theorem 21.14

α(n) =

Theorem 21.14

α(n) =

More than the number of atoms in the universe!

log∗(n), the iterated logarithm, satifiesα(n) ≤ log∗(n), but still log∗(1080) = 5.

Theorem 21.14

α(n) =

More than the number of atoms in the universe!

log∗(n), the iterated logarithm, satifiesα(n) ≤ log∗(n), but still log∗(1080) = 5.

Theorem 21.14

α(n) =

Theorem 21.14

α(n) =

Simulating the Effects of Union by Rank and Path Compression

1. Initialise singletons 1, 2, . . . , 300

2. For every 1 ≤ i ≤ 300, pick a random 1 ≤ r ≤ 300, r 6= i andperform UNION(FIND(i), FIND(r))

3. Perform j ∈ {0, 100, 200, 300, 600, 900} many FIND(r), where1 ≤ r ≤ 300 is random

Experimental Setup

Simulating the Effects of Union by Rank and Path Compression

1. Initialise singletons 1, 2, . . . , 300

2. For every 1 ≤ i ≤ 300, pick a random 1 ≤ r ≤ 300, r 6= i andperform UNION(FIND(i), FIND(r))

3. Perform j ∈ {0, 100, 200, 300, 600, 900} many FIND(r), where1 ≤ r ≤ 300 is random

Experimental Setup

Union by Rank without Path Compression

22222221013211222

21221223112323322213122223313123133222213232221223312112213332322132222212232333

2233322333331223221322323122122233222222323332232133

23131223222 2 2 2 2 3 3 3 2 2 1 3 2 3 2 2 2 3 2 3 2 2 3 3 3 3 2 2 2 1 3 2 3 3 2 3 3 2 3 2 2 2 2 2 1 3 1 2 3 2 3 3 2 2 2 1 2 3 1 1 2122231232

32223221331221213131211313232323333

Union by Rank with Path Compression

22211121012211121

21121113111122322112122222213122133222213222221223211112212332222132122212222233

2223222223221213221121322122122211121212222232222111

12131223122 2 2 2 1 3 2 3 2 2 1 3 2 3 2 1 1 2 2 1 1 2 3 3 3 2 2 2 1 1 2 2 2 2 1 3 2 2 1 2 2 2 2 2 1 2 1 2 3 2 3 1 2 2 1 1 2 1 1 1 2122221222

32222221221221212131211113222121121

After 100 additional FINDs

12111111012211111

21111113111112212112112122212112132222213212121223211111111332212122112212222233

2212222222121113221121321122122211121112222232122111

12131212122 1 2 2 1 2 1 3 1 2 1 3 2 3 2 1 1 1 2 1 1 1 1 3 3 2 1 1 1 1 2 2 1 2 1 3 2 2 1 1 2 1 2 2 1 2 1 2 2 1 1 1 1 1 1 1 2 1 1 1 2112121122

12122221221121212121211113222121111

12111111011111111

11111113111111212112111112212112122121112112121213211111111312212112111212212223

2111122212121113211121221122122211121112222212121111

12131112122 1 2 2 1 2 1 3 1 2 1 3 2 2 2 1 1 1 1 1 1 1 1 3 1 2 1 1 1 1 2 2 1 1 1 3 1 1 1 1 2 1 2 2 1 2 1 2 1 1 1 1 1 1 1 1 2 1 1 1 2112121122

12121211221121112121211111112121111

11111111011111111

11111111111111112112111112211112122121111111111112111111111211212112111212212221

2111112212121113211121221111122111111112222212111111

12121112112 1 2 2 1 1 1 3 1 2 1 2 2 2 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 2 2 1 1 1 1 1 1 1 1 2 1 1 2 1 2 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1111121121

12121111221121112111211111112121111

11111111011111111

11111111111111111111111111211111111111111111111111111111111111211112111212112121

2111112111111112111111211111111111111111221112111111

12121111111 1 1 2 1 1 1 1 1 2 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 2 1 1 2 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1111111111

11121111111111111111111111111121111

11111111011111111

11111111111111111111111111111111111111111111111111111111111111211112111112111111

1111112111111111111111211111111111111111111111111111

12111111111 1 1 2 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 1 1111111111

11111111111111111111111111111111111

Outline

Disjoint Sets

Introduction to Graphs and Graph Searching

Origin of Graph Theory

Leonhard Euler (1707-1783)

Seven Bridges at Königsberg 1737

Is there a tour which crosseseach bridge exactly once?

Is there a tour which visits everyisland exactly once?

1B course: Complexity Theory

Source: Wikipedia

Leonhard Euler (1707-1783)Seven Bridges at Königsberg 1737

Source: Wikipedia Source: Wikipedia

What is a Graph?

A graph G = (V ,E) consists of:

V : the set of vertices

E : the set of edges (arcs)

Directed Graph

E : the set of (undirected) edges

Undirected Graph

A sequence of edges betweentwo vertices forms a path

If each pair of vertices has apath linking them, then G isconnected

Paths and Connectivity

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

Path p = (1, 2, 3, 4)Path p = (1, 2, 3, 1), which is a cycle

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is not (strongly)connected

G is connected

Later: edge-weighted graphs G = (V ,E ,w)

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

Path p = (1, 2, 3, 4)

Path p = (1, 2, 3, 1), which is a cycle

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

Path p = (1, 2, 3, 4)

Path p = (1, 2, 3, 1), which is a cycle

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

What is a Graph?

Directed Graph

Undirected Graph

V = {1, 2, 3, 4}E = {(1, 2), (1, 3), (2, 3), (3, 1), (3, 4)}

V = {1, 2, 3, 4}E = {{1, 2}, {1, 3}, {2, 3}, {3, 4}}

G is not a DAG

G is connected

Representations of Directed and Undirected Graphs590 Chapter 22 Elementary Graph Algorithms

2 51224 1 2

45 31 0 0 10 1 1 11 0 1 01 1 0 11 0 1 0

1 2 3 4 512345

(a) (b) (c)

Figure 22.1 Two representations of an undirected graph. (a)An undirected graph G with 5 verticesand 7 edges. (b) An adjacency-list representation of G. (c) The adjacency-matrix representationof G.

2 456246

1 0 1 00 0 0 10 0 0 11 0 0 00 0 1 0

1 2 3 4 512345

(a) (b) (c)

6 0 0 0 0 0 100100

Figure 22.2 Two representations of a directed graph. (a) A directed graph G with 6 vertices and 8edges. (b) An adjacency-list representation of G. (c) The adjacency-matrix representation of G.

shortest-paths algorithms presented in Chapter 25 assume that their input graphsare represented by adjacency matrices.

The adjacency-list representation of a graph G D .V; E/ consists of an ar-ray Adj of jV j lists, one for each vertex in V . For each u 2 V , the adjacency listAdjŒu! contains all the vertices " such that there is an edge .u; "/ 2 E. That is,AdjŒu! consists of all the vertices adjacent to u in G. (Alternatively, it may containpointers to these vertices.) Since the adjacency lists represent the edges of a graph,in pseudocode we treat the array Adj as an attribute of the graph, just as we treatthe edge set E. In pseudocode, therefore, we will see notation such as G:AdjŒu!.Figure 22.1(b) is an adjacency-list representation of the undirected graph in Fig-ure 22.1(a). Similarly, Figure 22.2(b) is an adjacency-list representation of thedirected graph in Figure 22.2(a).

If G is a directed graph, the sum of the lengths of all the adjacency lists is jEj,since an edge of the form .u; "/ is represented by having " appear in AdjŒu!. If G is

2 51224 1 2

45 31 0 0 10 1 1 11 0 1 01 1 0 11 0 1 0

1 2 3 4 512345

(a) (b) (c)

2 456246

1 0 1 00 0 0 10 0 0 11 0 0 00 0 1 0

1 2 3 4 512345

(a) (b) (c)

6 0 0 0 0 0 100100

2 51224 1 2

45 31 0 0 10 1 1 11 0 1 01 1 0 11 0 1 0

1 2 3 4 512345

(a) (b) (c)

2 456246

1 0 1 00 0 0 10 0 0 11 0 0 00 0 1 0

1 2 3 4 512345

(a) (b) (c)

6 0 0 0 0 0 100100

5.3: disjoint sets - university of cambridge · disjoint sets data structure x h0=makeset(x) h0...

Documents