introduction to algorithms - duke universityreif/courses/alglectures/...© 2004 by erik demaine and...
TRANSCRIPT
![Page 1: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/1.jpg)
Introduction to Algorithms6.046J/18.401J/SMA5503
Lecture 10Prof. Piotr Indyk
![Page 2: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/2.jpg)
Introduction to Algorithms October 18, 2004 L10.2© 2004 by Erik Demaine and Piotr Indyk
Today
• A data structure for a new problem• Amortized analysis
![Page 3: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/3.jpg)
Introduction to Algorithms October 18, 2004 L10.3© 2004 by Erik Demaine and Piotr Indyk
2-3 Trees: Deletions
• Problem: there is an internal node that has only 1child
• Solution: delete recursively
9 1251 6 7
5.5 12
5.5
12
6
6
12
12
![Page 4: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/4.jpg)
Introduction to Algorithms October 18, 2004 L10.4© 2004 by Erik Demaine and Piotr Indyk
Example
9 1251 6 7 8
5.5 8 12
5.5
12
6
6
12
12
![Page 5: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/5.jpg)
Introduction to Algorithms October 18, 2004 L10.5© 2004 by Erik Demaine and Piotr Indyk
Example, ctd.
9 1251 6 7
5.5 8 12
5.5
12
6
6
12
12
![Page 6: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/6.jpg)
Introduction to Algorithms October 18, 2004 L10.6© 2004 by Erik Demaine and Piotr Indyk
Example, ctd.
9 1251 6 7
5.5 12
5.5
12
12
6
12
12
![Page 7: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/7.jpg)
Introduction to Algorithms October 18, 2004 L10.7© 2004 by Erik Demaine and Piotr Indyk
Example, ctd.
9 1251 6 7
5.5 12
5.5
12
12
6
![Page 8: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/8.jpg)
Introduction to Algorithms October 18, 2004 L10.8© 2004 by Erik Demaine and Piotr Indyk
Procedure for Delete(x)• Let y=p(x)• Remove x• If y≠root then
– Let z be the sibling of y.– Assume z is the right sibling of y, otherwise the code is
symmetric.– If y has only 1 child w left
Case 1: z has 3 children • Attach left[z] as the rightmost child of y• Update y.max and z.maxCase 2: z has 2 children:• Attach the child w of y as the leftmost child of z• Update z.max• Delete(y) (recursively*)
– Else• Update max of y, p(y), p(p(y)) and so on until root
• Else – If root has only one child u
• Remove root• Make u the new root
*Note that the input of Delete does not have to be a leaf
x
zy
![Page 9: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/9.jpg)
Introduction to Algorithms October 18, 2004 L10.9© 2004 by Erik Demaine and Piotr Indyk
2-3 Trees
• The simplest balanced trees on the planet!(but, nevertheless, not completely trivial)
![Page 10: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/10.jpg)
Introduction to Algorithms October 18, 2004 L10.10© 2004 by Erik Demaine and Piotr Indyk
Dynamic Maintenance of Sets• Assume, we have a collection
of elements• The elements are clustered• Initially, each element forms
its own cluster/set• We want to enable two
operations:– FIND-SET(x): report the
cluster containing x– UNION(C1, C2): merges
the clusters C1, C2
1 2
3
5
6
![Page 11: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/11.jpg)
Introduction to Algorithms October 18, 2004 L10.11© 2004 by Erik Demaine and Piotr Indyk
Disjoint-set data structure(Union-Find)
Problem:• Maintain a collection of pairwise-disjointsets S = {S1, S2, …, Sr}.• Each Si has one representative element x=rep[Si].• Must support three operations:
• MAKE-SET(x): adds new set {x} to Swith rep[{x}] = x (for any x ∉ Si for all i).
• UNION(x, y): replaces sets Sx, Sy with Sx ∪ Syin S for any x, y in distinct sets Sx, Sy .
• FIND-SET(x): returns representative rep[Sx]of set Sx containing element x.
rep.WEAK
![Page 12: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/12.jpg)
Introduction to Algorithms October 18, 2004 L10.12© 2004 by Erik Demaine and Piotr Indyk
Quiz
• If we have a WEAKUNION( x, y) that works only if x, y are representatives, how can we implement UNION that works for any x, y ?
• UNION( x, y)=WEAKUNION( FIND-SET(x) , FIND-SET(y) )
![Page 13: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/13.jpg)
Introduction to Algorithms October 18, 2004 L10.13© 2004 by Erik Demaine and Piotr Indyk
Representation
x
Other fields containingdata of our choice
Data
![Page 14: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/14.jpg)
Introduction to Algorithms October 18, 2004 L10.14© 2004 by Erik Demaine and Piotr Indyk
Applications
• Data clustering• Killer App: Minimum
Spanning Tree (Lecture 13)
• Amortized analysis
1 2
3
5
6
![Page 15: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/15.jpg)
Introduction to Algorithms October 18, 2004 L10.15© 2004 by Erik Demaine and Piotr Indyk
Ideas ?
• How can we implement this data structure efficiently ?– MAKE-SET
– UNION
– FIND-SET
![Page 16: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/16.jpg)
Introduction to Algorithms October 18, 2004 L10.16© 2004 by Erik Demaine and Piotr Indyk
Bad case for UNION or FIND
1 2 … n n+1 … 2n
![Page 17: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/17.jpg)
Introduction to Algorithms October 18, 2004 L10.17© 2004 by Erik Demaine and Piotr Indyk
Simple linked-list solutionStore set Si = {x1, x2, …, xk} as an (unordered) doubly linked list. Define representative elementrep[Si] to be the front of the list, x1.
…Si : x1 x2 xk
rep[Si]
• MAKE-SET(x) initializes x as a lone node.• FIND-SET(x) walks left in the list containing x
until it reaches the front of the list.• UNION(x, y) concatenates the lists containing
x and y, leaving rep. as FIND-SET[x].
Θ(1)
Θ(n)
Θ(n)
How can we improve it ?
![Page 18: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/18.jpg)
Introduction to Algorithms October 18, 2004 L10.18© 2004 by Erik Demaine and Piotr Indyk
Augmented linked-list solution
…Si : x1 x2 xk
rep[Si]
Store set Si = {x1, x2, …, xk} as unordered doublylinked list. Each xj also stores pointer rep[xj] to head.
• FIND-SET(x) returns rep[x].• UNION(x, y) concatenates the lists containing
x and y, and updates the rep pointers forall elements in the list containing y.
![Page 19: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/19.jpg)
Introduction to Algorithms October 18, 2004 L10.19© 2004 by Erik Demaine and Piotr Indyk
Example ofaugmented linked-list solution
Sx : x1 x2
rep[Sx]
rep
Sy : y1 y2 y3
rep[Sy]
rep
![Page 20: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/20.jpg)
Introduction to Algorithms October 18, 2004 L10.20© 2004 by Erik Demaine and Piotr Indyk
Example ofaugmented linked-list solution
Sx ∪ Sy :x1 x2
rep[Sx]
rep
y1 y2 y3
rep[Sy]
rep
![Page 21: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/21.jpg)
Introduction to Algorithms October 18, 2004 L10.21© 2004 by Erik Demaine and Piotr Indyk
Example ofaugmented linked-list solution
Sx ∪ Sy :
x1 x2
rep[Sx ∪ Sy]y1 y2 y3
rep
![Page 22: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/22.jpg)
Introduction to Algorithms October 18, 2004 L10.22© 2004 by Erik Demaine and Piotr Indyk
Augmented linked-list solution
…Si : x1 x2 xk
rep[Si]
Store set Si = {x1, x2, …, xk} as unordered doublylinked list. Each xj also stores pointer rep[xj] to head.
• FIND-SET(x) returns rep[x].• UNION(x, y) concatenates the lists containing
x and y, and updates the rep pointers forall elements in the list containing y.
Θ(1)
Θ(n)
?
![Page 23: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/23.jpg)
Introduction to Algorithms October 18, 2004 L10.23© 2004 by Erik Demaine and Piotr Indyk
Amortized analysis
• So far, we focused on worst-case time of each operation.– E.g., UNION takes Θ(n) time for some operations
• Amortized analysis: count the total time spent by any sequence of operations
• Total time is always at mostworst-case-time-per-operation * #operations
but it can be much better!• E.g., if times are 1,1,1,…,1,n,1,…,1• Can we modify the linked-list data structure so that any
sequence of m MAKE-SET, FIND-SET, UNION operations cost less than m*Θ(n) time?
![Page 24: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/24.jpg)
Introduction to Algorithms October 18, 2004 L10.24© 2004 by Erik Demaine and Piotr Indyk
Alternative
Sx : x1 x2
rep[Sy]
UNION(x, y) :• concatenates the lists containing y and x, and• update the rep pointers for all elements in the
list containing
y1 y2 y3
rep
rep[Sx]rep
Sy :
xy
![Page 25: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/25.jpg)
Introduction to Algorithms October 18, 2004 L10.25© 2004 by Erik Demaine and Piotr Indyk
Alternative concatenation
Sx ∪ Sy :x1 x2
rep[Sy]
UNION(x, y) could instead• concatenate the lists containing y and x, and• update the rep pointers for all elements in the
list containing x.
y1 y2 y3
rep[Sx]rep
rep
![Page 26: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/26.jpg)
Introduction to Algorithms October 18, 2004 L10.26© 2004 by Erik Demaine and Piotr Indyk
Alternative concatenation
Sx ∪ Sy :x1 x2
UNION(x, y) could instead• concatenate the lists containing y and x, and• update the rep pointers for all elements in the
list containing x.
y1 y2 y3
rep
rep
rep[Sx ∪ Sy]
![Page 27: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/27.jpg)
Introduction to Algorithms October 18, 2004 L10.27© 2004 by Erik Demaine and Piotr Indyk
Smaller into larger• Concatenate smaller list onto the end of the larger list (each list stores its weight = # elements)
• Cost = Θ(length of smaller list).
Let n denote the overall number of elements(equivalently, the number of MAKE-SET operations).Let m denote the total number of operations.
Theorem: Cost of all UNION’s is O(n lg n).Corollary: Total cost is O(m + n lg n).
![Page 28: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/28.jpg)
Introduction to Algorithms October 18, 2004 L10.28© 2004 by Erik Demaine and Piotr Indyk
Total UNION cost is O(n lg n)Proof:• Monitor an element x and set Sx containing it• After initial MAKE-SET(x), weight[Sx] = 1• Consider any time when Sx is merged with set Sy
– If weight[Sy] ≥ weight[Sx]• pay 1 to update rep[x]• weight[Sx] at least doubles (increasing by weight[Sy])
– Otherwise• pay nothing• weight[Sx] only increases
• Thus:– Each time we pay 1, the weight doubles– Maximum possible weight is n– Maximum pay ≤ lg n for x , or O(n log n) overall
![Page 29: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/29.jpg)
Introduction to Algorithms October 18, 2004 L10.29© 2004 by Erik Demaine and Piotr Indyk
Final Result
• We have a data structure for dynamic sets which supports:– MAKE-SET: O(1) worst case– FIND-SET: O(1) worst case– UNION:
• Any sequence of any m operations* takes O(m log n) time, or• … the amortized complexity of the operations* is O(log n)
* I.e., MAKE-SET, FIND-SET or UNION
![Page 30: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/30.jpg)
Introduction to Algorithms October 18, 2004 L10.30© 2004 by Erik Demaine and Piotr Indyk
Amortized vs Average
• What is the difference between average case complexity and amortized complexity ?– “Average case” assumes random
distribution over the input (e.g., random sequence of operations)
– “Amortized” means we count the totaltime taken by any sequence of moperations (and divide it by m)
![Page 31: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/31.jpg)
Introduction to Algorithms October 18, 2004 L10.31© 2004 by Erik Demaine and Piotr Indyk
Can we do better ?
• One can do:– MAKE-SET: O(1) worst case– FIND-SET: O(lg n) worst case– WEAKUNION: O(1) worst case– Thus, UNION: O(lg n) worst case
![Page 32: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/32.jpg)
Introduction to Algorithms October 18, 2004 L10.32© 2004 by Erik Demaine and Piotr Indyk
Representing sets as trees• Each set Si = {x1, x2, …, xk} stored as a tree• rep[Si] is the tree root.
S1 = {x1, x2, x3, x4, x5 , x6}S2 = {x7}
• MAKE-SET(x) initializes xas a lone node.
• FIND-SET(x) walks up thetree containing x until itreaches the root.
• UNION(x, y) concatenatesthe trees containingx and y
x1
x4 x3
x2 x5
rep[S1]
x6
x7
rep[S2]
UNION(rep[S1] ,rep[S1]): rep[S1 ∪ S2]
![Page 33: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/33.jpg)
Introduction to Algorithms October 18, 2004 L10.33© 2004 by Erik Demaine and Piotr Indyk
Time Analysis
O(1)• MAKE-SET(x) initializes xas a lone node.
• FIND-SET(x) walks up thetree containing x until itreaches the root.
• WEAKUNION(x, y)concatenatesthe trees containing x and y
O(depth)
O(1)
= ?
![Page 34: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/34.jpg)
Introduction to Algorithms October 18, 2004 L10.34© 2004 by Erik Demaine and Piotr Indyk
“Smaller into Larger” in trees
y1
y4 y3
Algorithm: Merge tree with smaller weight into tree withlarger weight.• Height of tree increases only when its sizedoubles• Height logarithmic in weight
x1
x4 x3
x2 x5 x6
![Page 35: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/35.jpg)
Introduction to Algorithms October 18, 2004 L10.35© 2004 by Erik Demaine and Piotr Indyk
“Smaller into Larger” in trees
Proof:• Monitor the height of an element z• Each time the height of z increases, the
weight of its tree doubles• Maximum weight is n• Thus, height of z is ≤log n
![Page 36: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/36.jpg)
Introduction to Algorithms October 18, 2004 L10.36© 2004 by Erik Demaine and Piotr Indyk
Tree implementation
• We have:– MAKE-SET: O(1) worst case– FIND-SET: O(depth) = O(lg n) worst case– WEAKUNION: O(1) worst case
• Can amortized analysis buy us anything ? • Need another trick…
![Page 37: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/37.jpg)
Introduction to Algorithms October 18, 2004 L10.37© 2004 by Erik Demaine and Piotr Indyk
Trick 2: Path compressionWhen we execute a FIND-SET operation and walkup a path to the root, we know the representativefor all the nodes on the path.
y1
y4 y3
y2 y5
x1
x4 x3
x2 x5 x6
Path compression makesall of those nodes directchildren of the root.
FIND-SET(y2)
![Page 38: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/38.jpg)
Introduction to Algorithms October 18, 2004 L10.38© 2004 by Erik Demaine and Piotr Indyk
Trick 2: Path compressionWhen we execute a FIND-SET operation and walkup a path to the root, we know the representativefor all the nodes on the path.
y1
y4 y3
y2 y5
x1
x4 x3
x2 x5 x6
Path compression makesall of those nodes directchildren of the root.
FIND-SET(y2)
![Page 39: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/39.jpg)
Introduction to Algorithms October 18, 2004 L10.39© 2004 by Erik Demaine and Piotr Indyk
Trick 2: Path compressionWhen we execute a FIND-SET operation and walkup a path p to the root, we know the representativefor all the nodes on path p.
y1
y4
y3y2
y5
x1
x4 x3
x2 x5 x6
FIND-SET(y2)
Path compression makesall of those nodes directchildren of the root.
Cost of FIND-SET(x)is still Θ(depth[x]).
![Page 40: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/40.jpg)
Introduction to Algorithms October 18, 2004 L10.40© 2004 by Erik Demaine and Piotr Indyk
Theorem: In general, amortized cost is
The Theorem
where α(n) grows really, really, really slow.O(α(n)),
![Page 41: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/41.jpg)
Introduction to Algorithms October 18, 2004 L10.41© 2004 by Erik Demaine and Piotr Indyk
Ackermann’s function A
Define
≥=+
= +− 1 if
0 if )(
1)( )1(
1 kk
jAj
jA jk
k
Define α(n) = min {k : Ak(1) ≥ n}.
A0( j) = j + 1A1( j) = A0(…(A0( j)…) ~2jA2( j) = A1(…A1( j)…) ~2j 2j
A3( j ) >
A4( j) is a lot bigger.
22
2
2 j
...
j
A0(1) = 2A1(1) = 3A2(1) = 7A3(1) = 2047
A4(1) >
-iterate Ak-1() j+1 times
22
2
22047
...
2048
![Page 42: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/42.jpg)
Introduction to Algorithms October 18, 2004 L10.42© 2004 by Erik Demaine and Piotr Indyk
Theorem: In general, amortized cost is
The Theorem
where α(n) grows really, really, really slow.O(α(n)),
Proof: Really, really, really long (CLRS, p. 509)
![Page 43: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/43.jpg)
Introduction to Algorithms October 18, 2004 L10.43© 2004 by Erik Demaine and Piotr Indyk
Application:Dynamic connectivity
Suppose a graph is given to us incrementally by• ADD-VERTEX(v)• ADD-EDGE(u, v)
and we want to support connectivity queries:• CONNECTED(u, v):Are u and v in the same connected component?
For example, we want to maintain a spanning forest,so we check whether each new edge connects apreviously disconnected pair of vertices.
![Page 44: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/44.jpg)
Introduction to Algorithms October 18, 2004 L10.44© 2004 by Erik Demaine and Piotr Indyk
Application:Dynamic connectivity
Sets of vertices represent connected components.Suppose a graph is given to us incrementally by
• ADD-VERTEX(v) – MAKE-SET(v)• ADD-EDGE(u, v) – if not CONNECTED(u, v)
then UNION(v, w)and we want to support connectivity queries:
• CONNECTED(u, v): – FIND-SET(u) = FIND-SET(v)Are u and v in the same connected component?
For example, we want to maintain a spanning forest,so we check whether each new edge connects apreviously disconnected pair of vertices.
![Page 45: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/45.jpg)
Introduction to Algorithms October 18, 2004 L10.45© 2004 by Erik Demaine and Piotr Indyk
Simple balanced-tree solutionStore each set Si = {x1, x2, …, xk} as a balanced tree(ignoring keys). Define representative elementrep[Si] to be the root of the tree.
x1
x4 x3
x2 x5
• MAKE-SET(x) initializes xas a lone node.
• FIND-SET(x) walks up thetree containing x until itreaches the root.
• UNION(x, y) concatenatesthe trees containing x and y,changing rep.
Si = {x1, x2, x3, x4, x5}
rep[Si]– Θ(1)
– Θ(lg n)
– Θ(lg n)
![Page 46: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/46.jpg)
Introduction to Algorithms October 18, 2004 L10.46© 2004 by Erik Demaine and Piotr Indyk
Plan of attackWe will build a simple disjoint-union data structurethat, in an amortized sense, performs significantlybetter than Θ(lg n) per op., even better thanΘ(lg lg n), Θ(lg lg lg n), etc., but not quite Θ(1).
To reach this goal, we will introduce two key tricks.Each trick converts a trivial Θ(n) solution into asimple Θ(lg n) amortized solution. Together, thetwo tricks yield a much better solution.
First trick arises in an augmented linked list.Second trick arises in a tree structure.
![Page 47: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/47.jpg)
Introduction to Algorithms October 18, 2004 L10.47© 2004 by Erik Demaine and Piotr Indyk
Each element xj stores pointer rep[xj] to rep[Si].
UNION(x, y)• concatenates the lists containing x and y,
and• updates the rep pointers for all elements
in thelist containing y.
![Page 48: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/48.jpg)
Introduction to Algorithms October 18, 2004 L10.48© 2004 by Erik Demaine and Piotr Indyk
Analysis of Trick 2 aloneTheorem: Total cost of FIND-SET’s is O(m lg n).Proof: Amortization by potential function.The weight of a node x is # nodes in its subtree.Define φ(x1, …, xn) = Σi lg weight[xi].UNION(xi, xj) increases potential of root FIND-SET(xi)by at most lg weight[root FIND-SET(xj)] ≤ lg n.Each step down p → c made by FIND-SET(xi),except the first, moves c’s subtree out of p’s subtree.Thus if weight[c] ≥ ½ weight[p], φ decreases by ≥ 1,paying for the step down. There can be at most lg nsteps p → c for which weight[c] < ½ weight[p].
![Page 49: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/49.jpg)
Introduction to Algorithms October 18, 2004 L10.49© 2004 by Erik Demaine and Piotr Indyk
Analysis of Trick 2 aloneTheorem: If all UNION operations occur beforeall FIND-SET operations, then total cost is O(m).
Proof: If a FIND-SET operation traverses a pathwith k nodes, costing O(k) time, then k – 2 nodesare made new children of the root. This changecan happen only once for each of the n elements,so the total cost of FIND-SET is O(f + n).
![Page 50: Introduction to Algorithms - Duke Universityreif/courses/alglectures/...© 2004 by Erik Demaine and Piotr Indyk Introduction to Algorithms October 18, 2004 L10.8 Procedure for Delete(x)](https://reader030.vdocuments.us/reader030/viewer/2022040915/5e8c29ba41b2762302398175/html5/thumbnails/50.jpg)
Introduction to Algorithms October 18, 2004 L10.50© 2004 by Erik Demaine and Piotr Indyk
UNION(x, y)
• Every tree has a rank• Rank is an upper bound for height• When we take UNION(x, y):
– If rank[x] >rank[y] then link y to x– If rank[x] <rank[y] then link x to y– If rank[x]=rank[y] then
• link x to y• rank[y]=rank[y]+1
• Can show that 2rank(x) ≤ #elements in x (Exercise 21.4-2)• Therefore, height is O(log n)