priority queues - university of haifaoren/ads/lecture 3 - binomial and fibonacci heaps.pdf · n $ b...
TRANSCRIPT
2
Priority Queues
Supports the following operations. ! Insert element x. ! Return min element. ! Return and delete minimum element. ! Decrease key of element x to k.
Applications. ! Dijkstra's shortest path algorithm. ! Prim's MST algorithm. ! Event-driven simulation. ! Huffman encoding. ! Heapsort. ! . . .
Thursday, December 13, 12
4
Dijkstra/Prim 1 make-heap |V| insert |V| delete-min |E| decrease-key
Priority Queues
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1
Binary
log N 1
log N
N log N log N
1
Binomial
log N log N log N
log N log N log N
1
Fibonacci *
1 1
log N
1 1
log N
1
Relaxed
1 1
log N
1 1
log N
1
Linked List
1 N N
1 1 N
is-empty 1 1 1 1 1
Heaps
O(|E| + |V| log |V|) O(|E| log |V|) O(|V|2)
Thursday, December 13, 12
20
Priority Queues
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1
Binary
log N 1
log N
N log N log N
1
Binomial
log N log N log N
log N log N log N
1
Fibonacci *
1 1
log N
1 1
log N
1
Relaxed
1 1
log N
1 1
log N
1
Linked List
1 N N
1 1 N
is-empty 1 1 1 1 1
Heaps
Thursday, December 13, 12
21
Binomial Tree
Binomial tree. ! Recursive definition:
Bk-1
Bk-1
B0 Bk
B0 B1 B2 B3 B4
Thursday, December 13, 12
22
Binomial Tree
Useful properties of order k binomial tree Bk. ! Number of nodes = 2k. ! Height = k. ! Degree of root = k. ! Deleting root yields binomial
trees Bk-1, … , B0.
Proof. ! By induction on k.
B0 B1 B2 B3 B4
B1
Bk
Bk+1
B2 B0
Thursday, December 13, 12
24
Binomial Heap
Binomial heap. Vuillemin, 1978. ! Sequence of binomial trees that satisfy binomial heap property.
– each tree is min-heap ordered – 0 or 1 binomial tree of order k
B4 B0 B1
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3 18
Thursday, December 13, 12
26
Binomial Heap: Properties
Properties of N-node binomial heap. ! Min key contained in root of B0, B1, . . . , Bk. ! Contains binomial tree Bi iff bi = 1 where bn$ b2b1b0 is binary
representation of N. ! At most !log2 N" + 1 binomial trees. ! Height % !log2 N".
B4 B0 B1
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3 18
N = 19 # trees = 3 height = 4 binary = 10011
Thursday, December 13, 12
27
Binomial Heap: Union
Create heap H that is union of heaps H' and H''. ! "Mergeable heaps." ! Easy if H' and H'' are each order k binomial trees.
– connect roots of H' and H'' – choose smaller key to be root of H
H'' 55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
H'
Thursday, December 13, 12
28
Binomial Heap: Union
0 0 1 1
1 0 0 1 +
0 1 1 1
1 1
1 1 0
1
19 + 7 = 26
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3 18
41
33 28
15
25
7 12
+
Thursday, December 13, 12
29
Binomial Heap: Union
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3 18
41
33 28
15
25
7 12
+
Thursday, December 13, 12
30
Binomial Heap: Union
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3
41
33 28
15
25
7
+
12
18
18
12
Thursday, December 13, 12
31
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3
41
33 28
15
25
7
+
12
18
25
37 7
3
18
12
18
12
Thursday, December 13, 12
32
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3
41
33 28
15
25
7
12
+
18
25
37 7
3
41
28 33 25
37 15 7
3
18
12
18
12
Thursday, December 13, 12
33
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3
41
33 28
15
25
7
+
18
12
41
28 33 25
37 15 7
3
12
18
25
37 7
3
41
28 33 25
37 15 7
3
18
12
Thursday, December 13, 12
34
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
37
3
41
33 28
15
25
7
+
18
12
41
28 33 25
37 15 7
3
12
18
25
37 7
3
41
28 33 25
37 15 7
3
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
6
18
12
Thursday, December 13, 12
35
Binomial Heap: Union
Create heap H that is union of heaps H' and H''. ! Analogous to binary addition.
Running time. O(log N) ! Proportional to number of trees in root lists % 2( !log2 N" + 1).
0 0 1 1
1 0 0 1 +
0 1 1 1
1 1
1 1 0
1
19 + 7 = 26
Thursday, December 13, 12
36
3
37
6 18
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
H
Binomial Heap: Delete Min
Delete node with minimum key in binomial heap H. ! Find root x with min key in root list of H, and delete ! H' & broken binomial trees ! H & Union(H', H)
Running time. O(log N)
Thursday, December 13, 12
37
Binomial Heap: Delete Min
Delete node with minimum key in binomial heap H. ! Find root x with min key in root list of H, and delete ! H' & broken binomial trees ! H & Union(H', H)
Running time. O(log N)
55
45 32
30
24
23 22
50
48 31 17
37
6 18
44 8 29 10
H
H'
Thursday, December 13, 12
38
3
37
6 18
55
x 32
30
24
23 22
50
48 31 17
44 8 29 10
H
Binomial Heap: Decrease Key
Decrease key of node x in binomial heap H. ! Suppose x is in binomial tree Bk. ! Bubble node x up the tree if x is too small.
Running time. O(log N) ! Proportional to depth of node x % !log2 N" .
depth = 3
Thursday, December 13, 12
39
Binomial Heap: Delete
Delete node x in binomial heap H. ! Decrease key of x to -'. ! Delete min.
Running time. O(log N)
Thursday, December 13, 12
40
Binomial Heap: Insert
Insert a new node x into binomial heap H. ! H' & MakeHeap(x) ! H & Union(H', H)
Running time. O(log N)
3
37
6 18
55
45 32
30
24
23 22
50
48 31 17
44 8 29 10
H
x
H'
Thursday, December 13, 12
41
Binomial Heap: Sequence of Inserts
Insert a new node x into binomial heap H. ! If N = .......0, then only 1 steps. ! If N = ......01, then only 2 steps. ! If N = .....011, then only 3 steps. ! If N = ....0111, then only 4 steps.
Inserting 1 item can take #(log N) time. ! If N = 11...111, then log2 N steps.
But, inserting sequence of N items takes O(N) time! ! (N/2)(1) + (N/4)(2) + (N/8)(3) + . . . % 2N ! Amortized analysis. ! Basis for getting most operations
down to constant time.
50
48 31 17
44 29 10
3
37
6 x
Thursday, December 13, 12
1
Priority Queues
make-heap
Operation
insert
find-min
delete-min
union
decrease-key
delete
1 Binary
log N 1
log N
N log N log N
1 Binomial
log N log N log N
log N log N log N
1 Fibonacci †
1 1
log N
1 1
log N
1 Relaxed
1 1
log N
1 1
log N
1 Linked List
1 N N
1 1 N
is-empty 1 1 1 1 1
Heaps
this time † amortized
3
Fibonacci Heaps: Structure Fibonacci heap. � Set of min-heap ordered trees.
7 23
30
17
35
26 46
24
H 39
41 18 52
3
44
min
marked
4
Fibonacci Heaps: Implementation Implementation. � Represent trees using left-child, right sibling pointers and circular,
doubly linked list. – can quickly splice off subtrees
� Roots of trees connected with circular doubly linked list. – fast union
� Pointer to root of tree with min element. – fast find-min
7 23
30
17
35
26 46
24
H 39
41 18 52
3
44
min
5
Fibonacci Heaps: Potential Function Key quantities. � Degree[x] = degree of node x. � Mark[x] = mark of node x (black or gray). � t(H) = # trees. � m(H) = # marked nodes. � )(H) = t(H) + 2m(H) = potential function.
7 23
30
17
35
26 46
24
H
t(H) = 5, m(H) = 3 )(H) = 11
39
41 18 52
3
44
min degree = 3
6
Fibonacci Heaps: Insert Insert. � Create a new singleton tree. � Add to left of min pointer. � Update min pointer.
7 23
30
17
35
26 46
24
H 39
41 18 52
3
44
min 21
Insert 21
7
Fibonacci Heaps: Insert Insert. � Create a new singleton tree. � Add to left of min pointer. � Update min pointer.
39
41
7 23
18 52
3
30
17
35
26 46
24
44
min
H
21
Insert 21
8
Fibonacci Heaps: Insert Insert. � Create a new singleton tree. � Add to left of min pointer. � Update min pointer.
Running time. O(1) amortized � Actual cost = O(1). � Change in potential = +1. � Amortized cost = O(1).
39
41
7
18 52
3
30
17
35
26 46
24
44
min
H
21 23
Insert 21
9
Fibonacci Heaps: Union Union. � Concatenate two Fibonacci heaps. � Root lists are circular, doubly linked lists.
39
41
7 17
18 52
3
30
23
35
26 46
24
44
min
H' H''
21
min
10
Fibonacci Heaps: Union Union. � Concatenate two Fibonacci heaps. � Root lists are circular, doubly linked lists.
Running time. O(1) amortized � Actual cost = O(1). � Change in potential = 0. � Amortized cost = O(1).
39
41
7 17
18 52
3
30
23
35
26 46
24
44
min
H' H''
21
11
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 18 52
3
44
min
17 23
30
7
35
26 46
24
12
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 17 23 18 52
30
7
35
26 46
24
44
current min
13
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 17 23 18 52
30
7
35
26 46
24
44
current
0 1 2 3
min
14
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 17 23 18 52
30
7
35
26 46
24
44
current
0 1 2 3
min
15
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 17 23 18 52
30
7
35
26 46
24
44 current
0 1 2 3
min
16
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 17 23 18 52
30
7
35
26 46
24
44 current
0 1 2 3
Merge 17 and 23 trees.
min
17
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 17
23
18 52
30
7
35
26 46
24
44
current
0 1 2 3
Merge 7 and 17 trees.
min
18
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 7
30
18 52
17
35
26 46
24
44
current
0 1 2 3
23 Merge 7 and 24 trees.
min
19
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 7
30
18 52
23
17
35
26 46
24 44
current
0 1 2 3
min
20
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 7
30
18 52
23
17
35
26 46
24 44
current
0 1 2 3
min
21
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 7
30
18 52
23
17
35
26 46
24 44
current
0 1 2 3
min
22
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39
41 7
30
18 52
23
17
35
26 46
24 44
current
0 1 2 3
Merge 41 and 18 trees.
min
23
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39 41
7
30
18 52
23
17
35
26 46
24
44
current
0 1 2 3
min
24
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39 41
7
30
18 52
23
17
35
26 46
24
44
current
0 1 2 3
min
25
Fibonacci Heaps: Delete Min Delete min. � Delete min and concatenate its children into root list. � Consolidate trees so that no two roots have same degree.
39 41
7
30
18 52
23
17
35
26 46
24
44
min
Stop.
26
Fibonacci Heaps: Delete Min Analysis Notation. � D(n) = max degree of any node in Fibonacci heap with n nodes. � t(H) = # trees in heap H. � )(H) = t(H) + 2m(H).
Actual cost. O(D(n) + t(H)) � O(D(n)) work adding min's children into root list and updating min.
– at most D(n) children of min node � O(D(n) + t(H)) work consolidating trees.
– work is proportional to size of root list since number of roots decreases by one after each merging
– d D(n) + t(H) - 1 root nodes at beginning of consolidation
Amortized cost. O(D(n)) � t(H') d D(n) + 1 since no two trees have same degree. � ')(H) d D(n) + 1 - t(H).
27
Fibonacci Heaps: Delete Min Analysis Is amortized cost of O(D(n)) good?
� Yes, if only Insert, Delete-min, and Union operations supported.
– in this case, Fibonacci heap contains only binomial trees since we only merge trees of equal root degree
– this implies D(n) d ¬log2 N¼
� Yes, if we support Decrease-key in clever way. – we'll show that D(n) d ¬logI N¼, where I is golden ratio – I2 = 1 + I – I = (1 + �5) / 2 = 1.618… – limiting ratio between successive Fibonacci numbers!
28
Decrease key of element x to k. � Case 0: min-heap property not violated.
– decrease key of x to k – change heap min pointer if necessary
24
46
17
30
23
7
88
26
21
52
39
18
41
38
Decrease 46 to 45. 72
Fibonacci Heaps: Decrease Key
45
35
min
29
Decrease key of element x to k. � Case 1: parent of x is unmarked.
– decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer
24
45
17
30
23
7
88
26
21
52
39
18
41
38
Decrease 45 to 15. 72
Fibonacci Heaps: Decrease Key
15
35
min
30
Decrease key of element x to k. � Case 1: parent of x is unmarked.
– decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer
24
15
17
30
23
7
88
26
21
52
39
18
41
38
Decrease 45 to 15. 72
24
Fibonacci Heaps: Decrease Key
35
min
31
Decrease key of element x to k. � Case 1: parent of x is unmarked.
– decrease key of x to k – cut off link between x and its parent – mark parent – add tree rooted at x to root list, updating heap min pointer
24 17
30
23
7
88
26
21
52
39
18
41
38
Decrease 45 to 15.
24
Fibonacci Heaps: Decrease Key
35
min 15
72
32
35
Decrease key of element x to k. � Case 2: parent of x is marked.
– decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list � If p[p[x]] unmarked, then mark it. � If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
24
15
17
30
23
7
88
26
21
52
39
18
41
38
Decrease 35 to 5.
72 24
Fibonacci Heaps: Decrease Key
5
min
33
Decrease key of element x to k. � Case 2: parent of x is marked.
– decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list � If p[p[x]] unmarked, then mark it. � If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
24 17
30
23
7
26
21
52
39
18
41
38
Decrease 35 to 5.
24
5
Fibonacci Heaps: Decrease Key
88
parent marked
15
72
min
34
Decrease key of element x to k. � Case 2: parent of x is marked.
– decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list � If p[p[x]] unmarked, then mark it. � If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
24
26
17
30
23
7
21
52
39
18
41
38
Decrease 35 to 5.
88 24
5
Fibonacci Heaps: Decrease Key
15
72
parent marked
min
35
Decrease key of element x to k. � Case 2: parent of x is marked.
– decrease key of x to k – cut off link between x and its parent p[x], and add x to root list – cut off link between p[x] and p[p[x]], add p[x] to root list � If p[p[x]] unmarked, then mark it. � If p[p[x]] marked, cut off p[p[x]], unmark, and repeat.
26
17
30
23
7
21
52
39
18
41
38
Decrease 35 to 5.
88
5
Fibonacci Heaps: Decrease Key
15 24
72
min
36
Notation. � t(H) = # trees in heap H. � m(H) = # marked nodes in heap H. � )(H) = t(H) + 2m(H).
Actual cost. O(c) � O(1) time for decrease key. � O(1) time for each of c cascading cuts, plus reinserting in root list.
Amortized cost. O(1) � t(H') = t(H) + c � m(H') d m(H) - c + 2
– each cascading cut unmarks a node – last cascading cut could potentially mark a node
� ') d c + 2(-c + 2) = 4 - c.
Fibonacci Heaps: Decrease Key Analysis
37
Delete node x. � Decrease key of x to -f. � Delete min element in heap.
Amortized cost. O(D(n)) � O(1) for decrease-key. � O(D(n)) for delete-min. � D(n) = max degree of any node in Fibonacci heap.
Fibonacci Heaps: Delete
38
Fibonacci Heaps: Bounding Max Degree Definition. D(N) = max degree in Fibonacci heap with N nodes. Key lemma. D(N) d logI N, where I = (1 + �5) / 2. Corollary. Delete and Delete-min take O(log N) amortized time.
Lemma. Let x be a node with degree k, and let y1, . . . , yk denote the children of x in the order in which they were linked to x. Then:
Proof. � When yi is linked to x, y1, . . . , yi-1 already linked to x, � degree(x) = i - 1 � degree(yi) = i - 1 since we only link nodes of equal degree
� Since then, yi has lost at most one child – otherwise it would have been cut from x
� Thus, degree(yi) = i - 1 or i - 2
¯®
t�
t1if21if0
)(degreeiii
yi
39
Fibonacci Heaps: Bounding Max Degree Key lemma. In a Fibonacci heap with N nodes, the maximum degree of any node is at most logI N, where I = (1 + �5) / 2. Proof of key lemma. � For any node x, we show that size(x) t Idegree(x) .
– size(x) = # node in subtree rooted at x – taking base I logs, degree(x) d logI (size(x)) d logI N.
� Let sk be min size of tree rooted at any degree k node. – trivial to see that s0 = 1, s1 = 2 – sk monotonically increases with k
� Let x* be a degree k node of size sk, and let y1, . . . , yk be children in order that they were linked to x*.
Assume k t 2
¦�
¦�t
¦�t
¦�
�
�
2
0
22
2]deg[
2
2
2
2
)(2
*)(size
k
ii
k
ii
k
iy
k
ii
k
s
s
s
ysize
xs
i
≥
40
Fibonacci Facts Definition. The Fibonacci sequence is: � 1, 2, 3, 5, 8, 13, 21, . . . • Slightly nonstandard definition.
Fact F1. Fk t Ik, where I = (1 + �5) / 2 = 1.618… Fact F2. Consequence. sk t Fk t Ik. � This implies that size(x) t Idegree(x)
for all nodes x.
°̄
°®
t�
2ifFF1if20if1
F
2-k1-k
k
kkk
¦� t�
2
02,2For
k
iik F Fk
¦�
¦�t
¦�t
¦�
�
�
2
0
22
2]deg[
2
2
2
2
)(2
*)(size
k
ii
k
ii
k
iy
k
ii
k
s
s
s
ysize
xs
i
≥
41
Golden Ratio Definition. The Fibonacci sequence is: 1, 2, 3, 5, 8, 13, 21, . . . Definition. The golden ratio I = (1 + �5) / 2 = 1.618… � Divide a rectangle into a square and smaller rectangle such that the
smaller rectangle has the same ratio as original one.
Parthenon, Athens Greece
43
Fibonacci Numbers and Nature
Pinecone
Cauliflower
44
Fibonacci Proofs Fact F1. Fk t Ik. Proof. (by induction on k) � Base cases:
– F0 = 1, F1 = 2 t I. � Inductive hypotheses:
– Fk t Ik and Fk+1 t Ik+1 Fact F2. Proof. (by induction on k) � Base cases:
– F2 = 3, F3 = 5 � Inductive hypotheses:
¦� t�
2
02,2For
k
iik F Fk
2
2
112
)()1(
�
���
� �t�
k
k
k
kkkkk FFF
MMM
MMMM
I2 = I + 1
¦� �
2
02
k
iik FF ¦�
¦ ��
�
�
�
��
k
ik
k
iki
kkk
F
FF
FFF
0
2
01
12
2
2
45
On Complicated Algorithms "Once you succeed in writing the programs for [these] complicated algorithms, they usually run extremely fast. The computer doesn't need to understand the algorithm, its task is only to run the programs."
R. E. Tarjan