Download - heaps.pdf
-
8/9/2019 heaps.pdf
1/72
111111Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
2/72
Motivation
Queues are a standard mechanism for ordering taskson a first-come, first-served basis
However, some tasks may be more important ortimely than others (higher priority)
Store tasks using a partial ordering based on priority
Ensure highest priority task at head of queue
Heaps are the underlying data structure of priorityqueues
22222Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
3/72
Priority Queues: Specification
Main operations insert (i.e., enqueue)
ynam c nser
specification of a priority level (0-high, 1,2.. Low)
deleteMin (i.e., dequeue)
Finds the current minimum element (read: “highest priority”) inthe queue, deletes it from the queue, and returns it
Performance goal is for operations to be “fast”
3Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
4/72
Using priority queues
5 3
1019
4
8
2211
insert()
deleteMin()
2Dequeues the next element
4Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
5/72
Can we build a data structure better suited to store and retrieve priorities?
Simple Implementations
Unordered linked list O(1) insert
5 2 103…
Ordered linked list O(n) insert
2 3 5 10…
e e e n
Ordered array O(lg n + n) insert
2 3 5 … 10
O(n) deleteMin
Balanced BST
55Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
6/72
nary eap
A priority queue data structure
6Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
7/72
Binary Heap
A binary heap is a binary tree with two
Structure property
-
7Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
8/72
Structure Property
A binary heap is a complete binary tree Each level exce t ossibl the bottom most level
is completely filled
The bottom most level may be partially filledrom e o r g
Height of a complete binary tree with Nelements is N 2log
88Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
9/72
Structure property
Binary Heap Example
N=10
Every level
(except last)
saturated
Array representation:
9Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
10/72
Heap-order Property
Heap-order property (for a “MinHeap”) ,
Except root node, which has no parent
,
Alternatively, for a “MaxHeap”, always
Insert and deleteMin must maintain-
10Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
11/72
Heap Order Property
Minimum
element
No order implied for elements which do not
-
11Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
12/72
Implementing CompleteBinary Trees as Arrays
Given element at position i in the array=
Right child(i) = at position 2i + 1
2i
2i + 1
i
12Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
13/72
insert
without deleting it
deleteMin
Note: a general delete()function is not as important
for heaps
but could be implemented
Stores the heap as
a vector
Fix hea after
13
deleteMin
Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
14/72
Heap Insert
Insert new element into the heap at the “ ”
According to maintaining a complete binary
tree
Then, “percolate” the element up the
- satisfied
14Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
15/72
Percolating Up
Heap Insert: Example
Insert 14:
hole14
15Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
16/72
Percolating Up
Heap Insert: Example
Insert 14:(1)
14 vs. 31
hole
14
14
1616Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
17/72
Percolating Up
Heap Insert: Example
Insert 14:(1)
14 vs. 31
hole
14
14
14 vs. 21
14
1717Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
18/72
Percolating Up
Heap Insert: Example
Insert 14:(1)
14 vs. 31
hole14
(3) Heap order prop14
14 vs. 21
vs. ruc ure prop
Path of ercolation u
1818Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
19/72
Heap Insert: Implementation
// assume array implementationvoid insert( const Comparable &x) {
}
19Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
20/72
Heap Insert: Implementation
2020Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
21/72
Heap DeleteMin
Minimum element is always at the root
Move last element into hole at root Percolate down while heap-order
property not satisfied
21Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
22/72
Percolating down…
Heap DeleteMin: Example
Make thisposition
empty
22Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
23/72
Percolating down…
Heap DeleteMin: Example
Copy 31 temporarilyhere and move it dow
Is 31 > min(14,16)?
•Yes - swap 31 with min(14,16)
Make thisposition
empty
2323Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
24/72
Percolating down…
Heap DeleteMin: Example
31
Is 31 > min(19,21)?
•Yes - swap 31 with min(19,21)
24Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
25/72
Percolating down…
Heap DeleteMin: Example
31
31
Is 31 > min(65,26)?
•Yes - swap 31 with min(65,26)Is 31 > min(19,21)?
•Yes - swap 31 with min(19,21)
25Percolating down… 25Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
26/72
Percolating down…
Heap DeleteMin: Example
31
26Percolating down…Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
27/72
Percolating down…
Heap DeleteMin: Example
31
Heap order prop
27
27Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
28/72
Heap DeleteMin:Implementation
28
O(log N) time
Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
29/72
Heap DeleteMin:Implementation
Percolate
Left child
down
Right child
Pick child toswap with
29Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
30/72
Other Heap Operations
decreaseKey(p,v) Lowers the current value of item p to new priority value v Need to ercolate u E.g., promote a job
increaseKey(p,v) Increases the current value of item to new riorit value v Need to percolate down E.g., demote a job
remove Run-times for all three functions? First, decreaseKey(p,-∞) Then, deleteMin E.g., abort/cancel a job
O(lg n)
30Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
31/72
Improving Heap Insert Time
What if all N elements are all availableupfront?
o build a hea with N elements: Default method takes O(N lg N) time
We will now see a new method called buildHeap()
t at wi ta e O N time - i.e., optima
31Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
32/72
Building a Heap
Construct heap from initial set of N items Solution 1
Perform N inserts
O(N log2 N) worst-case Solution 2 (use buildHeap() )
Randomly populate initial heap with structure
property Perform a percolate-down from each internal node
To take care of heap order property32Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
33/72
BuildHeap Examplenput: , , , , , , , , , , , , , ,
valid heaps
(implicitly)
• Arbitrarily assign elements to heap nodes
• Structure property satisfied
• Hea order ro ert violated
So, let us look at eachinternal node,
from bottom to top,
33
• Leaves are all valid heaps (implicit)
Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
34/72
BuildHeap ExampleSwap
Nothingto do
with left
child
• Randomly initialized heap
34
• Structure property satisfied
• Heap order property violated
• Leaves are all valid heaps (implicit) 34Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
35/72
BuildHeap Example Swapwith right
childNothing
to do
Dotted lines show path of percolating down
3535Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
36/72
Swa with
BuildHeap Exampleright child
& then with 60
o ng
to do
Dotted lines show path of percolating down
36Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
37/72
BuildHeap Example
Swap path
Dotted lines show path of percolating down
Final Heap
37Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
38/72
BuildHeap Implementation
Start withlowest,
rightmost
38
internal node
Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
39/72
BuildHeap() : Run-time Analysis
Run-time = ? O(sum of the heights of all the internal nodes)
ecause we may ave o perco a e a e waydown to fix every internal node in the worst-case
heorem 6.1 HOW? For a perfect binary tree of height h, the sum of
heights of all nodes is 2 h+1 – 1 – (h + 1)
nce = g , en sum o e g s s Will be slightly better in practice
Implication: Each insertion costs O(1) amortized time
39Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
40/72
40Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
41/72
Binary Heap OperationsWorst-case Analysis
Height of heap is insert: O l N for each insert
N 2log
In practice, expect less
buildHea insert: O N for N inserts deleteMin: O(lg N)
r K : l N
increaseKey: O(lg N)
41Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
42/72
Applications
Operating system scheduling
Graph algorithms
Event simulation
Instead of checking for events at each timeclick, look up next event to happen
42Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
43/72
An Application:he Selection Problem
Given a list of n elements, find the k th
smallest element
Sort the list => O(n log n)
Pick the k th element => O 1
A better algorithm:
43Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
44/72
Selection using a MinHeap
Input: n elements
Algorithm:1. u eap n == n
2. Perform k deleteMin() operations ==> O(k log n)
3.
Report the k th
deleteMin output ==> O(1)
Total run-time = O(n + k log n)
If k = O(n/log n) then the run-time becomes O(n)
44Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
45/72
Other Types of Heaps
Binomial Heaps
- eaps Generalization of binary heaps (ie., 2-Heaps)
Leftist Heaps Supports merging of two heaps in o(m+n) time (ie., sub-
linear)
Skew Heaps O(log n) amortized run-time
Fibonacci Heaps45Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
46/72
Run-time Per Operation
Insert DeleteMin Merge (=H1+H2)
Binary heap O(log n) worst-case O(log n) O(n)
O(1) amortized forbuildHeap
Leftist Heap O(log n) O(log n) O(log n)
Skew Heap O(log n) O(log n) O(log n)
nom aHeap
og n wors case
O(1) amortized forsequence of n inserts
og n og n
Fibonacci Heap O(1) O(log n) O(1)
46Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
47/72
Priority Queues in STL
Uses Binary heap Default is MaxHea
#include
int main ()
Methods
Push to o
{
priority_queue Q;
Q.push (10);
empty, clear .
Q.pop ();
}Calls DeleteMax()
For MinHeap: declare priority_queue as:
priority_queue Q;
47
Refer to Book Chapter 6, Fig 6.57 for an example
Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
48/72
Binomial Heaps
48Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
49/72
Binomial Heap
A binomial heap is a forest of heap-orderedbinomial trees, satisfying:
,ii) Heap order property
A binomial heap is different from binary heapin that:
Its heap-order property (within each binomial
tree) is the same as in a binary heap
49Cpt S 223. School of EECS, WSU
Note: A binomial tree need not be a binary tree
-
8/9/2019 heaps.pdf
50/72
y
Definition: A “Binomial Tree” Bk
A binomial tree of height k is called Bk : It has 2k nodes
The number of nodes at depth d = ( )k
d
-k d
3
Depth: #nodes:B3:
=
d=1
0
( 31 )3
d=3
2
( 33 ) 50Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
51/72
Wh will Bin mi l H wi h n= 1 nodes look like?
We know that:i) A binomial heap should be a forest of binomial
trees
ii) Each binomial tree has power of 2 elements
B0B1B2B3B4
o ow many inomia trees o we nee
n = =2
51Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
52/72
nom a eap w n= no es
n = 31 = (1 1 1 1 1)2
Bi == Bi-1 + Bi-1
, B 2 ,
B 3 ,
B 4
}
BB
B1B0
t r e e s { B 0 ,
B 1
t o f b i n o m
i a l
F o r e s
52Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
53/72
Binomial Heap Property
Lemma: There exists a binomial heap for everypositive value of n
Proof:
Have one binomial tree for each power of two with co-efficientof 1
= == ==
53Cpt S 223. School of EECS, WSU
Binomial Heaps: Heap Order
-
8/9/2019 heaps.pdf
54/72
Binomial Heaps: Heap-OrderProperty
Each binomial tree should contain theminimum element at the root of everysubtree
Just like binary heap, except that the treeere s a nom a tree structure an not acomplete binary tree)
The order of elements across binomial
54Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
55/72
Definition: Binomial Heaps
A binomial heap of n nodes is:
(Structure Property) A forest of binomial trees as dictated by
(Heap-Order Property) Each binomial tree is a min-heap or amax- eap
55Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
56/72
Binomial Heaps: Examples
Two different heaps:
56Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
57/72
Key Properties
Could there be multiple trees of the same height in abinomial heap?
no
What is the upper bound on the number of binomial
g n
Given n , can we tell (for sure) if Bk exists?
Bk exists if and only if:
the kth least significant bit is 1
in the binary representation of n
57Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
58/72
An Im lementation of a Binomial HeaMaintain a linked list of
tree pointers (for the forest)
B0
B1
B2
B3
B4
B5
B6
B7
2
Shown using the
Analo ous to a bit-based re resentation of a
- , -
binary number n
58Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
59/72
Binomial Heap: Operations
x
-
8/9/2019 heaps.pdf
60/72
DeleteMin()
Goal: Given a binomial heap, H, find theminimum and delete it
Observation: The root of each binomial tree
in H contains its minimum element Approach: Therefore, return the minimum of
all the roots (minimums)
Complexity: O(log n) comparisons(because there are only O(log n) trees)
60Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
61/72
FindMin() & DeleteMin() Example
B0 B2B3
B1’ B2’B0’
For DeleteMin(): After delete, how to adjust the heap?
’ ’ ’ 0, 2 0 , 1 , 2
61Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
62/72
Insert(x ) in Binomial Heap
Goal: To insert a new element x into a
Observation:
element binomial heap
= == ,
So, if we decide how to do merge we will automatically
figure out how to implement both insert() and deleteMin()
62Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
63/72
Merge(H1,H2)
Let n1 be the number of nodes in H1 Let n2 be the number of nodes in H2
, 1 2nodes Assume n = n1 + n2
Logic: Merge trees of same height, starting from lowest height
If only one tree of a given height, then just copy that Otherwise, need to do carryover (just like adding two binary
numbers)
63Cpt S 223. School of EECS, WSU
Idea: merge tree of same heights
-
8/9/2019 heaps.pdf
64/72
Merge: Example
+
B0 B1 B2
13
?? 64Cpt S 223. School of EECS, WSU
How to Merge Two Binomial
-
8/9/2019 heaps.pdf
65/72
How to Merge Two Binomial
rees of the Same Height?
+
B2:
B2: B3:
Simply compare the roots
Note: Merge is defined for only binomial trees with the same height 65Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
66/72
1, 2 carryover
+
13 14 ?
26
26
66Cpt S 223. School of EECS, WSU
How to Mer e more than two
-
8/9/2019 heaps.pdf
67/72
How to Mer e more than two binomial trees of the same height?
Merging more than 2 binomial trees ofthe same height could generate carry-overs
+14
26 16
26
-
67Cpt S 223. School of EECS, WSU
Input:
-
8/9/2019 heaps.pdf
68/72
+
Merge(H1,H2) : Example
Output:
w w
= ,
68Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
69/72
Run-time Complexities
Merge takes O(log n) comparisons
Insert and DeleteMin also take O(log n)
Insert operations takes only O(m) time per operation, implyingO(1) amortize time per insert
Proof Hint: For each insertion, if i is the least significant bit position with a 0, then
number of comparisons required to do the next insert is i+1 If you count the #bit flips for each insert, going from insert of the first
element to the insert of the last (nth) element, thenaffected
unaffected
= amor ze run- me o per nser
1
--------------
10011000 69Cpt S 223. School of EECS, WSU
Binomial Queue Run-time
-
8/9/2019 heaps.pdf
70/72
Q
Summary
Insert O(lg n) worst-case
O(1) amortized time if insertion is done in anuninterrupted sequence (i.e., without being
intervened by deleteMins) DeleteMin, FindMin
O(lg n) worst-case
Merge O(lg n) worst-case
70Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
71/72
Run-time Per Operation
Insert DeleteMin Merge (=H1+H2)
Binary heap O(log n) worst-case O(log n) O(n)
O(1) amortized forbuildHeap
Leftist Heap O(log n) O(log n) O(log n)
Skew Heap O(log n) O(log n) O(log n)
nom a
Heap
og n wors case
O(1) amortized forsequence of n inserts
og n og n
Fibonacci Heap O(1) O(log n) O(1)
71Cpt S 223. School of EECS, WSU
-
8/9/2019 heaps.pdf
72/72
Summary
Priority queues maintain the minimum
Support O(log N) operations worst-case
, ,
Many applications in support of othera or ms
72Cpt S 223. School of EECS, WSU