algorithms and data structures - bfhhnr1/algdata/slides/05heaps.pdf · priority queues and heaps...

Priority Queues and Heaps

Berner Fachhochschule - Technik und Informatik

Algorithms and Data Structures


Dr. Rolf Haenni

Fall 2008

Berner Fachhochschule Rolf Haenni

Technik und Informatik Algorithms and Data Structures


Outline

Priority Queues

Heaps

Heap-Based Priority Queues

Bottom-Up Heap Construction



Priority Queues and Heaps Priority Queues

Outline

Priority Queues

Heaps






Priority Queue ADT

I A priority queue stores a collection of items

I An item is a pair (key , element)I Characteristic operations:

Ý insertItem(k,e): inserts an item with key k and element eÝ removeMin(): removes the item with the smallest key and

returns its elementÝ minKey(): returns the smallest key of an item (no removal)Ý minElement(): returns the element of an item with smallest

key (no removal)

I General operations:

Ý size(): returns the number of itemsÝ isEmpty(): indicates whether the priority queue is empty

I Two distinct items in a priority queue can have the same key




Total Order

I Keys in a priority queue can be arbitrary objects on which atotal order is defined

I Mathematically, a total order is a binary relation � defined ona set K which satisfies three properties:

Ý Totality: x � y or y � x , for all x , y ∈ KÝ Antisymmetry: x � y and y � x implies x = y , for all x , y ∈ KÝ Transitivity: x � y and y � z implies x � z , for all x , y , z ∈ K

I Totality implies Reflexivity: x � x , for all x ∈ K

I Examples: ≤ or ≥ for R, alphabetical or reverse alphabeticalorder for {A, . . . ,Z}, lexicographical order for {A, . . . ,Z}∗,etc.




Comparator ADT

I A comparator encapsulates the action of comparing two keysaccording to a given total order relation

I A generic priority queue uses an auxiliary comparator (passedas a parameter to the constructor)

I When the priority queue needs to compare two keys, it uses itscomparator

I Operations (all with Boolean return type):

Ý isLessThan(x,y)Ý isLessThanOrEqualTo(x,y)Ý isEqualTo(x,y)Ý isGreaterThan(x,y)Ý isGreaterThanOrEqualTo(x,y)Ý isComparable(x)




Sequence-Based Priority Queue

I There are two ways to implement a priority queue with asequence (list or vector)

I Using an unsorted sequence

Ý insertItem(k,e) runs in O(1) time, since we can insert theitem at the beginning of the sequence

Ý removeMin(), minKey(), minElement() run in O(n) timesince we have to traverse the entire sequence to find thesmallest key

I Using a sorted sequence

Ý insertItem(k,e) runs in O(n) time, since we have to findthe place where to insert the item

Ý removeMin(), minKey(), minElement() run in O(1) timesince the smallest key is at the beginning or end of thesequence




UML Diagram: Composition

<<interface>>PriorityQueue

insertItem(k,e)removeMin()minKey()minElement()

<<interface>>Position

etc.

<<interface>>Comparator

isLessThan(x,y)isEqualTo(x,y)etc.

<<interface>>List

etc.

<<interface>>BasicSequence

isEmpty()size()

SinglyLinkedListfirst: ListNoden: Integer

ListNodeelement: Objectnext: ListNode

SortedPQlist: SinglyLinkedListC: Comparator

UnsortedPQlist: SinglyLinkedListC: Comparator




UML Diagram: Inheritance



<<interface>>Position

etc.



<<interface>>List

etc.


isEmpty()size()

SinglyLinkedListfirst: ListNoden: Integer

ListNodeelement: Objectnext: ListNode

SortedPQC: Comparator

UnsortedPQC: Comparator




Sorting with a Priority Queue

I We can use a priority queue to sort a collection of comparableelements

Ý Insert the elements one by one with a series ofinsertItem(e,e) operations

Ý Remove the elements in sorted order with a series ofremoveMin() operations

I Depending on how the priority queue is implemented (sortedor unsorted), this leads to two well-known sorting algorithms

Ý Selection-SortÝ Insertion-Sort




PQ-Sort Algorithm

Solution in pseudo-code:

Algorithm PQ−Sort(S ,C )P ← new priority queue with comparator Cwhile not S .isEmpty() do

e ← S .removeElement(S .first())P.insertItem(e,e)

while not P.isEmpty() doe ← P.removeMin()S .insertLast(e)

return S




Selection-Sort

I Selection-sort is the variation of PQ-sort where the priorityqueue is implemented with an unsorted sequence

I The main task is to select the elements from the unsortedsequence in the right order

I Running time analysis:

Ý Inserting the elements into the priority queue with ninsertItem(e,e) operations runs in O(n) time

Ý Removing the elements in sorted order from the priority queuewith n removeMin() operations takes time proportional ton + . . . + 2 + 1, and thus runs in O(n2) time

I Selection-sort runs in O(n2) time




Insertion-Sort

I Selection-sort is the variation of PQ-sort where the priorityqueue is implemented with an sorted sequence

I The main task is to insert the elements at the right placeI Running time analysis:

Ý Inserting the elements into the priority queue with ninsertItem(e,e) operations takes time proportional to1 + 2 + . . . + n, and thus runs in O(n2) time

Ý Removing the elements in sorted order from the priority queuewith n removeMin() operations runs in O(n) time

I In general (worst case), insertion-sort runs in O(n2) time

I If the input sequence is in reverse order (best case), it runs inO(n) time




In-Place Sorting

I Instead of using external datastructures, we can implementinsertion- and selection-sortin-place

I A portion of the input sequenceitself serves as priority queue

I For in-place insertion-sort we

Ý keep the initial portion of thesequence sorted

Ý use swapElements(p,q) insteadof modifying the sequence

4 5 2 3 1

2 4 5 3 1

2 3 4 5 1

1 2 3 4 5

5 4 2 3 1

5 4 2 3 1



Priority Queues and Heaps Heaps

Outline

Priority Queues

Heaps






The Heap ADT

I A heap is a specialized binary tree storing keys at its nodes(positions) and satisfying the following properties:

Ý Heap-Order: key(p) � key(parent(p)), for every node p otherthan the root

Ý Completeness: let h be the height of the tree1) there are 2i nodes of depth i for i = 0, . . . , h − 12) at depth h, nodes are filled up from the left

I The last node of a heap is the rightmost node of depth hI Heap operations:

Ý insertKey(k): inserts a key k into the heapÝ removeMin(): removes smallest key and returns itÝ minKey(): returns the smallest key (no removal)




Heap Example

52

6

9 7 11 8

13 10 9 8

Last node

01

2

312 14 9 11

17 15 412

Depth

I A heap storing n keys has height O(log n)




Insertion to a Heap

I The insertion of a key kconsists of 3 steps:

Ý Add a new node q (the newlast node)

Ý Store k at qÝ Restore the heap order

property (discussed next)

52

6

9 7

52

6

9 7 1

q




Upheap

I Algorithm upheap restores theheap-order property

Ý Swap k along an upward pathfrom the insertion node

Ý Stop when k reaches the rootor a node whose parent has akey smaller than or equal to k

Ý Since the height of the heapis O(log n), upheap runs inO(log n) time

I insertKey(k) runs in O(log n)time

52

1

9 7

51

2

9 7 6

6




Removal from a Heap

I The removal algorithm consistsof 4 steps:

Ý Return the key of the rootÝ Replace the root key with the

key k of the last nodeÝ Delete the last nodeÝ Restore the heap order

property (discussed next)

52

6

9 7

57

6

9

Last node




Downheap

I Algorithm downheap restores theheap-order property

Ý Swap k along an downwardpath from the root (always withthe smaller key of its children)

Ý Stop when k reaches a leaf or anode whose children have keysgreater than or equal to k

Ý Since the height of the heap isO(log n), downheap runs inO(log n) time

I removeMin() runs in O(log n)time

57

6

9

75

6

9




Finding the Insertion Node

I Starting from the last node, the insertion node can be foundby traversing a path of O(log n) nodes

Ý If the last node is a left child, return the parent nodeÝ While the current node is a right child, go to the parent nodeÝ If the current node is a left child, go to the right childÝ While the current node is internal, go to the left child

52

6

9 7 11 8

13 10 9 8 Last node Insertion node




Array-Based Heap Implementation

I We can represent a heap with n keys directly by means of anarray of length N > n

Ý By-pass the binary tree ADTÝ No explicit position ADT needed

I Idea similar to array-based implementation of binary trees

Ý The array cell at index 0 is unusedÝ The root of the heap is at index 1Ý The left child of the node at index i is at index 2iÝ The right child of the node at index i is at index 2i + 1

I The insertion operations corresponds to inserting at indexn + 1 and thus runs in O(1) time




Array-Based Heap: Example

52

6

9 7 11 8

13 10 9 8

52 9 7 11 8 13 10 9 861 2 3 4 5 6 7 8 9 10 11 12 13 140

Last node



Priority Queues and Heaps Heap-Based Priority Queues

Outline

Priority Queues

Heaps







I We can use a heap to implement a priority queue by storing apair item = (key , element) at each node of the heap

I Running times for different implementations

Unsorted SortedOperation Sequence Sequence Heapsize, isEmpty 1 1 1minElement, minKey n 1 1insertItem 1 n log nremoveMin n 1 log nPQ-Sort n2 n2 n· log n

I In the long run, the heap-based implementation beats anysequence-based implementation




UML Diagram: Composition

<<interface>>BinaryTree

etc.

<<interface>>Tree

etc.

ArrayBinaryTreeT: Arrayn: Integer

<<interface>>Heap

insertKey(k)removeMin()minKey()

BinaryTreeHeapT: ArrayBinaryTreeC: Comparator




isEmpty()size()

HeapPQH: ArrayHeapC: Comparator



ArrayHeapA: ArrayC: Comparator




Heap-Sort

I Heap-sort is the variation of PQ-sort where the priority queueis implemented with a heap

I Running time analysis:

Ý Inserting the elements into the priority queue with ninsertItem(e,e) operations runs in O(n log n) time

Ý Removing the elements in sorted order from the priority queuewith n removeMin() operations runs in O(n log n) time

I In general (worst case), heap-sort runs in O(n log n) time

I For large n, heap-sort is much faster than quadratic sortingalgorithms such as insertion-sort or selection-sort




In-Place Heap-Sort

I To implement Heap-Sort in-place, use the front of the originalarray to implement the heap

Ý Step 1: Use the reverse comparator (e.g. � instead of �) tobuild up the heap (by swapping elements)

Ý Step 2: Iteratively remove the maximum element from theheap and insert it in front of the sequence

4 7 2 5 3 4 7 2 5 3 7 4 2 5 3 7 4 2 5 3 7 5 2 4 3 7 5 2 4 34 7 7

4 4 27

5 24

75 2

4 3

7 5 2 4 3

75 2

4 35 4 2 3 7

54 2

34 3 2 5 7

43 2

3 2 4 5 7

32

2 3 4 5 72

2 3 4 5 7

Phase 1:

Phase 2:



Priority Queues and Heaps Bottom-Up Heap Construction

Outline

Priority Queues

Heaps






Bottom-up Heap Construction

I We can construct a heap storing n = 2h − 1 keys using arecursive bottom-up construction with h − 1 phases

I In phase i , pairs of heaps with 2i − 1 keys are merged intoheaps with 2i+1 − 1 keys

I Merging two heaps (and a new key k):

Ý Create a new heap with the root node storing k and with thetwo heaps as subtrees

Ý Perform downheap to restore the heap-order property

2i–1 2i–12i+1–1




Recursive Algorithm in Pseudo-Code

Algorithm bottomUpHeap(S) // sequence of keys of length 2ˆh−1if S . isEmpty() thenreturn new heap

elsek ← S . elemAtRank(0)S . removeAtRank(0)H1 ←bottomUpHeap(firstHalf(S)) // 1st recursive callH2 ←bottomUpHeap(secondHalf(S)) // 2nd recursive callreturn mergeHeaps(k,H1,H2) // merge the results

In the general case, i.e. if the sequence is not of size n = 2h − 1,we can adapt the splitting of the sequence accordingly




Example: Phase 1

16 15 4 12 6 9 23 20

25 5 11 27

16 15 4 12 6 9 23 20

15 4 6 20

16 25 5 12 11 9 23 27




Example: Phase 2

7 8

15 4 6 20

16 25 5 12 11 9 23 27

15 4 6 20

16 25 5 12 11 9 23 27

4 6

15 5 8 20

16 25 7 12 11 9 23 27




Example: Phase 3

4 6

15 5 8 20

16 25 7 12 11 9 23 27

410

6

15 5 8 20

16 25 7 12 11 9 23 27

54

6

15 7 8 20

16 25 10 12 11 9 23 27




Complexity Analysis

I The recursive BottomUpHeap algorithm decomposes aproblem of size n into two problems of size n−1

2

Ý Rule D2 with q = r = 2 (see Topic 2, Page 31)

I The running time f (n) of each recursive step is O(log n)

Ý with an array-based sequence implementation, firstHalf()and secondHalf() run in O(1) time

Ý the necessary downheap in mergeHeaps runs in O(log n) time

I Apply the second column of D2, i.e. bottom-up heapconstruction runs in O(n)

I Speeds up the heap construction (Phase 1) of the Heap-Sortalgorithm, but not Phase 2



algorithms and data structures - bfhhnr1/algdata/slides/05heaps.pdf · priority queues and heaps...

Documents