TreesData Structures and Algorithms (60-254)

2

Recursive Definition of a Tree

A tree T is either empty or it consists of a root and zero or more nonempty subtrees T1,T2,…Tk each of whose roots are connected by an edge from the root of T.

…

nodes, verticesarcs, edges

3

The terminology of trees

B

HGF

EDC

K

JI

A: Root of the treeB,C,D,E: Children of the rootB,C,D,E: SibblingsE is an ancestor of KK is a descendant of EF,G,C,H,I,K are the leaves of the treeThe depth of a node is the length of the path from the root to the nodeLength of path = # of edgesDepth of E =1Height of a node = length of the longest path from that node to a leafHeight of E = 2Height of tree = height of rootHeight of example tree is 3

A

4

Applications

Trees are a ubiquitous data structure in Computer Science.The Unix directory structure is a tree.Allows user to store files in a logical/organized fashion. Issues:

How to implement tree structureHow to insert and delete nodes in a tree structureHow to find nodes in a tree structureHow to traverse the entire tree structureHow to organize structure for efficiency

These issues will be at the core of much of the discussion of trees.

5

Binary Trees

A

B

ED

C

H

GF

Root

TRightTLeft

A binary tree T is either empty or it consists of a root and two (possibly empty) binary subtrees TLeft and TRight each of whose roots are connected by an edge from the root of T.

If the left (right) binary subtree is not empty, its root is called the left (right) child of the root of the tree.The degree of a node is the number of children it has.

6

Problem

Show by induction that the number of degree 2 nodes (nT2) in any binary tree is 1 less than the number of leaves (nT0).Proof: The induction is on the number of nodes n in the binary tree. For n = 1, there is only the root node with 0 children. Hence, 1 leaf node and 0 nodes of degree 2.Suppose the claim is true for all binary trees with n or less nodes. Consider a binary tree with n+1 nodes. It has one of the following structures:

Let nL and nR be the number of nodes in the left and right subtrees respectively.

(i) (ii) (iii)

7

Proof - continued

Since nL £ n and nR £ n, the claim is true for the left subtree in case (i) and the right subtree in case (ii). Since in both these cases the degree 2 nodes and the leaves belong to these subtrees alone, the claim holds for the entire tree.In case (iii), the claim is true for both subtrees. If we add the root to the count of degree 2 nodes of these trees, the claim follows.nL2 = nL0 -1 nR2 = nR0 -1 nT2 = nL2+nR2+1 (root is degree 2)

= nL0 - 1 + nR0 - 1 + 1 = nL0 +nR0 - 1 = nT0 – 1

Inductive Hypothesis

Inductive Step

8

Exercise

Show by induction that a binary tree with n nodes has height at least ëlog nû.

9

Operations that we would like to do on a binary tree

Find its heightFind its size (number of nodes)Visit its nodes in a systematic wayEtc.

10

Height of a binary tree

Height(T) = 1 + max(Height(TL),Height(TR))Height(Æ) = -1 Write a recursive function based on the above definition. height(T)

if (T is empty) return –1 else return 1 + max(height(T.left),height(T.right))

11

Size of a binary tree

Size(Æ) = 0

Size(T) = 1 + Size(TL) + Size(TR) Write a recursive function based on the above definition. size(T)

if (T is empty) return 0else return 1 + size(T.left) + size(T.right)

12

Tree traversals

Many problems require that we visit the nodes of a binary tree structure in a systematic way. Three such ways are:

1. Preorder traversal2. Postorder traversal3. Inorder traveral

13

Preorder traversal

1. Visit the root2. Visit the left subtree in preorder3. Visit the right subtree in preorder

For example, printing the labels of the nodes in the given tree using a preorder traversal gives: A,B,D,E,C,F,G,H

14

Postorder traversal

1. Visit the left subtree in postorder2. Visit the right subtree in postorder3. Visit the root

For example, printing the labels of the nodes in the given tree using a postorder traversal gives: D,E,B,F,H,G,C,A

15

Inorder traversal

1. Visit the left subtree in inorder2. Visit the root3. Visit the right subtree in inorder

For example, printing the labels of the nodes in the given tree using an inorder traversal gives: D,B,E,A,F,C,H,G

16

Printing

printPreorder(T) print T.data // visit root if (T.left is not empty) printPreorder(T.left) // visit left if (T.right is not empty) printPreorder(T.right) // visit right printPostorder(T) if (T.left is not empty) printPostorder(T.left) // visit left if (T.right is not empty) printPostorder(T.right) // visit right print T.data // visit root printInorder(T) if (T.left is not empty) printInorder(T.left) // visit left print T.data // visit root if (T.right is not empty) printInorder(T.right) // visit right

17

Heapsort

We can define another sorting algorithm that uses a binary tree.Definition:An (essentially) complete binary tree is a binary tree that is complete at all levels except possibly the last.Examples:

Definition: A heap is an essentially complete binary tree with the additional property: The key stored at any node is the keys stored at its children, if any.

18

Replacing by

Example:

Array Representation of a heap:

Key values placed in array level-by-level and left to right.Children of node at index i placed at indices 2i+1 and 2i+2.Given, an array of n elements, how can we heapify them?

16 11 9 10 5 6 8 1 2 4 0 1 2 3 4 5 6 7 8 9

19

Algorithm Heapify

Input: Indices i and j. An array A[0..n-1] of key values.

Output: A[i] sinks to appropriate place in A[i..j]

Procedure Heapify(i, j)

if A[i] is a leaf node then return

Set A[k] ¬ max{A[2i+1],A[2i+2]} // k is index of max.

where 2i+1 and 2i+2 j

if (A[i] < A[k]) then Exchange(A[i],A[k])

Heapify(k, j)

20

Example

12 10 20 19 8 7 15 0 1 2 3 4 5 6

10 20

19 8 7 15

Heapify(1,6): causes 10 to sink

12 19 20 10 8 7 15 0 1 2 3 4 5 6

19 20

10 8 7 15

Heapify(0,6): causes 12 to sink

20 19 12 10 8 7 15 0 1 2 3 4 5 6

19 12

10 8 7 15

21

Example

Heapify(2,6): causes 12 to continue sinking

20 19 15 10 8 7 12 0 1 2 3 4 5 6

19 15

10 8 7 12

22

Heapify and BuildHeap

Heapify(0,n-1) is O(log n) Heapify(i,n-1) only fixes element i

To build the entire heap, we must apply heapify to all of the elements in turn.

23

Algorithm BuildHeap

Input: An array A[0..n-1] of key values.Output: An array A[0..n-1] that satisfies the heap property.Procedure BuildHeap(A)

for i n div 2 - 1 downto 0Heapify(i, n-1)

Initial array:

12 10 20 19 8 7 15 0 1 2 3 4 5 6

10 20

19 8 7 15

i: 7 div 2 – 1 = 2BuildHeap -Heapify(2,6): no

change

24

BuildHeap Example12 10 20 19 8 7 15 0 1 2 3 4 5 6

10 20

19 8 7 15 Heapify(1,6): causes 10 to

sink 12 19 20 10 8 7 15 0 1 2 3 4 5 6

19 20

10 8 7 15 Heapify(0,6): causes 12 to sink

20 19 12 10 8 7 15 0 1 2 3 4 5 6

19 12

10 8 7 15

Heapify(2,6): causes 12 to continue sinking

20 19 15 10 8 7 12 0 1 2 3 4 5 6

19 15

10 8 7 12

25

Algorithm Heapsort

Input: An array A[0..n-1] of key-values.Output: A sorted array A[0..n-1] of key-values. BuildHeap(A)for i n-1 downto 1

Exchange(A[0],A[i])Heapify(0,i-1)

26

Example

27

Example Continued

28

29

30

31

Complexity of Heapsort

BuildHeap is O(n) : True but difficult to prove!!

Heapify is O(log n)

Heapify is called n times in heapsort.

Therefore, Heapsort is O(n log n)

32

Binary Search Trees

Definition:A Binary Search Tree (BST) is a binary tree that satisfiesthe property that for every node x in the BST,• x.Key > every key in x.Left (left subtree), and• x.Key < every key in x.Right (right subtree).A node of a binary search tree may be represented with 5 fields:

For simplicity, we usually omit the data field in examples.Also, the parent field can often be omitted depending on theintended application and/or the particular implementation.

33

An example of a BST

BST property ⇒Inorder traversal of a BST visits the nodes in sorted order:2,3,4,5,7,8 (ascending order)

34

1. Searching for a node with a given key, k:

TreeSearch(x,k) // x is the node where search startsif (x == NULL or k == x.Key) return xelse if (k < x.Key) return TreeSearch(x.Left,k)else return TreeSearch(x.Right,k)

35

2. Minimum and Maximum keys

TreeMin(x)while (x.Left != NULL)

x = x.Leftreturn x;

TreeMax(x)while (x.Right != NULL)

x = x.Rightreturn x;

36

3. Successor of a key

Definition: The successor of a node x is the node with the smallest key greater than x.Key

TreeSuccessor(x)if (x.Right != NULL)

return TreeMin(x.Right)else

p = x.Parentwhile (p != NULL and x == p.Right)

x = pp = x.Parent

return p

37

Example: Successor of 13?

Note that 13 is the maximum key in the BST rooted at 6

38

4. Predecessor of a key

Definition: The predecessor of a node x is the node with thelargest key smaller than x.Key.

TreePredecessor(x)if (x.Left != NULL)

return TreeMax(x.Left)else

p = x.Parentwhile (p != NULL and x == p.Left)

x = pp = x.Parent

return p

39

Example: Predecessor of 13?

Note that 13 is the minimum key in the BST rooted at 16

40

Insertion

Node TreeInsert(x, k)if (x == NULL)

Create new node t with key kelse if (k < x.Key)

x.Left ← TreeInsert(x.Left, k)else if (k > x.Key)

x.Right ← TreeInsert(x.Right, k)else

Throw exception “Existing key!”return t

41

Example: Insert key 13 into existing BST

42

Deletion

Assumption: The node to be deleted is given.Case 1: Node has no childrenSimply remove the node.Example: Delete 13

43

Deletion

Case 2: Node has one childReplace node with its child.Example: Delete 13

44

Deletion

Case 3: Node has two childrenReplace node with minimum key min right subtree and recursivelydelete m in right subtree.Example: Delete 13

45

Balanced Search Trees

Definition:

Binary Search Trees with a worst-case height of O(log n) are called balanced trees

We can guarantee O(log n) performance for each search tree operation for balanced trees.

We will discuss two kinds of balanced trees(a) AVL trees(b) Red-black trees

46

AVL Trees

Named after its discoverers:Adelson-Velskii and Landis, 1962

Definition: An empty binary tree is an AVL tree. If T is anon-empty binary tree with TL and TR as its leftand right sub-trees, then T is an AVL tree iff(a) TL and TR are AVL trees and(b) | hL − hR | ≤ 1 where hL and hR are the heights ofTL and TR respectively.

47

AVL search tree

Definition: An AVL search tree is a binary search tree that is also an AVL treeHenceforth, by an AVL tree we shall mean an AVL search tree.

balance = height of left subtree – height of right subtree

Node of an AVL tree:

48

Examples of AVL search treesBalance factor = left balance– right balance

49

Height of an AVL tree

Let Nh be the minimum number of nodes in an AVL tree of height h.

Then Nh = Nh−1+ Nh−2 +1, N0=1, N1=2assuming the worst-case scenario in which one sub-tree hasheight h-1 and the other h-2The solution to the recurrence given above is:

Nh = Fh+3 -1 ,

where Fn is the nth Fibonacci number

50

51

Insertion into an AVL tree

Inserting key 32 causes the tree tobecome unbalanced

52

Imbalance due to:Insertion into left sub-tree of left child of a node

53

Balanced by LL-rotation

54

Tree balanced by LL rotation

55

Pseudocode: LL rotation

Input: Binary Node k2Procedure withLeftChild( k2)

Set k1 ← k2.Leftif (k2 is Left child)

k2.Parent.Left ← k1else

k2.Parent.Right ← k1k2.Left ← k1.Rightk1.Right ← k2// Also, update Parent fields!return k1

56

Imbalance due toInsertion into right sub-tree of right child of a node

57

Balanced by RR-rotation

58

Example of RR rotation: Insert 45Tree unbalanced after insertion

59

After the RR rotation:

60

Pseudocode: RR rotation

Input: Binary Node k1Procedure withRightChild( k1)

Set k2 ← k1.Rightif (k1 is Left child)

k1.Parent.Left ← k2else

k2.Parent.Right ← k2k1.Right ← k2.Leftk2.Left ← k1// Update Parent fields too!return k2

61

Imbalance caused byInsertion into right sub-tree of the left child of a node

62

Balanced by LR-rotation

63

Example of LR rotation:

64

Remark: An LR rotation is often called a double rotation because it can be viewed as an RR rotation, followed by an LL rotation

On the example:After the RR-rotation

65

After the LL-rotation

66

Imbalance caused byInsertion into left sub-tree of right child

Balance is restored by an RL rotation

Details are the same as in the case of a LR rotation

67

An example of RL rotation:

68

After the RL rotation:

69

70

Pseudocode: LR rotation

Input: Binary Node k3Procedure doubleWithLeftChild( k3)

Set k3.Left ← withRightChild(k3.Left)return withLeftChild(k3)

71

Pseudocode: RL rotation

Input: Binary Node k3Procedure doubleWithRightChild( k3)

Set k3.Right ← withLeftChild(k3.Right)return withRightChild(k3)

72

Balance:

73

Balance:

74

75

Deletion from an AVL tree

We delete a node just as in a binary search tree• Let q be the parent of the physically deleted node.• We might need to go all the way from q to the root in order to balance the tree and restore AVL-balance

76

Three cases arise:

C1: Balance at q has changed to 0 height has decreased⇒by 1 (of the tree rooted at q) and we need to check thebalance factor of the ancestors of q.

C2: Balance at q has changed to +1 or –1 height of subtrees⇒at q is unchanged, so the balances of all theancestors of q remain unchanged

C3: Balance at q is ±2 tree needs to be balanced at ⇒ q

77

Three cases: example

Case C1, delete rootCase C2, delete node circled in greenCase C3, delete node circled in blue

78

Types of Imbalance

Let A be the first node on the path from q to the root where the balance has changed to ±2

The imbalance is of type

L if deletion has taken from the left sub-tree of AR if deletion has taken from the right sub-tree of A

We discuss the R-type imbalance

79

80

81

82

83

84

85

86

A Red-Black Tree

87

88

89

Important conclusion

For a red-black tree with n nodes

90

Insertion into a red-black tree (Method 1)Insert node as a leafRestore properties P1-P4, bottom-upOK?ObservationsInserted node must be colored redRed-black property P3 is violated if it has a red parent6 cases. We’ll study 3 of them:

Inserted node is in the “left” subtree of G.

91

Case 1: Inserted node has a red parent, which has a black siblingSub-case 1.1: X is an outer child

Single rotation restoration (LL)

92

Sub-case 1.2: X is an inner child

Double rotation restoration (LR)

93

Case 2: Inserted node has a red parent, which has a red sibling.

Single rotation restoration: LL

94

More on Case 2

This case is problematic because:P could have a red parent, andwe need to propagate the restoration up to the root

95

96

97

98

99

Top-down insertion

Rule: Walk down the tree (starting from root).If we hit a node X with (two) red children,then flip the color of X and its children.

Claim. If the parent of X is red,Then X’s parent’s sibling cannot be red, andthe single rotation (LL) orthe double rotation (LR) can be applied.

100

101

102

103

Deletion from a red-black tree

Deletion is as the case of a binary search tree, but we need to guarantee that the color of the physically deleted node is red

Start from the root…

As we search down the tree,we maintain the invariant that

the color of the current node X is red

104

105

Let X be the current nodeLet P be its parentLet T be its siblingAlways ensure that, P is red

⇒ both X and T are black … why?

Two cases:Case 1: X has 2 black children

106

Case 1.1: T has 2 black children

107

Case 1.2: T has an outer child that is red

108

Case 1.3: T has an inner red child

109

Case 2: X has a red child

110

Deletion from a red-black tree

Note that the rotation is the main case when X has two black children, always when X red.

When X has a red child we have to move one level down on the search path (whether the child along the search path is red, the rotation is both).

Note that in Case 1.2, if both children of T are red we can still apply the case.