1

CSC 421: Algorithm Design & Analysis

Spring 2014

Divide & conquer divide-and-conquer approach familiar examples: merge sort, quick sort other examples: closest points, large integer multiplication tree operations: binary trees, BST

2

Divide & conquer

probably the best-known problem-solving paradigm1. divide the problem into several subproblems of the same type (ideally of

about equal size)2. solve the subproblems, typically using recursion3. combine the solutions to the subproblems to obtain a solution to the original

problem

Not always a win

some problems can be thought of as divide & conquer, but are not efficient to implement that way

e.g., counting the number of 0 elements in a list of numbers:1. recursively count the number of 0's in the 1st half of the list2. recursively count the number of 0's in the 2nd half of the list3. add the two counts

3

Cost(N) = 2 Cost(N/2) + C= 2 [2 Cost(N/4) +

C] + C= 4 Cost(N/4) + 3C= 4 [2 Cost(N/8) +

C] + 3C= 8 Cost(N/8) + 7C= . . .= N Cost(N/N) +

(N-1)C= NC' + (N-1)C= (C + C')N - C

O(N)

the overhead of recursion makes this much slower than a simple iteration (i.e., decrease & conquer)

Master Theorem

Suppose Cost(N) = a Cost(N/b) + C nd. that is, divide & conquer is performed with polynomial-time overhead

O(Nd) if a < bd

Cost(N) = O(Nd log N) if a = bd

O(N logb a) if a > bd

4

e.g., zero count algorithm has Cost(N) = 2Cost(N/2) + C

a = 2, b = 2, d = 0 2 > 20 O(N log2 2) O(N)

Merge sort

we have seen divide-and-conquer algorithms that are more efficient than brute force

e.g., merge sort list[0..N-1]

1. if list N <= 1, then DONE2. otherwise,

a) merge sort list[0..N/2]b) merge sort list[N/2+1..N-1]c) merge the two sorted halves

recall: Cost(N) = 2Cost(N/2) +CN merging is O(N), but requires O(N) additional storage and copying

we can show this is O(N log N) by unwinding, or a = 2, b = 2, d = 1 2 = 21 O(N log N) by Master Theorem

5

Why is halving most common?

consider a variation of merge sort that divides the list into 3 parts

1. if list N <= 1, then DONE2. otherwise,

a) merge sort list[0..N/3]b) merge sort list[N/3+1..2N/3]c) merge sort list[2N/3+1..N-1]d) merge the three sorted parts

Cost(N) = 3Cost(N/3) + CN

a = 3, b = 3, d = 1 3 = 31 O(N log N)

in general, X Cost(N/X) + CN O(N log N)X Cost(N/X) + C O(N)

dividing into halves is simplest, and just as efficient as thirds, quarters, …6

Quick sort

Collections.sort implements quick sort, another O(N log N) sort which is faster in practice

e.g., quick sort list[0..N-1]

1. if list N <= 1, then DONE2. otherwise,

a) select a pivot element (e.g., list[0], list[N/2], list[random], …)b) partition list into [items < pivot] + [items == pivot] + [items > pivot]c) quick sort the < and > partitions

best case: pivot is medianCost(N) = 2Cost(N/2) +CN O(N log N)

worst case: pivot is smallest or largest valueCost(N) = Cost(N-1) +CN O(N2)

7

Quick sort (cont.)

average case: O(N log N)

there are variations that make the worst case even more unlikely switch to selection sort when small median-of-three partitioning

instead of just taking the first item (or a random item) as pivot, take the median of the first, middle, and last items in the list

if the list is partially sorted, the middle element will be close to the overall median

if the list is random, then the odds of selecting an item near the median is improved

refinements like these can improve runtime by 20-25%

however, O(N2) degradation is still possible8

9

Closest pair

given a set of N points, find the pair with minimum distance

brute force approach: consider every pair of points, compare distances & take minimum

big Oh?

there exists an O(N log N) divide-and-conquer solution

Assume the points are sorted by x-coordinate (can be done in O(N log N))1. partition the points into equal parts using a vertical line in the plane2. recursively determine the closest pair on left side (Ldist) and the

closest pair on the right side (Rdist)3. find closest pair that straddles the line (Cdist), each within

min(Ldist,Rdist) of the line (can be done in O(N))4. answer = min(Ldist, Rdist, Cdist)

O(N2)

Cost(N) = 2 Cost(N/2) + CN O(N log N)

Multiplying large integers

many CS applications, e.g., cryptography, involve manipulating large integers

long multiplication: 123456 × 213121

123456 + 2469120 + 12345600 + 370368000 + 1234560000 + 24691200000 26311066176

long multiplication of two N-digit integers requires N2 digit multiplications

10

Divide & conquer multiplication

can solve (a x b) by splitting the numbers in half & factoringconsider a1 = 1st N/2 digits of a, a0 = 2nd N/2 digits of a (similarly for b1 and b0)

(a x b) = (a110N/2 + a0) x (b110N/2 + b0)= (a1 x b1)10N + (a1 x b0 + a0 x b1)10N/2 + (a0 x b0)= c210N + c110N/2 + c0

wherec2 = a1 x b1c0 = a0 x b0c1 = (a1 + a0) x (b1 + b0) – (c2 + c0)

11

EXAMPLE: a = 123456 b = 213121 c2 = a1 x b1 = 123 x 213 = 26199 c0 = a0 x b0 = 456 x 121 = 55176 c1 = (a1 + a0) x (b1 + b0) – (c2 + c0) = (123 + 456) x (213 + 121) – (26199 + 55176)

= 579 x 334 – 81375 = 193386 – 81375 = 112011(a x b) = c210N + c110N/2 + c0 = 26199000000 + 55176000 + 112011 = 26311066176

Efficiency of d&c multiplication

in order to multiply N-digit numbers calculate c2, c1, and c0, each of which involves multiplying N/2-digit numbers

MultCost(N) = 3 MultCost(N/2) + C

from Master Theorem: a = 3, b = 2, d = 0 3 > 20 O(Nlog2 3) ≈ O(N1.585)

12

worry: in minimizing the number of multiplications, we have increased the number of additions & subtractions a similar analysis can show that AddCost(N) is also O(Nlog

2 3)

does O(N log2

3) really make a difference vs. O(N2) ? has been shown to improve performance for as small as N = 8 for N = 300, can run more than twice as fast

13

Dividing & conquering trees

since trees are recursive structures, most tree traversal and manipulation operations can be classified as divide & conquer algorithms can divide a tree into root + left subtree + right subtree most tree operations handle the root as a special case, then recursively process

the subtrees

e.g., to display all the values in a (nonempty) binary tree, divide into 1. displaying the root2. (recursively) displaying all the values in the left subtree3. (recursively) displaying all the values in the right subtree

e.g., to count number of nodes in a (nonempty) binary tree, divide into1. (recursively) counting the nodes in the left subtree2. (recursively) counting the nodes in the right subtree3. adding the two counts + 1 for the root

BinaryTree class

public class BinaryTree<E> {

protected TreeNode<E> root;

public BinaryTree() { this.root = null; }

public void add(E value) { … }

public boolean remove(E value) { … }

public boolean contains(E value) { … }

public int size() { … }

public String toString() { … }

}

14

to implement a binary tree, need to store the root node the root field is "protected"

instead of "private" to allow for inheritance

the empty tree has a null root

then, must implement methods for basic operations on the collection

15

size method

public int size() { return this.size(this.root); }

private int size(TreeNode<E> current) { if (current == null) { return 0; } else { return this.size(current.getLeft()) + this.size(current.getRight()) + 1; } }

divide-and-conquer approach:BASE CASE: if the tree is empty, number of nodes is 0

RECURSIVE: otherwise, number of nodes is (# nodes in left subtree) + (# nodes in right subtree) + 1 for the root

note: a recursive implementation requires passing the root as parameter will have a public "front" method, which calls the recursive "worker" method

16

contains method

public boolean contains(E value) { return this.contains(this.root, value); }

private boolean contains(TreeNode<E> current, E value) { if (current == null) { return false; } else { return value.equals(current.getData()) || this.contains(current.getLeft(), value) || this.contains(current.getRight(), value); } }

divide-and-conquer approach:BASE CASE: if the tree is empty, the item is not foundBASE CASE: otherwise, if the item is at the root, then found

RECURSIVE: otherwise, search the left and then right subtrees

17

toString method

public String toString() { if (this.root == null) { return "[]"; } String recStr = this.toString(this.root); return "[" + recStr.substring(0,recStr.length()-1) + "]"; }

private String toString(TreeNode<E> current) { if (current == null) { return ""; } return this.toString(current.getLeft()) + current.getData().toString() + "," + this.toString(current.getRight()); }

must traverse the entire tree and build a string of the items there are numerous patterns that can be used, e.g., in-order traversal

BASE CASE: if the tree is empty, then nothing to traverse

RECURSIVE: recursively traverse the left subtree, then access the root,then recursively traverse the right subtree

Other tree operations?

add?

remove?

numOccur?

height?

weight?

18

19

Binary search trees

a binary search tree is a binary tree in which, for every node: the item stored at the node is ≥ all items stored in its left subtree the item stored at the node is < all items stored in its right subtree

are BST operations divide & conquer?

• contains?

• add?

• remove?

what about balanced BSTs? (e.g., red-black trees, AVL trees)