242-535 ada: 4. divide/conquer1 objective o look at several divide and conquer examples (merge sort,...

242-535 ADA: 4. Divide/Conquer1 Objective o look at several divide and conquer examples (merge sort, binary search), and 3 approaches for calculating their running time Algorithm Design and Analysis (ADA) 242-535, Semester 1 2014-2015 4. Divide and Conquer

242-535 ADA: 4. Divide/Conquer2 1.Divide and Conquer 2.A Faster Sort: merge sort 3.The Iteration Method 4.Recursion Trees 5.Merge Sort vs Insertion Sort 6.Binary Search 7.Recursion Tree Examples 8.Iteration Method Examples 9.The Master MethodOverview

242-535 ADA: 4. Divide/Conquer3 1.Divide the problem into subproblems 2.Conquer the subproblems by solving them recursively 3.Combine subproblem solutions. 1. Divide and Conquer

2. A Faster Sort: Merge Sort M ERGE S ORT( A, left, right) 1.If left < right, // if left right, do nothing 2. mid := floor(left+right)/2) 3. MergeSort(A, left, mid) 4. MergeSort( A, mid+1,right) 5. Merge(A, left, mid, right) 6.return Initial call: M ERGE S ORT (A, 1, n)

242-535 ADA: 4. Divide/Conquer5 A faster sort: MergeSort A[1.. mid ] A[ mid +1.. n] input A[1.. n] M ERGE S ORT Sorted A[1.. mid ] Sorted A[ mid +1.. n] M ERGE output

242-535 ADA: 4. Divide/Conquer6 merge Tracing MergeSort()

20 13 7 2 12 11 9 1 Merging two sorted arrays The merge() function 2 7 13 201 9 11 12 1 2 7 9 11 12 13 20

1 20 13 7 2 12 11 9 Merging two sorted arrays

1 20 13 7 2 12 11 9 20 13 7 2 12 11 9 Merging two sorted arrays

20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7 20 13 12 11 9

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7 20 13 12 11 9 9

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7 20 13 12 11 9 9 20 13 12 11

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7 20 13 12 11 9 9 20 13 12 11 20 13 12

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7 20 13 12 11 9 9 20 13 12 11 20 13 12 1 2 7 9 11 12 13 20

Merging two sorted arrays 20 13 7 2 12 11 9 1 1 20 13 7 2 12 11 9 2 20 13 7 12 11 9 7 20 13 12 11 9 9 20 13 12 11 20 13 12 Time = one pass through each array = (n) to merge a total of n elements (linear time).

242-535 ADA: 4. Divide/Conquer20 StatementEffort Analysis of Merge Sort MergeSort(A, left, right) T(n) if (left < right) {O(1) mid = floor((left+right)/2); O(1) MergeSort(A, left, mid);T(n/2) MergeSort(A, mid+1, right);T(n/2) Merge(A, left, mid, right);O(n) } As shown on the previous slides mid) temp[i] = A[bIdx++]; // copy 2nd range else if (bIdx > right) temp[i] = A[aIdx++]; // copy 1st range else if (a[aIdx]

242-535 ADA: 4. Divide/Conquer24 Recursive T() equation: o T(1) = O(1) o T(n) = 2T(n/2) + O(n), for n > 1 Convert to algebra o T(1) = a o T(n) = 2T(n/2) + cn MergeSort Running Time

242-535 ADA: 4. Divide/Conquer25 The expression: is called a recurrence. A recurrence is an equation that describes a function in terms of its value for smaller function calls. Recurrence for Merge Sort

242-535 ADA: 4. Divide/Conquer26 T(n) = 2T(n/2) + cn 2(2T(n/2/2) + cn/2) + cn 2 2 T(n/2 2 ) + cn2/2 + cn 2 2 T(n/2 2 ) + cn(2/2 + 1) 2 2 (2T(n/2 2 /b) + cn/2 2 ) + cn(2/2 + 1) 2 3 T(n/2 3 ) + cn(2 2 /2 2 ) + cn(2/2 + 1) 2 3 T(n/2 3 ) + cn(2 2 /2 2 +2/2 + 1) 2 k T(n/2 k ) + cn(2 k-1 /2 k-1 + 2 k-2 /2 k-2 + + 2 2 /2 2 + 2/2 + 1)

242-535 ADA: 4. Divide/Conquer27 So we have o T(n) = 2 k T( n/2 k ) + cn(2 k-1 /2 k-1 +... + 2 2 /2 2 + 2/2 + 1) For k = log 2 n o n = 2 k, so T() argument becomes 1 o T(n)= 2 k T( 1 ) + cn(k-1+1) = na + cn(log 2 n) = O(n) + O(n log 2 n) = O(n log 2 n) k-1 of these

242-535 ADA: 4. Divide/Conquer28 A graphical technique for finding a big-oh solution to a recurrence o Draw a tree of recursive function calls o Each tree node gets assigned the big-oh work done during its call to the function. o The big-oh equation is the sum of work at all the nodes in the tree. 4. Recursion Trees

MergeSort Recursion Tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. We usually omit stating the base case because our algorithms always run in time (1) when n is a small constant.

MergeSort Recursion Tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. T(n)T(n)

MergeSort Recursion Tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. T(n/2) cn

MergeSort Recursion Tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn T(n/4) cn/2

MergeSort Recursion Tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. cn cn/4 cn/2 (1) h = log n cn #leaves = n (n)(n) Total (n log n)

242-535 ADA: 4. Divide/Conquer34 Height and no. of Leaves h steps why?

242-535 ADA: 4. Divide/Conquer35 Logarithm Equalities because of this

242-535 ADA: 4. Divide/Conquer36 O(n lg n) grows more slowly than O( n 2 ). In other words, merge sort is asymptotically faster (runs faster) than insertion sort in the worst case. In practice, merge sort beats insertion sort for n > 30 or so. 5. Merge Sort vs Insertion Sort

242-535 ADA: 4. Divide/Conquer37 Running time estimates: o Laptop executes 10 8 compares/second. o Supercomputer executes 10 12 compares/second. Timing Comparisons Lesson 1. Good algorithms are better than supercomputers.

242-535 ADA: 4. Divide/Conquer38 Binary Search from part 3 is a divide and conquer algorithm. Find an element in a sorted array: 1.Divide: Check middle element. 2.Conquer: Recursively search 1 subarray. 3.Combine: Easy; return index 6. Binary Search Example: Find 9 357891215

Binary Search Example: Find 9 357891215 Find an element in a sorted array: 1.Divide: Check middle element. 2.Conquer: Recursively search 1 subarray. 3.Combine: Trivial.

Binary Search Find an element in a sorted array: 1.Divide: Check middle element. 2.Conquer: Recursively search 1 subarray. 3.Combine: Trivial. Example: Find 9 357891215

242-535 ADA: 4. Divide/Conquer44 Binary Search Code (again) int binSrch(char A[], int i,int j, char key) { int k; if (i > j) /* key not found */ return -1; k = (i+j)/2; if (key == A[k]) /* key found */ return k; if (key < A[k]) j = k-1; /* search left half */ else i = k+1; /* search right half */ return binSrch(A, i, j, key); }

242-535 ADA: 4. Divide/Conquer45 Using big-oh. o Basis: T(1) = O(1) o Induction: T(n) = O(1) + T( ), for n > 1 As algebra o Basis: T(1) = a o Induction: T(n) = c + T( ), for n > 1 Running time for binary search is O(log 2 n). Running Time (again) n == the range of the array being looked at

Recurrence for Binary Search T(n) = 1 T(n/2) + (1) # subproblems subproblem size work dividing and combining

BS Recursion tree Solve T(n) = T(n/2) + c, where c > 0 is constant. We usually don't bother with the base case because our algorithms always run in time (1) when n is a small constant.

BS Recursion tree Solve T(n) = T(n/2) + c, where c > 0 is constant. T(n)T(n)

BS Recursion tree Solve T(n) = T(n/2) + c, where c > 0 is constant. c T(n/2)

BS Recursion tree Solve T(n) = T(n/2) + c, where c > 0 is constant. c c T(n/4)

BS Recursion tree Solve T(n) = 2T(n/2) + cn, where c > 0 is constant. (1) h = log 2 n c c c a Total = c log 2 n + a = O(log 2 n) c c c

242-535 ADA: 4. Divide/Conquer52 Merge Sort T(n) = 2T(n/2) + (n) = (n log n) Binary Search T(n) = T(n/2) + (1) = (log n) The big-oh running times were calculated in two ways: the iteration method and using recursion trees. Let's do some more example of both. Two recurrences so far

7. Recursion Tree Examples 1 Solve T(n) = T(n/4) + T(n/2) + n 2 :

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : T(n)

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : n2n2 T(n/4) T(n/2)

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : n2n2 (n/4) 2 (n/2) 2 T(n/16)T(n/8) T(n/4)

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : n2n2 (n/4) 2 (n/2) 2 (n/16) 2 (n/8) 2 (n/4) 2 O (1)

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : n2n2 (n/4) 2 (n/16) 2 (n/8) 2 O (1) 2 n (n/2) 2 (n/8) 2 (n/4) 2

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/4) 2 (n/16) 2 (n/8) 2 O (1) n2n2 (n/2) 2 (n/8) 2 (n/4) 2 5 16 2 n 2 n

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/4) 2 (n/16) 2 (n/8) 2 O (1) n2n2 (n/2) 2 (n/8) 2 (n/4) 2 5 16 25 256 2 n 2 n 2 n

Example 1 Solve T(n) = T(n/4) + T(n/2) + n 2 : (n/4) 2 (n/16) 2 (n/8) 2 n2n2 (n/2) 2 (n/8) 2 (n/4) 2 5 16 25 256 2 n 2 n 2 n O (1) 2 Total = n ( 1 + = O(n 2 ) 5 16 2 3 5 + ( ) + ( ) + ) 16 geometric series = 16/11*n 2

Geometric Series Reminder for |x| < 1 for x 1

242-535 ADA: 4. Divide/Conquer63 T(n) = 3T(n/4) + cn 2 Recursion Tree 2

242-535 ADA: 4. Divide/Conquer64 T(n) = 3T(n/4) + cn 2

242-535 ADA: 4. Divide/Conquer65 height (h) = no. of leaves = T(n) = 3T(n/4) + cn 2

242-535 ADA: 4. Divide/Conquer67 Add the cost of all the levels: Cost of the Tree leaves level leaves level next to bottom level next to bottom level

242-535 ADA: 4. Divide/Conquer68 T(n) = T(n/3) + T(2n/3) + cn Recursion Tree 3 height =

242-535 ADA: 4. Divide/Conquer69 Height and no. of Leaves h steps for the longest path why?

242-535 ADA: 4. Divide/Conquer70 Since the tree is smaller than a complete binary tree, then the cost of all the level will be: T(n) cn * log 3/2 n T(n) is O(n log 3/2 n) is O(n log n) Since log 3/2 n = log 2 n / log 2 3/2 = c log 2 n // see slide 35 Cost of the Tree

242-535 ADA: 4. Divide/Conquer71 8. Iteration Method Examples 1 1 2 2 3 3

242-535 ADA: 4. Divide/Conquer72 T(n) = c + T(n-1) = c + c + T(n-2) = 2c + T(n-2) = 2c + c + T(n-3) = 3c + T(n-3) = kc + T(n-k) = ck + T(n-k) Example 1

242-535 ADA: 4. Divide/Conquer73 When k == n o T(n) = cn + T(0) = cn The conversion back to big-oh: o T(n) is O(n)

242-535 ADA: 4. Divide/Conquer74 T(n) =n + T(n-1) =n + n-1 + T(n-2) =n + n-1 + n-2 + T(n-3) =n + n-1 + n-2 + n-3 + T(n-4) == = n + n-1 + n-2 + n-3 + + n-(k-1) + T(n-k) Example 2

242-535 ADA: 4. Divide/Conquer75 T(n) =n + T(n-1) =n + n-1 + T(n-2) =n + n-1 + n-2 + T(n-3) =n + n-1 + n-2 + n-3 + T(n-4) = = n + n-1 + n-2 + n-3 + + n-(k-1) + T(n-k) =

242-535 ADA: 4. Divide/Conquer76 When k = n, T(n) = In general, T(n) is O(n 2 )

242-535 ADA: 4. Divide/Conquer77 T(n) = aT(n/b) + cn a(aT(n/b/b) + cn/b) + cn a 2 T(n/b 2 ) + cna/b + cn a 2 T(n/b 2 ) + cn(a/b + 1) a 2 (aT(n/b 2 /b) + cn/b 2 ) + cn(a/b + 1) a 3 T(n/b 3 ) + cn(a 2 /b 2 ) + cn(a/b + 1) a 3 T(n/b 3 ) + cn(a 2 /b 2 + a/b + 1) a k T(n/b k ) + cn(a k-1 /b k-1 + a k-2 /b k-2 + + a 2 /b 2 + a/b + 1) Example 3

242-535 ADA: 4. Divide/Conquer78 So we have o T(n) = a k T(n/b k ) + cn(a k-1 /b k-1 +... + a 2 /b 2 + a/b + 1) For k = log b n o n = b k o T(n)= a k T(1) + cn(a k-1 /b k-1 +... + a 2 /b 2 + a/b + 1) = a k d + cn(a k-1 /b k-1 +... + a 2 /b 2 + a/b + 1) ~= ca k + cn(a k-1 /b k-1 +... + a 2 /b 2 + a/b + 1) = cna k /b k + cn(a k-1 /b k-1 +... + a 2 /b 2 + a/b + 1) = cn(a k /b k +... + a 2 /b 2 + a/b + 1)

242-535 ADA: 4. Divide/Conquer79 With k = log b n o T(n) = cn(a k /b k +... + a 2 /b 2 + a/b + 1) There are three cases at this stage depending on if a == b, a b If a == b o T(n)= cn(k + 1) = cn(log b n + 1) = O(n log b n) Case 1

242-535 ADA: 4. Divide/Conquer80 With k = log b n o T(n) = cn(a k /b k +... + a 2 /b 2 + a/b + 1) If a < b o Recall that (x k + x k-1 + + x + 1) = (x k+1 -1)/(x-1) // slide 62 o So: o T(n) = cn * O(1) = O(n) Case 2

242-535 ADA: 4. Divide/Conquer81 With k = log b n o T(n) = cn(a k /b k +... + a 2 /b 2 + a/b + 1) If a > b? Case 3

242-535 ADA: 4. Divide/Conquer82 why?

242-535 ADA: 4. Divide/Conquer83 So e.g. merge sort (a = b = 2 and c = 1) e.g. merge sort (a = b = 2 and c = 1)

9. The Master Method The master method only applies to divide and conquer recurrences of the form: T(n) = a T(n/b) + f (n) where a 1, b > 1, and f (n) > 0 for all n > n 0 The Master method gives us a cookbook solution for an algorithms running time plug in the numbers, get the equation this is a more general version of the last example this is a more general version of the last example

242-535 ADA: 4. Divide/Conquer85 >a>a When T(n) = aT(n/b) + f(n) then Three cases Case 1 Case 2 Case 3

Example 3 E X. T(n) = 4T(n/2) + n 3 a = 4, b = 2 so n log b a = n 2 ; f (n) = n 3. C ASE 3 since f(n) > n log b a (n 3 > n 2 ) and 4(n/2) 3 cn 3 (reg. cond.) for c = 1/2 T(n) is (n 3 )

Example 4 (fail) E X.T(n) = 4T(n/2) + n 2 /log n a = 4, b = 2 so n log b a = n 2 ; f (n) = n 2 / log n. The master method does not apply because n 2 /log n n 2- for any. f(n) must be a simple polynominal function for the master method to be applicable

242-535 ADA: 4. Divide/Conquer90 Example 5

242-535 ADA: 4. Divide/Conquer91 Common Examples Cannot use Master method since f(n) is not a polynomial

n log b a (1) f (n/b) (1) f (n)f (n) a f (n/b 2 ) a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) #leaves = n log b a Recursion Tree for Master T() T(n) = aT(n/b) + f(n)

f (n/b) (1) f (n)f (n) a f (n/b 2 ) a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) The sums increase geometrically from the root to the leaves. The leaves hold the biggest part of the total sum. (n log b a ) n log b a (1) Case 1 Explained The means that f(n) is smaller than leaf sum.

f (n/b) (1) f (n)f (n) a f (n/b 2 ) a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) The sums are approximately the same on each of the levels ((total of all sums)). (n log b a * log n) n log b a (1) Case 2 Explained No means that f(n) is roughly equal to the leaf sum.

f (n/b) (1) f (n)f (n) a f (n/b 2 ) a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) The sums decrease geometrically from the root to the leaves. The root holds the biggest part of the total sum. n log b a (1) ( f (n)) Case 3 Explained af(n/b) is getting smaller at lower levels (see def n )

242-535 ada: 4. divide/conquer1 objective o look at several divide and conquer examples (merge sort,...

Documents