csc 331: algorithm analysis divide-and-conquer algorithms
TRANSCRIPT
![Page 1: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/1.jpg)
CS
C 3
31
: A
lgori
thm
An
aly
sis
Divide-and-Conquer Algorithms
![Page 2: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/2.jpg)
The divide-and-conquer strategy solves a problem by:
Breaking it into subproblems that are themselves smaller instances of the same type
of problem.
Recursively solving these subproblems.
Appropriately combining their answers.
![Page 3: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/3.jpg)
CS
C 3
31
: A
lgori
thm
An
aly
sis
Recurrence Relations
![Page 4: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/4.jpg)
Divide-and-conquer algorithms often follow a generic pattern:
They tackle a problem of size n by recursively solving a subproblems of size n/b and then
combining these answers in O(nd) time, for some a, b, d > 0.
Their running time can therefore be captured by the equation T(n) = aT(⎡n/b⎤) + O(nd).
![Page 5: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/5.jpg)
If T(n) = aT(⎡n/b⎤) + O(nd) for some constants a > 0, b > 1, and d ≥ 0, then
The Master Theorem tells us the running times of most of the divide-
and-conquer procedures we are likely to use.
![Page 6: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/6.jpg)
CS
C 3
31
: A
lgori
thm
An
aly
sis
Binary Search
![Page 7: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/7.jpg)
Given a sorted array of numbers, return the index of the number k or -1 if it’s not there.
We could go through each number and check to see if that number is k. If so, return the index.
What’s the running time?
Instead, we want to USE the fact that the array is sorted.
What is a better way? (Think high/low game).
![Page 8: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/8.jpg)
2 3 5 6 7 13 19 100
Input: Look for the 13 in:
13 19 100
13
Look at middle number, 7. 13>7, so recurse on second half:
Look at middle number, 19. 13<19, so recurse on first half:
Look at middle number, 13. l found it, with only 3 recursions.
![Page 9: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/9.jpg)
Search in part of Array A for element k, and return its index. Assumes A is sorted.
function binarySearch(A[left..right], k)
// Base case: 0 elements left, I didn’t find it.if right< left return Not foundmid<- (right-left/2) + leftif k = A[mid] return midif k < A[mid] return A[left..mid-1]if k > A[mid] return A[mid+1..right]
![Page 10: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/10.jpg)
What is the recurrence relation?
What is the running time?
function binarySearch(A[left..right], k)
// Base case: 0 elements left, I didn’t find it.if right< left return Not foundmid<- (right-left)/2 + leftif k = A[mid] return midif k < A[mid] return A[left..mid-1]if k > A[mid] return A[mid+1..right]
![Page 11: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/11.jpg)
CS
C 3
31
: A
lgori
thm
An
aly
sis
Mergesort
![Page 12: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/12.jpg)
The problem of sorting a list of numbers lends itself immediately to a divide-and-conquer
strategy.
Split the list into two halves.
Recursively sort each half.
Merge the two sorted sublists.
![Page 13: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/13.jpg)
10 2 5 3 7 13 1 6
Input:
10 2 5 3 7 13 1 6
10 2 5 3 7 13 1 6
10 2 5 3 7 13 1 6
![Page 14: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/14.jpg)
10 2 5 3 7 13 1 6
1 2 3 5 6 7 10 13
2 3 5 10 1 6 7 13
2 10 3 5 7 13 1 6
![Page 15: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/15.jpg)
function mergesort(a[1...n])
if n > 1:return merge(mergesort(a[1... n/2 ]),⎣ ⎦ mergesort(a[ n/2 + 1...n]))⎣ ⎦
else:return a
![Page 16: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/16.jpg)
function merge(x[1...k], y[1...m])
if k = 0: return y[1...m]if m = 0: return x[1...k]
if x[1] ≤ y[1]:return x[1] merge(x[2...k], y[1...m])✪
else:return y[1] merge(x[1...k], y[2...m])✪
where is concatenation.✪
![Page 17: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/17.jpg)
function merge(x[1...k], y[1...m])
if k = 0: return y[1...m]if m = 0: return x[1...k]
if x[1] ≤ y[1]:return x[1] merge(x[2...k], y[1...m])✪
else:return y[1] merge(x[1...k], y[2...m])✪
This merge procedure does a constant amount of work per recursive call (provided the required
array space is allocated in advance), for a total running time of O(k + m).
![Page 18: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/18.jpg)
Thus merge’s are linear, and the overall time take by mergesort is
T(n) = 2T(n/2) + O(n).
a = 2, b = 2, d = 1 ➠ O(n log n)
![Page 19: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/19.jpg)
CS
C 3
31
: A
lgori
thm
An
aly
sis
Medians
![Page 20: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/20.jpg)
What is the median of [45, 1, 10, 30, 25]?
How do you compute the median of n numbers?
Sorting takes O(n log n), can we do better?
25
![Page 21: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/21.jpg)
Sorting is doing far more work than we really need -- we just want the middle element and don’t care about the relative ordering of the
rest of them.
When looking for a recursive solution, it is paradoxically often easier to work with a more general version of the
problem.
![Page 22: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/22.jpg)
Selection
Input: A list of numbers S; an integer k
Output: The kth smallest element of S
For instance, if k = 1, the minimum of S is sought.
For median, k = ⎣⎜S⎟/ 2⎦.
![Page 23: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/23.jpg)
For any number v, split list S into three categories:
elements smaller than v (SL)
elements equal to v (SV)
elements greater than v (SR)
The three sublists can be computed in linear time O(n).
![Page 24: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/24.jpg)
For instance, if the array
2 36 5 21 8 13 11 20 5 4 1S:
is split on v = 13, the three subarrays generated are:
2 5 8 11 5 4 1SL:
13Sv:
36 21 20SR:
![Page 25: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/25.jpg)
Now we can instantly narrow down the search to one of these sublists.
If we wanted, say, the ninth-smallest element of S, we know it must be the first-
smallest element of SR since | SL | + | SV | = 8.
2 5 8 11 5 4 1SL:
13SV:
36 21 20SR:
selection(S,9) = selection(SR,1)
![Page 26: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/26.jpg)
if k ≤ | SL |selection(S,k) = selection(SL, k)
if | SL | < k ≤ | SL | + | SV |selection(S,k) = v
if k > | SL | + | SV |selection(S,k) = selection(SR, k - | SL | - | SV |)
![Page 27: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/27.jpg)
If we could always pick v so that ⎜SL⎟, ⎟ SR⎟ ≈ 1/2(S)
T(n) = T(n/2) + O(n).
a = 1, b = 2, d = 1 ➠ O(n)
But this requires picking v to be the median, which is our ultimate goal!
![Page 28: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/28.jpg)
We will pick v randomly from S.
n + (n - 1) + (n - 2) + ... + n/2 = Θ(n2)
Naturally, the running time of our algorithm depends on the random choice of v.
The worst-case scenario would be if we kept picking v to be the largest (or
smallest) element of the array.
![Page 29: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/29.jpg)
Θ(n)
The best-case scenario would be if we picked v to be the median element of the
array.
Where does the average running time lie?
![Page 30: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/30.jpg)
v is good if it lies within the 25th to 75th percentile of the array.
A good v ensures that the sublists SL and SR have size at most three-fourths that of
S.
Half of the elements of any list must fall between the 25th and 75th percentile.
![Page 31: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/31.jpg)
Lemma: On average a fair coin needs to be tossed two
times before a “heads” is seen.
After two split operations on average, the array will shrink to at most three-fourths of its size.
![Page 32: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/32.jpg)
Letting T(n) be the expected running time on an array of size n, we get
T(n) ≤ T(3n/4) + O(n).
a = 1, b = 4/3, d = 1 O(n)➠
![Page 33: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/33.jpg)
So, the overall ideas:
Often the obvious way to solve a problem is not the most efficient.
Try to find a way to divide the problem into pieces.
IF you know the answer (recursively) to the piece, does that help you find the overall answer?
Once you write a recursive function, see if its solution is quicker than your iterative way.
![Page 34: CSC 331: Algorithm Analysis Divide-and-Conquer Algorithms](https://reader036.vdocuments.us/reader036/viewer/2022062304/56649cef5503460f949bdb40/html5/thumbnails/34.jpg)
Practice:
(Book, problem 2.23)
An array A[1..n] has a majority element if one element occurs in more than half of the entries. Design an O(nlgn) algorithm to decide if the array has a majority element and if so, what the element is. You cannot sort the array, or ask if A[i]>A[j], since comparisons are not supported on the array type. (For example, if the array were of GIF files.) You can, however, ask if A[i] = A[j].