sorting and searching · searching algorithms sorting algorithms merge sort radboud university...
TRANSCRIPT
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Sorting and searching
Alexandra Silva
http://www.cs.ru.nl/~alexandra
Institute for Computing and Information SciencesRadboud University Nijmegen
13 February 2013
Alexandra 13 February 2013 Lesson 2 1 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Recap
Last week’s message
• There are different types of functions
• They have very different growths
• Different algorithms are associated with different functions
• Why analyzing time complexity is important
• Which type of abstractions are done in the analysis
Alexandra 13 February 2013 Lesson 2 2 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Recap
Last week’s message
• There are different types of functions
• They have very different growths
• Different algorithms are associated with different functions
• Why analyzing time complexity is important
• Which type of abstractions are done in the analysis
Alexandra 13 February 2013 Lesson 2 2 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Recap
Last week’s message
• There are different types of functions
• They have very different growths
• Different algorithms are associated with different functions
• Why analyzing time complexity is important
• Which type of abstractions are done in the analysis
Alexandra 13 February 2013 Lesson 2 2 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example I : Linear search in a vector
int linsearch(int *v, int a, int b, int k)
int i;
i=a;
while ((i<=b) && (v[i]!=k))
i++;
if (i>b)
return -1;
else return i;
Cost
c1c2c3c4c5c5
repetitions
1m+1m111
m is the number of time i++ executes.Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.
Alexandra 13 February 2013 Lesson 2 3 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example I : Linear search in a vector
int linsearch(int *v, int a, int b, int k)
int i;
i=a;
while ((i<=b) && (v[i]!=k))
i++;
if (i>b)
return -1;
else return i;
Cost
c1c2c3c4c5c5
repetitions
1m+1m111
m is the number of time i++ executes.Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.
Alexandra 13 February 2013 Lesson 2 3 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example I : Linear search in a vector
int linsearch(int *v, int a, int b, int k)
int i;
i=a;
while ((i<=b) && (v[i]!=k))
i++;
if (i>b)
return -1;
else return i;
Cost
c1c2c3c4c5c5
repetitions
1m+1m111
m is the number of time i++ executes.
Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.
Alexandra 13 February 2013 Lesson 2 3 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example I : Linear search in a vector
int linsearch(int *v, int a, int b, int k)
int i;
i=a;
while ((i<=b) && (v[i]!=k))
i++;
if (i>b)
return -1;
else return i;
Cost
c1c2c3c4c5c5
repetitions
1m+1m111
m is the number of time i++ executes.Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.
Alexandra 13 February 2013 Lesson 2 3 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total Execution Time
T (n) = c1 + c2(m + 1) + c3m + c4 + c5
For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why? Depends on when the element is found.
Best case scenario: The value k is in the first position (k =
v[a]). Then:T (n) = (c1 + c2 + c4 + c5)
T(n) is constant
Worst case scenario: the value in not found.
T (n) = c1+c2(n+1)+c3(n)+c4+c5 = (c2+c3)n+(c1+c2+c4+c5)
T(n) is linear
Alexandra 13 February 2013 Lesson 2 4 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total Execution Time
T (n) = c1 + c2(m + 1) + c3m + c4 + c5
For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why?
Depends on when the element is found.
Best case scenario: The value k is in the first position (k =
v[a]). Then:T (n) = (c1 + c2 + c4 + c5)
T(n) is constant
Worst case scenario: the value in not found.
T (n) = c1+c2(n+1)+c3(n)+c4+c5 = (c2+c3)n+(c1+c2+c4+c5)
T(n) is linear
Alexandra 13 February 2013 Lesson 2 4 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total Execution Time
T (n) = c1 + c2(m + 1) + c3m + c4 + c5
For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why? Depends on when the element is found.
Best case scenario: The value k is in the first position (k =
v[a]). Then:T (n) = (c1 + c2 + c4 + c5)
T(n) is constant
Worst case scenario: the value in not found.
T (n) = c1+c2(n+1)+c3(n)+c4+c5 = (c2+c3)n+(c1+c2+c4+c5)
T(n) is linear
Alexandra 13 February 2013 Lesson 2 4 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total Execution Time
T (n) = c1 + c2(m + 1) + c3m + c4 + c5
For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why? Depends on when the element is found.
Best case scenario: The value k is in the first position (k =
v[a]). Then:T (n) = (c1 + c2 + c4 + c5)
T(n) is constant
Worst case scenario: the value in not found.
T (n) = c1+c2(n+1)+c3(n)+c4+c5 = (c2+c3)n+(c1+c2+c4+c5)
T(n) is linear
Alexandra 13 February 2013 Lesson 2 4 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example II : search for duplicates
void dup(int *v,int a,int b)
int i,j;
for (i=a;i<b;i++)
for (j=i+1;j<=b;j++)
if (v[i]==v[j])
printf("%d equals to %d\n",i,j);
Cost
c1c2c3c4
repetitions
NS1S2S2
N = b− a + 1 (input size); S1 =∑b−1
i=a (ni + 1); S2 =∑b−1
i=a ni andni = b − i number of times the inner for executes, for each i .
Alexandra 13 February 2013 Lesson 2 5 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example II : search for duplicates
void dup(int *v,int a,int b)
int i,j;
for (i=a;i<b;i++)
for (j=i+1;j<=b;j++)
if (v[i]==v[j])
printf("%d equals to %d\n",i,j);
Cost
c1c2c3c4
repetitions
NS1S2S2
N = b− a + 1 (input size); S1 =∑b−1
i=a (ni + 1); S2 =∑b−1
i=a ni andni = b − i number of times the inner for executes, for each i .
Alexandra 13 February 2013 Lesson 2 5 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example II : search for duplicates
void dup(int *v,int a,int b)
int i,j;
for (i=a;i<b;i++)
for (j=i+1;j<=b;j++)
if (v[i]==v[j])
printf("%d equals to %d\n",i,j);
Cost
c1c2c3c4
repetitions
NS1S2S2
N = b− a + 1 (input size); S1 =∑b−1
i=a (ni + 1); S2 =∑b−1
i=a ni andni = b − i number of times the inner for executes, for each i .
Alexandra 13 February 2013 Lesson 2 5 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total time of execution
T (N) = c1N + c2S1 + c3S2 + c4S2
In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.Let us take a = 1 and b = N. Then:
S2 =N−1∑i=1
N − i = (N − 1)N − (N−1)N2 = 1
2N2 − 1
2N
S1 =N−1∑i=1
(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1
2N2 − 1
2N − 1
T (N) is a quadratic function: T (N) = k2N2 + k1N + k0.
Alexandra 13 February 2013 Lesson 2 6 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total time of execution
T (N) = c1N + c2S1 + c3S2 + c4S2
In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.
Let us take a = 1 and b = N. Then:
S2 =N−1∑i=1
N − i = (N − 1)N − (N−1)N2 = 1
2N2 − 1
2N
S1 =N−1∑i=1
(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1
2N2 − 1
2N − 1
T (N) is a quadratic function: T (N) = k2N2 + k1N + k0.
Alexandra 13 February 2013 Lesson 2 6 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total time of execution
T (N) = c1N + c2S1 + c3S2 + c4S2
In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.Let us take a = 1 and b = N. Then:
S2 =N−1∑i=1
N − i = (N − 1)N − (N−1)N2 = 1
2N2 − 1
2N
S1 =N−1∑i=1
(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1
2N2 − 1
2N − 1
T (N) is a quadratic function: T (N) = k2N2 + k1N + k0.
Alexandra 13 February 2013 Lesson 2 6 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total time of execution
T (N) = c1N + c2S1 + c3S2 + c4S2
In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.Let us take a = 1 and b = N. Then:
S2 =N−1∑i=1
N − i = (N − 1)N − (N−1)N2 = 1
2N2 − 1
2N
S1 =N−1∑i=1
(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1
2N2 − 1
2N − 1
T (N) is a quadratic function: T (N) = k2N2 + k1N + k0.
Alexandra 13 February 2013 Lesson 2 6 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total time of execution
T (N) = c1N + c2S1 + c3S2 + c4S2
In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.Let us take a = 1 and b = N. Then:
S2 =N−1∑i=1
N − i = (N − 1)N − (N−1)N2 = 1
2N2 − 1
2N
S1 =N−1∑i=1
(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1
2N2 − 1
2N − 1
T (N) is a quadratic function: T (N) = k2N2 + k1N + k0.
Alexandra 13 February 2013 Lesson 2 6 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Some remarks
We simplified the analysis:
• we used abstract costs ci instead of concrete times
We simplify even further: for a large enough N the term with N2 isso much larger than the other terms that we can say that theexecution time of dup is approximated by
T (N) = kN2
And in fact k is typically also not relevant. So, using the notationfrom last time, we can say that the execution time is
Θ(N2)
Alexandra 13 February 2013 Lesson 2 7 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Some further remarks
Analisys of worst case is useful:
• it’s a guarantee: upper limit for any input
• it occurs often (more than one would wish; e.g. search ofinexistent information)
• often the average case is very close to the worst case
Analysis of the average case involves (advanced) probabilistic studyof the dimension and type of input (e.g. what is the probability avector is almost sorted?; what is the average size? )
Simplifying further: we will in general assume that, for a givendimension, all inputs occur with the same probability.
Alexandra 13 February 2013 Lesson 2 8 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Some further remarks
Analisys of worst case is useful:
• it’s a guarantee: upper limit for any input
• it occurs often (more than one would wish; e.g. search ofinexistent information)
• often the average case is very close to the worst case
Analysis of the average case involves (advanced) probabilistic studyof the dimension and type of input (e.g. what is the probability avector is almost sorted?; what is the average size? )
Simplifying further: we will in general assume that, for a givendimension, all inputs occur with the same probability.
Alexandra 13 February 2013 Lesson 2 8 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Intermezzo
We can classify algorithms in terms of the strategy they use tosolve a problem:
• Incremental – (e.g insertion sort, which we will see next).
• Divide and conquer – (e.g mergesort and quicksort).
• Greedy – (e.g Minimum Spanning Tree).
• Dinamyc programming – (e.g All-Pairs-Shortest- Path).
• Randomized algorithms – (e.g randomized quicksort).
Alexandra 13 February 2013 Lesson 2 9 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Case study I: Insertion sort
Problem: You’re given a list of numbers (integers) and you wantto order them. We want to do this on a list given in an array whichis to be re-ordered ( 6= construct a new list).
• look at the cards
• pick the lower one of a club;put it to the left; search forthe next card of that cluband continue until you haveall the cards inside each clubordered.
Alexandra 13 February 2013 Lesson 2 10 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Case study I: Insertion sort
Problem: You’re given a list of numbers (integers) and you wantto order them. We want to do this on a list given in an array whichis to be re-ordered ( 6= construct a new list).
• look at the cards
• pick the lower one of a club;put it to the left; search forthe next card of that cluband continue until you haveall the cards inside each clubordered.
Alexandra 13 February 2013 Lesson 2 10 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Case study I: Insertion sort –example
Alexandra 13 February 2013 Lesson 2 11 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Case study I: Insertion sort – algorithm
void insertion_sort(int A[])
for (j=2 ; j<=N ; j++)
key = A[j]; // store the current elm in a tmp record
i = j-1;
while (i>0 && A[i] > key)
/* search for the position where A[j] belongs */
A[i+1] = A[i]; /* shift the rest of the greater
elements down */
i--;
A[i+1] = key; // place the element
Alexandra 13 February 2013 Lesson 2 12 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Case study I: Insertion sort – analysis
Execution time will depend on several things.
• dimension of the list
• how ordered it is already
But let’s again take it a step at a time . . .
Alexandra 13 February 2013 Lesson 2 13 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Case study I: Insertion sort – analysis
void insertion_sort(int A[])
for (j=2 ; j<=N ; j++)
key = A[j];
i = j-1;
while (i>0 && A[i] > key)
A[i+1] = A[i];
i--;
A[i+1] = key;
Cost
c1c2c3c4c5c6
c7
repetitions
NN − 1N − 1S1S2S2
N − 1
where S1 =∑N
j=2 nj ; S2 =∑N
j=2(nj − 1), where nj is the numberof times the while is executed, for each value of j
Alexandra 13 February 2013 Lesson 2 14 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
T (N) = c1N+c2(N−1)+c3(N−1)+c4S1+c5S2+c6S2+c7(N−1)
Depending on the input the time can vary.
Best case:
The vector is already sorted
nj = 1 for j = 2, . . . ,N; Hence S1 = N − 1 and S2 = 0;
Thus:
T (N) = (c1 + c2 + c3 + c4 + c7)N − (c2 + c3 + c4 + c7)
T (N) linear: in the best case we have an execution time in Θ(N).
Alexandra 13 February 2013 Lesson 2 15 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
T (N) = c1N+c2(N−1)+c3(N−1)+c4S1+c5S2+c6S2+c7(N−1)
Depending on the input the time can vary.
Best case: The vector is already sorted
nj = 1 for j = 2, . . . ,N; Hence S1 = N − 1 and S2 = 0;
Thus:
T (N) = (c1 + c2 + c3 + c4 + c7)N − (c2 + c3 + c4 + c7)
T (N) linear: in the best case we have an execution time in Θ(N).
Alexandra 13 February 2013 Lesson 2 15 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
T (N) = c1N+c2(N−1)+c3(N−1)+c4S1+c5S2+c6S2+c7(N−1)
Depending on the input the time can vary.
Best case: The vector is already sorted
nj = 1 for j = 2, . . . ,N; Hence S1 = N − 1 and S2 = 0;
Thus:
T (N) = (c1 + c2 + c3 + c4 + c7)N − (c2 + c3 + c4 + c7)
T (N) linear: in the best case we have an execution time in Θ(N).
Alexandra 13 February 2013 Lesson 2 15 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
Worst case:
The vector is ordered backwards (from the greatestto the smallest element.)
nj = j for j = 2, . . . ,N; Hence
S1 =N∑j=2
j =N(N + 1)
2− 1 and S2 =
N∑j=2
(j − 1) =N(N − 1)
2;
Thus:
T (N) = 12(c4 + c5 + c6)N2 + (c1 + c2 + c3 + .5c4 − .5c5 − .5c6 + c7)N−(c2 + c3 + c4 + c7)
T (N) quadratic: worst case has an execution time in Θ(N2).We can say the time of execution is in Ω(N) and O(N2).
Alexandra 13 February 2013 Lesson 2 16 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
Worst case: The vector is ordered backwards (from the greatestto the smallest element.)
nj = j for j = 2, . . . ,N; Hence
S1 =N∑j=2
j =N(N + 1)
2− 1 and S2 =
N∑j=2
(j − 1) =N(N − 1)
2;
Thus:
T (N) = 12(c4 + c5 + c6)N2 + (c1 + c2 + c3 + .5c4 − .5c5 − .5c6 + c7)N−(c2 + c3 + c4 + c7)
T (N) quadratic: worst case has an execution time in Θ(N2).We can say the time of execution is in Ω(N) and O(N2).
Alexandra 13 February 2013 Lesson 2 16 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
Worst case: The vector is ordered backwards (from the greatestto the smallest element.)
nj = j for j = 2, . . . ,N; Hence
S1 =N∑j=2
j =N(N + 1)
2− 1 and S2 =
N∑j=2
(j − 1) =N(N − 1)
2;
Thus:
T (N) = 12(c4 + c5 + c6)N2 + (c1 + c2 + c3 + .5c4 − .5c5 − .5c6 + c7)N−(c2 + c3 + c4 + c7)
T (N) quadratic: worst case has an execution time in Θ(N2).
We can say the time of execution is in Ω(N) and O(N2).
Alexandra 13 February 2013 Lesson 2 16 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Total execution time
Worst case: The vector is ordered backwards (from the greatestto the smallest element.)
nj = j for j = 2, . . . ,N; Hence
S1 =N∑j=2
j =N(N + 1)
2− 1 and S2 =
N∑j=2
(j − 1) =N(N − 1)
2;
Thus:
T (N) = 12(c4 + c5 + c6)N2 + (c1 + c2 + c3 + .5c4 − .5c5 − .5c6 + c7)N−(c2 + c3 + c4 + c7)
T (N) quadratic: worst case has an execution time in Θ(N2).We can say the time of execution is in Ω(N) and O(N2).
Alexandra 13 February 2013 Lesson 2 16 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
More algorithms!
• Up to here: how to analyze the execution time of anon-recursive algorithm.
• Now: Divide and conquer – (mergesort and quicksort).
1 algorithms2 solving recurrence relations
• Randomized algorithms – (randomized quicksort).
Alexandra 13 February 2013 Lesson 2 17 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Dive and Conquer
Divide and conquer strategy
• Divide the problem into n sub-problems (smaller instances ofthe original problem)
• Conquer the sub-problems• trivial for small sizes• use the same strategy, otherwise
• Combine the solutions of the sub-problems into the solution ofthe original problem
Implementation is typically recursive (why?)
Alexandra 13 February 2013 Lesson 2 18 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge sort
It uses a divide and conquer strategy
• Divide the vector in two vectors of similar size
• Conquer: recursively order the two vectors (trivialfor. . .
vectors of size 1)
• Combine two ordered vectors into a new ordered vector.Auxiliary merge function.
1 The merge function gets as input two ordered sequencesA[p...q] and A[q+1...r]
2 In the end of the execution we get the sequence A[p...r]
ordered.
Alexandra 13 February 2013 Lesson 2 19 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge sort
It uses a divide and conquer strategy
• Divide the vector in two vectors of similar size
• Conquer: recursively order the two vectors (trivialfor. . . vectors of size 1)
• Combine two ordered vectors into a new ordered vector.Auxiliary merge function.
1 The merge function gets as input two ordered sequencesA[p...q] and A[q+1...r]
2 In the end of the execution we get the sequence A[p...r]
ordered.
Alexandra 13 February 2013 Lesson 2 19 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Mergesort
void merge_sort(int A[], int p, int r)
if (p < r) /* base case ? */
q = (p+r)/2; /* Divide */
merge_sort(A,p,q); /* Conquer */
merge_sort(A,q+1,r); /* Conquer */
merge(A,p,q,r); /* Combine */
Question: What is the dimension of each sequence in the dividestep? q = b(p + r)/2c
Initial call on an array with n elements: merge sort(A,1,n).
Alexandra 13 February 2013 Lesson 2 20 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Mergesort
void merge_sort(int A[], int p, int r)
if (p < r) /* base case ? */
q = (p+r)/2; /* Divide */
merge_sort(A,p,q); /* Conquer */
merge_sort(A,q+1,r); /* Conquer */
merge(A,p,q,r); /* Combine */
Question: What is the dimension of each sequence in the dividestep?
q = b(p + r)/2c
Initial call on an array with n elements: merge sort(A,1,n).
Alexandra 13 February 2013 Lesson 2 20 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Mergesort
void merge_sort(int A[], int p, int r)
if (p < r) /* base case ? */
q = (p+r)/2; /* Divide */
merge_sort(A,p,q); /* Conquer */
merge_sort(A,q+1,r); /* Conquer */
merge(A,p,q,r); /* Combine */
Question: What is the dimension of each sequence in the dividestep? q = b(p + r)/2c
Initial call on an array with n elements: merge sort(A,1,n).
Alexandra 13 February 2013 Lesson 2 20 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example I
Alexandra 13 February 2013 Lesson 2 21 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example II
Array with number of elements not a multiple of 2
Alexandra 13 February 2013 Lesson 2 22 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Remains to do. . .
merge procedure
Input: Array A and indices p, q, r such that
• p ≤ q < r .
• Subarray A[p..q] is sorted and subarray A[q + 1..r ] is sorted.By the restrictions on p, q, r , neither subarray is empty.
Output: The two subarrays are merged into a single sortedsubarray in A[p..r ].
Alexandra 13 February 2013 Lesson 2 23 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge
Idea behind merging: Think of two piles of cards.Each pile is sorted and placed face-up on a table with the smallestcards on top. We will merge these into a single sorted pile,face-down on the table. A basic step:
• Choose the smaller of the two top cards.
• Remove it from its pile, thereby exposing a new top card.
• Place the chosen card face-down onto the output pile.
• Repeatedly perform basic steps until one input pile is empty.Once one input pile empties, just take the remaining input pileand place it face-down onto the output pile.
What is the complexity of this algorithm?
Alexandra 13 February 2013 Lesson 2 24 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge
Idea behind merging: Think of two piles of cards.Each pile is sorted and placed face-up on a table with the smallestcards on top. We will merge these into a single sorted pile,face-down on the table. A basic step:
• Choose the smaller of the two top cards.
• Remove it from its pile, thereby exposing a new top card.
• Place the chosen card face-down onto the output pile.
• Repeatedly perform basic steps until one input pile is empty.Once one input pile empties, just take the remaining input pileand place it face-down onto the output pile.
What is the complexity of this algorithm?
Alexandra 13 February 2013 Lesson 2 24 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge
• Each basic step should take constant time, since we check justthe two top cards.
• There are ≤ n basic steps, since each basic step removes onecard from the input piles, and we started with n cards in theinput piles.
• Therefore, this procedure should take Θ(n) time.
Alexandra 13 February 2013 Lesson 2 25 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
The merge procedurevoid merge(int A[], int p, int q, int r)
int L[MAX], R[MAX];
int n1 = q-p+1; /* length of array A[p..q] */
int n2 = r-q; /* length of array A[q+1..r] */
for (i=1 ; i<=n1 ; i++) L[i] = A[p+i-1]; /* array LEFT */
for (j=1 ; j<=n2 ; j++) R[j] = A[q+j]; /* array RIGHT */
L[n1+1] = MAXINT; R[n2+1] = MAXINT; /* sentinels */
what are the sentinels good for?
i = 1; j = 1;
for (k=p ; k<=r ; k++)
if (L[i] <= R[j]) /* put the next smallest
A[k] = L[i]; i++; element in the array */
else
A[k] = R[j]; j++;
merge executes in Θ(n) time, where n = r − p + 1 = the numberof elements being merged.
Alexandra 13 February 2013 Lesson 2 26 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
The merge procedurevoid merge(int A[], int p, int q, int r)
int L[MAX], R[MAX];
int n1 = q-p+1; /* length of array A[p..q] */
int n2 = r-q; /* length of array A[q+1..r] */
for (i=1 ; i<=n1 ; i++) L[i] = A[p+i-1]; /* array LEFT */
for (j=1 ; j<=n2 ; j++) R[j] = A[q+j]; /* array RIGHT */
L[n1+1] = MAXINT; R[n2+1] = MAXINT; /* sentinels */
what are the sentinels good for?
i = 1; j = 1;
for (k=p ; k<=r ; k++)
if (L[i] <= R[j]) /* put the next smallest
A[k] = L[i]; i++; element in the array */
else
A[k] = R[j]; j++;
merge executes in Θ(n) time, where n = r − p + 1 = the numberof elements being merged.
Alexandra 13 February 2013 Lesson 2 26 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
The merge procedurevoid merge(int A[], int p, int q, int r)
int L[MAX], R[MAX];
int n1 = q-p+1; /* length of array A[p..q] */
int n2 = r-q; /* length of array A[q+1..r] */
for (i=1 ; i<=n1 ; i++) L[i] = A[p+i-1]; /* array LEFT */
for (j=1 ; j<=n2 ; j++) R[j] = A[q+j]; /* array RIGHT */
L[n1+1] = MAXINT; R[n2+1] = MAXINT; /* sentinels */
what are the sentinels good for?
i = 1; j = 1;
for (k=p ; k<=r ; k++)
if (L[i] <= R[j]) /* put the next smallest
A[k] = L[i]; i++; element in the array */
else
A[k] = R[j]; j++;
merge executes in Θ(n) time, where n = r − p + 1 = the numberof elements being merged.
Alexandra 13 February 2013 Lesson 2 26 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
The merge procedurevoid merge(int A[], int p, int q, int r)
int L[MAX], R[MAX];
int n1 = q-p+1; /* length of array A[p..q] */
int n2 = r-q; /* length of array A[q+1..r] */
for (i=1 ; i<=n1 ; i++) L[i] = A[p+i-1]; /* array LEFT */
for (j=1 ; j<=n2 ; j++) R[j] = A[q+j]; /* array RIGHT */
L[n1+1] = MAXINT; R[n2+1] = MAXINT; /* sentinels */
what are the sentinels good for?
i = 1; j = 1;
for (k=p ; k<=r ; k++)
if (L[i] <= R[j]) /* put the next smallest
A[k] = L[i]; i++; element in the array */
else
A[k] = R[j]; j++;
merge executes in Θ(n) time, where n = r − p + 1 = the numberof elements being merged.Alexandra 13 February 2013 Lesson 2 26 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example
merge(A,9,12,16)
Alexandra 13 February 2013 Lesson 2 27 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Example c’d
merge(A,9,12,16)
Alexandra 13 February 2013 Lesson 2 28 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Algorithm with a recursive call: running time can usually bedescribed as a recurrence.
How does that work?
Let’s for a moment assume that the size of the input is a power of2. In each division step the sub-arrays have exactly size n/2.
T (n): time of execution (in the worst case) for a input of size n;
If n = 1 then T (n) is constant: T (n) ∈ Θ(1).
Alexandra 13 February 2013 Lesson 2 29 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Algorithm with a recursive call: running time can usually bedescribed as a recurrence. How does that work?
Let’s for a moment assume that the size of the input is a power of2. In each division step the sub-arrays have exactly size n/2.
T (n): time of execution (in the worst case) for a input of size n;
If n = 1 then T (n) is constant: T (n) ∈ Θ(1).
Alexandra 13 February 2013 Lesson 2 29 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Algorithm with a recursive call: running time can usually bedescribed as a recurrence. How does that work?
Let’s for a moment assume that the size of the input is a power of2. In each division step the sub-arrays have exactly size n/2.
T (n): time of execution (in the worst case) for a input of size n;
If n = 1 then T (n) is constant: T (n) ∈ Θ(1).
Alexandra 13 February 2013 Lesson 2 29 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Algorithm with a recursive call: running time can usually bedescribed as a recurrence. How does that work?
Let’s for a moment assume that the size of the input is a power of2. In each division step the sub-arrays have exactly size n/2.
T (n): time of execution (in the worst case) for a input of size n;
If n = 1 then T (n) is constant: T (n) ∈ Θ(1).
Alexandra 13 February 2013 Lesson 2 29 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Otherwise:
1 Divide computing the middle of the vector is done in constanttime: Θ(1).
2 Conquer two problems of size n/2 are solved: 2T (n/2).
3 Compose the merge function executes in linear time: Θ(n)
Hence:
T (n) =
Θ(1) if n = 1
Θ(1) + 2T (n/2) + Θ(n) if n > 1
and now we want to conclude that T (n) ∈ Θ(??).
Alexandra 13 February 2013 Lesson 2 30 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Otherwise:
1 Divide computing the middle of the vector is done in constanttime: Θ(1).
2 Conquer two problems of size n/2 are solved: 2T (n/2).
3 Compose the merge function executes in linear time: Θ(n)
Hence:
T (n) =
Θ(1) if n = 1
Θ(1) + 2T (n/2) + Θ(n) if n > 1
and now we want to conclude that T (n) ∈ Θ(??).
Alexandra 13 February 2013 Lesson 2 30 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Complexity analysis
Otherwise:
1 Divide computing the middle of the vector is done in constanttime: Θ(1).
2 Conquer two problems of size n/2 are solved: 2T (n/2).
3 Compose the merge function executes in linear time: Θ(n)
Hence:
T (n) =
Θ(1) if n = 1
Θ(1) + 2T (n/2) + Θ(n) if n > 1
and now we want to conclude that T (n) ∈ Θ(??).
Alexandra 13 February 2013 Lesson 2 30 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge vs Insertion sort
Running ahead: we will show that T (n) ∈ Θ(n lg n).
Cf. Insertion sort in Θ(n2).
Alexandra 13 February 2013 Lesson 2 31 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge vs Insertion sort
Running ahead: we will show that T (n) ∈ Θ(n lg n).
Cf. Insertion sort in Θ(n2).
Alexandra 13 February 2013 Lesson 2 31 / 31
IntroductionSearching algorithms
Sorting algorithmsMerge Sort
Radboud University Nijmegen
Merge vs Insertion sort
Running ahead: we will show that T (n) ∈ Θ(n lg n).
Cf. Insertion sort in Θ(n2).
Alexandra 13 February 2013 Lesson 2 31 / 31