Last lesson Arrays for implementing collection classes Performance analysis (review)
Today Performance analysis Logarithm
Complexity Theory
Three different symbols will be discussed in class: O (called the Big Oh) (upper case Greek letter OMEGA) (upper case Greek letter THETA)
Weiss, chapter 5
O (Big Oh)
The Big Oh is the upper bound of a function. In the case of algorithm analysis, we use it to bound the
worst-case running time, or the longest running time possible for any input of size n.
We can say that the maximum running time of the algorithm is in the order of Big Oh.
T(n) = O(f(n)) if there are positive constants c and n0 such that T(n) c*f(n) for all n n0
is is the lower bound of a function. We can say that the minimum running time of the
algorithm is in the order of
T(n) = (f(n)) if there are positive constants c and n0 such that T(n) c*f(n) for all n n0
is composed of both O and . T(n) = (f(n)) if and only if T(n) is O(f(n)) and T(n)
is (f(n))
n
T
upper bound
lower bound
A Few Remarks
We only care what happens for large n (n n0) O, , "hide" the constant c from us. Find simple and small functions
E.g. (n2 + 3n + 4) is O(n3), since (n2 + 3n + 4) < n3 for all n > 3,
but it is also O(n2), since (n2 + 3n + 4) < 2n2 for all n > 10
The Logarithm
For any b, n>0: logbn=k if bk=n
We use log instead of log2 (as in Weiss) Be careful, often
lg instead of log2
log instead of log10
Properties of logarithms logb(x*y) = logbx + logby
logb(x/y) = logbx - logby
logbxa = a*logbx
logbx= (logax)/(logab)
For complexity analysis the base does not matter For any constant b>1: logbn = O(log n)
Repeated Doubling and Halving
Repeated doubling We can repeatedly double only logarithmically many
times until we reach n
Repeated halving Starting at n, we can halve only logarithmically many
times before we reach 1
Static Searching Problem
Given an integer num and an array arr, return the position of num in arr or an indication that it is not present. If num occurs more than once, return any occurrence. The array is never altered (is static).
Weiss, chapter 5.6
4
“7”“3” “12” “42” “56” “75” “77”
4
Linear (Sequential) Search
for (int i = 0; i <= last; i++) { if (arr[i] = num) {
return i; }
}
Complexity E.g. search for “56” O(n) (linear)
“7”“3” “12” “42” “56” “75” “77”
Linear (Sequential) Search
for (int i = 0; i <= last; i++) { if (arr[i] = num) {
return i; }
}
Complexity E.g. search for “98” O(n) (linear)
“7”“3” “12” “42” “56” “75” “77”
Binary Search (1)
Array must be sorted Routine to return index where element is equal to
argument Look in middle of occupied part of array If that is the element we are looking for, we found it Otherwise, decide whether the element must be in front
half or back half (do recursive call to look there) E. g. search for “56”
This is a divide-and-conquer algorithm
“7”“3” “12” “42” “56” “75” “77”
Binary Search (1)
Array must be sorted Routine to return index where element is equal to
argument Look in middle of occupied part of array If that is the element we are looking for, we found it Otherwise, decide whether the element must be in front
half or back half (do recursive call to look there) E. g. search for “56”
This is a divide-and-conquer algorithm
“7”“3” “12” “42” “56” “75” “77”
Binary Search (1)
Array must be sorted Routine to return index where element is equal to
argument Look in middle of occupied part of array If that is the element we are looking for, we found it Otherwise, decide whether the element must be in front
half or back half (do recursive call to look there) E. g. search for “56”
This is a divide-and-conquer algorithm
“7”“3” “12” “42” “56” “75” “77”
Binary Search (1)
Array must be sorted Routine to return index where element is equal to
argument Look in middle of occupied part of array If that is the element we are looking for, we found it Otherwise, decide whether the element must be in front
half or back half (do recursive call to look there) E. g. search for “56”
This is a divide-and-conquer algorithm
“7”“3” “12” “42” “56” “75” “77”
Binary Search (1)
Array must be sorted Routine to return index where element is equal to
argument Look in middle of occupied part of array If that is the element we are looking for, we found it Otherwise, decide whether the element must be in front
half or back half (do recursive call to look there) E. g. search for “56”
This is a divide-and-conquer algorithm
“7”“3” “12” “42” “56” “75” “77”
Binary Search (3)
It can also be written as a loop (non recursive) Warning: the idea (binary search) is simple, but
the code is tricky to get right Avoid infinite recursion or infinite loop Treat boundary cases properly Avoid looking at elements which are not in the sequence
Each time we deal with half as much of the array as the previous call (or iteration) So we make log n calls or iterations Each time we do const work besides the recursion So the total cost is O(log n) (much faster than simple
searching or searching in an unsorted array)
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
73 12 42 56 75 77
low high
find 56
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
73 12 42 56 75 77
low high
find 56
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
73 12 42 56 75 77
low highmid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
highmid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
highmid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
highmid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
high
mid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
high
mid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
high
mid
find 56
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
high
mid
find 56
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
73 12 42 56 75 77
low high
find 98
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
73 12 42 56 75 77
low high
find 98
Code
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
73 12 42 56 75 77
low highmid
find 98
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
highmid
find 98
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
highmid
find 98
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
highmid
find 98
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
low
mid
find 98
high
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
find 98
midhigh
low
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
find 98
high
low
mid
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
find 98
high
low
mid
public static int binarySearch(Comparable [ ] a, Comparable x) {
int low = 0; int high = a.length - 1; int mid;
while(low <= high) {mid = (low + high) / 2; if(a[mid].compareTo(x) < 0)
low = mid + 1;else if(a[mid].compareTo(x) > 0)
high = mid - 1;else
return mid;}return NOT_FOUND; // NOT_FOUND = -1
}
Code
73 12 42 56 75 77
find 98
high
low
mid
Simplifying Rules
If f(n) is in O(g(n)) and g(n) is in O(h(n)), then f(n) is in O(h(n)).
If f(n) is in O(k*g(n)) for any constant k > 0, then f(n) is in O(g(n)).
If f1(n) is in O(g1(n)) and f2(n) is in O(g2(n)), then f1(n)+f2(n) is in max{O(g1(n)),
O(g2(n))}.
If f1(n) is in O(g1(n)) and f2(n) is in O(g2(n)), then f1(n)*f2(n) is in O(g1(n)*g2(n)).
Analysing Control Statements
While loop Analyse like a for loop
If statement Take greater complexity of then/else clauses
Switch statement Take complexity of most expensive case
Subroutine call Complexity of the subroutine
Limitations of Big Oh
Not for small amount of input Is designed for theoretical analysis
E.g. memory access is O(1), does not differentiate between main memory and disc
Constants may be to large for practical use Worst case is sometimes uncommon an can be ignored
Recursion
This is a review of the main ideas We expect some experience from first year Read Weiss Sections 7.1 - 7.3
Key Ideas
A method includes one or more calls to the same method The result of the nested calls is used to compute the
result of the outer call There is a base case The call may be indirectly
Understand the method by assuming the nested calls work correctly
First, let us look at a non-programming example
Recursive Definition
Define a list of number separated by commas Example: 23, 27, 31, 1
Like this LIST is number or LIST is number, LIST
Example
Try to see whether “23, 27, 31, 1” is a list Apply definition
LIST: number
or LIST: number, LIST23, 27, 31, 1
23, LIST
27, LIST
31, LIST
1
Base Case
Notice that we had one case (LIST: number) that does no recursion That is called the base case Otherwise, we would have infinite recursion
LIST: number
or LIST: number, LIST
31, LIST
1
…
What About Programs?
We can solve a lot of mathematical problems recursively
E.g. i=0 i = 1 + 2 + 3 + … + n
public static int sum (int n) {
if (n < 1)
return 0;
else
return(sum(n – 1) + n);
}
n
Example
public static int sum(int n) {
if(n < 1)
return 0;
return(sum(n – 1) + n);
}
sum(4)
sum(3)
sum(2)
sum(1)
sum(0)
0
0 + 1
1 + 2
3 + 3
6 + 4
Progress
Notice that we must make progress toward base case Sum(0) There is steady reduction in size of problem Otherwise, trouble