introduction to complexity. 2 analysis of algorithms why do we need to analyze algorithms? –to...
TRANSCRIPT
Introduction to complexity
2
Analysis of Algorithms
• Why do we need to analyze algorithms?– To measure the performance
– Comparison of different algorithms
– Can we find a better one? Is this the best solution?
– Running time analysis
– Memory usage
3
Complexity: a measure of the performance of an algorithm
An algorithm’s performance depends on internal and external factors
External• Size of the input to the algorithm• Speed of the computer on which it is run• Quality of the compiler
InternalThe algorithm’s efficiency, in terms of:• Time required to run• Space (memory storage) required to run
Complexity measures the internal factors(usually more interested in time than space)
4
Running Time Analysis
N
Algorithm 1
Running Time T1(n)
N
Algorithm 2
Running Time T2(n)10001010101000111110001100011101010101010101010010001010101000100000000000011110101000111010
Algorithm 1 T1(N)=1000N
Algorithm 2 T2(N)=N2
5
Running Time Analysis
N
Run
ning
Tim
e T
(N)
Algorithm 2
Algorithm 1
1000
6
Summary of Running TimesN T1 T2
10 10-2 sec 10-4 sec
100 10-1 sec 10-2 sec
1000 1 sec 1 sec
10000 10 sec 100 sec
100000 100 sec 10000 sec
For the values of N that are less than 1000 the difference bw. running times of the two algorithms can be ignored
7
Growth rates and big-O notation
• Growth rates capture the essence of an algorithm’s performance
• Big-O notation indicates the growth rate. It is the class of mathematical formula that best describes an algorithm’s performance, and is discovered by looking inside the algorithm
• Big-O is a function with parameter N, where N is usually the size of the input to the algorithm– For example, if an algorithm depending on the value n has performance
an2 + bn + c (for constants a, b, c) then we say the algorithm has performance O(N2)
• For large N, the N2 term dominates. Only the dominant term is included in big-O
8
big-O notation – Asympthotic Upper Limit
• Running time of an algorithm
T(N)=O(f(n))
•If there are c and n0 values satisfying T(N) c f(n) for N n0 then T(N) c f(n) • f(n), is an asymptotic upper bound for T(N) and T(N)=O(f(n)).
This is not a function, but a notation.
9
big-O notation• big-O notation is written in the simplest form
– Example
• 3n2+2n+5 = O(n2)
– The followings are correct, but not used
• 3n2+2n+5 = O(3n2+2n+5)
• 3n2+2n+5 = O(n2+n)
• 3n2+2n+5 = O(3n2)
10
Common growth rates
Time complexity Example
O(1) constant Adding to the front of a linked list
O(log N) log Finding an entry in a sorted array
O(N) linear Finding an entry in an unsorted array
O(N log N) n-log-n Sorting n items by ‘divide-and-conquer’
O(N2) quadratic Shortest path between two nodes in a graph
O(N3) cubic Simultaneous linear equations
O(2N) exponential The Towers of Hanoi problem
11
Growth rates
Number of Inputs
Tim
e
O(N2)
O(Nlog N)
For a short time N2 isbetter than NlogN
12
Calculating the actual time taken by a program (example)
• A program takes 10ms to process one data item (i.e. to do one operation on the data item)
• How long would the program take to process 1000 data items, if time is proportional to:– log10 N
– N
– N log10 N
– N2
– N3
• (time for 1 item) x (big-O( ) time complexity of N items)
13
How do we calculate big-O?
1 Loops
2 Nested loops
3 Consecutive statements
4 If-then-else statements
5 Logarithmic complexity
Five guidelines for finding out the time complexity of a piece of code
14
Guideline 1: Loops
The running time of a loop is, at most, the running time of the statements inside the loop (including tests) multiplied by the number of iterations.
for (i=1; i<=n; i++){ m = m + 2;}
constant timeexecutedn times
Total time = a constant c * n = cn = O(N)
15
Guideline 2: Nested loops
Analyse inside out. Total running time is the product of the sizes of all the loops.
for (i=1; i<=n; i++) { for (j=1; j<=n; j++) { k = k+1; }} constant time
outer loopexecutedn times
inner loopexecutedn times
Total time = c * n * n * = cn2 = O(N2)
16
Guideline 3: Consecutive statements
Add the time complexities of each statement.
Total time = c0 + c1n + c2n2 = O(N2)
x = x +1;for (i=1; i<=n; i++) { m = m + 2;}for (i=1; i<=n; i++) { for (j=1; j<=n; j++) { k = k+1; }}
inner loopexecutedn times
outer loopexecutedn times constant time
executedn times
constant time
constant time
17
Guideline 4: If-then-else statements
Worst-case running time: the test, plus either the then part or the else part (whichever is the larger).
if (depth( ) != otherStack.depth( ) ) { return false;}else { for (int n = 0; n < depth( ); n++) { if (!list[n].equals(otherStack.list[n])) return false; }}
then part:constant
else part:(constant +constant) * n
test:constant
another if :constant + constant(no else part)
Total time = c0 + c1 + (c2 + c3) * n = O(N)
18
Guideline 5: Logarithmic complexity
An algorithm is O(log N) if it takes a constant time to cut the problem size by a fraction (usually by ½)
Example algorithm (binary search):finding a word in a dictionary of n pages
• Look at the centre point in the dictionary• Is word to left or right of centre?• Repeat process with left or right part of dictionary until the word is found
19
big-O notation - Example 1
• Prove/disprove that 3n2+2n+5 = O(n2) 10 n2 = 3n2 + 2n2 + 5n2
3n2 + 2n + 5 for n 1
c = 10, n0 = 1 The number of (n0,c) pairs is not important. Having only one pair satisfying the condition is enough.
20
big-O notation - Example 2
• If T(N) can be expressed as T(N)=O(7n2+5n+4), T(N) can be anything from the followings
• T(N)=n2
• T(N)=4n+7• T(N)=1000n2+2n+300• T(N)= O(7n2+5n+4) =O(n2)
21
big-O notation - Example 3• Write the big-O notations for the follwing running times
• f1(n) = 10 n + 25 n2
• f2(n) = 20 n log n + 5 n
• f3(n) = 12 n log n + 0.05 n2
• f4(n) = n1/2 + 3 n log n
• O(n2)
• O(n log n)
• O(n2)
• O(n log n)
22
Analysis of Running Times
• Strategy: Upper and lower bounds
Upper bound
Running time function
Lower bound
23
Analysis of Running Times
• Determing the running times exactly is a difficult task. Therefore:– Analysis of the best cases
– Analysis of the average cases which is quite difficult
– Analysis of the worst cases which is easier
have to be considered
24
• In some cases, it is important to consider the best, worst and/or average (or typical) performance of an algorithm:
• E.g., when sorting a list into order, if it is already in order then the algorithm may have very little work to do
• The worst-case analysis gives a bound for all possible input (and may be easier to calculate than the average case)
Best, average, worst-case complexity
– Worst, O(N) or o(N): or > true function *– Typical, Θ(N): true function *– Best, Ω(N): true function *
* These approximations are true only after N has passed some value
25
Notation- Asymptotic Bound
f(n)
c g(n)
Reverse of big-O notation If there exist positive c and n0 constatnts can be found for T(N) c f(n) where N n0 then T(N)=(f(n)) f(n) is an asymptotic lower bound of T(N)
n0
26
Notation- Example 1
• 7n2+3n+5 = O(n4)
• 7n2+3n+5 = O(n3)
• 7n2+3n+5 = O(n2)
• 7n2+3n+5 = (n2)
• 7n2+3n+5 = (n)
• 7n2+3n+5 = (1)
27
Running Time 6N+4=O(N)
1
1
int Sum (int N) { int i, PartialSum; PartialSum=0;
for(i=1 ;i<=N ; i++)
PartialSum+=i*i*i; return PartialSum; }
Algorithm 1
N+N+2N
1+(N+1)+N
28
Running Time O(N2)
for(i=0; i<N; i++) for(j=1; j<=N; i++) k++;
Algorithm 2
for(i=0; i<N; i++) A[i]=0;for(i=0; i<N; i++) for(j=1; j<=N; i++) A[i]+=A[j]+i+j;
Running Time O(N2)
29
Running Timemax_ Running_Time (S1,S2)
If( condition )
S1
Else
S2
Algorithm 3
30
• int binary search(A,key,N)• low=0, high=N-1 • while(lowhigh)• mid=(low+high)/2 • if(A[mid]<key)• low=mid+1• if(A[mid]>key)• high=mid-1;• if(A[mid]=key)• return mid• Return not found
Algorithm 4
Running Time O(logN). Since the number of input is halved in each iteration.
31
• int binarysearch(A,key,low,high)• if (low>high)• Return not found• else• mid=(low+high)/2 • if(A[mid]<key) • Return binarysearch(A,key,mid+1,high)• if(A[mid]>key) • Return binary search(A,key,low,mid-1)• if (A[mid]=key) • Return mid
Algorithm 5
T(N)=T(N/2)+O(1)
32
Running Time O(N)
MaxSubsequenceSum(const int A[], int n)
ThisSum=MaxSum=0;
for(j=0;j<N;j++)
ThisSum+=A[j];
if (ThisSum<MaxSum)
MaxSum=ThisSum;
else if(ThisSum<0)
ThisSum=0;
Return MaxSum;
Algorithm 6
33
Performance isn’t everything!
• There can be a tradeoff between:– Ease of understanding, writing and debugging
– Efficient use of time and space
• So, maximum performance is not always desirable
• However, it is still useful to compare the performance of different algorithms, even if the optimal algorithm may not be adopted