asymptotic analysis of algorithms (how to evaluate your programming power tools)

CS 307 Fundamentals of Computer Science

1

Asymptotic Analysis of Algorithms(how to evaluate your programming power tools)

based on presentation material by

Mike Scott

Honors CS 102

Sept. 4, 2007


2

Algorithmic Analysis A technique used to characterize the

execution behavior of algorithms in a manner independent of a particular platform, compiler, or language.

Abstract away the minor variations and describe the performance of algorithms in a more theoretical, processor independent fashion.

A method to compare speed of algorithms against one another.


3

Different Algorithms Why are the words in a dictionary in

alphabetical order? A brute force approach

– linear search– worst case is (d x N)

Another way– divide and conquer– worst case is (c x log N)

Constants are unknown and largely irrelevant.


4

Big O The most common method and notation for

discussing the execution time of algorithms is "Big O”.

For the alphabetized dictionary the algorithm requires O(log N) steps.

For the unsorted list the algorithm requires O(N) steps.

Big O is the asymptotic execution time of the algorithm.


5

Formal Definition of Big O T(N) is O( F(N) ) if there are positive constants c

and N0 such that T(N) < cF(N) when N > N0

– There is a point N0 such that for all values of N that are past this point, T(N) is bounded by some multiple of F(N).

– Thus if T(N) of the algorithm is O( N2 ) then, ignoring constants, at some point we can bound the running time by a quadratic function of the input size.

– Given a linear algorithm, it is technically correct to say the running time is O(N2). O(N) is a more precise answer as to the Big O bound of a linear algorithm.


6

Big O Examples

3n3 = O(n3) 3n3 + 8 = O(n3) 8n2 + 10n * log(n) + 100n + 1020 = O(n2) 3log(n) + 2n1/2 = O(n1/2) 2100 = O(1) TlinearSearch(n) = O(n)

TbinarySearch(n) = O(log(n))


7

Other Algorithmic Analysis Tools Big Omega T(N) is ( F(N) ) if there are

positive constants c and N0 such that T(N) > cF( N )) when N > N0

– Big O is similar to less than or equal, an upper bound.

– Big Omega is similar to greater than or equal, a lower bound.

Big Theta T(N) is ( F(N) ) if and only if T(N) is O( F(N) )and T( N ) is ( F(N) ).– Big Theta is similar to equals.


8

Relative Rates of GrowthAnalysis

TypeMathematicalExpression

Relative Rates of Growth

Big O T(N) = O( F(N) ) T(N) < F(N)

Big T(N) = ( F(N) ) T(N) > F(N)

Big T(N) = ( F(N) ) T(N) = F(N)

"In spite of the additional precision offered by Big Theta,Big O is more commonly used, except by researchersin the algorithms analysis field" - Mark Weiss


9

What it All Means T(N) is the actual growth rate of the

algorithm. F(N) is the function that bounds the growth

rate.– may be upper or lower bound

T(N) may not equal F(N).– constants and lesser terms ignored because it is

a bounding function


10

Assumptions in Big O Analysis Once found accessing the value of a

primitive is constant timex = y;

mathematical operations are constant timex = y * 5 + z % 3;

if statement: constant time if test and maximum time for each alternative are constantsif( iMySuit ==DIAMONDS || iMySuit == HEARTS)return RED;

elsereturn BLACK;


11

Fixed-Size Loops

Loops that perform a constant number of iteration are considered to execute in constant time. They don't depend on the size of some data set

for(int suit = Card.CLUBS; suit <= Card.SPADES; suit++) { for(int value = Card.TWO; value <= Card.ACE; value++)

{ myCards[cardNum] = new Card(value, suit); cardNum++; }

}


12

Loops That Work on a Data Set Loops like on the previous slide are fairly rare. Normally a loop operates on a data set which can

vary is size. public double minimum(double[] values)

{ int n = values.length; double minValue = values[0]; for(int i = 1; i < n; i++) if(values[i] < minValue) minValue = values[i]; return minValue;}

The number of executions of the loop depends on the length of the array, values. The actual number of executions is (length - 1).

The run time is O(N).


13

Nested Loops

Number of executions?

public void bubbleSort(double[] data){ int n = data.length; for(int i = n - 1; i > 0; i--) for(int j = 0; j < i; j++)

if(data[j] > data[j+1]) { double temp = data[j];

data[j] = data[j + 1]; data[j + 1] = temp;

}}


14

Summing Execution Times If an algorithm’s execution time is N2 + N

then it is said to have O(N2) execution time, not O(N2 + N).

When adding algorithmic complexities the larger value dominates.

Formally, a function f(N) dominates a function g(N) if there exists a constant value n0 such that for all values N > N0 it is the case that g(N) < f(N).


15

Example of Dominance Suppose we go for precision and determine

how fast an algorithm executes based on the number of items in the data set.

x2/10000 + 2x log10 x + 100000

Is it plausible to say the x2 term dominates even though it is divided by 10000?

What if we separate the equation into (x2/10000 ) and (2x log x + 100000)?


16

Summing Execution Times

For large values the x2 term dominates so the algorithm is O(N2).


17

Function Common Name

N! factorial

2N Exponential

Nd, d > 3 Polynomial

N3 Cubic

N2 Quadratic

N N

N log N

N Linear

N Root - n

log N Logarithmic

1 Constant

Ranking of Algorithmic Behaviors


18

Running Times Assume N = 100,000 and processor speed is

1,000,000 operations per second

Function Running Time

2N over 100 years

N3 31.7 years

N2 2.8 hours

N N 31.6 seconds

N log N 1.2 seconds

N 0.1 seconds

N 3.2 x 10-4 seconds

log N 1.2 x 10-5 seconds


19

Graphical Results


20

Determining Big O A DNA problem Is a number prime? Hand actions

– inserting card– determining if card is in Hand– combining Hands– copying Hand– retrieving Card


21

Putting it in Context Why worst-case analysis? Wouldn’t

average-case analysis be more useful? A1: No. E.g., doesn’t matter if a cashier’s

terminal handles 2 transactions per second or 3, as long as it never takes more than 20.

A2: Well, OK, sometimes, yes. However, this often requires much more sophisticated mathematics. In fact, for some common algorithms, nobody has been able to do an average case analysis.

asymptotic analysis of algorithms (how to evaluate your programming power tools)

Documents