Question of the Day

A friend tells the truth when saying:

A road near my house runs directly north-south; I get on the road facing north, drive for a mile, & end up south of where I started

How does he do it?

Question of the Day

A friend tells the truth when saying:

A road near my house runs directly north-south; I get on the road facing north, drive for a mile, & end up south of where I started

How does he do it?

Note: He (anyone) doesn't live near the north pole

Question of the Day

A friend tells the truth when saying:

A road near my house runs directly north-south; I get on the road facing north, drive for a mile, & end up south of where I started

How does he do it?

Note: He (anyone) doesn't live near the north pole

Your friend drives in reverse!

LECTURE 15:BIG-OH NOTATION

Analysis Techniques

Running time is critical, … …but comparing times impossible in

many cases Single problem may have lots of ways to be

solved Many implementations possible for each

solution

Pseudo-Code

Only for human eyes Unimportant & implementation details

ignored Serves very real purpose, even if it is

not real Useful for tasks like outlining, designing, &

analyzing Language-like manner to system, though

not formal

Pseudo-Code

Only needs to include details needed for tracing Loops, assignments, calls to methods, etc. Anything that would be helpful analyzing

algorithm Understanding algorithm is only goal

Feel free to ignore punctuation & other formalisms

Understanding & analysis is only goal of using this

Pseudo-Code Example

Algorithm factorial(int n)

returnVariable = 1

while (n > 0)

returnVariable = returnVariable * n

n = n – 1endwhile

return returnVariable

“Anything that can go wrong…” Expresses an algorithm’s complexity

Worst-case analysis of algorithm performance

Usually reasonably correlated with execution time

Not always right to consider only worst-case May be situation where worst-case is very

rare Solve for other cases similarly, but almost

never done

“Should I Even Bother?”

Compare algorithms using big-Oh notation Could use to compare implementations,

also Saves time implementing all the algorithms Biases like CPU, typing speed, cosmic rays

ignored

Algorithmic Analysis

Algorithm Analysis

Execution time with n inputs on 4GHz machine:n = 10 n = 50 n = 100 n = 1000 n = 106

O(n log n) 9 ns 50 ns 175 ns 2500 ns 5 ms

O(n2) 25 ns 625 ns 2250 ns 250000 ns 4 min

O(n5) 25000 ns 72.5 ms 2.7 s 2.9 days1x1013

yrs

O(2n) 2500 ns 3.25 days 1 x 1014 yrs1 x 10285

yrsToo long!

O(n!) 1 ms1.4 x 1058

yrs 7 x 10141

yrsToo long! Too long!

Algorithm Analysis

Execution time with n inputs on 4GHz machine:n = 10 n = 50 n = 100 n = 1000 n = 106

O(n log n)

2.6 m 15 m 52 m1.53x

height of CN Tower

Buffalo to Minneapoli

s

O(n2) 7.5 m 187 m 674 mBuffalo to Fredonia

Sun to Mercury

O(n5) 4 miles1/17th of Earth to

Moon

2x Earth to

Moon

10x Sun to

Pluto

833x older than

universe

O(2n) 749 m12x

Sun to Pluto

8333x older than

universe

10275 x older than universe

Too long!

O(n!)Buffalo

to DC

1048 x older than

universe

10131 x older than

universe Too long! Too long!

Want results for large data sets Nobody cares about 2 minutes; too small

to matter For this process, only really major details

considered Ignore multipliers

So, O(⅛n) = O(5n) = O(50000n) = O(n) Multipliers usually implementation-specific Going from Sun to Mercury v. going to

Minneapolis Ignore lesser terms

So, O(⅚n5 + 23402n2) = O(n5 + n2) = O(n5) Job 300x older than universe v. 17 minutes?

Big-Oh Notation

What is n?

Big-Oh analysis always relative to input size But determining input size is not always

clear Quick rules of thumb:

Need to consider what algorithm is processingAnalyze values below x: n = xAnalyze data in an array: n = size of arrayAnalyze 2 arrays: n = sum of array sizes

Big-Oh computes primitive operations executed Assignments Calling a method (but NOT executing the

method) Performing arithmetic operation Comparing two values Getting entry from an array Following a reference Returning a value from a method Accessing a field

Analyzing an Algorithm

Primitive Statements

Basis of programming, take constant time: O(1) Fastest possible big-Oh notation

Time to run sequence of primitive statements, too But only if the input does not affect

sequence

Ignore constant multiplierO(5) = O(5 * 1) = O(1)

Simple Loops

for (int i = 0; i < n.length; i++){}

-or-while (i < n) { i++; }

Each loop executed n times Primitive statements only within body

of loop Big –oh complexity of single loop iteration:

O(1)

Either loop runs O(n) iterations So loop has O(n) * O(1) = O(n) complexity

total

for (int i = 0; i < n.length; i++){}

int i = 0;

while (i < n) { i++; }

Add complexities of sequences to compute total

For this example, total big-Oh complexity is:= O(n) + O(1) + O(n)

= O(2 * n + 1)

= O(n + 1)

Loops In a Row

for (int i = 0; i < n.length; i++){}

int i = 0;

while (i < n) { i++; }

Add complexities of sequences to compute total

For this example, total big-Oh complexity is:= O(n) + O(1) + O(n)

= O(2 * n + 1)

= O(n + 1)

Loops In a Row

for (int i = 0; i < n.length; i++){}

int i = 0;

while (i < n) { i++; }

Add complexities of sequences to compute total

For this example, total big-Oh complexity is:= O(n) + O(1) + O(n)

= O(2 * n + 1)

= O(n)

Loops In a Row

More Complicated Loops

for (int i = 0; i < n; i += 2) { }

i 0, 2, 4, 6, ..., n

In above example, loop executes n/2 iterations

Iterations takes O(1) time, so total complexity:= O(n/2) * O(1)

= O(n * ½ * 1)

= O(n)

Really Complicated Loops

for (int i = 1; i < n; i *= 2) { }

i 1, 2, 4, 8, ..., n

In above code, loop executes log2 n iterations

Iterations takes O(1) time, so total complexity:= O(log2 n) * O(1)

= O(log2 n * 1)

= O(log2 n)

Really Complicated Loops

for (int i = 1; i < n; i *= 3) { }

i 1, 3, 9, 27, ..., n

In above code, loop executes log3 n iterations

Iterations takes O(1) time, so total complexity:= O(log3 n) * O(1)

= O(log3 n * 1)

= O(log3 n)

Math Moment

All logarithms are related, no matter the base Change base for an answer using constant

multiple But ignore constant multiple using big-Oh

notation So can consider all O(log n) solutions

identical

Nested Loops

for (int i = 0; i < n; i++){

for (int j = 0; j < n; j++) { }

}

Program would execute outer loop n times Inner loop run n times each iteration of

outer loop O(n) iterations doing O(n) work each

iteration So loop has O(n) * O(n) = O(n2) complexity

total Loops complexity multiplies when

nested

+

Only care about approximates on huge data sets Ignore constant multiples Drop lesser terms (& n! > 2n > n5 > n2 > n >

log n > 1) O(1) time for primitive statements to

execute Change by constant amount in loop:

O(n) time O(log n) time if multiply by constant in

loop Ignore constants: does not matter what

constant is When code is sequential, add their

complexities Complexities are multiplied when code is

nested

Your Turn

Get into your groups and complete activity

For Next Lecture

Read 2.4 for class on Wednesday How do we go about proving big-Oh

calculations

Week #5 weekly assignment due Tuesday Get started soon; I will be leaving at 12:30

today

Midterm #1 in class on Friday; start studying Think of questions to ask; may have review

time Wed.