Big OhData Structures and Algorithms

CS 244

Brent M Dingle PhD

Department of Mathematics Statistics and Computer Science

University of Wisconsin ndash Stout

Based on the book Data Structures and Algorithms in C++ (Goodrich Tamassia Mount)Some content derivedtaken from httpwwwstroustrupcomProgramming and some from Data Structures for Game Programmers (Penton)

This presentationrequires Sound Enabled

Things to Note

bull Homework 5 is Due Soon

bull Homework 6 is Posted on D2Lndash Do NOT delay in starting it

bull Do not forget to look at the Meta-Info files

From Last Timendash Templates

ndash Algorithmsbull Review

ndash Big-Ohbull Introductionbull Linear Searchbull Binary Search

For Today

bull More on Big-Ohndash This now will start to relate to

the official book for the class Chapter 4bull Data Structures amp Algorithms in C++ 2nd Edition

ndash Goodrich Tamassia Mount

bull Side Commentndash There are many ways to approach Big-Ohndash You will see several of the possible approachesndash Some are mathlogic based some more lsquohuman intuitionrsquo basedndash Some are faster less error prone and better than othersndash NONE are randomly guessing

Marker Slide

bull Any General Questions

bull Next upndash Searching Arrays

bull Summary Review from last time

ndash Big-Ohbull The Math side of things

ndash Asymptotic Analysis

bull The Code side of thingsndash Examples how to get the math function

Searching Arrays Linear Search and Binary Search

bull Searchingndash Find an element in a large amount of data

bull Typically checking if an array contains a value matching the sought elementrsquos key value

bull So far you have seen at leastndash Linear searching (sequential search)ndash Binary searching

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Things to Note

bull Homework 5 is Due Soon

bull Homework 6 is Posted on D2Lndash Do NOT delay in starting it

bull Do not forget to look at the Meta-Info files

From Last Timendash Templates

ndash Algorithmsbull Review

ndash Big-Ohbull Introductionbull Linear Searchbull Binary Search

For Today

bull More on Big-Ohndash This now will start to relate to

the official book for the class Chapter 4bull Data Structures amp Algorithms in C++ 2nd Edition

ndash Goodrich Tamassia Mount

bull Side Commentndash There are many ways to approach Big-Ohndash You will see several of the possible approachesndash Some are mathlogic based some more lsquohuman intuitionrsquo basedndash Some are faster less error prone and better than othersndash NONE are randomly guessing

Marker Slide

bull Any General Questions

bull Next upndash Searching Arrays

bull Summary Review from last time

ndash Big-Ohbull The Math side of things

ndash Asymptotic Analysis

bull The Code side of thingsndash Examples how to get the math function

Searching Arrays Linear Search and Binary Search

bull Searchingndash Find an element in a large amount of data

bull Typically checking if an array contains a value matching the sought elementrsquos key value

bull So far you have seen at leastndash Linear searching (sequential search)ndash Binary searching

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

From Last Timendash Templates

ndash Algorithmsbull Review

ndash Big-Ohbull Introductionbull Linear Searchbull Binary Search

For Today

bull More on Big-Ohndash This now will start to relate to

the official book for the class Chapter 4bull Data Structures amp Algorithms in C++ 2nd Edition

ndash Goodrich Tamassia Mount

bull Side Commentndash There are many ways to approach Big-Ohndash You will see several of the possible approachesndash Some are mathlogic based some more lsquohuman intuitionrsquo basedndash Some are faster less error prone and better than othersndash NONE are randomly guessing

Marker Slide

bull Any General Questions

bull Next upndash Searching Arrays

bull Summary Review from last time

ndash Big-Ohbull The Math side of things

ndash Asymptotic Analysis

bull The Code side of thingsndash Examples how to get the math function

Searching Arrays Linear Search and Binary Search

bull Searchingndash Find an element in a large amount of data

bull Typically checking if an array contains a value matching the sought elementrsquos key value

bull So far you have seen at leastndash Linear searching (sequential search)ndash Binary searching

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

For Today

bull More on Big-Ohndash This now will start to relate to

the official book for the class Chapter 4bull Data Structures amp Algorithms in C++ 2nd Edition

ndash Goodrich Tamassia Mount

bull Side Commentndash There are many ways to approach Big-Ohndash You will see several of the possible approachesndash Some are mathlogic based some more lsquohuman intuitionrsquo basedndash Some are faster less error prone and better than othersndash NONE are randomly guessing

Marker Slide

bull Any General Questions

bull Next upndash Searching Arrays

bull Summary Review from last time

ndash Big-Ohbull The Math side of things

ndash Asymptotic Analysis

bull The Code side of thingsndash Examples how to get the math function

Searching Arrays Linear Search and Binary Search

bull Searchingndash Find an element in a large amount of data

bull Typically checking if an array contains a value matching the sought elementrsquos key value

bull So far you have seen at leastndash Linear searching (sequential search)ndash Binary searching

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slide

bull Any General Questions

bull Next upndash Searching Arrays

bull Summary Review from last time

ndash Big-Ohbull The Math side of things

ndash Asymptotic Analysis

bull The Code side of thingsndash Examples how to get the math function

Searching Arrays Linear Search and Binary Search

bull Searchingndash Find an element in a large amount of data

bull Typically checking if an array contains a value matching the sought elementrsquos key value

bull So far you have seen at leastndash Linear searching (sequential search)ndash Binary searching

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Searching Arrays Linear Search and Binary Search

bull Searchingndash Find an element in a large amount of data

bull Typically checking if an array contains a value matching the sought elementrsquos key value

bull So far you have seen at leastndash Linear searching (sequential search)ndash Binary searching

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Sequential (Linear) Search

bull Begins at the start of the list and continues until the item is found or the entire list has been searchedndash Compare each array element with search key

bull If search key found return element indexbull If search key not found return ndash1 (invalid index)

ndash Works best for small or unsorted arraysndash Inefficient for larger arrays

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Binary Search on Ordered Lists

bull Uses a divide-and-conquer strategy to find an element in a list (eliminates half for each pass)ndash Compare middle array element to search key

bull If element equals key then return array indexbull If element is less than key then repeat search on first half of

arraybull If element is greater then key then repeat search on second

half of arraybull Continue search until

ndash Element equals search key (success)

ndash Efficient for large sorted arrays

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Searching Big-Oh Reference

bull Linear Searchndash O(n)

bull Binary Searchndash O(lg n)

bull yet to be provenshown howbull spoiler

ndash (integer-wise) lg n is the number of times n can be divided by 2

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slide

bull Any Questions Onndash Searching Arrays

bull Summary Review from last time

bull Next upndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons (input growth versus time growth)

bull The Code side of thingsndash Examples how to get the math function

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Definitionsbull An algorithm is a sequence of steps to solve a

problemndash The performance of an algorithm when implemented

on a computer may vary depending on the approach used to solve the problem and the actual steps taken

bull To compare the performance of algorithms computer scientists use big-O notationndash We will study several algorithms during this course

and analyze their performance using big-O notation

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Measuring Algorithm Efficiencybull There are different mechanisms for measuring efficiency

ndash Measure actual system characteristics (practical)bull processor time used bull memory size used bull and execution time (practical)

ndash Measure Time to develop the program (developer time)

ndash Analyze the number of operations an algorithm performs to measure its complexity (theoretical)

ndash You can analyze the space (memory or disk) that the algorithm uses to perform its computation (theoretical)

bull Often algorithms make a time vs space trade-offbull That is the algorithm may run faster if given more space

Most of this class will put focus here

>

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Best Worst and Average Casebull Few algorithms

ndash have the exact same performance every time bull performance of an algorithm depends on the size of the inputs it processes

bull The best case performance of the algorithm is the most efficient execution of the algorithm on the best data inputs

bull The worst case performance of the algorithm is the least efficient execution of the algorithm on the worst data inputs

bull The average case performance of the algorithm is the average efficiency of the algorithm on the set of all data inputs

bull The analysis of all cases typically ndash expresses efficiency in terms of the input size n of the data

Most of this class will put focus here ndash on worst case

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Used for What (again)

bull Big-O notation is a mechanism for quickly communicating the efficiency of an algorithm ndash Big-O notation measures the worst case

performance of the algorithm by bounding the formula expressing the efficiency

cg(n) bounds f(n)

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Asymptotic Execution Timebull Suppose f(n) is a formula describing the exact

execution run time of some algorithm for an input size of n

bull That algorithm is then O( g(n) )bull Big Oh of g of nbull Order g of n

bull If there exist constants c and n0 such thatbull f(n) le cg(n) for all n gt n0

Searching a dictionary is O( log n )Searching an unordered list is O( n )

For Examplehellip

cg(n) bounds f(n)

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

In other wordsa function f(n) is O(g(n)) if f(n) is bounded above by some constant multiple of g(n) for all large nSo f(n) le cg(n) for all n gt no

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Notationbull Given functions f(n) and g(n)

we say that f(n) is O(g(n)) if there are positive constantsc and n0 such that

f(n) cg(n) for n n0

bull Example 2n + 10 is O(n)ndash 2n + 10 cnndash (c 2) n 10ndash n 10(c 2)ndash Pick c = 3 and n0 = 10 1

10

100

1000

10000

1 10 100 1000n

3n

2n+10

n

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

7 Functions

bull Seven functions commonly used to examine the complexity of an algorithm arendash constant log2n n n log2n n2 n3 2n

bull These are listed in lowest complexity to highest complexityndash More details follow

In this class the subscript 2 may be omittedlog2n log n but in the context of this class log n should be taken to mean log base 2 of nunless explicitly stated otherwise

You may also see log2n abbreviated lg nNote your calculator will not understand this

Base Conversion Reminder

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

1)(nLog

n

)(nLogn

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Common Big-Oh Functionsbull Seven functions that often

appear in algorithm analysisndash Constant 1ndash Logarithmic log nndash Linear nndash N-Log-N n log nndash Quadratic n2

ndash Cubic n3

ndash Exponential 2n

)(nLogn

2n

3nn2

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Running Time Comparisons

just too big of a number to calculate

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Ranking of Execution Times

function common name

n factorial

2n exponential

nd d gt 3 polynomial

n3 cubic

n2 quadratic

n

n log n

n linear

square root

log n logarithmic

1 constant

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Constant Factors and Big Oh

bull With regard to Big Oh Analysisbull The growth rate is not affected by

ndash constant factors c or ndash lower-order terms

bull Examplesndash 100 n + 105 is a linear function O(n)

ndash 40 n2 + 108n is a quadratic function O(n2)

constant factor lower order term

constant factors lower order term

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

bull 7n ndash 2

bull 3n3 + 20n2 + 5

bull 3 (log n) + 5

Big Oh Examples Applying Definition

7n-2 is O(n)

need c gt 0 and n0 1 such that 7n-2 cbulln for n n0

this is true for c = 7 and n0 = 1

3n3 + 20n2 + 5 is O(n3)

need c gt 0 and n0 1 such that 3n3 + 20n2 + 5 cbulln3 for n n0

this is true for c = 4 and n0 = 21

3 log n + 5 is O(log n)

need c gt 0 and n0 1 such that 3 log n + 5 cbulllog n for n n0

this is true for c = 8 and n0 = 2

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysis

bull Next upndash Big-Oh

bull The Math side of thingsndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Relates to Growth Ratebull As Big-Oh notation is based on asymptotic analysis

ndash Big-Oh notation gives an upper bound on the growth rate of a function

ndash The statement ldquof(n) is O(g(n))rdquo means that the growth rate of f(n) is no more than the growth rate of g(n) (f(n) cg(n) for n n0 frsquo(n) c grsquo(n) )

bull Thusndash We can use the big-Oh notation

to rank functions according to their growth rate

f(n) is O(g(n)) g(n) is O(f(n))

g(n) grows more

Yes No

f(n) grows more No Yes

Same growth Yes Yes

Are we allowed to use derivativesin a Computer Science Class

YES

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

8 times longer = 38 = 24 minutes

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

64 times longer = 364 = 192 minutes

64 times LONGER

8 times LONGER

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Running Time Comparisons

ie 8n versus n

Say it takes 3 minutes for an input size of 10 (ie n = 10)

Now if we had an input size of 80 (ie 8n)How long would it take

3 1 = 3 minutessame time

8 times LONGER

64 times LONGER

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Running Time Comparisons

ie 8n versus n

And these follow in similar fashion

same time

8 times LONGER

64 times LONGER

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Summarizing Why Growth Rate Matters

if runtime is time for n + 1 time for 2 n time for 4 n

c lg n c lg (n + 1) c (lg n + 1) c(lg n + 2)

c n c (n + 1) 2c n 4c n

c n lg n~ c n lg n

+ c n2c n lg n +

2cn4c n lg n +

4cn

c n2 ~ c n2 + 2c n 4c n2 16c n2

c n3 ~ c n3 + 3c n2 8c n3 64c n3

c 2n c 2 n+1 c 2 2n c 2 4n

runtimequadrupleswhen problemsize doubles

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull Next upndash Big-Oh

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Examplebull Determine the performance using big-O notation for the

following code based on the number of cout executions

bull Determine the number of operations performedndash 1) The inner loop counts from 1 to n =gt n operationsndash 2) The outer loop counts from 1 to n =gt n operationsndash 3) The outer loop performs the inner loop n timesndash 4) Thus the cout statement is executed n n times = O(n2)

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Sigma Summationbull To more formally analyze algorithms we typically need to perform mathematical

summations

bull The summation sign is used to denote the summation of a large series of numbers The summation sign is the Greek letter sigma

bull Example

n

i

in1

321

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Sigma Summation Rulesbull The sum of numbers from 1 to n has a simple formula

bull Multiplied constants can be moved outside the summation

bull Added constants can be summed separately

2

)1(

1

nni

n

i

2

)1(333

11

nnii

n

i

n

i

n

i

n

i

n

i

n

i

n

i

nnn

iii11111

32

)1(133)3(

This would be a good formula tohave memorized

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Executes n times (worst case) Uses n units of time

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Constant time Mark it as using ONE unit of time

1

n n

math function

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

n is a lsquoconstantrsquo take it out of the summation

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = =

Simplifies to n

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Multiply out

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Example Using the Sumbull Using summations determine the Big-Oh

of the following code

for (int i =1 i lt= n i++)for (int j=1 j lt= n j++)

cout ltlt j ltlt endl

= = = =

Math function for this is n2

Algorithm is O(n2)

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notation

bull Next upndash Big-Oh

bull The Code side of thingsndash Practice Exercisesndash Book Example

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-O Exercises Warm Up1 What is the Big-Oh for the following formula

4n3 + 3n2 + 6nRemove lower order terms

Eliminate constants

O(n3)

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-O Exercises Warm Up2 What is the Big-Oh for the following formula

n + n (log n)Remove lower order terms

Eliminate constants

O(n log n)

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-O Exercises Code3 Analyze using Big-Oh and Summations

(count number of cout calls)

for (int i =1 i lt= n i++) for (int j=1 j lt= i j++) cout ltlt j ltlt endl

i = 1 inner loops runs 1 timei = 2 inner loops runs 2 timesi = 3 inner loops runs 3 timesi = 4 inner loops runs 4 times i = n inner loops runs n times

1 + 2 + 3 + 4 + hellip + n

sum119894=1

119899

119894=iquest 119899(119899+1)2

O(n2)frac12 n2 + frac12 n

Remove lower order termsEliminate constants

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-O Exercises Code4 Analyze using Big-Oh

and Summationsint i=1 j=1 while (i lt= 2n) cout ltlt i ltlt endl j=1 while (j lt= n2) cout ltlt j cout ltlt j2 ltlt endl j++ i++

Assume n = 8Outer loop runs 16 times Inner loop runs 4 times each time164 = 64 82

Assume n = 10Outer loop runs 20 times Inner loop runs 5 times each time205 = 100 102

Assume n = 9Outer loop runs 18 times Inner loop runs 4 times each time184 = 72 98 lt 81 = 92

Assume n = 1024Outer loop runs 2048 times Inner loop runs 512 times each time2048512 = 1048675 = 10242

Wait a minutehellip

Assume n = nOuter loop runs 2n times Inner loop runs n2 times each time2n n2 = n2 O(n2

)

>

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercises

bull Next upndash Big-Oh

bull The Code side of thingsndash Book Example

bull More Examples

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Rules (yet another refresher)

bull If is f(n) a polynomial of degree d then f(n) is O(nd) ie

1 Drop lower-order terms2 Drop constant factors

bull Use the smallest possible class of functionsndash Say ldquo2n is O(n)rdquo instead of ldquo2n is O(n2)rdquo

bull Use the simplest expression of the classndash Say ldquo3n + 5 is O(n)rdquo instead of ldquo3n + 5 is O(3n)rdquo

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Big-Oh Constant Time O(1)

bull We assume most primitive operationsndash addition multiplication assignment ops etc

bull Can be performed in constant time cndash The exact number does not really matter

bull Constants get knocked out during Big-Oh analysis

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Constant Time Loops

bull A loop that performs a constant number of iterations is still considered a constant

Initialize a deck by creating all 52 cardsDeckDeck() topCard = 0 for (int i = 1 i lt= 13 i++) Card c1(diamond i) c2(spade i) c3(heart i) c4(club i) cards[topCard++] = c1 cards[topCard++] = c2 cards[topCard++] = c3 cards[topCard++] = c4

Everything is constant in the loopAnd the loop runs from 1 to 13 always

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Computing Prefix Averages(stuff from the book)

bull We further illustrate asymptotic analysis with two algorithms for prefix averages

bull Definendash The i-th prefix average of an array X is the

average of the first (i + 1) elements of XA[i] = (X[0] + X[1] + hellip + X[i]) (i + 1)

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

Yet another way of looking at thingsCount the operationsThis initializes n elements

This will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] for j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This loop runs n timesThis will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do

s s + X[j]A[i] s (i + 1)

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes once for the first iteration of the outer loop (i == 1)Thus using 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes twice for the second iteration of the outer loop (i == 2)Thus using 2 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes 3 times for the third iteration of the outer loop (i == 3)Thus using 3 more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j]

A[i] s (i + 1) return A

This line executes (n-1) times for the last iteration of the outer loop (i == n-1)Thus using (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1)

return A

This line is just like the for-j loop it first uses 1 then 2 more then 3 more hellipand eventually uses (n-1) more operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A

This line executes once for each iterationof the outer loop (ie n times)So this will take a total of n operations

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

This line executes only one timeIt is outside both loopsSo this will take a total of 1 operation

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Quadratic)The following algorithm computes prefix averages in quadratic time by applying the definition

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+ (n

1)s s + X[j] 1 + 2 + 3 +hellip+ (n

1)A[i] s (i + 1) n

return A 1

Clearly either of these can be taken as the LARGEST number of operations

So we will use 1 + 2 + 3 + hellip+ (n-1)to characterize the runtime of this algorithmhellip see next page

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages End of 1st Method

bull The running time of prefixAverages1 isO(1 + 2 + hellip+ n)

bull The sum of the first n integers is n(n + 1) 2bull Thus algorithm prefixAverages1 runs in O(n2) time

Algorithm prefixAverages1(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers nfor i 0 to n 1 do n

s X[0] nfor j 1 to i do 1 + 2 + 3 +hellip+

(n 1)s s + X[j] 1 + 2 + 3 +hellip+

(n 1)A[i] s (i + 1) n

return A 1

This is from the formula you memorized

2

)1(

1

nni

n

i

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages 2nd Implementation

bull This is a transition slide

bull Do you feel things transitioning

bull Move onto the 2nd implementation of the Prefix Averages Code

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integerss 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i]A[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1)

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Largest of all these is n

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Prefix Averages (Linear)The following algorithm computes prefix averages in linear time by keeping a running sum

Algorithm prefixAverages2(X n)Input array X of n integersOutput array A of prefix averages of X operations

A new array of n integers ns 0 1for i 0 to n 1 do n

s s + X[i] nA[i] s (i + 1) n

return A 1

Algorithm prefixAverages2 runs in O(n) time

Largest of all these is n

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

So Many Ways to Big-Ohbull You have now seen a variety of ways to approach

calculating Big-Oh for various algorithms and source code

bull Why so many waysndash Are they really differentndash Or just different implementations of the same processndash Think of it as a ldquometardquo example of algorithm

implementationndash And hopefully at least one of the ways makes

ldquothe most senserdquo to you

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slidebull Any Questions On

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull Next upndash Big-Oh

bull More Examples

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

n

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 1

for ( i=0 i lt n i++ )m += i

Operations

nn

Answer O(n)

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 2

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j]

Operations

n

n2

n2

Answer O(n2)

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

i = 1hellip j-loop goes 1i = 2hellip j-loop goes 2i = 3hellip j-loop goes 3 i = (n-1) hellip j-loop goes (n-1)

1 +2 +3 + (n-1)

sum119894=1

119899minus 1

119894=(119899minus1 )119899

2

Operations

n

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 3

for ( i=0 i lt n i++ )

for( j=0 j lt i j++ ) m += j

Operations

n

1+2++n-1

1+2++n-1

= n2

Answer O(n2)

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

This looks tricky

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

i starts equal to n We divide by 2 until i lt= 0

So the real question is How many times can n be divided by 2

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

How many times can n be divided by 2

Or rather Find k such that n - 2k lt= 0

and k+1 will be the number of times thisloop executes

the +1 is by experiment n = 1 n ndash 20 = 0 loop executes 1 time n = 2 n ndash 21 = 0 loop executes 2 times

It really does not matter though -it will not change the asymptotic bound

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 4

int i = nwhile (i gt 0) tot += i i = i 2

Assume positive integersso 1 2 = 05 =gt 0

Answer O(lg n)

Find k such that n - 2k lt= 0n lt= 2k

lg n lt= k

k gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) + 1 times

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Equivalent of Coding example 4

i = 1while (i lt n) tot += i i = i 2

Answer O(lg n)

Find k such that 2k gt= nk gt= lg n

Pick the minimum k value = lg n

Loop executes (lg n) times

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

n

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 5

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) for( k=0 k lt n k++ ) sum[i][j] += entry[i][j][k]

Operations

nn^2n^3n^3

Answer O(n3)

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

n

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 6

for ( i=0 i lt n i++ ) for( j=0 j lt n j++ ) sum[i] += entry[i][j][0] for ( i=0 i lt n i++ ) for( k=0 k lt n k++ ) sum[i] += entry[i][0][k]

Calculate the total These 4 for-loops are the function

Operations

nn2

n2

nn2

n2

Answer n + n2 + n2 + n + n2 + n2

4n2 + 2n 4n2

O(n2)

Showing more of the implied steps for illustration

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

n

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 7

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(n) j++ ) m += j

Operations

nn32

n32

Answer O(n32)

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 8

for ( i=0 i lt n i++ ) for( j=0 j lt sqrt(995) j++ ) m += j

Operations

n31n31n

Answer O(n)

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 8 Equivalent code

for ( i=0 i lt n i++ ) m += j m += j m += j hellip m += j 31 times

Answer O(n)

Operations

n

nnn

n

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

First rearrange the function to be after main- cosmetic reasons

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int n ) int subtotal = 0 for( i=0 i lt n i++) subtotal += i return subtotal

Coding example 9Change the parameter nameused in the total() function

m

m

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( i=0 i lt m i++) subtotal += i return subtotal

Coding example 9Change the loop variableused in the total function

k k kk

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) tot += total(i)

int total(int m ) int subtotal = 0 for( k=0 k lt m k++) subtotal += k return subtotal

Nextinline the total function

m becomes i

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Coding example 9main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Graded In-Class Activity BigOhEx09main() int tot = 0 for ( i=0 i lt n i++ ) int subtotal = 0 for( k=0 k lt i k++) subtotal += k tot += subtotal

Operations

1n

n1 + 2 + 3 + hellip + (n-1)1 + 2 + 3 + hellip + (n-1)

n

Create a MS-Word document named BigOhEx09docx Copy the above code and operations table into it Complete the exercise by including the corresponding Sigma Summation its simplification and the final Big-Oh runtime for this codeSubmit the document to the appropriate D2L dropbox before class ends

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Marker Slidebull Any Questions on

ndash Searching Arraysndash Big-Oh

bull The Math side of thingsndash Asymptotic Analysisndash Growth Rate Comparisons

bull The Code side of thingsndash Nested For-Loops Sigma Notationndash Practice Exercisesndash Book Example

bull More Examples

bull Next upndash Free Play

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

Free Play ndash Things to Work On

bull Graded In-Class Activitybull Homework 5bull Homework 6

bull Various In-Class Activities to revisit

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End

The End

bull Or is it

• Big Oh
• Things to Note
• From Last Time
• For Today
• Marker Slide
• Searching Arrays Linear Search and Binary Search
• Sequential (Linear) Search
• Binary Search on Ordered Lists
• Searching Big-Oh Reference
• Marker Slide (2)
• Definitions
• Measuring Algorithm Efficiency
• Best Worst and Average Case
• Big-Oh Used for What (again)
• Asymptotic Execution Time
• Big-Oh Notation
• Big-Oh Notation (2)
• 7 Functions
• Common Big-Oh Functions
• Common Big-Oh Functions (2)
• Running Time Comparisons
• Ranking of Execution Times
• Constant Factors and Big Oh
• Big Oh Examples Applying Definition
• Marker Slide (3)
• Big-Oh Relates to Growth Rate
• Running Time Comparisons (2)
• Running Time Comparisons (3)
• Running Time Comparisons (4)
• Running Time Comparisons (5)
• Summarizing Why Growth Rate Matters
• Marker Slide (4)
• Big-Oh Example
• Sigma Summation
• Sigma Summation Rules
• Example Using the Sum
• Example Using the Sum (2)
• Example Using the Sum (3)
• Example Using the Sum (4)
• Example Using the Sum (5)
• Example Using the Sum (6)
• Example Using the Sum (7)
• Example Using the Sum (8)
• Example Using the Sum (9)
• Example Using the Sum (10)
• Example Using the Sum (11)
• Example Using the Sum (12)
• Marker Slide (5)
• Big-O Exercises Warm Up
• Big-O Exercises Warm Up (2)
• Big-O Exercises Code
• Big-O Exercises Code (2)
• Marker Slide (6)
• Big-Oh Rules (yet another refresher)
• Big-Oh Constant Time O(1)
• Constant Time Loops
• Computing Prefix Averages (stuff from the book)
• Slide 58
• Slide 59
• Slide 60
• Slide 61
• Slide 62
• Slide 63
• Slide 64
• Slide 65
• Slide 66
• Slide 67
• Slide 68
• Prefix Averages End of 1st Method
• Prefix Averages 2nd Implementation
• Slide 71
• Slide 72
• Slide 73
• Slide 74
• Slide 75
• Slide 76
• Slide 77
• Slide 78
• Slide 79
• So Many Ways to Big-Oh
• Marker Slide (7)
• Coding example 1
• Coding example 1 (2)
• Coding example 1 (3)
• Coding example 1 (4)
• Coding example 2
• Coding example 2 (2)
• Coding example 2 (3)
• Coding example 2 (4)
• Coding example 2 (5)
• Coding example 3
• Coding example 3 (2)
• Coding example 3 (3)
• Coding example 3 (4)
• Coding example 3 (5)
• Coding example 4
• Coding example 4 (2)
• Coding example 4 (3)
• Coding example 4 (4)
• Equivalent of Coding example 4
• Coding example 5
• Coding example 5 (2)
• Coding example 5 (3)
• Coding example 5 (4)
• Coding example 5 (5)
• Coding example 5 (6)
• Coding example 6
• Coding example 6 (2)
• Coding example 6 (3)
• Coding example 6 (4)
• Coding example 6 (5)
• Coding example 6 (6)
• Coding example 6 (7)
• Coding example 7
• Coding example 7 (2)
• Coding example 7 (3)
• Coding example 7 (4)
• Coding example 7 (5)
• Coding example 8
• Coding example 8 (2)
• Coding example 8 (3)
• Coding example 8 (4)
• Coding example 8 (5)
• Coding example 8 Equivalent code
• Coding example 9
• Coding example 9 (2)
• Coding example 9 (3)
• Coding example 9 (4)
• Coding example 9 (5)
• Coding example 9 (6)
• Coding example 9 (7)
• Coding example 9 (8)
• Coding example 9 (9)
• Coding example 9 (10)
• Coding example 9 (11)
• Graded In-Class Activity BigOhEx09
• Marker Slide (8)
• Free Play ndash Things to Work On
• The End