reynolds 2006 complexity1 complexity analysis algorithm: –a sequence of computations that operates...

Reynolds 2006 Complexity 1

Complexity Analysis

• Algorithm: – A sequence of computations that operates on

some set of inputs and produces a result in a finite period of time.

• Representing algorithms:– Pseudocode

• Characterizing performance of algorithms:– Theta, or “Big-O” notation


Sequential Search Pseudocode

Read all the names into array NAMES.index = 0matchFound = falselocationOfMatch = -1 while( matchFound == false and index < length_of_NAMES_array ) do if( NAMES[ index ] == "Debbie Drawe" ) then matchFound = true locationOfMatch = index else index = index + 1endWhile if( matchFound == true ) then print( "Match found at name " & index )else print( "Match not found" )EndOfAlgorithm


Pair up!• Count the primitive operations in the

sequential search pseudocode

• What is the worst case scenario?

• Develop a formula for the total number of operations in a sequential search (worst case).


Sequential Search Performance• The while loop will dominate execution

time when the number of names in the list becomes large.– The number of iterations will increase

proportionally to the size of the list.

• The “order of growth” of time to search will be n, the number of names in the list.– Θ( n ) – Sequential Search has a Theta of n.


Sequential Search Performance

• What is best case performance?

• 1*(while loop time) + overhead

• What is average case performance?

• (n/2)*(while loop time) + overhead

• What is worst case performance?

• n*(while loop time) + overhead


Insertion Sort

• Given a set to be sorted, use the card player’s approach: – Add each new item at the appropriate place in

the set of already sorted items.


Insertion Sort PseudocodeRead all the numbers into array NUMS. numberIndex = 1sortedIndex = 0 while( numberIndex < length_of_NUMS_array ) do newNum = NUMS[ numberIndex ] sortedIndex = numberIndex - 1 /* From high to low, look for the place for the new number. If the previously sorted numbers are larger, move them up in the array NUMS. */ while( NUMS[ sortedIndex ] > newNum and sortedIndex >= 0 ) do NUMS[ sortedIndex + 1 ] = NUMS[ sortedIndex ] sortedIndex = sortedIndex - 1 endWhile /* We found the place for the new number, so insert it. */ NUMS[ sortedIndex + 1 ] = newNum numberIndex++ sortedIndex = numberIndex -1;endWhile EndOfAlgorithm


Performance of Insertion Sort• Execution time will increase with the size of

the set to be sorted (outer while loop)

• Each element to be sorted must be compared one or many times with the elements already sorted (inner while loop).


Insertion Sort Performance• What is best case performance?• Items already sorted• n*(outerWhile) + 1*(innerWhile) + overhead

• What is average case performance?• n*(outerWhile) + n*(n/2)*(innerWhile) + overhead

• What is worst case performance?• Items already sorted in reverse order

• n*(outerWhile) + ((n2 – n)/2)*(innerWhile) + overhead

• What is Theta?• Θ(n2).


Merge Sort• Divide the problem into many smaller problems

before recombining the elements for the full solution.– Use recursion to create the smaller problems (stay

tuned…)

• Imagine two piles of playing cards:– Each pile is sorted from smallest to largest, with the cards

face up in two piles.

– The two smallest cards are showing.

– The merge routine compares the two cards that are showing, and places the smaller card face down in the merged pile.


Merge Pseudocode• The following pseudocode takes:

– A single array NUMS[], which includes two sorted sub arrays.

– The first sorted subarray is NUMS[first] through NUMS[second-1]

– The second sorted subarray is NUMS[second] through NUMS[last]

• Merge is only the merge part of the merge sort.


Merge PseudocodeProcedure merge( NUMS[ ], first, second, last ) if( second == first ) return index1 = first index2 = second while( index1 < index2 and index2 <= last ) do /* If the number in the first array is smaller, leave it in that position, and check the next number in the first array. */ if( NUMS[ index1 ] <= NUMS[ index2 ] ) then index1 = index1 + 1 else /* If the number in the second array is smaller, save it, move the array elements up one, and put the number from the second array in place. */ temp = NUMS[ index2 ] shiftIndex = index2 while( shiftIndex > index1 ) do NUMS[ shiftIndex ] = NUMS[ shiftIndex - 1 ] shiftIndex = shiftIndex - 1 endWhile /* Insert the smaller element in the hole */ NUMS[ index1 ] = temp index1 = index1 + 1 index2 = index2 + 1 endIf endWhileendProcedure


Merge Sort• Use recursion to divide the problem into

subproblems.

• Merge sort repeatedly calls itself until the recursion "bottoms out" with lists whose lengths are one.

• Then the recursion "returns," reassembling the numbers in sorted order as it does.


Merge Sort Pseudocode

Procedure mergeSort( NUMS[ ], first, last ) if( first < last ) then middle = truncate( ( last + first ) / 2 ) call mergeSort( NUMS, first, middle ) call mergeSort( NUMS, middle + 1, last ) merge( NUMS, first, middle + 1, last )endProcedure

The NUMS[] array is the set of items to be sorted.The first subscript of the items to be sorted is first.The last subscript of the items to be sorted is last.


Following the Merge Sort for NUMS = { 1, 6, 4, 2 }

• Call mergeSort passing NUMS, 0, and 3.

• Middle = (0+3)/2 = 1.

• Call mergeSort again passing NUMS, 0, & 1

• Middle = (0+1)/2 = 0.

• Call mergeSort again passing NUMS, 0, and 0

• Since 0 is not less than 0, mergeSort simply returns


(cont.)Following the Merge Sort for NUMS = { 1, 6, 4, 2 }

• Call mergeSort passing NUMS,– middle + 1 (1) as first, and last (1)

• Since 1 is not less than 1, it simply returns

• Call procedure merge passing NUMS, – first (0) – middle + 1 (1) – and last (1)– This is a call to order the first two elements (1,6)



• Having completed ordering the front half of the list, the algorithm returns to call mergeSort passing NUMS, – middle + 1 (2) – last (3).

• mergeSort calculates middle as (2+3)/2 = 2

• Call mergeSort passing NUMS, 2, and 2

• Since 2 is not less than 2, it simply returns


(cont.)Following the Merge Sort for NUMS = { 1, 6, 4, 2 }• call mergeSort passing NUMS,

– middle + 1 (3) as first,

– last (3).

• Since 3 is not less than 3, it mergeSort returns • call procedure merge passing NUMS,

– first (2),

– middle + 1 (3), and

– last (3).

– This is a call to order the last two elements ( 4, 2 ).



• Having returned from ordering the front and back halves of the list, the algorithm calls merge passing NUMS, – first (0),

• List 1 consists of elements 0 & 1 (1, 6)

– middle + 1 (2), • List 2 consists of elements 2 & 3 (2, 4)

– and last (3).

• Merge completes by ordering the list {1, 2, 4, 6}.


Merge Sort Performance

• Total time T consists of the time to recursively solve 2 problems of half the size, and then to combine the results: T = 2T(n/2) + merge

• Since the time for merge is Θ(n):T = 2T(n/2) + Θ(n)

• How can we characterize T in general?


Merge Sort Recursion Tree Sum

Θ(n) Θ(n)

Θ(n/2) Θ(n/2) Θ(n)

Θ(n/4) Θ(n/4) Θ(n/4) Θ(n/4) Θ(n)

...

...


Total Merge Sort Run Time

• For each level of recursion:– Θ(n)

• How many levels of recursion?– log2( n )

– lg( n )– A problem with 8 numbers will have 3 levels

• What is the Theta of merge sort?– Θ( n( lg n ) )


What is the Significance of Θ( n( lg n ) ) vs. Θ( n2 )?

• Suppose 1 million numbers must be sorted – insertion sort will require ~(106)2, or

1,000,000,000,000 cycles – merge sort will require ~106( lg 106), or 106( 20 ),

or 20,000,000 cycles – Merge sort is faster by 5 orders of magnitude!


Binary Search• Sequential Search performs with Θ(n)

• If we know the list is ordered, we can search much more efficiently.– The difference between a phone book with

entries in sorted order and a phone book with entries in random order!

• Binary Search performs with Θ(lg n) – Works by repetitively dividing the list in half.– Search a list

25

Binary Search PseudocodeRead all the names into array NAMES.begin = 0end = length_of_NAMES_array - 1matchFound = falselocationOfMatch = -1searchItem = "Mark Waldman" while( matchFound == false and begin <= end ) do midpoint = truncate( ( begin + end ) / 2 ) if( NAMES[ midpoint ] == searchItem ) then matchFound = true locationOfMatch = midpoint else if( searchItem < NAMES[ midpoint ] ) then end = midpoint - 1 else begin = midpoint + 1 endIfendWhile if( matchFound == true ) then print( "Match found at name " & locationOfMatch )else print( "Match not found" )EndOfAlgorithm


Binary Search PerformanceWith each iteration, the binary search reduces the size of the list to be searched by a factor of 2.

So, the binary search will find the search item, or conclude that the search item is not in the list, when the algorithm has executed (lg n) iterations or fewer.

If there are 7 items in the list, the algorithm will complete in 3 iterations or fewer.

If there are 1,000,000 items in the list, the algorithm will complete in 20 iterations or fewer.

reynolds 2006 complexity1 complexity analysis algorithm: –a sequence of computations that operates...

Documents