FIRST QUESTIONS FOR ALGORITHM ANALYSIS

WHAT IS AN ALGORITHM?

From the text (p. 3):“An algorithm is a sequence of unambiguous instructions for solving a problem, i.e., for obtaining a required output for any legitimate input in a finite amount of time.”

From the preceding line:“Although there is no universally agreed-on wording to describe this notion, there is general agreement about what the concept means:”

PROBLEMS WITH THE WORDING

What do you see as problematic???

Some might include:

• Unambiguous• Instructions• Solving• Problem• Finite time

Is this the only discipline with difficult primitives?

• Geometry: Point, line, and plane; axioms (Euclid, Reimann, others)

• Set Theory: Set

YET ANOTHER PERSPECTIVE

Set and its members

Set operations:

• union, • intersection, • complement, (problem of having

universe)• cross product of two sets

Relation

Function

• Special type of relation• Sometimes expressed by a “rule” of

relationship

Could algorithms simply be functions?

COMPARE ALGORITHMS AND FUNCTIONS

Sequentiality

Lack of ambiguity

Instructions

Solving

Problem

Input – Output relationship

Finiteness of time

Uniqueness of expression

Computation

SOME COMMON PROBLEMS

???

Sorting

• What does it mean formally?

Searching

String processing

• Matching• Translation

Graph

• TSP• Connectedness

Combinatorial

• Optimization: TSP, QAP• Constraint satisfaction: SAT, Queen Placement,

Knight’s Tour

Geometric

• Path planning

Numerical (CSC 340)

Etc.

DESCRIBING ALGORITHMIC BEHAVIOR

Strategic

• Greedy• Brute force

• Enumeration (exhaustive)

• Graph• DFS, BFS• Connectedness, planarity, etc.

• Divide and conquer• Dynamic programming• Backtracking

• Branch and bound

• Recursive• Domain transformation

Operational

• Deterministic versus randomized• Exact (find an optimum or find a compliant)• Approximate (estimate pi)• Heuristic (choose quickly, WTA)

Efficiency

• Space• Time

http://en.wikipedia.org/wiki/List_of_algorithms

MEASUREMENT UNITS

Space

• Input magnitude (the number of bits needed to represent the input

• Number of inputs• Space required to store the inputs, perform intermediate

computations, and store outputs

Time

BIG O(.)

Definition of g(n) ε O(f(n)).

The function g(n) is in O(f(n)) if there exist non negative constants c and such that

g(n) ≤ c f(n) for all n ≥ .

FIND ALGORITHMS THAT ARE O(F(N))

DO THINGS EVER GET WORSE?

Is the O() = Φ?

Are there decision questions that cannot be computed by any algorithm?

OTHER BOUNDS

Definition: g(n) ε Ω(f(n)). What does it mean to be bounded below?

• The function g(n) is in Ω(f(n)) if there exist constants c>0 and nonnegative such that

g(n) ≥ c f(n) for all n ≥ .

Definition: g(n) ε Θ(f(n)). What does it mean to be bounded above and below?

• The function g(n) is in Θ(f(n)) if there exist non negative constants c1, c2 and such that

c1 f(n) ≤g(n) ≤ c2 f(n) for all n ≥ .

EXAMPLE WITH F(N)=N*N

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96 101 106 111 116 121 1260

5000

10000

15000

20000

25000

g(n)

c1*f(n)

c2*f(n)

ANALYZING NON-RECURSIVE ALGORITHMS

How should input size be measured?

• Number of items in a list• Value of a parameter or the number of bits needed to store

the parameter

What is the program’s most frequently occurring operation?

• Look inside loops

Are there other factors than input size that affect the frequency of the most frequent operation?

• Separate best, worst and average case analyses

How to tally the frequency?

• Summations• Profiling

EXAMPLE 1

Find max

Given a list containing n>0 integers (It is a separate question to ask the time for set-up. What would it be for this example?)

Pseudo-code

max = list[0]

for i = 0 to n-1

if list[i]>max

max = list[i]

Analysis?

EXAMPLE 1 ANALYSIS

max = list[0]

• O(1) = load + store

if…

• Evaluate condition (a difference)• Conditional transfer (a jump, branch or skip depending upon

flag values)• Assignment = load + store• O(??)

for…

• Iterates n times• Total accumulated time = ?

ANALYZING RECURSIVE ALGORITHMS

To understand recursion one must first understand recursion

Questions

• How will one measure input size?• What is the core operation?• Do other factors than input size influence the number of

repetitions?• What is a recurrence expressing the repetition of the core

operation?• Solve the recurrence?

EXAMPLE 2:A RECURSIVE ALGORITHM

import random

import math

c = [0, 0]

alist=[]

n = int(input('Number of items for list = ' ))

for i in range(n):

x = random.randint(1, 1000)

alist.append(x)

##print(len(alist))

##print('The initial list = ', alist)

def recursfindmax(alis):

print(alis)

if len(alis) == 1:

c[0]+=1

return alis[0]

else:

c[0]+=1

amax = max(alis[0], recursfindmax(alis[1:len(alis)]))

return(amax)

def findmax(alis):

amax = alis[0]

for i in range(len(alis)):

c[1]+=1

if amax < alis[i]:

amax = alis[i]

return (amax)

print('Recursive max = ', recursfindmax(alist))

print('Iterative max = ', findmax(alist))

print('Count recursive, iterative ', c)

ANALYSISt(n) = 1 + t(n-1)

t(n-1) = 1 + t(n-2)

t(n-2) = 1 + t(n-3)

t(n-3) = 1 + t(n-4)

…

t(n-(n-2)) = 1 + t(n-1)

t(n-n-1)) = t(1) = 1; this is the anchor

t(n) = 1 + t(n-1) = 2 + t(n-2) = 3 + t(n-3) = …= (n-1) + t(n-(n-1))= n

COMPARISON

Both methods are O(n), where n is the number of items in the list

How do the algorithms compare with respect to space utilization?

FUN FACT

Given a homogeneous, second order, linear recurrence with constant coefficients (a≠0) of the form

a y[n] + b y[n-1] + c y[n-2] = 0

The recurrence has a closed form solution depending upon the roots and of the characteristic equation

and the initial conditions for y[0] and y[1]

FUN FACT (CONT’D)If the roots are real and distinct

If there is a repeated root

If the roots are imaginary

, and

CONSIDER THE FIBONACCI NUMBER RECURRENCE

Fib(0) = 0

Fib(1) = 1

Fib(n) = Fib(n-1) + Fib(n-2)

Produces the homogeneous recurrence:

Fib(n) – Fib(n-1) – Fib(n-2) = 0

With characteristic equation:

Where a = 1, b = -1 and c = -1

SOLVING THE EQUATION

Thus,

The roots are real and distinct so that the recurrence is expressed by

APPLYING INITIAL CONDITIONS

Recall that Fib(0) = 0, Fib(1) = 1, and

Applying the first condition yields:

Applying the second condition yields:

Since , the second becomes:

Thus, and so that

NOTES

Given

Recall and note that,

Thus, and

COUNTING ADDITIONS IN RECURSIVE FIBONACCI

A(n) = 1 (addition)+1 A(n-1)+ 1A(n-2)=1 + 1 +A(n-2)+A(n-3)+ A(n-2)= 2 + 2 A(n-2) + 1 A(n-3)= 2 + 2(1+A(n-3)+A(n-4))+A(n-3)= 4 + 3 A(n-3) + 2 A(n-4)= 4 + 3(1+A(n-4)+A(n-5))+2A(n-4)= 7 + 5 A(n-4)+3 A(n-5)= 7 + 5(1 + A((n-5)+A(n-6))+ 3 A(n-5)= 12 + 8 A(n-5) + 5 A(n-6)= …= (Fib(i+2) -1)+Fib(i+1)A(n-i)+Fib(i)A(n-(i+1))

Letting i = n-1, yields A(n) = Fib(n+1)-1,since A(0)=A(1)=0

EMPIRICAL VERFICIATIONc=[0,0,0,0,0]

def fib(n):

c[0]+=1

if n>1:

c[1]+=1

return fib(n-1)+fib(n-2)

elif n==0:

c[2]+=1

return 0

else:

c[3]+=1

return 1

c[4]+=1

print(fib(int(input("Supply a number: "))))

print(c)