cs 221
DESCRIPTION
CS 221. Analysis of Algorithms Instructor: Don McLaughlin. Algorithms. What are they and Why do we care?. Algorithms. “A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output” - PowerPoint PPT PresentationTRANSCRIPT
Algorithms
“A well defined computational procedure that takes some value or set of values, as input and produces some value or set of values as output”
“A sequence of computational steps that transforms input to the output”
From Cormen, Leiserson, Rivest and Stein
Algorithms
“Algorithms + Data Structures = Programs”
Niklaus Wirth, Swiss Computer Scientist
Or, perhaps- Programs = Programming_language(Algoritms +
Data Structures)
Algorithms
So you might think of an algorithm as the equation behind a function
y=f(x)
Just as an equation is a computational procedure for creating a certain output from a certain input
Algorithms “A sequence of unambiguous instructions for
solving a problem, i.e. for obtaining a required output for any legitimate input in a finite amount of time” From Levitin, 2007
Algorithms
So, why do we want to study them? Because there may be many algorithms
intended to solve a problem… …, some are correct and some are not
correct …and of the correct ones some are
better than others (as you will see)
Algorithms
We are largely going to be concerned about the efficiency of algorithms What do we mean by efficiency?
Memory
Time
Algorithms
Mostly we are going to explore the efficiency of algorithms in terms of run time (T)
In particular we are going to study the growth of algorithms in relation to the size of the problem
Theoretical Analysis of Algorithms
To do this we need a few things- A computational model (what is the
computer) A language for expressing the algorithm A metric and methodology for measuring
the performance and efficiency of an algorithm
Theoretical Analysis of Algorithms
RAM – Random Access Machine Model essentially Von Neumann architecture A CPU… …with a fixed and finite set of primitive
instructions …which run in a fixed amount of time …connected to a bank of memory… And the CPU can read or write any
memory location in one primitive operation
Theoretical Analysis of Algorithms
RAM – Random Access Machine Model Primitive operations might be
Assign a value to variable (memory location)
Call a method or subroutine Arithmetic operation Compare two values Access elements of an array via index Return from method Return from algorithm
Theoretical Analysis of Algorithms
Language for expressing an algorithm Programming Language
C, VB, Java,… Flowcharts
Theoretical Analysis of Algorithms
Language for expressing an algorithm Pseudocode
Natural language expression of algorithm Really mix of natural language and
language from programming language constructs
Intended for human consumption, not computers
Theoretical Analysis of Algorithms Pseudo-code – has some rules
Algorithm declaration – must name algorithm and its parameters
Must define range of input, and domain of output Expressions – use <- for assignment, = for logical
comparison Scope denoted by indentation Decision structures
If – Then – Else While – Do loops Repeat – until loops For loops
Method calls Method returns (including return from algorithm)
Theoretical Analysis of Algorithms Pseudo-code – an example arrayMax – find the largest value in an array of
values
Algorithm arrayMax(A, n)Input:An array A storing n >= 1 integers)Output: the maximum value in the array AcurrentMax <- A[0]for i <-1 to n – 1 do
if currentMax < A[i] thencurrentMax <- A[i]
return currentMax
Theoretical Analysis of Algorithms
Insertion-Sort
Algorithm Insertion-sort(A, n)Input:An array A storing n >= 1 integers)Output: the array A sorted such that A0 < A1 < A2 < …An
for j <- 2 to ndo key <- A[j]#insert A[j] into the sorted sequencei <- j – 1while i > 0 and A[j] > key do
A[j +1] <- A[j]i <- i -1
A[i + 1] <- keyreturn A
Theoretical Analysis of Algorithms
So now the question is - how do we determine the run-time T of these algorithms?
Keep in mind that we need to understand the run-time of algorithms in the context of the size of the problem …which we will express as n
So we are looking for T(n) … for any given algorithm
Theoretical Analysis of AlgorithmsPrimative Cost
Algorithm arrayMax(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0
currentMax <- A[0] 2
for i <- 1 to n – 1 do (init loop counter) (check loop boundary) (increment loop counter)
112
if currentMax < A[i] then 2
currentMax <- A[i] 2
return currentMax 1
Theoretical Analysis of AlgorithmsPrimitive Cost
Algorithm arrayMax(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0
currentMax <- A[0] 2
for i <- 1 to n – 1 do (init loop counter) (check loop boundary) (increment loop counter)
112
1nn-1
if currentMax < A[i] then 2 n-1
currentMax <- A[i] 2 0(n-1)
return currentMax 1 1
Theoretical Analysis of Algorithms
So T(n) for arrayMax is
T(n) = 2 + 1 + n + 4(n-1) + 1 = 5n
T(n) = 2 + 1 + n + 6(n-1) + 1 = 7n-2
Theoretical Analysis of Algorithms
So T(10) for arrayMax is
At least 5(10) = 50
Or
No worse than 7(10)-2 = 68
Theoretical Analysis of Algorithms
Primitive Cost
Algorithm arrayMax(A, n) 0 Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0 currentMax <- A[0] C1
for i <- 1 to n – 1 do C2
if currentMax < A[i] then C3
currentMax <- A[i] C4
return currentMax C5
Theoretical Analysis of Algorithms
Primitive Cost
Algorithm arrayMax(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0
currentMax <- A[0] C1
for i <- 1 to n – 1 do C2
if currentMax < A[i] then C3
currentMax <- A[i] C4
return currentMax C5
Theoretical Analysis of Algorithms
Primitive Cost
Algorithm arrayMax(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0
currentMax <- A[0] C1 1
for i <- 1 to n – 1 do C2 n-1
if currentMax < A[i] then C3 n-1
currentMax <- A[i] C4 n-1
return arrayMax C5 1
Theoretical Analysis of AlgorithmsPrimative Cost
Algorithm arrayMax(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0
currentMax <- A[0] C1
for i <- 1 to n – 1 do C2 n-1
if currentMax < A[i] then C3 n-1
currentMax <- A[i] C4 n-1
return arrayMax C5 1
So, what is the run-time T(n) for the algorithm arrayMax?
T(n)
T(n) = c1 + c2(n-1) + c3(n-1) + c4(n-1) + 1
T(n) = c1 + (n-1)(c2 + c3 +c4) + 1
Theoretical Analysis of AlgorithmsPrimitive Cost
Algorithm arrayMax(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: the maximum value in the array A 0
currentMax <- A[0] C1 1
for i <- 1 to n – 1 do C2 n-1
if currentMax < A[i] then C3 n-1
currentMax <- A[i] C4 n-1
return A C5 1
So, what would T(n) be for best-case?
T(n) = c1 + c2(n-1) + c3(n-1) + 1c5
T(n) = c1 + (n-1)(c2 + c3) + 1c5
Theoretical Analysis of Algorithms
Insertion-Sort
Algorithm Insertion-sort(A, n)Input:An array A storing n >= 1 integers)Output: the array A sorted such that A0 < A1 < A2 < …An
for j <- 2 to ndo key <- A[j]#insert A[j] into the sorted sequencei <- j – 1while i > 0 and A[j] > key do
A[j +1] <- A[j]i <- i -1
A[I + 1] <- keyreturn A
Theoretical Analysis of AlgorithmsPrimative Cost
Algorithm Insertion-Sort(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: array A sorted such that a0<a1<…an 0
for j <- 2 to n C1
do key <- A[j] C2
#insert A[j] into the sorted sequence 0
i <- j – 1 C4
while i > 0 and A[j] > key do C5
A[j +1] <- A[j]C6
i <- i -1C7
A[I + 1] <- keyC8
return A 1
Theoretical Analysis of AlgorithmsPrimative Cost
Algorithm Insertion-Sort(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: array A sorted such that a0<a1<…an 0
for j <- 2 to n C1 n
do key <- A[j] C2 n-1
#insert A[j] into the sorted sequence 0 n-1
i <- j – 1 C4 n-1
while i > 0 and A[j] > key do C5 Σ(tj)
A[i +1] <- A[i]c6 Σ(tj-1)
i <- i -1C7 Σ(tj-1)
A[i + 1] <- keyC8 n-1
return A
Theoretical Analysis of Algorithms
T(n) for Insertion-Sort
T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1)
So… what is the best-case scenario (shortest run-time)?
Theoretical Analysis of Algorithms
T(n) for Insertion-Sort
Ok, that would mean what in terms of the execution of the algorithm?
… that tj = 1 for all j
…therefore…… Σ(tj) = n – 1
…and c6 and c7 drop out
Theoretical Analysis of Algorithms
T(n) for Insertion-Sort
So, T(n) best-case is
T(n) = c1n + c2(n-1) + c4(n-1) + c5(n-1) + c8(n-1)
= c1n + (c2 + c4 + c5 + c8) (n-1)
= (c1+c2+ c4+c5+c8)n-(c2+c4+c5+c8)
Theoretical Analysis of AlgorithmsPrimative Cost
Algorithm Insertion-Sort(A, n) 0
Input:An array A storing n >= 1 integers) 0
Output: array A sorted such that a0<a1<…an 0
for j <- 2 to n C1 n
do key <- A[j] C2 n-1
#insert A[j] into the sorted sequence 0 n-1
i <- j – 1 C4 n-1
while i > 0 and A[j] > key do C5 Σ(tj)
A[i +1] <- A[i]c6 Σ(tj-1)
i <- i -1C7 Σ(tj-1)
A[i + 1] <- keyC8 n-1
return A
Theoretical Analysis of Algorithms
T(n) for Insertion-Sort
T(n) = c1n + c2(n-1) + c4(n-1) + C5Σ(tj) + c6Σ(tj-1) + c7Σ(tj-1) + c8(n-1)
So… what is the worst-case scenario (longest run-time)?
Theoretical Analysis of Algorithms
By the way - rules of thumb (for now)
Σnj=2j = n(n+1)/2 - 1
Σnj=2(j-1) = n(n-1)/2
Theoretical Analysis of Algorithms
So, the worst-case
T(n) = c1n + c2(n-1) + c4(n-1) + C5(n(n+1)/2-1) + c6(n(n-1)/2) + c7(n(n-1)/2) + c8(n-1)