yazici kalkan algorithmanalysis 2011 week2a
TRANSCRIPT
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
1/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 1
Analysis ofAlgorithm
Week 2a
Adnan YAZICISinan KALKAN
Dept. of Computer EngineeringMiddle East Technical Univ.
Ankara - TURKEY
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
2/33
Measure the height of a tall building
with a barometer
What would be your answer?
One student answered:
I would tie the barometer to the
end of a rope and release the rope
till it touches the ground. The
length of the rope then gives me the
height!
Of course, the instructor rejects theanswer since it doesnt include any
physics
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 2
Check the following for two different versions of the `legend:
http://www.snopes.com/college/exam/barometer.asp
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
3/33
Measure the height of a tall building
with a barometer (contd)
There are other ways:
1. Drop the barometer from the top, and measure the time
it takes to reach the ground.
2. Make a pendulum and time its frequency at the top and
the bottom of the building.
3. Walk down the stairs marking "barometer units" on the
wall.
4. Measure its shadow and the buildings shadow.
Workout the height of the building from barometers
height.
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 3
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
4/33
Analyze/Compare Algorithms
= Pros and Cons of Algorithms =
What are the disadvantages of these?
1. Drop the barometer from the top,
and measure the time it takes to
reach the ground.
2. Make a pendulum and time its
frequency at the top and the
bottom of the building.
3. Walk down the stairs marking
"barometer units" on the wall.
4. Measure its shadow and the
buildings shadow. Workout the
height of the building from
barometers height.
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 4
Cons:
1. You lose the barometer; it breaks.
2. You need a loooong rope and how
long that rope is going to be
depends on the answer to the
question.
3. It takes too long. It is too tiring.
4. What if there is no sun?
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
5/33
Where does the word algorithm
come from?
From a Persian mathematician, astronomer andgeographer: Mohammed ibn-Musa al-Khwarizmi
Algorithmi is the latin form of his name
He contributed to science by
Decimal positional number system(e.g., 32 = 101x3 + 100x2)
Presented the first systematic solutions to
linear and quadratic equations
In fact, the word Algebra comes from one of hisoperators (al-jabr: subtracting a number from both
sides of an equation) for solving equations
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 5
Mohammed ibn-Musa
al-Khwarizmi
(780-850)
Source: Wikipedia
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
6/33
Where does the word algorithm
come from? (contd)
al-Khwarizmi reduced equations to one of thefollowing six forms by using al-jabr (in Arabic:
restoring, completion):
squares equal roots (ax2 = bx)
squares equal number (ax2 = c) roots equal number (bx = c)
squares and roots equal number (ax2 + bx = c)
squares and number equal roots (ax2 + c = bx)
roots and number equal squares (bx + c = ax2)
For example,x2 = 40x 4x2 is reduced to 5x2 =
40x. From this reduced form, it is easily deducable
that the variable is either 0 or 8.
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 6
al-Khwarizmis
Compendious Book
on Calculation byCompletion and
Balancing
Source: Wikipedia
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
7/33
What does algorithm mean?
A procedure or formula for solving a
problem
A set ofinstructions to be followed tosolve a problem
an effective method expressed as a finite
list of well-defined instructions forcalculating a function
step-by-stepprocedure for calculations
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 7
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
8/33
A more formal definitionBetter
suited to us
Starting from an initial state and initial
input (perhaps empty), the instructions
describe a computation that, when executed,will proceed through a finite number of
well-defined successive states, eventually
producing "output" and terminating at afinal ending state.
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 8
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
9/33
A more formal definitionBetter
suited to us
There are three important aspects to thisdefinition:
Computation
Well-defined successive states
Terminate at a final state
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 9
Starting from an initial state and initial input (perhaps empty), the instructions describe a
computation that, when executed, will proceed through a finite number of well-defined
successive states, eventually producing "output" and terminating at a final ending state.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
10/33
Computation (cont.d)
Problem: Find temperature of the water if
A&B were mixed together.
Any suggestions on how to solve it?
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 10
A B
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
11/33
Computation
Digital vs. analog computation
Sequential vs. parallel computation
Batch vs. interactive computation Evolutionary, molecular, quantum
computation
Physical computation / Digital Physics The whole universe is itself a computation
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 11
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
12/33
Well-defined successive states
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 12
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
13/33
Terminate at a final state
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 13
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
14/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 14
Algorithms
An
algorithmis a set of instructions to be followed to solve aproblem. In other words, an algorithm is a complete set of rules
that transform the input into the output in a finite number ofsteps.
There can be more than one solution (more than onealgorithm) to solve a given problem.
An algorithm can be implemented using differentprogramming languages on different platforms.
An algorithm should correctly solve the problem.
e.g., for sorting, this means even if (1) the input is already
sorted, or (2) it contains repeated elements. Once we have a correct algorithm for a problem, we have todetermine the efficiency of that algorithm (computationalcomplexity).
i /
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
15/33
What properties/aspects
algorithms should have other than
performance?
In this course, we
concentrate on
performance.
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 15
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
16/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 16
Design and Analysis of Algorithms
Aspects of studying algorithms:
1. Designing algorithms: putting the pieces of the puzzles together,
choosing data structures,
selecting the basic approaches to the solution of theproblem,
The most popular design strategies are divide&conquer, greedy,dynamic prog., backtracking, and branch&bound.
2. Expressing and implementing the algorithm
Concerns are:
clearness conciseness
effectiveness
etc.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
17/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 17
Design and Analysis of Algorithms
3. Analyzing the algorithm
Algorithm analysis is assessing the time and space resourcesrequired by an algorithm as a function of the size of theproblem, without actually implementing the algorithm.
4. Compare UB and LB to see if your solution is good
enoughAnalyzing the algorithm gives us the upper and the lowerbound to solve the problem
5. Validate the algorithm
We show that the algorithm computes the correct answer forall possible legal (or given) inputs
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
18/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 18
Design and Analysis of Algorithms
6. Verifying the algorithm (or program)
An algorithm is said to be correct (verified) if, for every inputinstance, it halts with the correct output.
7. Testing algorithms
There are two phases; Debugging: The process of executing programs on sample data sets to
determine if faulty results occur, and if so, to correct them.
Debugging can only point to the presence of errors, but not to theirabsence
Edsger W. Dijkstra (19302002)
Profiling: the process of executing a correct program on various datasets and measuring the time (and space) it takes to compute the results.
http://en.wikipedia.org/wiki/1930http://en.wikipedia.org/wiki/2002http://en.wikipedia.org/wiki/2002http://en.wikipedia.org/wiki/1930 -
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
19/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 19
Algorithmic Performance
There are two aspects of algorithmic performance:
- Time & space
Time Instructions take time.
How fast does the algorithm perform?
What affects its runtime?
Space Data structures take space
What kind of data structures can be used?
How does choice of data structure affect the runtime?We will focus on time:
How to estimate the time required for an algorithm
How to reduce the time required
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
20/33
ANALYSIS OF ALGORITHMS
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 20
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
21/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 21
Analysis of Algorithms
Analysis of Algorithmsanalyze the efficiency of different methods of
solutions.
How do we compare the time efficiency of twoalgorithms that solve the same problem?
Nave Approach: implement these algorithms in aprogramming language (i.e., C++), and run them
to compare their time requirements.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
22/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 22
Analysis of Algorithms
Comparing the programs (instead of algorithms) has
difficulties.
What data should the program use?
Any analysis must be independent of specific data. Execution time issensitive to the amount of data manipulated, grows as the amount of dataincreases.
What computer should we use?
We should compare the efficiency of the algorithms independently of aparticular computer. Because of the execution speed of the processors, theexecution times for an algorithm on the same data set on two differentcomputers may differ.
How are the algorithms coded?
Comparing running times means comparing the implementations.
We should not compare implementations, because they are sensitive toprogramming style that may cloud the issue of which algorithm is inherentlymore efficient.
absolute measure for an algorithm is not appropriate.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
23/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 23
Analysis of Algorithms
Analyze algorithms independently of
specific implementations, computers, or data.
To analyze algorithms: First, we start to count the number of significant
operations in a particular solution to assess itsefficiency.
Then, we will express the efficiency of algorithms
using growth functions.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
24/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 24
What is Important?
An array-based list retrieve operation is O(1), a linked-
list-based list retrieve operation is O(n). But insert and delete operations are much easier on a
linked-list-based list implementation.
When selecting the implementation of an Abstract Data Type(ADT), we have to consider how frequently particular ADToperations occur in a given application.
If the problem size is always very small, we can probablyignore the algorithms efficiency.
We have to weigh the trade-offs between an algorithms timerequirement and its memory requirements.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
25/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 25
Analysis of Algorithms
A running time function, T(n), yieldsthe time required to execute the algorithm of
a problem of size n.
T(n) may contain unspecified constants, which
depend on the characteristics of the ideal
machine.
We cannot determine this function exactly.
T(n) = an2 + bn + c, where a, b and c are unspecified constants.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
26/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 26
The Execution Time of Algorithms (cont.)
Example: Simple If-Statement
Cost Times
if (n < 0) c1 1
absval = -n; c2 1
else
absval = n; c3 1
Total Cost
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
27/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 27
General Rules for Estimation
Loops
The running time of a loop is at most the running time of the
statements inside of that loop times the number of iterations.
Nested Loops
Running time of a nested loop containing a statement in theinner most loop is the running time of statement multiplied
by the product of the size of all loops.
Consecutive Statements
Just add the running times of those consecutive statements. If/Else
Never more than the running time of the test plus the larger
of running times of c1 and c2.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
28/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 28
The Execution Time of Algorithms
Example: Simple Loop
Cost Timesi = 1; c1 1
sum = 0; c2 1
while (i
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
29/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 29
The Execution Time of Algorithms (cont.)
Example: Nested Loop
Cost Timesi=1; c1 1
sum = 0; c2 1
while (i
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
30/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 30
Analysis of Algorithms
We measure the complexity of an algorithm by identifying a
basic operation and then counting how any times thealgorithm performs that basic operation for an input size n.
problem input of size n
searching a list lists with n elements
sorting a list lists with n elements
multiplying two matrices two n-by-n matrices
traversing a tree tree with n nodes
Towers of Hanoi n disks
Basic operation
comparisoncomparison
multiplication
accessing a node
moving a disk
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
31/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 31
Algorithm Growth Rates
An algorithms proportional time requirement isknown as growth rate.The growth rate of T(n) is referred to the
computational complexity of the algorithm.
The computational complexity gives a conciseway of saying how the running time, T(n),varies with n and is independent of any
particular implementation.
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
32/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 32
Definition of the Orders of an Algorithm
-Notation: Given two running time functions f(n) and g(n), we say that f(n)is (g(n)) if there exists a real constant c, and a positive integer n0, such
that f(n) c.g(n) for all n n0O-notation gives an upper bound for a function to within a constant factor.
Example: We want to show that 1/2n2+ 3n is O(n2)
f(n) = 1/2n2+ 3n g(n) = n2
to show desired result, need c and n0 such that 0 1/2n2 + 3n c.n2
try c =1
1/2n2 + 3n n2 3n 1/2n2 6 n i.e. n0 = 6.
cg(n)
f(n)
f(n) O(g(n))
Al i h G h R
-
8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a
33/33
Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 33
Algorithm Growth Rates
We can compare the efficiency of two algorithms bycomparing their growth rates. The lower the growth rate, the faster the algorithm, at least
for large values of n.
For example; T(n) = an2 + bn + c, the growth rate isO(n2)
The goal of the algorithm designer should be an
algorithm with as low a growth rate of the running timefunction, T(n), of that algorithm as possible.