comp 2402/2002 abstract data types and algorithms prof: office: email: office hours: eduardo mesa hp...
TRANSCRIPT
![Page 1: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/1.jpg)
COMP 2402/2002Abstract Data Types and
AlgorithmsProf:Office:Email:
Office hours:
Eduardo Mesa
HP 5347
[email protected] and Thursday 4:30pm – 5:30pm
Web Site: http://people.scs.carleton.ca/~eamesaba/
and also WebCT
Textbook:Open Data Structures (in Java). The pdf can be downloaded from the website
![Page 2: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/2.jpg)
TAs
Andrew Trenholm
• No office hours for TAs
Tawfic Abdul-Fatah
• Prompt answer to questions in:− Web-CT discussion − Carleton Computer Science Society forum.
![Page 3: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/3.jpg)
Evaluation
AssignmentsMidterm:Final Exam:Active Participation:
30%
30%
40%
5%
3 assignments 10% each
2 Midterms 15% each
bonus
![Page 4: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/4.jpg)
Assignments
• 1 Theory assignment
• 2 Programming Assignments
− Must be handled first thing in class.− All pages must be stapled
No late assignment will be accepted.
− Must be uploaded on Web CT− Make a folder (<Student ID>_<First Name>_<Last Name>_ <Assignment #>)− Put all the source files in the folder− Add a text file with your Id and full name− Zip the folder
![Page 5: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/5.jpg)
End of the Introduction
Begining of the Lecture
![Page 6: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/6.jpg)
a data storage format that can contain a specific type or range of values characterized by a set of operations that satisfy a set of specific properties.
Data Type
![Page 7: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/7.jpg)
ExampleData Type Int
Range of Value: -(231) to (231)
Properties:
•Symmetry: a+b = b+a, a*b = b*a •Associative a+(b+c) = (a+b)+c•Definition: a/b , b must not be 0
Operations: +, -, *, /
![Page 8: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/8.jpg)
Data Structures
How to organize data to be able to perform operations on those data efficiently.
A variable is the simplest data structure.
![Page 9: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/9.jpg)
ExampleElectronic Phone Book
Contains different DATA:- Names- phone numbers- addresses
Need to perform certain OPERATIONS:- add- delete- look for a phone number- look for an address
![Page 10: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/10.jpg)
ExampleElectronic Phone Book
How to organize the data so to optimize the efficiency of the
operations
AB
XZ
List Binary Search Tree Dictionary
![Page 11: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/11.jpg)
ExampleFinding the best route for an email
message in a network
Contains DATA:- Network + Traffic
Need to perform certain OPERATIONS:- Find the best route
12
217
18
8
10
43
16
149
![Page 12: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/12.jpg)
ExampleElectronic Phone Book
How to represent the data
Adjacency Matrix Adjacency List
![Page 13: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/13.jpg)
Abstract Data Type (ADT)(interfaces)
Define what operation can be done.
Define how each operation is performed.
Implementations(algorithms)
A same ADT could have several different implementations.
![Page 14: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/14.jpg)
Lucy
GetBook ( Book b )
Binary Search
ADT (Shelf)Peter
Random Search
![Page 15: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/15.jpg)
•Identify your data•Identify the operations you need to perform (and how often each operation is performed)
So to perform the operations efficiently we need:
Data Structures
•Define efficiency•Choose the best structure for your data.
![Page 16: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/16.jpg)
AlgorithmInput Output
Analysis of Algorithms
An algorithm is a step-by-step procedure forsolving a problem in a finite amount of time.
Analyze an algorithm = determine its efficiency
![Page 17: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/17.jpg)
Efficiency ?Running time …
Memory …
Quality of the result ….
Simplicity ….
Generally, while improving the efficiency in one of these aspects we diminish the efficiency in the others
![Page 18: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/18.jpg)
Running time
0
20
40
60
80
100
120
Ru
nn
ing
Tim
e
1000 2000 3000 4000
Input Size
Best caseAverage caseWorst caseThe running time depends
on the input size
It also depends on the input data:Different inputs can have different running times
![Page 19: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/19.jpg)
19
Running Time of an algorithm
• Average case time is often difficult to determine.
• We focus on the worst case running time.– Easier to analyze– Crucial to applications
such as games, finance and robotics
0
20
40
60
80
100
120
Runnin
g T
ime
1000 2000 3000 4000
Input Size
best caseaverage caseworst case
![Page 20: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/20.jpg)
20
If x is odd return xIf x is even
compute the sum Sof the first x integersreturn S
15 15
4 10
Example ….
![Page 21: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/21.jpg)
21
Measuring the Running Time
• How should we measure the running time of an algorithm?
• Approach 1: Experimental Study
50 1000
t (ms)
n
10
20
30
40
50
60
![Page 22: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/22.jpg)
22
Beyond Experimental Studies
• Experimental studies have several limitations:– need to implement– limited set of inputs– hardware and software environments.
![Page 23: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/23.jpg)
23
Theoretical Analysis
• We need a general methodology that: -
Uses a high-level description of the algorithm (independent of implementation).
Characterizes running time as a function of the input size.
Takes into account all possible inputs.
Is independent of the hardware and software environment.
![Page 24: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/24.jpg)
24
Analysis of Algorithms
• Primitive Operations: Low-level computations independent from the programming language can be identified in pseudocode.
• Examples:– calling a method and returning from a method– arithmetic operations (e.g. addition)– comparing two numbers, etc.
By inspecting the pseudo-code, we can count the number of primitive operations
executed by an algorithm.
![Page 25: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/25.jpg)
25
Example:
Algorithm arrayMax(A, n):Input: An array A storing n integers.Output: The maximum element in A.
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]return currentMax
![Page 26: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/26.jpg)
26
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]
return currentMax
5 13 4 7 6 2 3 8 1 2
currentMax
A
![Page 27: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/27.jpg)
27
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]
return currentMax
5 13 4 7 6 2 3 8 1 2
currentMax
A
5
![Page 28: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/28.jpg)
28
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]
return currentMax
5 13 4 7 6 2 3 8 1 2A
currentMax 5
![Page 29: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/29.jpg)
29
What are the primitive operations to count ?
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]
return currentMax
5 13 4 7 6 2 3 8 1 2ComparisonsAssignments to
currentMax
A
currentMax 13
![Page 30: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/30.jpg)
30
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]
return currentMax
1 assignment
n-1 comparisonsn-1 assignments
(worst case)
5 7 8 10 11 12 14 16 17 20
![Page 31: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/31.jpg)
currentMax A[0]for i 1 to n -1 doif currentMax < A[i] then
currentMax A[i]
return currentMax
In the best case ?
15 1 12 3 9 7 6 4 2 2
1 assignment
n-1 comparisons0 assignments
![Page 32: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/32.jpg)
Summarizing:
Worst Case:n-1 comparisonsn assignments
Best Case:n-1 comparisons1 assignment
Compute the exact number of primitive operations could be difficult
We compare the asymptotic behaviour of the running time when the size of the input rise.
![Page 33: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/33.jpg)
33
Big-Oh
– given two functions f(n) and g(n), we say that
f(n) is O(g(n)) if and only if there are positive constants c and n0 such that
f(n) c g(n) for n n0
n0
c • g(n)
f(n)
n
(upper bound)
![Page 34: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/34.jpg)
34
g(n) = n
n
What does it mean c g(n) ?
Example:
34
2 g(n) = 2 n
n
3 g(n) = 3 n
![Page 35: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/35.jpg)
35
f(n) = 2n+1
g(n) = n2
Graphical example …
f(n) is O(n2)
f(n) c g(n) for n n0
n
n0≈2.5 c = 1
?
![Page 36: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/36.jpg)
36
f(n) = 2n+1
g(n) = n
But also
f(n) c g(n) for n n0
n
![Page 37: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/37.jpg)
37
f(n) = 2n+1
2 g(n) = 2 n
But also
f(n) c g(n) for n n0
n
![Page 38: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/38.jpg)
38
f(n) = 2n+1
3 g(n) = 3 n
But also
f(n) c g(n) for n n0
f(n) is O(n)
nc = 3 and n0 = 1
![Page 39: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/39.jpg)
39
On the other hand…
n2 is not O(n) because there is no c and n0 such that: n2 cn for n n0
( no matter how large a c is chosen there is an n big enough (n > c) that n2 > c n ).
n2
n
n0 n
2n 3n 4n
![Page 40: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/40.jpg)
O(g(n)) = {f(n) : there exists positive constants c and n0
such that f(n) cg(n) for all n n0 }
Notice: O(g(n)) is a set of functions•When we say f(n) = O(g(n)) we really mean f(n) ϵ O(g(n))
Formal definition of big-Oh:
![Page 41: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/41.jpg)
41
Prove that f(n) = 60n2 + 5n + 1 is O(n2)
We must find a constant c and a constant n0 such that:
60n2 + 5n + 1 ≤ c n2 for all n≥n0
5n ≤ 5n2 for all n≥11 ≤ n2 for all n≥1
f(n) ≤ 60n2 +5n2 + n2 for all n≥1
f(n) ≤ 66n2 for all n≥1 c= 66 et n0=1 => f(n) = O(n2)
Example:
![Page 42: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/42.jpg)
f(n) ≤ 13n log2 n for all n ≥ 2
f(n) ϵ O(n log2 n ) [ c = 13, n0 = 2 ]
Prove f(n) = 5n log2 n + 8n - 200 = O(n log2 n)
5n log2 n + 8n - 200 ≤ 5n log2 n + 8n
≤ 5n log2 n + 8n log2 n for n ≥ 2 (log2 n ≥ 1)
≤ 13n log2 n
Example:
![Page 43: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/43.jpg)
We can multiply these to learn about other functions, O(an) = O(n) ⊂ O(n log n) ⊂ O(n1+b) ⊂ O(ncn)
Some commons relations
O(nc1) ⊂ O(nc
2) for any c1 < c2
For any constants a; b; c > 0,O(a) ⊂ O(log n) ⊂ O(nb) ⊂ O(cn)
These make things faster2 log2 n + 2 = O(log n)n + 2 = O(n)2n + 15n1/2 = O(n)
Examples: O(n1/5) ⊂ O(n1/5 log n)
![Page 44: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/44.jpg)
Ex 1:
2n3 + 3n2 = O (max(2n3, 3n2)) = O(2n3) = O(n3)
Theorem: If g(n) is O(f(n)) , then for any constant c > 0 g(n) is also O(c f(n))
Theorem: O(f(n) + g(n)) = O(max(f(n), g(n)))
Ex 2:
n2 + 3 log n – 7 = O(max(n2, 3 log n – 7)) = O(n2)
![Page 45: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/45.jpg)
45
Drop lower order terms and constant factors
7n-3 is O(n)
8n2log n + 5n2 + n is O(n2log n)
12n3 + 5000n2 + 2n4 is O(n4)
Simple Big Oh Rule:
![Page 46: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/46.jpg)
46
•Use the smallest possible class of functions
–Say “2n is O(n)” instead of “2n is O(n2)”
Other Big Oh Rules:
•Use the simplest expression of the class
–Say “3n + 5 is O(n)” instead of
“3n + 5 is O(3n)”
![Page 47: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/47.jpg)
Asymptotic Notation (terminology)
• Special classes of algorithms:constant: O(1)logarithmic: O(log n)linear: O(n)quadratic: O(n2)cubic: O(n3)polynomial: O(nk), k >0exponential: O(an), n > 1
![Page 48: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/48.jpg)
48
The i-th prefix average of an array X is average of the first (i 1) elements of X
X[0] X[1] … X[i]
Example of Asymptotic Analysis
An algorithm for computing prefix averages
(i 1)
5 13 4 8 6 2 3 8 1 2
5 9 7.3 7.5 … … … … … …
A[i]
![Page 49: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/49.jpg)
49
Example of Asymptotic Analysis
Algorithm prefixAverages1(X, n)
Input array X of n integersOutput array A of prefix averages of X #operations
A new array of n integers for i 0 to n 1 do
s X[0] for j 1 to i do
s s + X[j] A[i] s / (i + 1)
return A
![Page 50: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/50.jpg)
50
5 13 4 8 6 2 3 8 1 2
i
j = 0
5
![Page 51: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/51.jpg)
51
5 13 4 8 6 2 3 8 1 2
i
j = 0,1
5 9
![Page 52: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/52.jpg)
52
5 13 4 8 6 2 3 8 1 2
i
j = 0,1,2
5 9 7.3
![Page 53: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/53.jpg)
53
5 13 4 8 6 2 3 8 1 2
i
j = 0,1,2,3
5 9 7.3 7.5
![Page 54: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/54.jpg)
54
Example of Asymptotic Analysis
Algorithm prefixAverages1(X, n)
Input array X of n integersOutput array A of prefix averages of X #operations
A new array of n integers nfor i 0 to n 1 do n
s X[0] nfor j 1 to i do 1 + 2 + …+ (n 1)
s s + X[j] 1 + 2 + …+ (n 1)A[i] s / (i + 1) n
return A 1
![Page 55: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/55.jpg)
55
• The running time of prefixAverages1 isO(1 + 2 + …+ n)
• The sum of the first n integers is n(n + 1) / 2– There is a simple
visual proof of this fact
0
1
2
3
4
5
6
7
1 2 3 4 5 6
![Page 56: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/56.jpg)
56
Thus, algorithm prefixAverages1 runs in timeO(n(n + 1) / 2)
which is O(n2)
1 + 2 + …+ n = n(n+1)
TO REMEMBER
2
![Page 57: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/57.jpg)
57
Another Example:A better algorithm for computing prefix
averages
Algorithm prefixAverages2(X):Input: An n-element array X of numbers.Output: An n -element array A of numbers such that A[i] is the
average of elements X[0], ... , X[i]. Let X be an array of n numbers.
s 0 for i 0 to n-1 do
s s + X[i] A[i] s/(i+ 1)
return array A
![Page 58: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/58.jpg)
58
5 13 4 8 6 2 3 8 1 2
i
s=0
5
![Page 59: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/59.jpg)
59
5 13 4 8 6 2 3 8 1 2
i
s=5
5 9
![Page 60: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/60.jpg)
60
5 13 4 8 6 2 3 8 1 2
i
s=18
5 9 7.3
![Page 61: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/61.jpg)
61
5 13 4 8 6 2 3 8 1 2
i
s=22
5 9 7.3 7.5
![Page 62: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/62.jpg)
62
Let X be an array of n numbers. # operationss 0 1for i 0 to n-1 do n
s s + X[i] nA[i] s/(i+ 1) n
return array A 1
O(n) time
Another Example:A better algorithm for computing prefix
averages
![Page 63: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/63.jpg)
63
big-Omega (lower bound)
f(n) is (g(n))
if there exist c > 0 and n0 > 0 such that
f(n) c • g(n) for all n n0
nn0
c • g(n)f(n)
(thus, f(n) is (g(n)) iff g(n) is O(f(n)) )
![Page 64: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/64.jpg)
64
… is big theta …
g(n) is (f(n))
<===>
if g(n) O(f(n))
AND
f(n) O(g(n))
big-Theta
![Page 65: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/65.jpg)
65
We have seen that
f(n) = 60n2 + 5n + 1 is O(n2)
but 60n2 + 5n + 1 60n2 for n 1
So: with c = 60 and n0 = 1
f(n) c • n2 for all n 1
f(n) is (n2)
Example:
![Page 66: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/66.jpg)
66
Intuition for Asymptotic Notation
Big-Oh– f(n) is O(g(n)) if f(n) is
asymptotically less than or equal to g(n)
big-Omega– f(n) is (g(n)) if f(n) is
asymptotically greater than or equal to g(n)
big-Theta– f(n) is (g(n)) if f(n) is
asymptotically equal to g(n)
![Page 67: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/67.jpg)
67
Math You Need to Review Logarithms and Exponents
properties of logarithms:logb(xy) = logbx + logbylogb (x/y) = logbx - logbylogbxa = alogbxlogbax = (1/a)logbxlogba= logxa/logxb
Natural logarithm: ln k = ∫1
k(1/x)dxe = limk→∞(1+1/n)n ≈ 2.71828
![Page 68: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/68.jpg)
68
Math You Need to Review Logarithms and Exponents
properties of exponentials:a(b+c) = aba c
abc = (ab)c
ab /ac = a(b-c)
b = a logab
bc = a c*logab
![Page 69: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/69.jpg)
69
More Math to Review
• Floor: x = the largest integer ≤ x
• Ceiling: x = the smallest integer ≥ x
• Summations: – Arithmetic progression:
– Geometric progression:
![Page 70: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/70.jpg)
70
More Math to ReviewArithmetic Progression
n
S = di = 0 + d + 2d + … + nd i=0
= nd+(n-1)d+(n-2)d + … + 0
S = d/2 n(n+1)
for d=1 S = 1/2 n(n+1)
2S = nd + nd + nd + …+ nd
= (n+1) nd
![Page 71: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/71.jpg)
71
More Math to ReviewGeometric Progression
n
S = ri = 1 + r + r2 + … + rn
i=0
rS = r + r2 + … + rn + rn+1
If r=2,S = (2n+1-
1)
rS - S = (r-1)S = rn+1 - 1
S = (rn+1-1)/(r-1)
n
rS - S = ri = -1 - r - r2 - … - rn
i=0
r + r2 + … + rn + rn+1
![Page 72: COMP 2402/2002 Abstract Data Types and Algorithms Prof: Office: Email: Office hours: Eduardo Mesa HP 5347 eamesaba@connect.ca Tuesday and Thursday 4:30pm](https://reader036.vdocuments.us/reader036/viewer/2022062722/56649f2a5503460f94c43919/html5/thumbnails/72.jpg)
72
Math You Need to Review Randomization and Probability
Expected valueE[X] = ∑x ϵ U(x*Pr{X=x})
Properties E[X + Y ] = E[X] + E[Y ]
E[∑i =1..k(Xi)] = ∑i =1..k (E[Xi])