asymptotic complexity (big-o notation) cs 1037 fundamentals of computer science ii
TRANSCRIPT
![Page 1: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/1.jpg)
Asymptotic Complexity(Big-O Notation)
CS 1037 Fundamentals of Computer Science II
![Page 2: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/2.jpg)
2
What is a “Fast” Algorithm?
Best case
Average case
Worst case
Small problems
Large problems
Best case
Average case
Worst case
Small problems
Large problems
Takes the fewest seconds? Takes the fewest steps?
In practice, these matter. Choice depends on: • application requirements• computer architecture• expected input
P PP P
In theoretical comp. sci., algorithms ranked by:• fewest # of steps,• in the worst case,• on large problems
T
![Page 3: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/3.jpg)
3
Ranking Running-Time
• Suppose algorithm A takes an2+b steps on problem of size n in worst case
• Suppose algorithm B takes cnlgn+d steps• Which algorithm is faster in practice?– depends on a,b,c,d and typical input
• Which algorithm is faster in theory?– for large enough n coefficients a,b,c,d don’t matter– algorithm B therefore fundamentally faster than A
![Page 4: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/4.jpg)
4
Example: Algorithm A vs B
• Force cnlgn+d to have large c...
B faster than A for large enough
nA
B
![Page 5: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/5.jpg)
5
When is Algorithm A “as fast as” B?
• Let A(n) be # steps for A on problem size n• Let B(n) be # steps for algorithm B– e.g. define functions A(n)= 4n+2, B(n)= n2
A as fast as B if A(n) · kB(n) for all large enough n and some k > 0
4n+2 · n2 for all n ¸ 54n+2 · 2n2 for all n ¸ 320nlgn · n2 for all n ¸ 144
n
tim
e
# items 5
![Page 6: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/6.jpg)
6
Asymptotic Notation (“Big-O”)
• Let g(n) be some function of n• Define O(g(n)) to be set of all functions
as fast as g(n) in theoretical sense:
O(g(n)) = f(n) j there exist positive k, n0
such that 0 · f(n) · kg(n) for all n ¸n0
f
g
n
tim
e
# items
n2
f(n) 2 O(n2)n0
n 2 O(n) n 2 O(n2)n2 2 O(n)
e.g.
![Page 7: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/7.jpg)
7
Asymptotic Notation (“Big-O”)
• Intuition:
n+100 2 O(n) because n+100 · 50n for n ¸ 3 or because n+100 · 2n for n ¸ 100
n2 2 O(n) because for any k>0 eventually n2 > kn
for large n (i.e. “n2 slower than n”)
any constant c 2 O(n) because c · n for n ¸ c
f(n) 2 O(g(n))
“f(n) no slower than kg(n) for large n”
![Page 8: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/8.jpg)
8
Asymptotic Notation (“Big-O”)
• Claim: any function of the form an+b with a ¸0 is a member of the set O(n)
Proof: choose k = a+j b j, then an+b · (a+j b j)n 8 n ¸ 1
• Claim: any an2+bn+c 2 O(n2) if a ¸0 Proof: choose k = a+j b j+j c j, then an2+bn+c · (a+j b j+j c j)n2 8 n ¸ 1
![Page 9: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/9.jpg)
9
Asymptotic Notation (“Big-O”)
• Can say Insertion Sort runs in...– “quadratic time in the worst-case”, or– O(n2) time ... “oh of n squared time”
• Can say Binary Search runs in...– “logarithmic time in the worst-case”, or– O(lgn) time ... “oh of log n time”
• Can say Counting Sort runs in...– “time linear in n and k”, or– O(n+k) time ... “oh of n plus k time”
this k means “max range of items”
![Page 10: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/10.jpg)
10
Asymptotic Notation (“Big-O”)
• Claim: if f(n) 2 O(g(n)) and g(n) 2
O(h(n)) then f(n) 2 O(h(n)) Proof: We know for some n1,n2,k1,k2 > 0
0·f(n)·k1g(n) 8 n ¸ n1
0·g(n)·k2h(n) 8 n ¸ n2
and therefore 0·f(n)·k1k2h(n) 8 n ¸ maxfn1,n2g
![Page 11: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/11.jpg)
11
Big-O Notation Used Everywhere
DOCUMENTATION
COMP. SCI. COURSES
ALGORITHMS RESEARCH
![Page 12: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/12.jpg)
12
More Examples of Big-O Membership
5050
n+506n2+50
n2+10n+50n3¡10n
2lgn+50n+lgn+50
2nlgn+n+lgn
2n+500n3
cn+nc
22222222222
O(1)O(n)O(n)O(n2)O(n2)O(n3)O(lgn)O(n)O(nlgn)O(2n)O(cn) if c >
1
We say any constant c 2 O(1), i.e. O(1) = set of all constants
Such functions are “exponential in n” and grow extremely fast! Such algorithms take longer than age of universe to compute medium-sized problems
![Page 13: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/13.jpg)
13
History of Big-O
• Invented by mathematician Paul Bachmann in 1894 to characterize error bounds
• Don Knuth popularized use for running time bounds in 1970s
The Art of Computer Programming, Volume 1
![Page 14: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/14.jpg)
14
Knuth on Importance of Big-O“Students will be motivated to use O(n) notation for two important reasons. First, it significantly simplifies calculations because it allows us to be sloppy — but in a satisfactorily controlled way. Second, it appears in the power series
calculations of symbolic algebra systems like Maple and Mathematica, which today’s students will surely be using...”
—Don Knuth advocating Big-O in high school, Letter to The American Mathematical Society March 1998
http://micromath.wordpress.com/2008/04/14/donald-knuth-calculus-via-o-notation/
![Page 15: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/15.jpg)
15
END OF COURSE MATERIAL
hooray!!!
![Page 16: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/16.jpg)
16
variablesif-elseloops
arrayspointersfunctions
CS1036
Programming Skills Take Time!eff
ectiv
enes
s of
you
stuff in your brain
structsclasses
templates
sortinglists/queues
trees
CS1037
dictionarieshashingmore…
CS2210SE2205
![Page 17: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/17.jpg)
Big Universe of Ideas!
17
variablesloops if-else
arrayspointersfunctions
exceptions
structs
classes
templates
sortinglistsqueuestrees
big-O
recursion
polymorphism
inheritance
debuggingassertions
iterators
threads
networking
security
optimization
encapsulationsearching
hashing
clusteringheaps
balanced trees
caching
randomized algorithms
graphs
matching
NP-completeness
computability theory
finite automata
approximation algorithms
multisets
dynamic programming
compression
float arithmetic
symbolic computation
GPUs
distributed computing
compilers
operating systems
greedy algorithms
interpretersparsing
databases
dictionaries
stacks
user interfaces
JavaPython
virtual machinesC#
backtracking
profiling
reducibility
Cdata mining
assembly
file I/O
proof techniques
portabilitylinear algebra
rasterization
references
physics simulation
version control
concurrency
combinatorics
UML
![Page 19: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/19.jpg)
19
Goals of CS 1037
• Understand how to…– use data structures– implement data structures– characterize an algorithm’s performance– express computation in C++
• Develop intuition about…– best data structure / algorithm for situation– good code, poor code, and dangerous code
• Programming experience, confidence
![Page 20: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/20.jpg)
20
Pearls of Wisdom“Anybody who comes to you and says he has a perfect language is either naive or a salesman.” —Bjarne Stroustrup
at Waterloo Comp Sci Club Meeting
“A program that has not been tested does not work.” —Bjarne Stroustrup
![Page 21: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/21.jpg)
21
Pearls of Wisdom“The psychological profiling [of a programmer] is mostly the ability to shift levels of abstraction, from low level to high level. To see something in the small and to see something in the large.” —Don Knuth
“Premature optimization is the root of all evil.”
—Don Knuth
remember that one!!
![Page 22: Asymptotic Complexity (Big-O Notation) CS 1037 Fundamentals of Computer Science II](https://reader038.vdocuments.us/reader038/viewer/2022102906/56649d095503460f949da9bb/html5/thumbnails/22.jpg)
22
EXAM REVIEW TIME!
hooray?