![Page 1: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/1.jpg)
CSE 431/531: Analysis of Algorithms
Introduction and Syllabus
Lecturer: Shi Li
Department of Computer Science and EngineeringUniversity at Buffalo
![Page 2: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/2.jpg)
2/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 3: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/3.jpg)
3/69
CSE 431/531: Analysis of Algorithms
Course webpage:http://www.cse.buffalo.edu/~shil/courses/CSE531/
Please sign up the course on Piazza:http://piazza.com/buffalo/fall2016/cse431531
![Page 4: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/4.jpg)
4/69
CSE 431/531: Analysis of Algorithms
Time and locatiion:
MoWeFr, 9:00-9:50amCooke 121
Lecturer:
Shi Li, [email protected] hours: TBD
TAs
Di Wang, [email protected] Ye, [email protected] Stachnik, [email protected]
![Page 5: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/5.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 6: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/6.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 7: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/7.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 8: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/8.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 9: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/9.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 10: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/10.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 11: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/11.jpg)
5/69
You should know:
Mathematical Tools
Mathematical inductionsProbabilities and random variables
Data Structures
Stacks, queues, linked lists
Some Programming Experience
E.g., C, C++ or Java
You may know:
Asymptotic analysis
Simple algorithm design techniques such as greedy,divide-and-conquer, dynamic programming
![Page 12: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/12.jpg)
6/69
You Will Learn
Classic algorithms for classic problems
SortingShortest pathsMinimum spanning treeNetwork flow
How to analyze algorithms
CorrectnessRunning time (efficiency)Space requirement
Meta techniques to design algorithms
Greedy algorithmsDivide and conquerDynamic programmingReductions
NP-completeness
![Page 13: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/13.jpg)
6/69
You Will Learn
Classic algorithms for classic problems
SortingShortest pathsMinimum spanning treeNetwork flow
How to analyze algorithms
CorrectnessRunning time (efficiency)Space requirement
Meta techniques to design algorithms
Greedy algorithmsDivide and conquerDynamic programmingReductions
NP-completeness
![Page 14: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/14.jpg)
6/69
You Will Learn
Classic algorithms for classic problems
SortingShortest pathsMinimum spanning treeNetwork flow
How to analyze algorithms
CorrectnessRunning time (efficiency)Space requirement
Meta techniques to design algorithms
Greedy algorithmsDivide and conquerDynamic programmingReductions
NP-completeness
![Page 15: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/15.jpg)
6/69
You Will Learn
Classic algorithms for classic problems
SortingShortest pathsMinimum spanning treeNetwork flow
How to analyze algorithms
CorrectnessRunning time (efficiency)Space requirement
Meta techniques to design algorithms
Greedy algorithmsDivide and conquerDynamic programmingReductions
NP-completeness
![Page 16: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/16.jpg)
7/69
Textbook
Required Textbook:
Algorithm Design, 1st Edition, byJon Kleinberg and Eva Tardos
Other Reference Books
Introduction to Algorithms, Third Edition, Thomas Cormen,Charles Leiserson, Rondald Rivest, Clifford Stein
![Page 17: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/17.jpg)
8/69
Grading
20% for homeworks
5 homeworks, each worth 4%
20% for projects
2 projects, each worth 10%
30% for in-class exams
2 in-class exams, each worth 15%
30% for final exam
If to your advantage: each in-class exam is worth 5% and finalis worth 50%
![Page 18: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/18.jpg)
9/69
For Homeworks, You Are Allowed to
Use course materials (textbook, reference books, lecturenotes, etc)
Post questions on Piazza
Ask me or TAs for hints
Collaborate with classmates
Think about each problem for enough time before discussingMust write down solutions on your own, in your own wordsWrite down names of students you collaborated with
![Page 19: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/19.jpg)
10/69
For Homeworks, You Are Not Allowed to
Use external resources
Can’t Google or ask questions online for solutionsCan’t read posted solutions from other algorithm courses
Copy solutions from other students
If you are not following the rules, you will get an “F” for thecourse.
![Page 20: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/20.jpg)
10/69
For Homeworks, You Are Not Allowed to
Use external resources
Can’t Google or ask questions online for solutionsCan’t read posted solutions from other algorithm courses
Copy solutions from other students
If you are not following the rules, you will get an “F” for thecourse.
![Page 21: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/21.jpg)
11/69
Projects
Need to implement an algorithm for each of the two projects
Can not copy codes from others or the Internet
If you are not following the rules, you will get an “F” for thecourse.
![Page 22: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/22.jpg)
11/69
Projects
Need to implement an algorithm for each of the two projects
Can not copy codes from others or the Internet
If you are not following the rules, you will get an “F” for thecourse.
![Page 23: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/23.jpg)
12/69
Late policy
You have one late credit
turn in a homework or a project late for three days using thelate credit
no other late submissions will be accepted
![Page 24: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/24.jpg)
13/69
Exams
Closed-book
Can bring one A4 handwritten sheet
If you are caught cheating in exams, you will get an “F” for thecourse.
Questions?
![Page 25: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/25.jpg)
13/69
Exams
Closed-book
Can bring one A4 handwritten sheet
If you are caught cheating in exams, you will get an “F” for thecourse.
Questions?
![Page 26: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/26.jpg)
13/69
Exams
Closed-book
Can bring one A4 handwritten sheet
If you are caught cheating in exams, you will get an “F” for thecourse.
Questions?
![Page 27: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/27.jpg)
14/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 28: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/28.jpg)
15/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 29: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/29.jpg)
16/69
What is an Algorithm?
Donald Knuth: An algorithm is a finite, definite effectiveprocedure, with some input and some output.
Computational problem: specifies the input/outputrelationship.
An algorithm solves a computational problem if it producesthe correct output for any given input.
![Page 30: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/30.jpg)
16/69
What is an Algorithm?
Donald Knuth: An algorithm is a finite, definite effectiveprocedure, with some input and some output.
Computational problem: specifies the input/outputrelationship.
An algorithm solves a computational problem if it producesthe correct output for any given input.
![Page 31: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/31.jpg)
17/69
Examples
Greatest Common Divisor
Input: two integers a, b > 0
Output: the greatest common divisor of a and b
Example:
Input: 210, 270
Output: 30
Algorithm: Euclidean algorithm
gcd(270, 210) = gcd(210, 270 mod 210) = gcd(210, 60)
(270, 210)→ (210, 60)→ (60, 30)→ (30, 0)
![Page 32: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/32.jpg)
17/69
Examples
Greatest Common Divisor
Input: two integers a, b > 0
Output: the greatest common divisor of a and b
Example:
Input: 210, 270
Output: 30
Algorithm: Euclidean algorithm
gcd(270, 210) = gcd(210, 270 mod 210) = gcd(210, 60)
(270, 210)→ (210, 60)→ (60, 30)→ (30, 0)
![Page 33: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/33.jpg)
17/69
Examples
Greatest Common Divisor
Input: two integers a, b > 0
Output: the greatest common divisor of a and b
Example:
Input: 210, 270
Output: 30
Algorithm: Euclidean algorithm
gcd(270, 210) = gcd(210, 270 mod 210) = gcd(210, 60)
(270, 210)→ (210, 60)→ (60, 30)→ (30, 0)
![Page 34: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/34.jpg)
17/69
Examples
Greatest Common Divisor
Input: two integers a, b > 0
Output: the greatest common divisor of a and b
Example:
Input: 210, 270
Output: 30
Algorithm: Euclidean algorithm
gcd(270, 210) = gcd(210, 270 mod 210) = gcd(210, 60)
(270, 210)→ (210, 60)→ (60, 30)→ (30, 0)
![Page 35: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/35.jpg)
17/69
Examples
Greatest Common Divisor
Input: two integers a, b > 0
Output: the greatest common divisor of a and b
Example:
Input: 210, 270
Output: 30
Algorithm: Euclidean algorithm
gcd(270, 210) = gcd(210, 270 mod 210) = gcd(210, 60)
(270, 210)→ (210, 60)→ (60, 30)→ (30, 0)
![Page 36: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/36.jpg)
18/69
Examples
Sorting
Input: sequence of n numbers (a1, a2, · · · , an)
Output: a permutation (a′1, a′2, · · · , a′n) of the input sequence
such that a′1 ≤ a′2 ≤ · · · ≤ a′n
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
Algorithms: insertion sort, merge sort, quicksort, . . .
![Page 37: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/37.jpg)
18/69
Examples
Sorting
Input: sequence of n numbers (a1, a2, · · · , an)
Output: a permutation (a′1, a′2, · · · , a′n) of the input sequence
such that a′1 ≤ a′2 ≤ · · · ≤ a′n
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
Algorithms: insertion sort, merge sort, quicksort, . . .
![Page 38: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/38.jpg)
18/69
Examples
Sorting
Input: sequence of n numbers (a1, a2, · · · , an)
Output: a permutation (a′1, a′2, · · · , a′n) of the input sequence
such that a′1 ≤ a′2 ≤ · · · ≤ a′n
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
Algorithms: insertion sort, merge sort, quicksort, . . .
![Page 39: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/39.jpg)
19/69
Examples
Shortest Path
Input: directed graph G = (V,E), s, t ∈ V
Output: a shortest path from s to t in G
16 1
1 5 42
104
3
s
333 t
Algorithm: Dijkstra’s algorithm
![Page 40: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/40.jpg)
19/69
Examples
Shortest Path
Input: directed graph G = (V,E), s, t ∈ V
Output: a shortest path from s to t in G
16 1
1 5 42
104
3
s
333 t
Algorithm: Dijkstra’s algorithm
![Page 41: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/41.jpg)
19/69
Examples
Shortest Path
Input: directed graph G = (V,E), s, t ∈ V
Output: a shortest path from s to t in G
16 1
1 5 42
104
3
s
333 t
Algorithm: Dijkstra’s algorithm
![Page 42: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/42.jpg)
19/69
Examples
Shortest Path
Input: directed graph G = (V,E), s, t ∈ V
Output: a shortest path from s to t in G
16 1
1 5 42
104
3
s
333 t
Algorithm: Dijkstra’s algorithm
![Page 43: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/43.jpg)
20/69
Algorithm = Computer Program?
Algorithm: “abstract”, can be specified using computerprogram, English, pseudo-codes or flow charts.
Computer program: “concrete”, implementation ofalgorithm, associated with a particular programming language
![Page 44: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/44.jpg)
21/69
Pseudo-Code
Pseudo-Code:
Euclidean(a, b)
1 while b > 0
2 (a, b)← (b, a mod b)
3 return a
C++ program:
int Euclidean(int a, int b)int c;
while (b > 0)c = b;
b = a % b;
a = c;
return a;
![Page 45: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/45.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?
1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental4 it is fun!
![Page 46: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/46.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?
1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental4 it is fun!
![Page 47: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/47.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?
1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental4 it is fun!
![Page 48: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/48.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?
1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental4 it is fun!
![Page 49: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/49.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?1 feasible vs. infeasible
2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental4 it is fun!
![Page 50: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/50.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.
3 fundamental4 it is fun!
![Page 51: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/51.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental
4 it is fun!
![Page 52: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/52.jpg)
22/69
Theoretical Analysis of Algorithms
Main focus: correctness, running time (efficiency)
Sometimes: memory usage
Not covered in the course: engineering side
readabilityextensibilityuser-friendliness. . .
Why is it important to study the running time (efficiency) ofan algorithm?1 feasible vs. infeasible2 use efficiency to pay for user-friendliness, extensibility, etc.3 fundamental4 it is fun!
![Page 53: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/53.jpg)
23/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 54: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/54.jpg)
24/69
Sorting Problem
Input: sequence of n numbers (a1, a2, · · · , an)
Output: a permutation (a′1, a′2, · · · , a′n) of the input sequence
such that a′1 ≤ a′2 ≤ · · · ≤ a′n
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
![Page 55: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/55.jpg)
25/69
Insertion-Sort
At the end of j-th iteration, make the first j numbers sorted.
iteration 1: 53, 12, 35, 21, 59, 15
iteration 2: 12, 53, 35, 21, 59, 15
iteration 3: 12, 35, 53, 21, 59, 15
iteration 4: 12, 21, 35, 53, 59, 15
iteration 5: 12, 21, 35, 53, 59, 15
iteration 6: 12, 15, 21, 35, 53, 59
![Page 56: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/56.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
![Page 57: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/57.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 53 59 15↑i
![Page 58: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/58.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 53 59 59↑i
![Page 59: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/59.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 53 59 59↑i
![Page 60: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/60.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 53 53 59↑i
![Page 61: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/61.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 53 53 59↑i
![Page 62: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/62.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 35 53 59↑i
![Page 63: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/63.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 35 35 53 59↑i
![Page 64: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/64.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 21 35 53 59↑i
![Page 65: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/65.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 21 21 35 53 59↑i
![Page 66: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/66.jpg)
26/69
Example:
Input: 53, 12, 35, 21, 59, 15
Output: 12, 15, 21, 35, 53, 59
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
j = 6
key = 15
12 15 21 35 53 59↑i
![Page 67: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/67.jpg)
27/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 68: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/68.jpg)
28/69
Analysis of Insertion Sort
Correctness
Running time
![Page 69: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/69.jpg)
29/69
Correctness of Insertion Sort
Invariant: after iteration j of outer loop, A[1..j] is the sortedarray for the original A[1..j].
after j = 1 : 53, 12, 35, 21, 59, 15
after j = 2 : 12, 53, 35, 21, 59, 15
after j = 3 : 12, 35, 53, 21, 59, 15
after j = 4 : 12, 21, 35, 53, 59, 15
after j = 5 : 12, 21, 35, 53, 59, 15
after j = 6 : 12, 15, 21, 35, 53, 59
![Page 70: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/70.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 71: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/71.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 72: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/72.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 73: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/73.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 74: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/74.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 75: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/75.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 76: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/76.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 77: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/77.jpg)
30/69
Analyze Running Time of Insertion Sort
Q: Size of input?
A: Running time as function of size
possible definition of size : # integers, total length ofintegers, # vertices in graph, # edges in graph
Q: Which input?
A: Worst-case analysis:
Worst running time over all input instances of a given size
Q: How fast is the computer?
Q: Programming language?
A: Important idea: asymptotic analysis
Focus on growth of running-time as a function, not anyparticular value.
![Page 78: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/78.jpg)
31/69
Asymptotic Analysis: O-notation
Ignoring lower order terms
Ignoring leading constant
3n3 + 2n2 − 18n + 1028⇒ 3n3 ⇒ n3
3n3 + 2n2 − 18n + 1028 = O(n3)
2n/3+100 + 100n100 ⇒ 2n/3+100 ⇒ 2n/3
2n/3+100 + 100n100 = O(2n/3)
![Page 79: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/79.jpg)
31/69
Asymptotic Analysis: O-notation
Ignoring lower order terms
Ignoring leading constant
3n3 + 2n2 − 18n + 1028⇒ 3n3 ⇒ n3
3n3 + 2n2 − 18n + 1028 = O(n3)
2n/3+100 + 100n100 ⇒ 2n/3+100 ⇒ 2n/3
2n/3+100 + 100n100 = O(2n/3)
![Page 80: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/80.jpg)
31/69
Asymptotic Analysis: O-notation
Ignoring lower order terms
Ignoring leading constant
3n3 + 2n2 − 18n + 1028⇒ 3n3 ⇒ n3
3n3 + 2n2 − 18n + 1028 = O(n3)
2n/3+100 + 100n100 ⇒ 2n/3+100 ⇒ 2n/3
2n/3+100 + 100n100 = O(2n/3)
![Page 81: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/81.jpg)
31/69
Asymptotic Analysis: O-notation
Ignoring lower order terms
Ignoring leading constant
3n3 + 2n2 − 18n + 1028⇒ 3n3 ⇒ n3
3n3 + 2n2 − 18n + 1028 = O(n3)
2n/3+100 + 100n100 ⇒ 2n/3+100 ⇒ 2n/3
2n/3+100 + 100n100 = O(2n/3)
![Page 82: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/82.jpg)
31/69
Asymptotic Analysis: O-notation
Ignoring lower order terms
Ignoring leading constant
3n3 + 2n2 − 18n + 1028⇒ 3n3 ⇒ n3
3n3 + 2n2 − 18n + 1028 = O(n3)
2n/3+100 + 100n100 ⇒ 2n/3+100 ⇒ 2n/3
2n/3+100 + 100n100 = O(2n/3)
![Page 83: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/83.jpg)
32/69
Asymptotic Analysis: O-notation
Ignoring lower order terms
Ignoring leading constant
O-notation allows us to
ignore architecture of computer
ignore programming language
![Page 84: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/84.jpg)
33/69
Asymptotic Analysis of Insertion Sort
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
Worst-case running time for iteration j in the outer loop?
Answer: O(j)
Total running time =∑n
j=2O(j) = O(n2) (informal)
![Page 85: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/85.jpg)
33/69
Asymptotic Analysis of Insertion Sort
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
Worst-case running time for iteration j in the outer loop?
Answer: O(j)
Total running time =∑n
j=2O(j) = O(n2) (informal)
![Page 86: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/86.jpg)
33/69
Asymptotic Analysis of Insertion Sort
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
Worst-case running time for iteration j in the outer loop?Answer: O(j)
Total running time =∑n
j=2O(j) = O(n2) (informal)
![Page 87: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/87.jpg)
33/69
Asymptotic Analysis of Insertion Sort
insertion-sort(A, n)
1 for j ← 2 to n
2 key ← A[j]
3 i← j − 1
4 while i > 0 and A[i] > key
5 A[i + 1]← A[i]
6 i← i− 1
7 A[i + 1]← key
Worst-case running time for iteration j in the outer loop?Answer: O(j)
Total running time =∑n
j=2O(j) = O(n2) (informal)
![Page 88: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/88.jpg)
34/69
Computation Model
Random-Access Machine (RAM) model: read A[j] takesO(1) time.
Basic operations take O(1) time: addition, subtraction,multiplication, etc.
Each integer (word) has c log n bits, c ≥ 1 large enough
Precision of real numbers?
In most scenarios in the course, assuming real numbers arerepresented exactly
Can we do better than insertion sort asymptotically?
Yes: merge sort, quicksort, heap sort, ...
![Page 89: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/89.jpg)
34/69
Computation Model
Random-Access Machine (RAM) model: read A[j] takesO(1) time.
Basic operations take O(1) time: addition, subtraction,multiplication, etc.
Each integer (word) has c log n bits, c ≥ 1 large enough
Precision of real numbers?
In most scenarios in the course, assuming real numbers arerepresented exactly
Can we do better than insertion sort asymptotically?
Yes: merge sort, quicksort, heap sort, ...
![Page 90: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/90.jpg)
34/69
Computation Model
Random-Access Machine (RAM) model: read A[j] takesO(1) time.
Basic operations take O(1) time: addition, subtraction,multiplication, etc.
Each integer (word) has c log n bits, c ≥ 1 large enough
Precision of real numbers?In most scenarios in the course, assuming real numbers arerepresented exactly
Can we do better than insertion sort asymptotically?
Yes: merge sort, quicksort, heap sort, ...
![Page 91: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/91.jpg)
34/69
Computation Model
Random-Access Machine (RAM) model: read A[j] takesO(1) time.
Basic operations take O(1) time: addition, subtraction,multiplication, etc.
Each integer (word) has c log n bits, c ≥ 1 large enough
Precision of real numbers?In most scenarios in the course, assuming real numbers arerepresented exactly
Can we do better than insertion sort asymptotically?
Yes: merge sort, quicksort, heap sort, ...
![Page 92: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/92.jpg)
34/69
Computation Model
Random-Access Machine (RAM) model: read A[j] takesO(1) time.
Basic operations take O(1) time: addition, subtraction,multiplication, etc.
Each integer (word) has c log n bits, c ≥ 1 large enough
Precision of real numbers?In most scenarios in the course, assuming real numbers arerepresented exactly
Can we do better than insertion sort asymptotically?
Yes: merge sort, quicksort, heap sort, ...
![Page 93: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/93.jpg)
35/69
Remember to sign up for Piazza.
Questions?
![Page 94: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/94.jpg)
36/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 95: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/95.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30
Yes
2n − n20
Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 96: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/96.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30
Yes
2n − n20
Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 97: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/97.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30
Yes
2n − n20
Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 98: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/98.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30 Yes
2n − n20
Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 99: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/99.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30 Yes
2n − n20
Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 100: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/100.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30 Yes
2n − n20 Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 101: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/101.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30 Yes
2n − n20 Yes
100n− n2/10 + 50?
No
We only consider asymptotically positive functions.
![Page 102: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/102.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30 Yes
2n − n20 Yes
100n− n2/10 + 50? No
We only consider asymptotically positive functions.
![Page 103: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/103.jpg)
37/69
Asymptotically Positive Functions
Def. f : N→ R is an asymptotically positive function if:
∃n0 > 0 such that ∀n > n0 we have f(n) > 0
In other words, f(n) is positive for large enough n.
n2 − n− 30 Yes
2n − n20 Yes
100n− n2/10 + 50? No
We only consider asymptotically positive functions.
![Page 104: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/104.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 105: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/105.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 106: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/106.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 107: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/107.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 108: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/108.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 109: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/109.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 110: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/110.jpg)
38/69
O-Notation: Asymptotic Upper Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ O(g(n)) if f(n) ≤ cg(n) for some cand large enough n.
Informally, think of it as “f ≤ g”.
3n2 + 2n ∈ O(n3)
3n2 + 2n ∈ O(n2)
n100 ∈ O(2n)
n3 /∈ O(n2)
![Page 111: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/111.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 112: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/112.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 113: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/113.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 114: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/114.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 115: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/115.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 116: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/116.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 117: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/117.jpg)
39/69
Conventions
We use “f(n) = O(g(n))” to denote “f(n) ∈ O(g(n))”
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
There exists a function f(n) ∈ O(n3), such that4n3 + 3n2 + 2n = 4n3 + f(n).
n2 + O(n) = O(n2)
For every function f(n) ∈ O(n), there exists a functiong(n) ∈ O(n2), such that n2 + f(n) = g(n).
Rule: left side → ∀, right side → ∃
![Page 118: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/118.jpg)
40/69
Conventions
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
n2 + O(n) = O(n2)
“=” is asymmetric! Following statements are wrong:
O(n3) = 3n2 + 2n4n3 + O(n3) = 4n3 + 3n2 + 2nO(n2) = n2 + O(n)
Chaining is allowed:4n3 + 3n2 + 2n = 4n3 + O(n3) = O(n3) = O(n4)
![Page 119: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/119.jpg)
40/69
Conventions
3n2 + 2n = O(n3)
4n3 + 3n2 + 2n = 4n3 + O(n3)
n2 + O(n) = O(n2)
“=” is asymmetric! Following statements are wrong:
O(n3) = 3n2 + 2n4n3 + O(n3) = 4n3 + 3n2 + 2nO(n2) = n2 + O(n)
Chaining is allowed:4n3 + 3n2 + 2n = 4n3 + O(n3) = O(n3) = O(n4)
![Page 120: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/120.jpg)
41/69
Ω-Notation: Asymptotic Lower Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
Ω-Notation For a function g(n),
Ω(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≥ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ Ω(g(n)) if f(n) ≥ cg(n) for some cand large enough n.
Informally, think of it as “f ≥ g”.
![Page 121: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/121.jpg)
41/69
Ω-Notation: Asymptotic Lower Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
Ω-Notation For a function g(n),
Ω(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≥ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ Ω(g(n)) if f(n) ≥ cg(n) for some cand large enough n.
Informally, think of it as “f ≥ g”.
![Page 122: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/122.jpg)
41/69
Ω-Notation: Asymptotic Lower Bound
O-Notation For a function g(n),
O(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
Ω-Notation For a function g(n),
Ω(g(n)) =
function f : ∃c > 0, n0 > 0 such that
f(n) ≥ cg(n),∀n ≥ n0
.
In other words, f(n) ∈ Ω(g(n)) if f(n) ≥ cg(n) for some cand large enough n.
Informally, think of it as “f ≥ g”.
![Page 123: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/123.jpg)
42/69
Ω-Notation: Asymptotic Lower Bound
Again, we use “=” instead of ∈.
4n2 = Ω(n)3n2 − n + 10 = Ω(n2)Ω(n2) + n = Ω(n2) = Ω(n)
Theorem f(n) = O(g(n)) ⇔ g(n) = Ω(f(n)).
![Page 124: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/124.jpg)
42/69
Ω-Notation: Asymptotic Lower Bound
Again, we use “=” instead of ∈.
4n2 = Ω(n)3n2 − n + 10 = Ω(n2)Ω(n2) + n = Ω(n2) = Ω(n)
Theorem f(n) = O(g(n)) ⇔ g(n) = Ω(f(n)).
![Page 125: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/125.jpg)
43/69
Θ-Notation: Asymptotic Tight Bound
Θ-Notation For a function g(n),
Θ(g(n)) =
function f : ∃c2 ≥ c1 > 0, n0 > 0 such that
c1g(n) ≤ f(n) ≤ c2g(n),∀n ≥ n0
.
f(n) = Θ(g(n)), then for large enough n, we have“f(n) ≈ g(n)”.
Informally, think of it as “f = g”.
3n2 + 2n = Θ(n2)
2n/3+100 = Θ(2n/3)
Theorem f(n) = Θ(g(n)) if and only iff(n) = O(g(n)) and f(n) = Ω(g(n)).
![Page 126: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/126.jpg)
43/69
Θ-Notation: Asymptotic Tight Bound
Θ-Notation For a function g(n),
Θ(g(n)) =
function f : ∃c2 ≥ c1 > 0, n0 > 0 such that
c1g(n) ≤ f(n) ≤ c2g(n),∀n ≥ n0
.
f(n) = Θ(g(n)), then for large enough n, we have“f(n) ≈ g(n)”.
Informally, think of it as “f = g”.
3n2 + 2n = Θ(n2)
2n/3+100 = Θ(2n/3)
Theorem f(n) = Θ(g(n)) if and only iff(n) = O(g(n)) and f(n) = Ω(g(n)).
![Page 127: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/127.jpg)
43/69
Θ-Notation: Asymptotic Tight Bound
Θ-Notation For a function g(n),
Θ(g(n)) =
function f : ∃c2 ≥ c1 > 0, n0 > 0 such that
c1g(n) ≤ f(n) ≤ c2g(n),∀n ≥ n0
.
f(n) = Θ(g(n)), then for large enough n, we have“f(n) ≈ g(n)”.
Informally, think of it as “f = g”.
3n2 + 2n = Θ(n2)
2n/3+100 = Θ(2n/3)
Theorem f(n) = Θ(g(n)) if and only iff(n) = O(g(n)) and f(n) = Ω(g(n)).
![Page 128: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/128.jpg)
43/69
Θ-Notation: Asymptotic Tight Bound
Θ-Notation For a function g(n),
Θ(g(n)) =
function f : ∃c2 ≥ c1 > 0, n0 > 0 such that
c1g(n) ≤ f(n) ≤ c2g(n),∀n ≥ n0
.
f(n) = Θ(g(n)), then for large enough n, we have“f(n) ≈ g(n)”.
Informally, think of it as “f = g”.
3n2 + 2n = Θ(n2)
2n/3+100 = Θ(2n/3)
Theorem f(n) = Θ(g(n)) if and only iff(n) = O(g(n)) and f(n) = Ω(g(n)).
![Page 129: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/129.jpg)
43/69
Θ-Notation: Asymptotic Tight Bound
Θ-Notation For a function g(n),
Θ(g(n)) =
function f : ∃c2 ≥ c1 > 0, n0 > 0 such that
c1g(n) ≤ f(n) ≤ c2g(n),∀n ≥ n0
.
f(n) = Θ(g(n)), then for large enough n, we have“f(n) ≈ g(n)”.
Informally, think of it as “f = g”.
3n2 + 2n = Θ(n2)
2n/3+100 = Θ(2n/3)
Theorem f(n) = Θ(g(n)) if and only iff(n) = O(g(n)) and f(n) = Ω(g(n)).
![Page 130: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/130.jpg)
43/69
Θ-Notation: Asymptotic Tight Bound
Θ-Notation For a function g(n),
Θ(g(n)) =
function f : ∃c2 ≥ c1 > 0, n0 > 0 such that
c1g(n) ≤ f(n) ≤ c2g(n),∀n ≥ n0
.
f(n) = Θ(g(n)), then for large enough n, we have“f(n) ≈ g(n)”.
Informally, think of it as “f = g”.
3n2 + 2n = Θ(n2)
2n/3+100 = Θ(2n/3)
Theorem f(n) = Θ(g(n)) if and only iff(n) = O(g(n)) and f(n) = Ω(g(n)).
![Page 131: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/131.jpg)
44/69
Exercise
For each pair of functions f, g in the following table, indicatewhether f is O,Ω or Θ of g.
f g O Ω Θ
lg10 n n0.1
Yes No No
2n 2n/2
No Yes No
√n nsinn
No No No
n2 − 100n 5n2 + 30n
Yes Yes Yes
![Page 132: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/132.jpg)
44/69
Exercise
For each pair of functions f, g in the following table, indicatewhether f is O,Ω or Θ of g.
f g O Ω Θ
lg10 n n0.1 Yes No No
2n 2n/2
No Yes No
√n nsinn
No No No
n2 − 100n 5n2 + 30n
Yes Yes Yes
![Page 133: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/133.jpg)
44/69
Exercise
For each pair of functions f, g in the following table, indicatewhether f is O,Ω or Θ of g.
f g O Ω Θ
lg10 n n0.1 Yes No No
2n 2n/2 No Yes No√n nsinn
No No No
n2 − 100n 5n2 + 30n
Yes Yes Yes
![Page 134: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/134.jpg)
44/69
Exercise
For each pair of functions f, g in the following table, indicatewhether f is O,Ω or Θ of g.
f g O Ω Θ
lg10 n n0.1 Yes No No
2n 2n/2 No Yes No√n nsinn No No No
n2 − 100n 5n2 + 30n
Yes Yes Yes
![Page 135: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/135.jpg)
44/69
Exercise
For each pair of functions f, g in the following table, indicatewhether f is O,Ω or Θ of g.
f g O Ω Θ
lg10 n n0.1 Yes No No
2n 2n/2 No Yes No√n nsinn No No No
n2 − 100n 5n2 + 30n Yes Yes Yes
![Page 136: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/136.jpg)
45/69
Asymptotic Notations O Ω ΘComparison Relations ≤ ≥ =
Trivial Facts on Comparison Relations
f ≤ g ⇔ g ≥ f
f = g ⇔ f ≤ g and f ≥ g
f ≤ g or f ≥ g
Correct Analogies
f(n) = O(g(n)) ⇔ g(n) = Ω(f(n))
f(n) = Θ(g(n)) ⇔ f(n) = O(g(n)) and f(n) = Ω(g(n))
Incorrect Analogy
f(n) = O(g(n)) or g(n) = O(f(n))
![Page 137: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/137.jpg)
45/69
Asymptotic Notations O Ω ΘComparison Relations ≤ ≥ =
Trivial Facts on Comparison Relations
f ≤ g ⇔ g ≥ f
f = g ⇔ f ≤ g and f ≥ g
f ≤ g or f ≥ g
Correct Analogies
f(n) = O(g(n)) ⇔ g(n) = Ω(f(n))
f(n) = Θ(g(n)) ⇔ f(n) = O(g(n)) and f(n) = Ω(g(n))
Incorrect Analogy
f(n) = O(g(n)) or g(n) = O(f(n))
![Page 138: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/138.jpg)
45/69
Asymptotic Notations O Ω ΘComparison Relations ≤ ≥ =
Trivial Facts on Comparison Relations
f ≤ g ⇔ g ≥ f
f = g ⇔ f ≤ g and f ≥ g
f ≤ g or f ≥ g
Correct Analogies
f(n) = O(g(n)) ⇔ g(n) = Ω(f(n))
f(n) = Θ(g(n)) ⇔ f(n) = O(g(n)) and f(n) = Ω(g(n))
Incorrect Analogy
f(n) = O(g(n)) or g(n) = O(f(n))
![Page 139: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/139.jpg)
45/69
Asymptotic Notations O Ω ΘComparison Relations ≤ ≥ =
Trivial Facts on Comparison Relations
f ≤ g ⇔ g ≥ f
f = g ⇔ f ≤ g and f ≥ g
f ≤ g or f ≥ g
Correct Analogies
f(n) = O(g(n)) ⇔ g(n) = Ω(f(n))
f(n) = Θ(g(n)) ⇔ f(n) = O(g(n)) and f(n) = Ω(g(n))
Incorrect Analogy
f(n) = O(g(n)) or g(n) = O(f(n))
![Page 140: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/140.jpg)
46/69
Incorrect Analogy
f(n) = O(g(n)) or g(n) = O(f(n))
f(n) = n2
g(n) =
1 if n is odd
2n if n is even
![Page 141: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/141.jpg)
46/69
Incorrect Analogy
f(n) = O(g(n)) or g(n) = O(f(n))
f(n) = n2
g(n) =
1 if n is odd
2n if n is even
![Page 142: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/142.jpg)
47/69
Recall: informal way to define O-notation
ignoring lower order terms: 3n2 − 10n− 5→ 3n2
ignoring leading constant: 3n2 → n2
Thus 3n2 − 10n− 5 = O(n2)
Indeed, 3n2 − 10n− 5 = Ω(n2), 3n2 − 10n− 5 = Θ(n2)
Formally: if n > 10, then n2 < 3n2 − 10n− 5 < 3n2. So,3n2 − 10n− 5 ∈ Θ(n2).
![Page 143: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/143.jpg)
47/69
Recall: informal way to define O-notation
ignoring lower order terms: 3n2 − 10n− 5→ 3n2
ignoring leading constant: 3n2 → n2
Thus 3n2 − 10n− 5 = O(n2)
Indeed, 3n2 − 10n− 5 = Ω(n2), 3n2 − 10n− 5 = Θ(n2)
Formally: if n > 10, then n2 < 3n2 − 10n− 5 < 3n2. So,3n2 − 10n− 5 ∈ Θ(n2).
![Page 144: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/144.jpg)
47/69
Recall: informal way to define O-notation
ignoring lower order terms: 3n2 − 10n− 5→ 3n2
ignoring leading constant: 3n2 → n2
Thus 3n2 − 10n− 5 = O(n2)
Indeed, 3n2 − 10n− 5 = Ω(n2), 3n2 − 10n− 5 = Θ(n2)
Formally: if n > 10, then n2 < 3n2 − 10n− 5 < 3n2. So,3n2 − 10n− 5 ∈ Θ(n2).
![Page 145: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/145.jpg)
47/69
Recall: informal way to define O-notation
ignoring lower order terms: 3n2 − 10n− 5→ 3n2
ignoring leading constant: 3n2 → n2
Thus 3n2 − 10n− 5 = O(n2)
Indeed, 3n2 − 10n− 5 = Ω(n2), 3n2 − 10n− 5 = Θ(n2)
Formally: if n > 10, then n2 < 3n2 − 10n− 5 < 3n2. So,3n2 − 10n− 5 ∈ Θ(n2).
![Page 146: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/146.jpg)
48/69
o and ω-Notations
o-Notation For a function g(n),
o(g(n)) =
function f : ∀c > 0,∃n0 > 0 such that
f(n) ≤ cg(n),∀n ≥ n0
.
ω-Notation For a function g(n),
ω(g(n)) =
function f : ∀c > 0,∃n0 > 0 such that
f(n) ≥ cg(n),∀n ≥ n0
.
Example:
3n2 + 5n + 10 = o(n2 lg n).
3n2 + 5n + 10 = ω(n2/ lg n).
![Page 147: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/147.jpg)
49/69
Asymptotic Notations O Ω Θ o ωComparison Relations ≤ ≥ = < >
Questions?
![Page 148: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/148.jpg)
49/69
Asymptotic Notations O Ω Θ o ωComparison Relations ≤ ≥ = < >
Questions?
![Page 149: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/149.jpg)
50/69
Outline
1 Syllabus
2 IntroductionWhat is an Algorithm?Example: Insertion SortAnalysis of Insertion Sort
3 Asymptotic Notations
4 Common Running times
![Page 150: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/150.jpg)
51/69
O(n) (Linear) Running Time
Computing the sum of n numbers
sum(A, n)
1 S ← 0
2 for i← 1 to n
3 S ← S + A[i]
4 return S
![Page 151: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/151.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
![Page 152: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/152.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
![Page 153: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/153.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3
![Page 154: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/154.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3
![Page 155: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/155.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5
![Page 156: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/156.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5
![Page 157: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/157.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5 7
![Page 158: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/158.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5 7
![Page 159: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/159.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5 7 8
![Page 160: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/160.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5 7 8
![Page 161: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/161.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5 7 8 9 12 20 25 29
![Page 162: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/162.jpg)
52/69
O(n) (Linear) Running Time
Merge two sorted arrays
3 8 12 20 32 48
5 7 9 25 29
3 5 7 8 9 12 20 25 29 32 48
![Page 163: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/163.jpg)
53/69
O(n) (Linear) Running Time
merge(B,C, n1, n2) \\ B and C are sorted, with length n1
and n2
1 A← []; i← 1; j ← 1
2 while i ≤ n1 and j ≤ n2
3 if (B[i] ≤ C[j]) then
4 append B[i] to A; i← i + 1
5 else
6 append C[j] to A; j ← j + 1
7 if i ≤ n1 then append B[i..n1] to A
8 if j ≤ n2 then append C[j..n2] to A
9 return A
Running time = O(n) where n = n1 + n2.
![Page 164: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/164.jpg)
53/69
O(n) (Linear) Running Time
merge(B,C, n1, n2) \\ B and C are sorted, with length n1
and n2
1 A← []; i← 1; j ← 1
2 while i ≤ n1 and j ≤ n2
3 if (B[i] ≤ C[j]) then
4 append B[i] to A; i← i + 1
5 else
6 append C[j] to A; j ← j + 1
7 if i ≤ n1 then append B[i..n1] to A
8 if j ≤ n2 then append C[j..n2] to A
9 return A
Running time = O(n) where n = n1 + n2.
![Page 165: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/165.jpg)
54/69
O(n lg n) Running Time
merge-sort(A, n)
1 if n = 1 then
2 return A
3 else
4 B ← merge-sort(A[1..bn/2c
], bn/2c
)5 C ← merge-sort
(A[bn/2c+ 1..n
], n− bn/2c
)6 return merge(B,C, bn/2c, n− bn/2c)
![Page 166: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/166.jpg)
55/69
O(n lg n) Running Time
Merge-Sort
A[1..8]
A[1..4] A[5..8]
A[5..6] A[7..8]A[3..4]A[1..2]
A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8]
Each level takes running time O(n)
There are O(lg n) levels
Running time = O(n lg n)
![Page 167: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/167.jpg)
55/69
O(n lg n) Running Time
Merge-Sort
A[1..8]
A[1..4] A[5..8]
A[5..6] A[7..8]A[3..4]A[1..2]
A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8]
Each level takes running time O(n)
There are O(lg n) levels
Running time = O(n lg n)
![Page 168: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/168.jpg)
55/69
O(n lg n) Running Time
Merge-Sort
A[1..8]
A[1..4] A[5..8]
A[5..6] A[7..8]A[3..4]A[1..2]
A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8]
Each level takes running time O(n)
There are O(lg n) levels
Running time = O(n lg n)
![Page 169: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/169.jpg)
55/69
O(n lg n) Running Time
Merge-Sort
A[1..8]
A[1..4] A[5..8]
A[5..6] A[7..8]A[3..4]A[1..2]
A[1] A[2] A[3] A[4] A[5] A[6] A[7] A[8]
Each level takes running time O(n)
There are O(lg n) levels
Running time = O(n lg n)
![Page 170: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/170.jpg)
56/69
O(n2) (Quardatic) Running Time
Closest Pair
Input: n points in plane: (x1, y1), (x2, y2), · · · , (xn, yn)
Output: the pair of points that are closest
![Page 171: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/171.jpg)
56/69
O(n2) (Quardatic) Running Time
Closest Pair
Input: n points in plane: (x1, y1), (x2, y2), · · · , (xn, yn)
Output: the pair of points that are closest
![Page 172: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/172.jpg)
57/69
O(n2) (Quardatic) Running Time
Closest Pair
Input: n points in plane: (x1, y1), (x2, y2), · · · , (xn, yn)
Output: the pair of points that are closest
closest-pair(x, y, n)
1 bestd←∞2 for i← 1 to n− 1
3 for j ← i + 1 to n
4 d←√
(x[i]− x[j])2 + (y[i]− y[j])2
5 if d < bestd then
6 besti← i, bestj ← j, bestd← d
7 return (besti, bestj)
Closest pair can be solved in O(n lg n) time!
![Page 173: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/173.jpg)
57/69
O(n2) (Quardatic) Running Time
Closest Pair
Input: n points in plane: (x1, y1), (x2, y2), · · · , (xn, yn)
Output: the pair of points that are closest
closest-pair(x, y, n)
1 bestd←∞2 for i← 1 to n− 1
3 for j ← i + 1 to n
4 d←√
(x[i]− x[j])2 + (y[i]− y[j])2
5 if d < bestd then
6 besti← i, bestj ← j, bestd← d
7 return (besti, bestj)
Closest pair can be solved in O(n lg n) time!
![Page 174: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/174.jpg)
58/69
O(n3) (Cubic) Running Time
Multiply two matrices of size n× n
matrix-multiplication(A,B, n)
1 C ← matrix of size n× n, with all entries being 0
2 for i← 1 to n
3 for j ← 1 to n
4 for k ← 1 to n
5 C[i, k]← C[i, k] + A[i, j]×B[j, k]
6 return C
![Page 175: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/175.jpg)
59/69
O(nk) Running Time for Integer k ≥ 4
Def. An independent set of a graph G = (V,E) is a subsetS ⊆ V of vertices such that for every u, v ∈ S, we have(u, v) /∈ E.
Independent set of size k
Input: graph G = (V,E), an integer k
Output: whether there is an independent set of size k
![Page 176: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/176.jpg)
59/69
O(nk) Running Time for Integer k ≥ 4
Def. An independent set of a graph G = (V,E) is a subsetS ⊆ V of vertices such that for every u, v ∈ S, we have(u, v) /∈ E.
Independent set of size k
Input: graph G = (V,E), an integer k
Output: whether there is an independent set of size k
![Page 177: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/177.jpg)
59/69
O(nk) Running Time for Integer k ≥ 4
Def. An independent set of a graph G = (V,E) is a subsetS ⊆ V of vertices such that for every u, v ∈ S, we have(u, v) /∈ E.
Independent set of size k
Input: graph G = (V,E), an integer k
Output: whether there is an independent set of size k
![Page 178: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/178.jpg)
59/69
O(nk) Running Time for Integer k ≥ 4
Def. An independent set of a graph G = (V,E) is a subsetS ⊆ V of vertices such that for every u, v ∈ S, we have(u, v) /∈ E.
Independent set of size k
Input: graph G = (V,E), an integer k
Output: whether there is an independent set of size k
![Page 179: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/179.jpg)
60/69
O(nk) Running Time for Integer k ≥ 4
Independent Set of Size k
Input: graph G = (V,E)
Output: whether there is an independent set of size k
independent-set(G = (V,E))
1 for every set S ⊆ V of size k
2 b← true
3 for every u, v ∈ S
4 if (u, v) ∈ E then b← false
5 if b return true
6 return false
Running time = O(nk
k!× k2) = O(nk) (assume k is a constant)
![Page 180: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/180.jpg)
61/69
Beyond Polynomial Time: O(2n)
Maximum Independent Set Problem
Input: graph G = (V,E)
Output: the maximum independent set of G
max-independent-set(G = (V,E))
1 R← ∅2 for every set S ⊆ V
3 b← true
4 for every u, v ∈ S
5 if (u, v) ∈ E then b← false
6 if b and |S| > |R| then R← S
7 return R
Running time = O(2nn2).
![Page 181: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/181.jpg)
62/69
Beyond Polynomial Time: O(n!)
Hamiltonian Cycle Problem
Input: a graph with n vertices
Output: a cycle that visits each node exactly once,
or say no such cycle exists
![Page 182: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/182.jpg)
62/69
Beyond Polynomial Time: O(n!)
Hamiltonian Cycle Problem
Input: a graph with n vertices
Output: a cycle that visits each node exactly once,
or say no such cycle exists
![Page 183: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/183.jpg)
63/69
Beyond Polynomial Time: n!
Hamiltonian(G = (V,E))
1 for every permutation (p1, p2, · · · , pn) of V
2 b← true
3 for i← 1 to n− 1
4 if (pi, pi+1) /∈ E then b← false
5 if (pn, p1) /∈ E then b← false
6 if b then return (p1, p2, · · · , pn)
7 return “No Hamiltonian Cycle”
Running time = O(n!× n)
![Page 184: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/184.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
![Page 185: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/185.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
![Page 186: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/186.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 187: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/187.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 188: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/188.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 189: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/189.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
42 > 35
![Page 190: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/190.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 191: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/191.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 192: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/192.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
25 < 35
![Page 193: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/193.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 194: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/194.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 195: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/195.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
37 > 35
![Page 196: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/196.jpg)
64/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;Output: whether t appears in A.
E.g, search 35 in the following array:
3 8 10 25 29 37 38 42 46 52 59 61 63 75 79
![Page 197: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/197.jpg)
65/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;
Output: whether t appears in A.
binary-search(A, n, t)
1 i← 1, j ← n
2 while i ≤ j do
3 k ← b(i + j)/2c4 if A[k] = t return true
5 if A[k] < t then j ← k − 1 else i← k + 1
6 return false
Running time = O(lg n)
![Page 198: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/198.jpg)
65/69
O(lg n) (Logarithmic) Running Time
Binary search
Input: sorted array A of size n, an integer t;
Output: whether t appears in A.
binary-search(A, n, t)
1 i← 1, j ← n
2 while i ≤ j do
3 k ← b(i + j)/2c4 if A[k] = t return true
5 if A[k] < t then j ← k − 1 else i← k + 1
6 return false
Running time = O(lg n)
![Page 199: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/199.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 200: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/200.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 201: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/201.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 202: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/202.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 203: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/203.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 204: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/204.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 205: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/205.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 206: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/206.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 207: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/207.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 208: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/208.jpg)
66/69
Compare the Orders
Sort the functions from asymptotically smallest toasymptotically largest (informally, using “<” and “=”)n√n, lg n, n, n2, n lg n, n!, 2n, en, lg(n!), nn
lg n < n√n
lg n < n < n√n
lg n < n < n2 < n√n
lg n < n < n lg n < n2 < n√n
lg n < n < n lg n < n2 < n√n < n!
lg n < n < n lg n < n2 < n√n < 2n < n!
lg n < n < n lg n < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n!
lg n < n < n lg n = lg(n!) < n2 < n√n < 2n < en < n! < nn
![Page 209: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/209.jpg)
67/69
Terminologies
When we talk about upper bounds:
Logarithmic time: O(lg n)
Linear time: O(n)
Quadratic time O(n2)
Cubic time O(n3)
Polynomial time: O(nk) for some constant k
Exponential time: O(cn) for some c > 1
Sub-linear time: o(n)
Sub-quadratic time: o(n2)
When we talk about lower bounds:
Super-linear time: ω(n)
Super-quadratic time: ω(n2)
Super-polynomial time:⋂
k>0 ω(nk)
![Page 210: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/210.jpg)
67/69
Terminologies
When we talk about upper bounds:
Logarithmic time: O(lg n)
Linear time: O(n)
Quadratic time O(n2)
Cubic time O(n3)
Polynomial time: O(nk) for some constant k
Exponential time: O(cn) for some c > 1
Sub-linear time: o(n)
Sub-quadratic time: o(n2)
When we talk about lower bounds:
Super-linear time: ω(n)
Super-quadratic time: ω(n2)
Super-polynomial time:⋂
k>0 ω(nk)
![Page 211: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/211.jpg)
68/69
Goal of Algorithm Design
Design algorithms to minimize the order of the running time.
Using asymptotic analysis allows us to ignore the leadingconstants and lower order terms
Makes our life much easier! (E.g., the leading constantdepends on the implementation, complier and computerarchitecture of computer.)
![Page 212: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/212.jpg)
68/69
Goal of Algorithm Design
Design algorithms to minimize the order of the running time.
Using asymptotic analysis allows us to ignore the leadingconstants and lower order terms
Makes our life much easier! (E.g., the leading constantdepends on the implementation, complier and computerarchitecture of computer.)
![Page 213: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/213.jpg)
68/69
Goal of Algorithm Design
Design algorithms to minimize the order of the running time.
Using asymptotic analysis allows us to ignore the leadingconstants and lower order terms
Makes our life much easier! (E.g., the leading constantdepends on the implementation, complier and computerarchitecture of computer.)
![Page 214: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/214.jpg)
69/69
Q: Does ignoring the leading constant cause any issues?
e.g, how can we compare an algorithm with running time0.1n2 with an algorithm with running time 1000n?
A:
Sometimes yes
However, when n is big enough, 1000n < 0.1n2
For “natural” algorithms, constants are not so big!
For reasonable n, algorithm with lower order running timebeats algorithm with higher order running time.
![Page 215: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/215.jpg)
69/69
Q: Does ignoring the leading constant cause any issues?
e.g, how can we compare an algorithm with running time0.1n2 with an algorithm with running time 1000n?
A:
Sometimes yes
However, when n is big enough, 1000n < 0.1n2
For “natural” algorithms, constants are not so big!
For reasonable n, algorithm with lower order running timebeats algorithm with higher order running time.
![Page 216: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/216.jpg)
69/69
Q: Does ignoring the leading constant cause any issues?
e.g, how can we compare an algorithm with running time0.1n2 with an algorithm with running time 1000n?
A:
Sometimes yes
However, when n is big enough, 1000n < 0.1n2
For “natural” algorithms, constants are not so big!
For reasonable n, algorithm with lower order running timebeats algorithm with higher order running time.
![Page 217: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/217.jpg)
69/69
Q: Does ignoring the leading constant cause any issues?
e.g, how can we compare an algorithm with running time0.1n2 with an algorithm with running time 1000n?
A:
Sometimes yes
However, when n is big enough, 1000n < 0.1n2
For “natural” algorithms, constants are not so big!
For reasonable n, algorithm with lower order running timebeats algorithm with higher order running time.
![Page 218: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/218.jpg)
69/69
Q: Does ignoring the leading constant cause any issues?
e.g, how can we compare an algorithm with running time0.1n2 with an algorithm with running time 1000n?
A:
Sometimes yes
However, when n is big enough, 1000n < 0.1n2
For “natural” algorithms, constants are not so big!
For reasonable n, algorithm with lower order running timebeats algorithm with higher order running time.
![Page 219: Department of Computer Science and Engineering University ...shil/courses/CSE531-Fall2016/Slides/... · CSE 431/531: Analysis of Algorithms Introduction and Syllabus Lecturer: Shi](https://reader034.vdocuments.us/reader034/viewer/2022050414/5f8ab9b8604ed558f8298d1f/html5/thumbnails/219.jpg)
69/69
Q: Does ignoring the leading constant cause any issues?
e.g, how can we compare an algorithm with running time0.1n2 with an algorithm with running time 1000n?
A:
Sometimes yes
However, when n is big enough, 1000n < 0.1n2
For “natural” algorithms, constants are not so big!
For reasonable n, algorithm with lower order running timebeats algorithm with higher order running time.