comp26120: introducing complexity analysis...
TRANSCRIPT
![Page 2: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/2.jpg)
Introducing Complexity Analysis
• Textbook: § Algorithm Design and Applications, Goodrich, Michael
T. and Roberto Tamassia (Chapter 1) § Introduction to Algorithms, Cormen, Leiserson, Rivest,
Stein (Chapters 2 and 3)
![Page 3: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/3.jpg)
Motivating Example
…!for(i = 0; i < N-1; i++) { for(j = 0; j < N-1; j++) { if (a[j]> a[j+1]) { t = a[j]; a[j] = a[j+1]; a[j+1] = t; } } }…!
• What does this code fragment represent? What is its complexity?
§ This is bubble sort
§ There are two loops
§ Both loops make n-1 iterations so we have (n-1)*(n-1)
§ The complexity is O(n2)
Perform worst case analysis and ignore constants
![Page 4: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/4.jpg)
• Define asymptotic notation, functions, and running times
• Analyze the running time used by an algorithm via asymptotic analysis
• Provide examples of asymptotic analysis using the insertion sorting algorithm
Intended Learning Outcomes
![Page 5: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/5.jpg)
Asymptotic Performance
• In this course, we care most about asymptotic performance § We focus on the infinite set of large n ignoring small
values of n o The best choice for all, but minimal inputs
• How does the algorithm behave as the problem size gets huge? § Running time
§ Memory/storage requirements
§ Bandwidth/power requirements/logic gates/etc.
![Page 6: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/6.jpg)
Asymptotic Notation
• By now, you should have an intuitive feel for asymptotic (big-O) notation: § What does O(n) running time mean? O(n2)?
O(log n)?
§ How does asymptotic running time relate to asymptotic memory usage?
• Our first task is to define this notation more formally
![Page 7: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/7.jpg)
a1, a2, a3,….,an; q j
2 5 4 10 7; 5 2 2 5 4 10 7; 9 NIL
Search Problem (Arbitrary Sequence)
Input • sequence of numbers (a1,…an) • search for a specific number (q)
Output • index or NIL
![Page 8: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/8.jpg)
j=1 while j<=length(A) and A[j]!=q do j++ if j<=length(A) then return j else return NIL
• Worst case: f(n)=n, average case: n/2 • Can we do better using this approach?
§ this is a lower bound for the search problem in an arbitrary sequence
Linear Search
![Page 9: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/9.jpg)
a1, a2, a3,….,an; q j
2 4 5 7 10; 10 5 2 4 5 7 10; 8 NIL
A Search Problem (Sorted Sequence)
Input • sequence of numbers (a1<=a2,…, an-1<=an) • search for a specific number (q)
Output • index or NIL
Did the sorted sequence help in the search?
![Page 10: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/10.jpg)
left=1 right=length(A) do j=(left+right)/2 if A[j]==q then return j else if A[j]>q then right=j-1 else left=j+1
while left<=right return NIL
• Assume that the array is sorted and then perform successive divisions
Binary Search
![Page 11: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/11.jpg)
• How many times is the loop executed? § At each interaction, the number of positions n is cut in
half § How many times do we cut in half n to reach 1?
o lg2 n
38lg2 =
xnxn 2lg2 =⇔=
3 6 8 4 9 10 12 15
Binary Search Analysis
![Page 12: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/12.jpg)
• We perform the analysis concerning a computational model
• We will usually use a generic uniprocessor random-access machine (RAM) § All memory equally expensive to access § Instructions executed one after another (no
concurrent operations) § All reasonable instructions take unit time
o Except, of course, function calls
§ Constant word size o Unless we are explicitly manipulating bits
Analysis of Algorithms (COMP15111)
![Page 13: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/13.jpg)
Input Size
• Time and space complexity § This is generally a function of the input size
o E.g., sorting, multiplication
§ How we characterize input size depends: o Sorting: number of input items
o Multiplication: total number of bits
o Graph algorithms: number of nodes and edges
o Etc.
![Page 14: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/14.jpg)
Running Time • Number of primitive steps that are executed
§ Except for time of executing a function call most statements roughly require the same amount of time
o f = (g + h) – (i + j)
o y = m * x + b
o c = 5 / 9 * (t - 32 )
o z = f(x) + g(y)
• We can be more exact if needed
add t0, g, h # temp t0 = g + h add t1, i, j # temp t1 = i + j sub f, t0, t1 # f = t0 - t1
![Page 15: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/15.jpg)
Analysis
• Worst case § Provides an upper bound on running time
§ An (absolute) guarantee
• Average case § Provides the expected running time
§ Very useful, but treat with care: what is “average”? o Random (equally likely) inputs
o Real-life inputs
![Page 16: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/16.jpg)
An Example: Insertion Sort
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
![Page 17: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/17.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
30 10 40 20
1 2 3 4
i = ∅ j = ∅ key = ∅ A[j] = ∅ A[j+1] = ∅
An Example: Insertion Sort
![Page 18: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/18.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
30 10 40 20
1 2 3 4
i = 2 j = 1 key = 10 A[j] = 30 A[j+1] = 10
An Example: Insertion Sort
![Page 19: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/19.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
30 30 40 20
1 2 3 4
i = 2 j = 1 key = 10 A[j] = 30 A[j+1] = 30
An Example: Insertion Sort
![Page 20: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/20.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
30 30 40 20
1 2 3 4
i = 2 j = 1 key = 10 A[j] = 30 A[j+1] = 30
An Example: Insertion Sort
![Page 21: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/21.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
30 30 40 20
1 2 3 4
i = 2 j = 0 key = 10 A[j] = ∅ A[j+1] = 30
An Example: Insertion Sort
![Page 22: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/22.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
30 30 40 20
1 2 3 4
i = 2 j = 0 key = 10 A[j] = ∅ A[j+1] = 30
An Example: Insertion Sort
![Page 23: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/23.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 2 j = 0 key = 10 A[j] = ∅ A[j+1] = 10
An Example: Insertion Sort
![Page 24: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/24.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 3 j = 0 key = 10 A[j] = ∅ A[j+1] = 10
An Example: Insertion Sort
![Page 25: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/25.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 3 j = 0 key = 40 A[j] = ∅ A[j+1] = 10
An Example: Insertion Sort
![Page 26: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/26.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 3 j = 0 key = 40 A[j] = ∅ A[j+1] = 10
An Example: Insertion Sort
![Page 27: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/27.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 3 j = 2 key = 40 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 28: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/28.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 3 j = 2 key = 40 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 29: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/29.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 3 j = 2 key = 40 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 30: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/30.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 4 j = 2 key = 40 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 31: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/31.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 32: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/32.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 33: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/33.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 20
An Example: Insertion Sort
![Page 34: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/34.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 20
1 2 3 4
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 20
An Example: Insertion Sort
![Page 35: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/35.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 40
1 2 3 4
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 40
An Example: Insertion Sort
![Page 36: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/36.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 40
1 2 3 4
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 40
An Example: Insertion Sort
![Page 37: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/37.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 40
1 2 3 4
i = 4 j = 3 key = 20 A[j] = 40 A[j+1] = 40
An Example: Insertion Sort
![Page 38: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/38.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 40
1 2 3 4
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 39: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/39.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 40 40
1 2 3 4
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 40
An Example: Insertion Sort
![Page 40: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/40.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 30 40
1 2 3 4
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 30
An Example: Insertion Sort
![Page 41: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/41.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 30 40
1 2 3 4
i = 4 j = 2 key = 20 A[j] = 30 A[j+1] = 30
An Example: Insertion Sort
![Page 42: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/42.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 30 40
1 2 3 4
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 30
An Example: Insertion Sort
![Page 43: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/43.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 30 30 40
1 2 3 4
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 30
An Example: Insertion Sort
![Page 44: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/44.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 20 30 40
1 2 3 4
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 20
An Example: Insertion Sort
![Page 45: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/45.jpg)
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
10 20 30 40
1 2 3 4
i = 4 j = 1 key = 20 A[j] = 10 A[j+1] = 20
Done!
An Example: Insertion Sort
![Page 46: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/46.jpg)
Insertion Sort
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
What is the precondition for this loop?
![Page 47: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/47.jpg)
Insertion Sort
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
How many times will this loop execute?
![Page 48: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/48.jpg)
Insertion Sort
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
What is the post-condition for this loop?
![Page 49: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/49.jpg)
Insertion Sort
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
Invariant: A [1..i-1] consists of the elements originally in A [1..i-1], but in sorted order
![Page 50: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/50.jpg)
Insertion Sort
InsertionSort(A, n) { for i = 2 to n {
key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key
} }
Termination: when i == n+1 we have A[1..i-1] which leads to A[1..n]
![Page 51: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/51.jpg)
Insertion Sort
Statement Effort InsertionSort(A, n) { for i = 2 to n { c1n key = A[i] c2(n-1) j = i - 1; c3(n-1) while (j > 0) and (A[j] > key) { c4T A[j+1] = A[j] c5(T-(n-1)) j = j - 1 c6(T-(n-1)) } 0 A[j+1] = key c7(n-1) } 0
} T = t2 + t3 + … + tn where ti is number of while expression
evaluations for the ith for loop iteration
![Page 52: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/52.jpg)
Analysing Insertion Sort
• T(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1)
• What can T be? § Best case -- inner loop body never executed § T = t2 + t3 + … + tn
§ T(n) = an - b
1111322
−=+++=+++=∑=
ntttt n
n
jj ……
( ) ( ) ( ) ( )1111)( 74321 −+−+−+−+= ncncncncncnT
( ) ( )743274321)( ccccncccccnT +++−⋅++++=
![Page 53: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/53.jpg)
( )
( )
( ) ( ) ( ) ( )21
221
21...1
12132
2121
2
2
1
−=
−+=−
+=−
−+
=+++
+=+++
∑
∑
∑
nnnnnnnn=j
nnn=j
nnn=j
n
j=
n
j=
n
j=
…
!
Gaussian Closed Form can be defined as:
Thus, we have:
Similarly, we obtain:
Sum Review
![Page 54: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/54.jpg)
Analysing Insertion Sort
§ Worst case -- inner loop body executed for all previous elements
o T(n) = an2 + bn - c § Average case
o ???
( ) 121
2−
+∑
nn=jn
j=
( ) ( )211
2
−−∑
nn=jn
j=
( ) ( ) ( ) ( ) ( )
( ) ( )74327654
3212654
7
654321
2222221
21
211
2111)(
ccccncccccccncccnc
nncnncnncncncncnT
+++−⎟⎠
⎞⎜⎝
⎛ +−−++++⎟⎠
⎞⎜⎝
⎛ ++=−+
⎟⎠
⎞⎜⎝
⎛ −+⎟⎠
⎞⎜⎝
⎛ −+⎟⎠
⎞⎜⎝
⎛ −+
+−+−+=
![Page 55: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/55.jpg)
Simplifications • Abstract statement costs • Order of growth
average-case Average case
1n
2n
3n
4n
5n
6n
1 2 3 4 5 6 7 8 9 10 11 12 …
best-case
1n
2n
3n
4n
5n
6n
Best case
Worst case
Inputs size
Running tim
e
![Page 56: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/56.jpg)
Growth Functions
1,00E-01
1,00E+11
1,00E+23
1,00E+35
1,00E+47
1,00E+59
1,00E+71
1,00E+83
1,00E+95
1,00E+107
1,00E+119
1,00E+131
1,00E+143
1,00E+155
2 4 8 16 32 64 128 256 512 1024n
T(n)
nlog nsqrt nn log n100nn^2n^32^n
"E" represents "times ten raised to the power of"
![Page 57: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/57.jpg)
Thread T1
a1
a2
Thread T2
b1
b2
a1
a2
b1
b2
b1
a2 b2
a2 b2
b1
a1
a2 b2
a2 b2
a1
a2
b2
• the scheduler allows one thread to execute at a given time (emulate the execution on a single core)
• allow preemptions only before visible statements (global variables and synchronization points)
Thread interleavings:
a1; a2; b1; b2
a1; b1; a2; b2
…
Scheduler
![Page 58: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/58.jpg)
Exercise: Comparison of Running Times
• For each function f(n) and time t, determine the largest size n of a problem that can be solved in time t § the algorithm to solve the problem takes f(n) microseconds
log n n1/2 n n2 n3 2n n! 1 second
![Page 59: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/59.jpg)
Exercise: Comparison of Running Times
• For each function f(n) and time t, determine the largest size n of a problem that can be solved in time t § the algorithm to solve the problem takes f(n) microseconds
Assume a 30 day month and 365 day year
![Page 60: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/60.jpg)
Upper Bound Notation • InsertionSort’s runtime is
O(n2) § runtime is in O(n2) § Read O as “Big-O”
• In general, a function § f(n) is O(g(n)) if there exist
positive constants c and n0 such that f(n) ≤ c ⋅ g(n) for all n ≥ n0
)(nf( )c g n⋅
0n Input size
Running tim
e
![Page 61: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/61.jpg)
Insertion Sort Is O(n2)
• Proof: § Use the formal definition of O to demonstrate that an2
+ bn + c = O(n2)
§ If any of a, b, and c are less than 0 replace the constant with its absolute value
o 0 ≤ f(n) ≤ k ⋅ g(n) for all n ≥ n0 (k and n0 must be positive) o 0 ≤ an2 + bn + c <= kn2 o 0 ≤ a + b/n + c/n2 <= k
• Question § Is InsertionSort O(n)?
![Page 62: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/62.jpg)
Lower Bound Notation
• InsertionSort’s runtime is Ω(n)
• In general, a function § f(n) is Ω(g(n)) if there exist
positive constants c and n0 such that 0 ≤ c⋅g(n) ≤ f(n) ∀ n ≥ n0
• Proof: § Suppose runtime is an + b
o 0 ≤ cn ≤ an + b o 0 ≤ c ≤ a + b/n
Input size
Running tim
e )(nf( )c g n⋅
0n
![Page 63: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/63.jpg)
Asymptotic Tight Bound
• A function f(n) is Θ(g(n)) if there exist positive constants c1, c2, and n0 such that 0 ≤ c1 g(n) ≤ f(n) ≤ c2 g(n) for all n ≥ n0
• Theorem § f(n) is Θ(g(n)) iff f(n) is both
O(g(n)) and Ω(g(n)) Input size
Running tim
e )(nf
0n
)(ngc ⋅2
)(ngc ⋅1
![Page 64: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/64.jpg)
n Use the formal definition of Θ
to demonstrate that ( )22 321 nnn Θ=−
Exercise: Asymptotic Notation
1Within set notation, a colon means “such that”
![Page 65: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/65.jpg)
n Use the formal definition of Θ
to demonstrate that
Exercise: Asymptotic Notation
( )236 nn Θ≠
![Page 66: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/66.jpg)
Other Asymptotic Notations
• A function f(n) is o(g(n)) if ∃ positive constants c and n0 such that
f(n) < c g(n) ∀ n ≥ n0 • A function f(n) is ω(g(n)) if ∃ positive constants c
and n0 such that c g(n) < f(n) ∀ n ≥ n0
• Intuitively, § o() is like < § O() is like ≤
§ ω() is like > § Ω() is like ≥
§ Θ() is like =
![Page 67: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/67.jpg)
n We can draw an analogy between the asymptotic comparison of two functions f and g and the comparison of two real numbers a and b n f(n) = O(g(n)) is like a ≤ g
n f(n) = Ω(g(n)) is like a ≥ g
n f(n) = Θ(g(n)) is like a = g
n f(n) = o(g(n)) is like a < g
n f(n) = ω(g(n)) is like a > g
n Abuse of notation: n f(n) = O(g(n)) indica que f(n) ∈O(g(n))
Asymptotic Comparisons
![Page 68: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/68.jpg)
Check whether these statements are true: a) In the worst case, the insertion sort is
b) c) d) e) f)
)( 2nΘ
( ) ( ) ( )
O(n)=O(n)O(n))O(n=)O(n+)O(n
nn)O(=)O(=n+n
n
×
Θ=Θ+Θ222
1
2n
12222
Exercise: Asymptotic Notation
![Page 69: COMP26120: Introducing Complexity Analysis (2019/20)syllabus.cs.manchester.ac.uk/ugt/2019/COMP26120/2019... · 2019-10-22 · Complexity Analysis (2019/20) Lucas Cordeiro lucas.cordeiro@manchester.ac.uk](https://reader036.vdocuments.us/reader036/viewer/2022070904/5f7007c49731896d5b01e16a/html5/thumbnails/69.jpg)
Summary
• Analyse the running time used by an algorithm via asymptotic analysis § asymptotic (O, Ω, Θ, o, ω) notations
§ use a generic uniprocessor random-access machine
§ Time and space complexity (input size)
§ Best, average and worst-case