![Page 1: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/1.jpg)
Data Structures and Algorithms (CS210A)
Lecture 41 β’ Miscellaneous problems
1
![Page 2: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/2.jpg)
Order notation Definition: Let f(π) and g(π) be any two increasing functions of n.
f(π) is said to be of the order of g(π)
if there exist constants c and π0 such that
f(π) β€ c g(π) for all n > π0
2
π0
f(π)
c g(π)
f(π) = O(g(π))
![Page 3: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/3.jpg)
Order notation extended
Definition: Let f(π) and g(π) be any two increasing functions of n.
f(π) is said to be lower bounded by g(π)
if there exist constants c and π0 such that
f(π) β₯ c g(π) for all n > π0
π2
100= π(10000 π log π)
3
π0
c g(π)
f(π)
f(π) = π(g(π))
![Page 4: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/4.jpg)
Order notation extended
Observations:
β’ f(π) = O(g(π))
One more Notation:
If f(π) = O(g(π))
Examples:
β’π2
100= π―(10000 π2)
if and only if g(π) = π(f(π))
and g(π) = π(f(π)) , then
g(π) = π―(f(π))
![Page 5: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/5.jpg)
β’ Time complexity of Quick Sort is ?
β’ Time complexity of Merge sort is ?
π(π log π)
π―(π log π)
![Page 6: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/6.jpg)
Time complexity of a problem
Example: Sorting
β’ Algorithm 1 : Selection Sort with time complexity π(π2)
β’ Algorithm 2 : Merge Sort with time complexity π(π log π)
β’ Each comparison based sorting algorithm needs to
perform π(π log π) comparisons in the worst case.
β’ Sorting must takes π(π) time since
it has to read each item at least once.
Time complexity of sorting
π(π2)
π(π log π)
π(π log π)
π(π)
Sorting has time complexity of π―(π log π)
Upper bound
Lower bound
![Page 7: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/7.jpg)
Time complexity of a problem
Example: All-pairs shortest paths (APSP)
β’ Algorithm 1 : Floyd Warshal Algorithm with time complexity π(π3)
β’ Algorithm 2 : Johnsonβ algorithm with time complexity π(ππ log π)
β’ All-pairs shortest paths must require π(π2) time
Time complexity of APSP
π(π3)
π(ππ log π)
π(π2)
There is still a gap between upper and lower bounds for APSP.
Upper bound
Lower bound
![Page 8: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/8.jpg)
Aim of theoretical computer science
For any given computational problem P
β’ Get smallest possible upper bound on its time complexity
β’ Get largest possible lower bound on its time complexity.
Reduce the GAP
This requires designing better algorithm
?
![Page 9: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/9.jpg)
How to establish lower bound
Two ways:
β’ Adversarial approach
β’ Limitation of the model of computation
A gentle introduction today
CS345
![Page 10: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/10.jpg)
Adversarial approach
Key aspects Algorithm:
β’ Algorithm does not have free access to the input.
To access any item in the input, algorithm has to spend some time.
β’ The execution of an algorithm at any step is determined
only by the (partial) input it has seen till now.
Adversary
β’ Adversary has access to all possible inputs of a problem.
β’ The sole aim of adversary is to make an algorithm work really hard.
For this purpose, adversary discloses the input cleverly.
![Page 11: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/11.jpg)
Locating 1 problem
Input: An array A[0. . . π β π] with an unknown π s.t.
β’ For all π β π,
β’ A[π]= π
Aim: To locate/search 1 in A.
Upper bound: O(π)
Lower bound: π(π)
A[π]= π
![Page 12: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/12.jpg)
Lower bound on Locating 1 problem
Algorithm
Adversary
Input
A
What is A[π]?
π
π π
π
What is A[π] ?
π
What is A[π] ?
π
π π π π π π 1
![Page 13: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/13.jpg)
Miscellaneous problems
![Page 14: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/14.jpg)
Problem 1 Input: Given an array A storing π numbers,
there is an π < π (unknown) s.t.
A[π] < A[π] < β¦< A[π]
Aim:
To search efficiently
Answer : O(log π) is possible
> A[π + π] >β¦ >A[π β π]
π π ππ ππ ππ ππ ππ ππ ππ ππ πππ πππ ππ ππ ππ
sorted sorted
π
![Page 15: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/15.jpg)
Problem 2
Input:
Given an array A storing π numbers,
there is an π < π (unknown) s.t.
A[π] β€ A[π] β€ β¦β€ A[π]
Aim:
To search efficiently
Answer : π(π) time complexity
Locating 1 problem ? Problem 2
β₯ A[π + π] β₯β¦ β₯ A[π β π]
is a special case of
![Page 16: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/16.jpg)
Problem 3
Input:
Given an array A storing π numbers,
there are π < π < π (unknown) s.t.
A[π]< A[π]< β¦< A[π]
Aim:
To search efficiently
Answer : π(π) time complexity
> A[π + π] >β¦ >A[π] < A[π + π] <β¦ <A[π β π]
![Page 17: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/17.jpg)
Locating 0 problem
Input: An array A[0. . . π β π] with an unknown π s.t.
β’ For all π < π,
β’ A[π] = π
β’ For all π > π,
Aim: To locate/search 0 in A.
Upper bound: O(π)
Lower bound: π(π)
A[π]> π A[π β π] < A[π] and
A[π]> π A[π] < A[π + π] and
![Page 18: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/18.jpg)
Lower bound on Locating 0 problem
Algorithm
Adversary
Input
A
What is A[π]?
π
ππ ππ
π
What is A[π] ? What is A[π] ?
π
π ππ ππ
β
What is A[β] ?
ππ ππ πππ ππ ππ
![Page 19: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/19.jpg)
Problem 3
Input:
Given an array A storing π numbers,
there are π < π < π (unknown) s.t.
A[π]< A[π]< β¦< A[π]
Aim:
To search efficiently
Answer : π(π) time complexity
Locating 0 problem ? Problem 3
> A[π + π] >β¦ >A[π] < A[π + π] <β¦ <A[π β π]
is a special case of
![Page 20: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/20.jpg)
Problem 4
Input:
Given a sorted array A storing π numbers,
Aim:
To search efficiently
Answer : π(log π) time complexity (binary search)
π(log π) time complexity
![Page 21: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/21.jpg)
Lower bound on Problem 4
Algorithm
Adversary
Input A
What is A[π]?
π
ππ
π
What is A[π] ?
ππ ππ ππ ππ ππ πππ
Search for ππ
![Page 22: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/22.jpg)
Problem 5
Input:
Given a 2-dimensional square grid storing ππ distinct numbers,
Aim:
To find a Local minima efficiently
Answer :
π(π) time complexity (in the second week of the course)
π(π) time complexity
Prove it using Adversarial arguments during summer vacations
![Page 23: Data Structures and Algorithms - E & ICT Academy...β’ Algorithm 2 : Merge Sort with time complexity π( log ) β’ Each comparison based sorting algorithm needs to perform π(](https://reader034.vdocuments.us/reader034/viewer/2022042222/5ec965e2d52f7232643f3417/html5/thumbnails/23.jpg)
Final slide
Thatβs all. I hope you enjoyed this lecture.