asymptotic behavior algorithm : design & analysis [2]
TRANSCRIPT
![Page 1: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/1.jpg)
Asymptotic Behavior
Algorithm : Design & Analysis
[2]
![Page 2: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/2.jpg)
In the last class…
Goal of the Course Algorithm: the Concept Algorithm Analysis: the Contents Average and Worst-Case Analysis Lower Bounds and the Complexity of
Problems
![Page 3: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/3.jpg)
Asymptotic Behavior
Asymptotic growth rate The Sets , and Complexity Class An Example: Maximum Subsequence Sum
Improvement of Algorithm Comparison of Asymptotic Behavior
Another Example: Binary Search Binary Search Is Optimal
![Page 4: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/4.jpg)
How to Compare Two Algorithm?
Simplifying the analysis assumption that the total number of steps is roughly
proportional to the number of basic operations counted (a constant coefficient)
only the leading term in the formula is considered constant coefficient is ignored
Asymptotic growth rate large n vs. smaller n
![Page 5: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/5.jpg)
Relative Growth Rate
g
Ω(g):functions that grow at least as fast as g
Θ(g):functions that grow at the same rate as g
Ο(g):functions that grow no faster as g
![Page 6: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/6.jpg)
The Set “Big Oh”
Definition
Giving g:N→R+, then Ο(g) is the set of f:N→R+, such that for some cR+ and some n0N, f(n)cg(n) for all
nn0.
A function fΟ(g) if
Note: c may be zero. In that case, f(g), “little Oh”
cng
nfn )(
)(lim
![Page 7: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/7.jpg)
Example
Let f(n)=n2, g(n)=nlgn, then:
fΟ(g), since
gΟ(f), since
2ln11
lim
2lnln
limlg
limlg
lim2
nn
n
n
n
nn
nnnnn
Using L’Hopital’s RuleUsing L’Hopital’s Rule
02ln
1lim
2ln
lnlim
loglim
loglim
2
nn
n
n
n
n
nnnnnn
For your reference: L’Hôpital’s rule
)('
)('lim
)(
)(lim
ng
nf
ng
nfnn
with some constraints
![Page 8: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/8.jpg)
Logarithmic Functions and Powers
? or log2 nnWhich grows faster?
0lim)log2(
2
1
log
limlog
lim 2
2
2 n
ne
n
ne
n
nnnn
So, log2nO(n)
![Page 9: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/9.jpg)
The Result Generalized
The log function grows more slowly than any positive power of n
lgn o(n) for any >0
By the way:The power of n grows more slowly than any exponential function with base greater than 1
nk o(cn) for any c>1
![Page 10: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/10.jpg)
Dealing with big-O correctly
?10 if , and log :larger is which However,
0)any for 0log
lim (since
)(log :known that have We
100
0001.0
nnn
n
n
non
n
![Page 11: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/11.jpg)
Factorials and Exponential Functions
n! grows faster than 2n for positive integer n.
n
n
nn
n
nnn
e
nnn
e
nn
en
nn
2! :formular sStirling'
22lim
2
2lim
2
!lim
![Page 12: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/12.jpg)
The Sets and
Definition Giving g:N→R+, then (g) is the set of f:N→R+, such that
for some cR+ and some n0N, f(n)cg(n) for all nn0. A function f(g) if limn→[f(n)/g(n)]>0
Note: the limit may be infinity
Definition Giving g:N→R+, then (g) = Ο(g) (g)
A function f(g) if limn→[f(n)/g(n)]=c,0<c<
![Page 13: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/13.jpg)
Properties of O, and
Transitive property: If fO(g) and gO(h), then fO(h)
Symmetric properties fO(g) if and only if g(f) f(g) if and only if g(f)
Order of sum function O(f+g)=O(max(f,g))
![Page 14: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/14.jpg)
Complexity Class
Let S be a set of f:NR* under consideration, define the relation ~ on S as following: f~g iff. f(g)
then, ~ is an equivalence. Each set (g) is an equivalence class, called
complexity class. We usually use the simplest element as possible as
the representative, so, (n), (n2), etc.
![Page 15: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/15.jpg)
Effect of the Asymptotic Behavior
Run time in ns
1 2 3 4
1.3n3 10n2 47nlogn 48n
1.3s22m15d
41yrs41mill
10ms1s
1.7m2.8hrs1.7wks
0.4ms6ms78ms0.94s11s
0.05ms0.5ms5ms
48ms0.48s
103
104
105
106
107
time for size
maxSize in time
secminhr
day
9203,60014,00041,000
10,00077,000
6.0105
2.9106
1.0106
4.9107
2.4109
5.01010
2.1107
1.3109
7.61010
1.81012
time for 10 times size 1000
100 10+ 10
on 400Mhz Pentium II, in Cfrom: Jon Bentley: Programming Pearls
algorithm
![Page 16: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/16.jpg)
Searching an Ordered Array
The Problem: Specification Input:
an array E containing n entries of numeric type sorted in non-decreasing order
a value K
Output: index for which K=E[index], if K is in E, or, -1, if K is
not in E
![Page 17: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/17.jpg)
Sequential Search: the Procedure
The Procedure Int seqSearch(int[] E, int n, int K) 1. Int ans, index; 2. Ans=-1; // Assume failure 3. For (index=0; index<n; index++) 4. If (K= =E[index]) ans=index;//success! 5. break; 6. return ans
![Page 18: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/18.jpg)
Searching a Sequence
For a given K, there are two possibilities K in E (say, with probability q), then K may be any one of
E[i] (say, with equal probability, that is 1/n) K not in E (with probability 1-q), then K may be located in
any one of gap(i) (say, with equal probability, that is 1/(n+1))
E[1] E[2] E[i]E[i-1] E[I+1]
gap(i-1)
E[n]
gap(0) gap(i-1)
![Page 19: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/19.jpg)
Improved Sequential Search Since E is sorted, when an entry larger than K is met, no nore
comparison is needed Worst-case complexity: n, unchanged Average complexity
Note: A(n)(n)
)1(212
1
12
12)1(
2
)1()(
121
1)1(
1
1)(
2
1)1(
1)(
)()1()()(
1
0
1
0
On
n
nq
n
nn
n
nnq
nqnA
n
nnn
ni
nnA
ni
nnA
nAqnqAnA
n
ifail
n
isucc
failsucc
Roughly n/2Roughly n/2
![Page 20: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/20.jpg)
Divide and Conquer
If we compare K to every jth entry, we can locate the small section of E that may contain K.
To locate a section, roughly n/j steps at most To search in a section, j steps at most So, the worst-case complexity: (n/j)+j, with j selected
properly, (n/j)+j(n) However, we can use the same strategy in the small
sections recursively
Choose j = nChoose j = n
![Page 21: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/21.jpg)
Binary Searchint binarySearch(int[] E, int first, int last, int K)
if (last<first)index=-1;
elseint mid=(first+last)/2if (K==E[mid])
index=mid;else if (K<E[mid])
index=binarySearch(E, first, mid-1, K)else if (K<E[mid])
index=binarySearch(E, mid+1, last, K)return index;
![Page 22: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/22.jpg)
Worst-case Complexity of Binary Search
Observation: with each call of the recursive procedure, only at most half entries left for further consideration.
At most lg n calls can be made if we want to keep the range of the section left not less than 1.
So, the worst-case complexity of binary search is lg n+1= lg(⌈ n+1)⌉
![Page 23: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/23.jpg)
Average Complexity of Binary Search
Observation: for most cases, the number of comparison is or is
very close to that of worst-case particularly, if n=2k-1, all failure position need
exactly k comparisons Assumption:
all success position are equally likely (1/n) n=2k-1
![Page 24: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/24.jpg)
Average Complexity of Binary Search
Average complexity Note: We count the sum of st, which is the number of inputs
for which the algorithm does t comparisons, and if n=2k-1, st=2t-1
qnnA
nA
n
nOn
n
nk
n
kt
nn
stnA
nAqnqAnA
q
k
t
kk
t
tt
q
)1lg()(
)1lg(
log1)1lg(
1)1)(1(
12)1(2
1)(
)()1()()(
0
1 1
11
01
22)1(
)42()22(2)22(
)22)1(23222(
)22)1(2322(2
222
Series Geometric-Arithmetic :referenceyour For
1
11
2
1
)1(32
1432
1
1
1
k
kkk
i
ik
kk
kk
k
i
iik
i
i
k
kk
kk
kk
ii
![Page 25: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/25.jpg)
Decision Tree
A decision tree for A and a given input of size n is a binary tree whose nodes are labeled with numbers between 0 and n-1
Root: labeled with the index first compared If the label on a particular node is i, then the left child is labeled the index next compared if K<E[i], the right child the index next compared if K>E[i], and no branch for the case of K=E[i].
9
8
7
6
5
3
20
1
4
![Page 26: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/26.jpg)
Binary Search Is Optimal
If the number of comparison in the worst case is p, then the longest path from root to a leaf is p-1, so there are at most 2p-1 node in the tree.
There are at least n node in the tree.
(We can prove that For all i{0,1,…,n-1}, exist a node with the label i.)
Conclusion: n 2p-1, that is plg(n+1)
![Page 27: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/27.jpg)
Binary Search Is Optimal
For all i{0,1,…,n-1}, exist a node with the label i. Proof:
if otherwise, suppose that i doesn’t appear in the tree, make up two inputs E1 and E2, with E1[i]=K, E2[i]=K’, K’>K, for any
ji, (0jn-1), E1[j]=E2[j]. (Arrange the values to keeping the
order in both arrays). Since i doesn’t appear in the tree, for both K and K’, the algorithm behave alike exactly, and give the same outputs, of which at least one is wrong, so A is not a right algorithm.
![Page 28: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/28.jpg)
Home Assignment
pp.63 – 1.23 1.27 1.31 1.34 1.37 1.45
![Page 29: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/29.jpg)
Maximum Subsequence Sum The problem: Given a sequence S of integer, find the largest
sum of a consecutive subsequence of S. (0, if all negative items) An example: -2, 11, -4, 13, -5, -2; the result 20: (11, -4, 13)
A brute-force algorithm: MaxSum = 0; for (i = 0; i < N; i++) for (j = i; j < N; j++) { ThisSum = 0; for (k = i; k <= j; k++) ThisSum += A[k]; if (ThisSum > MaxSum) MaxSum = ThisSum; } return MaxSum;
……
i=0i=1
i=2
i=n-1
k
j=0 j=1 j=2
j=n-1
in O(n3)
the sequence
![Page 30: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/30.jpg)
More Precise Complexity
6
23
1)23(2
1)
2
3(
2
1
2
)1)(2(
2
))(1(
2
))(1()(...21)1(
11
1:iscost totalThe
23
1
2
11
2
1
1
0
1
1
0
1
nnn
nnini
inininin
inininij
ij
n
i
n
i
n
i
n
i
n
i
n
ij
j
ik
n
i
n
ij
j
ik
![Page 31: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/31.jpg)
Decreasing the number of loops
An improved algorithmMaxSum = 0; for (i = 0; i < N; i++) { ThisSum = 0; for (j = i; j < N; j++) { ThisSum += A[j]; if (ThisSum > MaxSum) MaxSum = ThisSum; } } return MaxSum;
the sequence
i=0i=1
i=2
i=n-1
j
in O(n2)
![Page 32: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/32.jpg)
Power of Divide-and-Conquer
Part 1 Part 2
the sub with largest sum may be in:
Part 1 Part 2or:
Part 1 Part 2
recursion
The largest is the result
in O(nlogn)
![Page 33: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/33.jpg)
Divide-and-Conquer: the ProcedureCenter = (Left + Right) / 2; MaxLeftSum = MaxSubSum(A, Left, Center); MaxRightSum = MaxSubSum(A, Center + 1, Right); MaxLeftBorderSum = 0; LeftBorderSum = 0; for (i = Center; i >= Left; i--) { LeftBorderSum += A[i]; if (LeftBorderSum > MaxLeftBorderSum) MaxLeftBorderSum = LeftBorderSum; } MaxRightBorderSum = 0; RightBorderSum = 0; for (i = Center + 1; i <= Right; i++) { RightBorderSum += A[i]; if (RightBorderSum > MaxRightBorderSum) MaxRightBorderSum = RightBorderSum; } return Max3(MaxLeftSum, MaxRightSum, MaxLeftBorderSum + MaxRightBorderSum);
Note: this is the core part of the procedure, with base case and wrap omitted.
Note: this is the core part of the procedure, with base case and wrap omitted.
![Page 34: Asymptotic Behavior Algorithm : Design & Analysis [2]](https://reader035.vdocuments.us/reader035/viewer/2022062409/5697bfa91a28abf838c99d83/html5/thumbnails/34.jpg)
A Linear Algorithm
ThisSum = MaxSum = 0; for (j = 0; j < N; j++) { ThisSum += A[j]; if (ThisSum > MaxSum) MaxSum = ThisSum; else if (ThisSum < 0) ThisSum = 0; } return MaxSum;
the sequence
j
Negative item or subsequence cannot be a prefix of the subsequence we want.
This is an example of “online algorithm”
in O(n)