introduction to algorithms 6.046j/18.401j/sma5503 lecture 2 prof. erik demaine
TRANSCRIPT
![Page 1: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/1.jpg)
Introduction to Algorithms
6.046J/18.401J/SMA5503
Lecture 2Prof. Erik Demaine
![Page 2: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/2.jpg)
Solving recurrences
• The analysis of merge sort fromLecture 1 required us to solve arecurrence.• Recurrences are like solving integrals,differential equations, etc.o Learn a few tricks.• Lecture 3: Applications of
recurrences.
![Page 3: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/3.jpg)
Substitution method
The most general method:1. Guess the form of the solution.2. Verify by induction.3. Solve for constants.Example: T(n) = 4T(n/2) + n• [Assume that T(1) = Θ(1).]• Guess O(n3) . (Prove O and Ω
separately.)• Assume that T(k) ≤ ck3 for k < n .• Prove T(n) ≤ cn3 by induction.
![Page 4: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/4.jpg)
Example of substitution
T(n) = 4T (n/2) + n ≦4c(n/2)3 + n = (c/2)n3 + n = cn3 – ((c/2)n3 –n) ← desired – residual
≦ cn3 ← desired whenever (c/2)n3 – n ≥ 0, for example,if c ≥ 2 and n ≥ 1.
residual
![Page 5: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/5.jpg)
Example (continued)
We must also handle the initial conditions,that is, ground the induction with basecases.• Base: T(n) = Θ(1) for all n < n0, where n0
is a suitable constant.• For 1 ≤ n < n0, we have “Θ(1)” ≤ cn3, if wepick c big enough.==========================
==This bound is not tight!
![Page 6: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/6.jpg)
A tighter upper bound?
We shall prove that T(n) = O(n2).Assume that T(k) ≤ ck2 for k < n:: T(n) = 4T(n/2) + n
≤ 4cn2 + n
= cn2 – (- n) [desired –residual] ≤ cn2for no choice of c > 0. Lose!
![Page 7: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/7.jpg)
A tighter upper bound!
IDEA: Strengthen the inductive hypothesis.• Subtract a low-order term.Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n. T(n) = 4T(n/2) + n
≤ 4(c1(n/2)2 –c2(n/2) + n = c1n2 –2 c2n – (c2n – n)
≤ c1n2 –c2n if c2 < 1Pick c1 big enough to handle the initial conditions.
![Page 8: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/8.jpg)
Recursion-tree method
• A recursion tree models the costs (time) of arecursive execution of an algorithm.• The recursion tree method is good forgenerating guesses for the substitution method.• The recursion-tree method can be unreliable,just like any method that uses ellipses (…).• The recursion-tree method promotes intuition,however.
![Page 9: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/9.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
![Page 10: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/10.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:T(n)
![Page 11: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/11.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
N2
T(n/4) T(n/2)
![Page 12: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/12.jpg)
Example of recursion tree
• Solve T(n) = T(n/4) + T(n/2) + n2:n2
(n/4)2 (n/2)2
T(n/16) T(n/8) T(n/8) T(n/4)
![Page 13: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/13.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
N2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
![Page 14: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/14.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2: n2 n2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
![Page 15: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/15.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:n2 n2 (n/4)2 (n/2)2 5/16*n2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
![Page 16: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/16.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:n2 n2
(n/4)2 (n/2)2 5/16*n2
(n/16)2 (n/8)2 (n/8)2 (n/4)2 25/256*n2
Θ(1)
![Page 17: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/17.jpg)
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:n2 n2
(n/4)2 (n/2)2 5/16*n2
(n/16)2 (n/8)2 (n/8)2 (n/4)2 25/256*n2
Θ(1) Total = n2(1+5/16+(5/16)2+(5/16)3+…) =Θ(n2) geometric series
![Page 18: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/18.jpg)
The master method
The master method applies to recurrences of
the formT(n) = aT(n/b) + f (n) ,
where a ≥ 1, b > 1, and f is asymptotically
positive.
![Page 19: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/19.jpg)
Three common cases
Compare f (n) with nlogba:1. f (n) =O(nlogba – ε)for some constantε> 0.• f (n) grows polynomially slower than nlogba(by an nε factor).Solution: T(n) = Θ(nlogba) .2.f (n)=Θ(nlogba lgkn) for some constant k ≥
0.• f (n) and nlogba grow at similar rates.Solution: T(n) = Θ(nlogba lgk+1n) .
![Page 20: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/20.jpg)
Three common cases (cont.)
Compare f (n) with nlogba:3.f (n) = Ω(nlogba +ε) for some
constantε> 0.• f (n) grows polynomially faster than
nlogba (byan nε factor),and f (n) satisfies the regularity
condition thata f (n/b) ≤ c f (n) for some constant c < 1.Solution: T(n) = Θ( f (n)) .
![Page 21: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/21.jpg)
Examples
Ex. T(n) = 4T(n/2) + na = 4, b = 2 ⇒ nlogba = n2; f (n) = n.CASE 1: f (n) = O(n2 – ε) for ε = 1.∴ T(n) = Θ(n2).Ex. T(n) = 4T(n/2) + n2a = 4, b = 2 ⇒ nlogba = n2; f (n) = n2.CASE 2: f (n) = Θ(n2lg0n), that is, k = 0.∴ T(n) = Θ(n2lgn).
![Page 22: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/22.jpg)
Examples
Ex. T(n) = 4T(n/2) + n3a = 4, b = 2 ⇒ nlogba = n2; f (n) = n3.CASE 3: f (n) = Ω(n2 + ε) for ε = 1and 4(cn/2)3 ≤ cn3 (reg. cond.) for c = 1/2.∴ T(n) = Θ(n3).Ex. T(n) = 4T(n/2) + n2/lgna = 4, b = 2 ⇒ nlogba = n2; f (n) = n2/lgn.Master method does not apply. In particular,for every constant ε > 0, we have nε= ω(lgn).
![Page 23: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/23.jpg)
General method (Akra-Bazzi)
Let p be the unique solution to
Then, the answers are the same as for themaster method, but with np instead of nlogba.(Akra and Bazzi also prove an even moregeneral result.)
![Page 24: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/24.jpg)
Idea of master theorem
![Page 25: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/25.jpg)
Idea of master theorem
![Page 26: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/26.jpg)
Idea of master theorem
![Page 27: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/27.jpg)
Idea of master theorem
![Page 28: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/28.jpg)
Conclusion
• Next time: applying the master method.
• For proof of master theorem, see CLRS.
![Page 29: Introduction to Algorithms 6.046J/18.401J/SMA5503 Lecture 2 Prof. Erik Demaine](https://reader033.vdocuments.us/reader033/viewer/2022061615/551502ae550346935c8b64bd/html5/thumbnails/29.jpg)
Appendix: geometric series