![Page 1: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/1.jpg)
1
CS473 - Algorithms I
CS 473 – Lecture 3
Lecture 3 Solving Recurrences
Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
![Page 2: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/2.jpg)
2 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Solving Recurrences
Reminder: Runtime (T(n)) of MergeSort was expressed as a recurrence
Solving recurrences is like solving differential equations, integrals, etc. Need to learn a few tricks
Θ(1) if n=1 2T(n/2) + Θ(n) otherwise
T(n) =
![Page 3: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/3.jpg)
3 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Recurrences
Recurrence: An equation or inequality that describes a function in terms of its value on smaller inputs.
+=
1)2/(1
)(nT
nT if n=1 if n >1
Example:
![Page 4: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/4.jpg)
4 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Recurrence - Example
Simplification: Assume n = 2k
Claimed answer: T(n) = lgn + 1 Substitute claimed answer in the recurrence:
+=
1)2/(1
)(nT
nT
if n = 1
if n > 1 True when n = 2k
if n=1 if n >1
![Page 5: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/5.jpg)
5 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Technicalities: Floor/Ceiling
Technically, should be careful about the floor and ceiling functions (as in the book).
e.g. For merge sort, the recurrence should in fact be:
if n = 1
if n > 1
But, it’s usually ok to: ignore floor/ceiling solve for exact powers of 2 (or another number)
![Page 6: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/6.jpg)
6 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Technicalities: Boundary Conditions
Usually assume: T(n) = Θ(1) for sufficiently small n Changes the exact solution, but usually the asymptotic
solution is not affected (e.g. if polynomially bounded)
For convenience, the boundary conditions generally
implicitly stated in a recurrence T(n) = 2T(n/2) + Θ(n) assuming that T(n) = Θ(1) for sufficiently small n
![Page 7: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/7.jpg)
7 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example: When Boundary Conditions Matter
Exponential function: T(n) = (T(n/2))2
Assume T(1) = c (where c is a positive constant). T(2) = (T(1))2 = c2
T(4) = (T(2))2 = c4
T(n) = Θ(cn)
e.g.
)1()1()(1)1( Θ=Θ=⇒= nnTT Difference in solution more dramatic when:
![Page 8: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/8.jpg)
8 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Solving Recurrences
We will focus on 3 techniques in this lecture:
1. Substitution method
2. Recursion tree approach
3. Master method
![Page 9: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/9.jpg)
9 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method
The most general method: 1. Guess 2. Prove by induction 3. Solve for constants
![Page 10: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/10.jpg)
10 CS 473 – Lecture 3
Solve T(n) = 4T(n/2) + n (assume T(1) = Θ(1)) 1. Guess T(n) = O(n3) (need to prove O and Ω separately) 2. Prove by induction that T(n) ≤ cn3 for large n (i.e. n ≥ n0)
Inductive hypothesis: T(k) ≤ ck3 for any k < n Assuming ind. hyp. holds, prove T(n) ≤ cn3
Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method: Example
![Page 11: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/11.jpg)
11 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method: Example – cont’d
Original recurrence: T(n) = 4T(n/2) + n From inductive hypothesis: T(n/2) ≤ c(n/2)3
Substitute this into the original recurrence: T(n) ≤ 4c (n/2)3 + n = (c/2) n3 + n = cn3 – ((c/2)n3 – n)
≤ cn3 when ((c/2)n3 – n) ≥ 0
desired - residual
![Page 12: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/12.jpg)
12 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method: Example – cont’d
So far, we have shown: T(n) ≤ cn3 when ((c/2)n3 – n) ≥ 0
We can choose c ≥ 2 and n0 ≥ 1 But, the proof is not complete yet. Reminder: Proof by induction:
1. Prove the base cases 2. Inductive hypothesis for smaller sizes 3. Prove the general case
haven’t proved the base cases yet
![Page 13: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/13.jpg)
13 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method: Example – cont’d
We need to prove the base cases Base: T(n) = Θ(1) for small n (e.g. for n = n0)
We should show that: “Θ(1)” ≤ cn3 for n = n0 This holds if we pick c big enough
So, the proof of T(n) = O(n3) is complete. But, is this a tight bound?
![Page 14: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/14.jpg)
14 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example: A tighter upper bound?
Original recurrence: T(n) = 4T(n/2) + n Try to prove that T(n) = O(n2),
i.e. T(n) ≤ cn2 for all n ≥ n0
Ind. hyp: Assume that T(k) ≤ ck2 for k < n Prove the general case: T(n) ≤ cn2
![Page 15: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/15.jpg)
15 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example (cont’d)
Original recurrence: T(n) = 4T(n/2) + n Ind. hyp: Assume that T(k) ≤ ck2 for k < n Prove the general case: T(n) ≤ cn2
T(n) = 4T(n/2) + n
≤ 4c(n/2)2 + n = cn2 + n = O(n2) Wrong! We must prove exactly
![Page 16: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/16.jpg)
16 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example (cont’d)
Original recurrence: T(n) = 4T(n/2) + n Ind. hyp: Assume that T(k) ≤ ck2 for k < n Prove the general case: T(n) ≤ cn2
So far, we have:
T(n) ≤ cn2 + n No matter which positive c value we choose, this does not show that T(n) ≤ cn2
Proof failed?
![Page 17: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/17.jpg)
17 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example (cont’d)
What was the problem? The inductive hypothesis was not strong enough
Idea: Start with a stronger inductive hypothesis Subtract a low-order term
Inductive hypothesis: T(k) ≤ c1k2 – c2k for k < n
Prove the general case: T(n) ≤ c1n2 - c2n
![Page 18: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/18.jpg)
18 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example (cont’d)
Original recurrence: T(n) = 4T(n/2) + n Ind. hyp: Assume that T(k) ≤ c1k2 - c2k for k < n Prove the general case: T(n) ≤ c1n2 – c2n
T(n) = 4T(n/2) + n ≤ 4 (c1(n/2)2 – c2(n/2)) + n = c1n2 – 2c2n + n
= c1n2 – c2n – (c2n – n)
≤ c1n2 – c2n for n(c2 – 1) ≥ 0 choose c2 ≥ 1
![Page 19: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/19.jpg)
19 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example (cont’d)
We now need to prove T(n) ≤ c1n2 – c2n for the base cases.
T(n) = Θ(1) for 1 ≤ n ≤ n0 (implicit assumption) “Θ(1)” ≤ c1n2 – c2n for n small enough (e.g. n = n0)
We can choose c1 large enough to make this hold
We have proved that T(n) = O(n2)
![Page 20: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/20.jpg)
20 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method: Example 2
For the recurrence T(n) = 4T(n/2) + n, prove that T(n) = Ω(n2) i.e. T(n) ≥ cn2 for any n ≥ n0
Ind. hyp: T(k) ≥ ck2 for any k < n Prove general case: T(n) ≥ cn2
T(n) = 4T(n/2) + n ≥ 4c (n/2)2 + n = cn2 + n ≥ cn2 since n > 0 Proof succeeded – no need to strengthen the ind. hyp as in the last example
![Page 21: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/21.jpg)
21 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example 2 (cont’d)
We now need to prove that T(n) ≥ cn2 for the base cases
T(n) = Θ(1) for 1 ≤ n ≤ n0 (implicit assumption) “Θ(1)” ≥ cn2 for n = n0
n0 is sufficiently small (i.e. constant)
We can choose c small enough for this to hold
We have proved that T(n) = Ω (n2)
![Page 22: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/22.jpg)
22 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Substitution Method - Summary
1. Guess the asymptotic complexity 1. Prove your guess using induction
1. Assume inductive hypothesis holds for k < n 2. Try to prove the general case for n
Note: MUST prove the EXACT inequality CANNOT ignore lower order terms If the proof fails, strengthen the ind. hyp. and try again
3. Prove the base cases (usually straightforward)
![Page 23: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/23.jpg)
23 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Recursion Tree Method
A recursion tree models the runtime costs of a recursive execution of an algorithm.
The recursion tree method is good for generating guesses for the substitution method.
The recursion-tree method can be unreliable. Not suitable for formal proofs
The recursion-tree method promotes intuition, however.
![Page 24: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/24.jpg)
24 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Solve Recurrence: T(n) = 2T (n/2) + Θ(n)
Θ(n)
T(n/2) T(n/2)
![Page 25: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/25.jpg)
25 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Solve Recurrence: T(n) = 2T (n/2) + Θ(n)
Θ(n)
Θ(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
T(n/2) Θ(n/2)
2x
subp
robs
each
size
ha
lved
![Page 26: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/26.jpg)
26 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Solve Recurrence: T(n) = 2T (n/2) + Θ(n)
Θ(n)
Θ(n/2) Θ(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1)
2lgn = n
lgn
Θ(n)
Θ(n)
Θ(n)
Total: Θ(nlgn)
![Page 27: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/27.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
27
Example of Recursion Tree Solve T(n) = T(n/4) + T(n/2) + n2:
![Page 28: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/28.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
28
Solve T(n) = T(n/4) + T(n/2) + n2: T(n)
Example of Recursion Tree
![Page 29: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/29.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
29
Solve T(n) = T(n/4) + T(n/2) + n2: n2
T(n/4) T(n/2)
Example of Recursion Tree
![Page 30: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/30.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
30
Solve T(n) = T(n/4) + T(n/2) + n2: n2
(n/4)2 (n/2)2
T(n/16) T(n/8) T(n/8) T(n/4)
Example of Recursion Tree
![Page 31: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/31.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
31
Solve T(n) = T(n/4) + T(n/2) + n2: n2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
Example of Recursion Tree
![Page 32: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/32.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
32
Solve T(n) = T(n/4) + T(n/2) + n2: n2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
n2
Example of Recursion Tree
![Page 33: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/33.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
33
Solve T(n) = T(n/4) + T(n/2) + n2: n2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
n2
5/16 n2
Example of Recursion Tree
![Page 34: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/34.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
34
Solve T(n) = T(n/4) + T(n/2) + n2: n2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
n2
5/16 n2
25/256 n2
Example of Recursion Tree
![Page 35: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/35.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
35
Solve T(n) = T(n/4) + T(n/2) + n2: n2
(n/4)2 (n/2)2
(n/16)2 (n/8)2 (n/8)2 (n/4)2
Θ(1)
n2
5/16 n2
25/256 n2
Total = n2 (1 + 5/16 + (5/16)2 + (5/16)2 + ...) = Θ(n2) geometric series
Example of Recursion Tree
![Page 36: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/36.jpg)
36 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
The Master Method
A powerful black-box method to solve recurrences.
The master method applies to recurrences of the form
T(n) = aT(n/b) + f (n)
where a ≥ 1, b > 1, and f is asymptotically positive.
![Page 37: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/37.jpg)
37 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
The Master Method: 3 Cases
Recurrence: T(n) = aT(n/b) + f (n)
Compare f (n) with Intuitively:
Case 1: f (n) grows polynomially slower than
Case 2: f (n) grows at the same rate as
Case 3: f (n) grows polynomially faster than
abnlog
abnlog
![Page 38: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/38.jpg)
38 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
The Master Method: Case 1
Recurrence: T(n) = aT(n/b) + f (n)
Case 1: for some constant ε > 0
i.e., f (n) grows polynomialy slower than (by an nε factor).
Solution: T(n) = Θ( )
abnlog
abnlog
![Page 39: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/39.jpg)
39 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
The Master Method: Case 2 (simple version)
Recurrence: T(n) = aT(n/b) + f (n)
Case 2: i.e., f (n) and grow at similar rates Solution: T(n) = Θ( lgn)
abnlog
abnlog
![Page 40: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/40.jpg)
40 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
The Master Method: Case 3
Case 3: for some constant ε > 0 i.e., f (n) grows polynomialy faster than (by an nε factor). and the following regularity condition holds: a f (n/b) ≤ c f (n) for some constant c < 1 Solution: T(n) = Θ( f(n) )
abnlog
![Page 41: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/41.jpg)
41 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example: T(n) = 4T(n/2) + n
a = 4 b = 2
f(n) = n
f(n) grows polynomially slower than
CASE 1
T(n) = Θ( ) abnlog
T(n) = Θ(n2)
for ε = 1
![Page 42: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/42.jpg)
42 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example: T(n) = 4T(n/2) + n2
a = 4 b = 2
f(n) = n2
f(n) grows at similar rate as
CASE 2
T(n) = Θ( lgn) abnlog
T(n) = Θ(n2lgn)
f(n) = Θ( ) = n2
![Page 43: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/43.jpg)
43 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example: T(n) = 4T(n/2) + n3
a = 4 b = 2
f(n) = n3
f(n) grows polynomially faster than
seems like CASE 3, but need to check the regularity condition
T(n) = Θ(f(n)) T(n) = Θ(n3)
for ε = 1
Regularity condition: a f (n/b) ≤ c f (n) for some constant c < 1
4 (n/2)3 ≤ cn3 for c = 1/2
CASE 3
![Page 44: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/44.jpg)
44 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
Example: T(n) = 4T(n/2) + n2/lgn
a = 4 b = 2
f(n) = n2/lgn
f(n) grows slower than
is not CASE 1
for any ε > 0
but is it polynomially slower?
Master method does not apply!
![Page 45: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/45.jpg)
45 CS 473 – Lecture 3 Cevdet Aykanat and Mustafa Ozdal Computer Engineering Department, Bilkent University
The Master Method: Case 2 (general version)
Recurrence: T(n) = aT(n/b) + f (n)
Case 2: for some constant k ≥ 0 Solution: T(n) = Θ ( lgk+1n)
abnlog
![Page 46: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/46.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
46
General Method (Akra-Bazzi)
Let p be the unique solution to Then, the answers are the same as for the master method, but with np instead of (Akra and Bazzi also prove an even more general result.)
∑=
+=k
iii nfbnTanT
1)()/()(
∑=
=k
ii
pi ba
11)/(
abnlog
![Page 47: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/47.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
47
Idea of Master Theorem Recursion tree:
)1(log Tn ab
T(1)
f (n/b)
f (n) f (n)
f (n/b) f (n/b) a
a f (n/b)
f (n/b2) f (n/b2) f (n/b2) a h= logbn
a2 f (n/b2) #leaves = a h
=
=
nba log
abnlog
![Page 48: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/48.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
48
Recursion tree:
)1(log Tn ab
T(1)
f (n/b)
f (n) f (n)
f (n/b) f (n/b) a
a f (n/b)
f (n/b2) f (n/b2) f (n/b2) a h= logbn
a2 f (n/b2)
CASE 1 : The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. Θ ( ) abnlog
Idea of Master Theorem
![Page 49: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/49.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
49
Recursion tree:
)1(log Tn ab
T(1)
f (n/b)
f (n) f (n)
f (n/b) f (n/b) a
a f (n/b)
f (n/b2) f (n/b2) f (n/b2) a h= logbn
a2 f (n/b2)
CASE 2 : (k = 0) The weight is approximately the same on
each of the logbn levels. Θ ( lgn) abnlog
Idea of Master Theorem
![Page 50: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/50.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
50
Recursion tree:
)1(log Tn ab
T(1)
f (n/b)
f (n) f (n)
f (n/b) f (n/b) a
a f (n/b)
f (n/b2) f (n/b2) f (n/b2) a h= logbn
a2 f (n/b2)
CASE 3 : The weight decreases geometrically from the root to the leaves. The root holds a constant
fraction of the total weight. Θ ( f (n) )
Idea of Master Theorem
![Page 51: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/51.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
51
Proof of Master Theorem: Case 1 and Case 2
• Recall from the recursion tree (note h = lgbn=tree height)
∑−
=
+Θ=1
0
log )/()()(h
i
iia bnfannT b
Leaf cost Non-leaf cost = g(n)
![Page 52: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/52.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
52
Proof of Case 1 for some ε > 0
)()(
logεn
nfn ab
Ω=
)()()()()()(
loglog
logεεε −− =⇒=⇒Ω= a
a
ab
b
b
nOnfnOn
nfnnf
n
( )
== ∑∑
−
=
−−
=
−1
0
log1
0
log )/()/()(h
i
aiih
i
aii bb bnaObnOang εε
= ∑
−
=
−1
0
loglog /h
i
aiiia bb bbanO εε
![Page 53: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/53.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
53
= An increasing geometric series since b > 1
∑∑∑∑−
=
−
=
−
=
===1
0
1
0log
1
0log )(
)()( h
i
ii
ii
h
iia
ii
h
iai
ii
baba
bba
bba
bb
εεεε
)(11
11)(
11)(
11 log
εε
ε
ε
ε
ε
ε
ε
ε
nObn
bb
bb
bb nhh b
=−−
=−
−=
−−
=−−
=
Case 1 (cont’)
![Page 54: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/54.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
54
( )
== − )()()(
loglog ε
εεε nO
nnOnOnOng
aa
bb
)()()()()( logloglog aaa bbb nOnngnnT +Θ=+Θ=
Case 1 (cont’)
)( log abnO=
)( log abnΘ= Q.E.D.
![Page 55: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/55.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
55
Proof of Case 2 (limited to k=0)
Θ=
Θ=
Θ= ∑∑∑
−
=
−
=
−
=
1
0
log1
0log
log1
0log
log 1)(
1 h
ii
iah
iia
iah
iai
ai
aan
ban
bna b
b
b
b
b
Θ=⇒Θ=⇒Θ=Θ= a
iia
abb
b bnbnfnnfn
nnf loglog0
log )()/()()()1()(lg)(
( )∑−
=
Θ=∴1
0
log)/()(h
i
aii bbnang
)lg()( loglog nnnnT aa bb Θ+=( )nn ab lglogΘ=
( ) ( )nnnnn ab
an
i
a bbb
b lglog1 loglog1log
0
log Θ=Θ=
Θ= ∑
−
=
Q.E.D.
![Page 56: Lecture 3 Solving Recurrences - cs.bilkent.edu.trcs.bilkent.edu.tr/~ozan.karsavuran/473/pdf/03-SolvingRecurrences.pdf · Solving Recurrences Reminder: Runtime (T(n)) of . MergeSort](https://reader034.vdocuments.us/reader034/viewer/2022051807/6003b90398f8c254db14b22f/html5/thumbnails/56.jpg)
CS473 – Lecture 3 Cevdet Aykanat - Bilkent University Computer Engineering Department
56
Conclusion
• Next time: applying the master method.