1 computer algorithms lecture 7 master theorem some of these slides are courtesy of d. plaisted, unc...
TRANSCRIPT
1
Computer AlgorithmsLecture 7
Master Theorem
Some of these slides are courtesy of D. Plaisted, UNC and M. Nicolescu, UNR
Recurrence Relations• Equation or an inequality that characterizes a function by its values on
smaller inputs.
• Solution Methods (Chapter 4)– Substitution Method.– Recursion-tree Method.– Master Method.
• Recurrence relations arise when we analyze the running time of iterative or recursive algorithms.– Ex: Divide and Conquer.
T(n) = (1) if n cT(n) = a T(n/b) + D(n) + C(n) otherwise
2
Master Method
• Based on the Master theorem.
• “Cookbook” approach for solving recurrences of the form T(n) = aT(n/b) + f(n)
• a 1, b > 1 are constants.– a subproblems of size n/b are solved recursively, each in time T(n/b)
• f(n) is asymptotically positive.– An asymptotically positive function is one that is positive for all– sufciently large n.– f(n) is the cost of dividing the problem and combining the results
• n/b may not be an integer, but we ignore floors and ceilings. Why?
• Requires memorization of three cases.• Idea: compare f(n) with
– f(n) is asymptotically smaller or larger than by a polynomial factor n
– f(n) is asymptotically equal with abnlog
abnlog
abnlog
3
Recursion tree view
Θ(1)
f(n)
f(n/b) f(n/b) f(n/b)
f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2) f(n/b2)
a
a a a…
… … …
a a a a a a
… … … … … …
Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1) Θ(1)
f(n)
af(n/b)
a2f(n/b2)
(nlogba)
Total:
1log
0
log )/()()(n
j
jjab
b bnfannTThere are leaveslog logb bn aa n
Split problem into a parts at logbn levels.
4
The master method
The master method applies to recurrences of the form
T(n) = a T(n/b) + f (n) ,
where a ³ 1, b > 1, and f is asymptotically positive.
5
Why ?
)()( nfb
naTnT
b
naTnT )(
• Assume n = bk k = logbn
• At the end of iterations, i = k:• to solve the recurrence, we need only to characterize the dominant term
anni
in bbbb naTa
b
bTanT loglogloglog )1()(
22
b
nTa
33
b
nTa
ib
nTanT
ii
)(
abnlog
Master Theorem
Theorem 4.1
Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:
1. If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba).
2. If f(n) = (nlogba), then T(n) = (nlogbalg n).3. If f(n) = (nlogba+) for some constant > 0,
and if, for some constant c < 1 and all sufficiently large n,
we have a·f(n/b) c f(n), then T(n) = (f(n)).
Theorem 4.1
Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:
1. If f(n) = O(nlogba–) for some constant > 0, then T(n) = (nlogba).
2. If f(n) = (nlogba), then T(n) = (nlogbalg n).3. If f(n) = (nlogba+) for some constant > 0,
and if, for some constant c < 1 and all sufficiently large n,
we have a·f(n/b) c f(n), then T(n) = (f(n)).
Whether the problem cost is dominated by the root or by the leaves
7
Master Theorem – What it means?
• Case 1: If f(n) = O(nlogba–) for some constant > 0,
then T(n) = (nlogba).– nlogba = alogbn : Number of leaves in the recursion tree.– f(n) = O(nlogba–) Sum of the cost of the nodes at each internal
level asymptotically smaller than the cost of leaves by a polynomial factor.
– Cost of the problem dominated by leaves, hence cost is (nlogba).
8
Master Theorem – What it means?
• Case 2: If f(n) = (nlogba), then T(n) = (nlogbalg n).– nlogba = alogbn : Number of leaves in the recursion tree.– f(n) = (nlogba) Sum of the cost of the nodes at each level
asymptotically the same as the cost of leaves.– There are (lg n) levels.
– Hence, total cost is (nlogba lg n).
9
Master Theorem – What it means?
• Case 3: If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)).
– nlogba = alogbn : Number of leaves in the recursion tree.– f(n) = (nlogba+) Cost is dominated by the root. Cost of the root is
asymptotically larger than the sum of the cost of the leaves by a polynomial factor.
– Hence, cost is (f(n)).
10
Summary)()( nf
b
naTnT
b
naTnT )(
• Assume n = bk k = logbn
• At the end of iterations, i = k:• to solve the recurrence, we need only to
characterize the dominant term
anni
in bbbb naTa
b
bTanT loglogloglog )1()(
22
b
nTa
33
b
nTa
ib
nTanT
ii
)(
• Case 1: Running time dominated by
cost at leaves
– If f(n) is dominated by :
• T(n) = ( )
• Case 3: Running evenly distributed
throughout the tree
– If f(n) dominates :
• T(n) = (f(n))
• Case 2: Running time dominated by
cost at root
– If f(n) = ( ):• T(n)
abnlog
abnlog
abnlog
)log( log nn ab
abnlog
11
Strategy
• Extract a, b, and f(n) from a given recurrence• Determine • Compare f(n) and asymptotically • Determine appropriate MT case, and apply
logb anlogb an
12
Master Method – Examples
• T(n) = 16T(n/4)+n– a = 16, b = 4, nlogba = nlog416 = n2.– f(n) = n = O(nlogba-) = O(n2- ), where = 1 Case 1.
– Hence, T(n) = (nlogba ) = (n2).
• T(n) = T(3n/7) + 1– a = 1, b=7/3, and nlogba = nlog 7/3 1 = n0 = 1– f(n) = 1 = (nlogba) Case 2.
– Therefore, T(n) = (nlogba lg n) = (lg n)
13
Master Method – Examples
• T(n) = 3T(n/4) + n lg n– a = 3, b=4, thus nlogba = nlog43 = O(n0.793)– f(n) = n lg n = (nlog43 + ) where 0.2 Case 3.– Therefore, T(n) = (f(n)) = (n lg n).
• T(n) = 2T(n/2) + n lg n– a = 2, b=2, f(n) = n lg n, and nlogba = nlog22 = n– f(n) is asymptotically larger than nlogba, but not polynomially larger.
The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here.
14
Master Theorem Summarized
• Given a recurrence of the form
• The master method cannot solve every recurrence of this form; there is a gap between cases 1 and 2, as well as cases 2 and 3
( ) ( / ) ( )T n aT n b f n
log
log
log
log
log0
1. ( )
( )
2. ( )
( ) lg
3. ( ) and ( / ) ( ), for some 1,
( ) ( )
b
b
b
b
b
a
a
a
a
a
f n O n
T n n
f n n
T n n n
f n n af n b cf n c n n
T n f n
15
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
T (1)
…
Recursion tree:
…
f (n)a
f (n/b2)f (n/b2) f (n/b2)…
ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogbaT (1)CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.
CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight.
Q(nlogba)
16
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
T (1)
…
Recursion tree:
…
f (n)a
f (n/b2)f (n/b2) f (n/b2)…
ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogbaT (1)CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.
CASE 2: (k = 0) The weight is approximately the same on each of the logbn levels.
Q(nlogbalg n)
17
f (n/b)
Idea of master theorem
f (n/b) f (n/b)
T (1)
…
Recursion tree:
…
f (n)a
f (n/b2)f (n/b2) f (n/b2)…
ah = logbn
f (n)
a f (n/b)
a2 f (n/b2)
…
nlogbaT (1)CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.
CASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight.
Q( f (n))
18
More Examples
19
Master Method Case 1
T(n) = a T(n/b) + f(n)
f(n) = O(nlogb a - ) for some >0 T(n) = (nlogb a)
T(n) = (nlg 7)
cn2 =? O(nlogb a - ) = O(nlog2 7 - ) O(n2.8 - )
Yes, for any 0.8.
T(n) = 7T(n/2) + cn2 a=7, b=2
20
Master Method Case 2
T(n) = a T(n/b) + f(n)
f(n) = (nlogb a) T(n) = (nlogb a lg n)
T(n) = 2T(n/2) + cn a=2, b=2E.g., mergesort.
cn =? (nlogb a) = (nlog2 2) = (n)
Yes.
T(n) = (n lg n)21
Master Method Case 3T(n) = a T(n/b) + f(n)
f(n) = (nlogb a + ) for some >0 and
af(n/b) cf(n) for some c<1 and all large enough n
T(n) = (f(n))
T(n) = 4T(n/2) + n3 a=4, b=2
n3 =? (nlogb a + ) = (nlog2 4 + ) = (n2 + )
Yes, for any 1.
4(n/2)3 = ½n3 ? cn3
Yes, for any c ½.
I.e., is the constant factor
shrinking?
T(n) = (n3)22
Master Method Case 4
T(n) = a T(n/b) + f(n)
None of previous apply. Master method doesn’t help.
T(n) = 4T(n/2) + n2/lg n a=4, b=2
Case 1?
n2/lg n =? O(nlogb a - ) = O(nlog2 4 - ) = O(n2 - ) = O(n2/n)
No, since lg n is asymptotically less than n.
Thus, n2/lg n is asymptotically greater than n2/n.23
Examples
Ex. T(n) = 4T(n/2) + na = 4, b = 2 nlogba = n2; f (n) = n.CASE 1: f (n) = O(n2 – e) for e = 1. T(n) = Q(n2).
Ex. T(n) = 4T(n/2) + n2
a = 4, b = 2 nlogba = n2; f (n) = n2. CASE 2: f (n) = Q(n2lg0n), that is, k = 0. T(n) = Q(n2lg n).
24
Examples
Ex. T(n) = 4T(n/2) + n3
a = 4, b = 2 nlogba = n2; f (n) = n3. CASE 3: f (n) = W(n2 + e) for e = 1and 4(n/2)3 £ cn3 (reg. cond.) for c = 1/2. T(n) = Q(n3).
25
Master Method Case 4
T(n) = a T(n/b) + f(n)
None of previous apply. Master method doesn’t help.
T(n) = 4T(n/2) + n2/lg n a=4, b=2
Case 2?
n2/lg n =? (nlogb a) = (nlog2 4) = (n2)
No.
26
Master Method Case 4
T(n) = a T(n/b) + f(n)
None of previous apply. Master method doesn’t help.
T(n) = 4T(n/2) + n2/lg n a=4, b=2
Case 3?
n2/lg n =? (nlogb a + ) = (nlog2 4 + ) = (n2 + )
No, since 1/lg n is asymptotically less than n.
27
Examples
2
3
log 1
log 9
2
( ) ( / 2) 1
1, 2; 1
also ( ) 1, ( ) (1)
( ) (lg )
(
Case 2:
Cas
) 9 ( / 3)
9, 3;
( ) , ( ) ( ) with 1
( )e 1:
T n T n
a b n
f n f n
T n n
T n T n n
a b
f n n f n O n
T n n
Binary-search(A, p, r, s): q¬(p+r)/2 if A[q]=s then return q else if A[q]>s then Binary-search(A, p, q-1, s) else Binary-search(A, q+1, r, s)
Binary-search(A, p, r, s): q¬(p+r)/2 if A[q]=s then return q else if A[q]>s then Binary-search(A, p, q-1, s) else Binary-search(A, q+1, r, s)
28
Examples (2)
4
4
2
log 3 0.793
log 3
log 2 1
( ) 3 ( / 4) lg
3, 4;
( ) lg , ( ) ( ) with 0.2
Regularity condition
( / ) 3( / 4) l
Case
g( / 4) (3 / 4) lg ( ) for 3 / 4
( ) ( lg )
( ) 2 ( / 2) lg
2, 2;
3:
T n T n n n
a b n n
f n n n f n n
af n b n n n n cf n c
T n n n
T n T n n n
a b n n
f
1
1
( ) lg , ( ) ( ) with ?
also l
neither Case 3 nor Case 2!
g / lg
n n n f n n
n n n n
29
Examples (3)
2
3
log 4 2
3 2
3
3 3
3 3
( ) 4 ( / 2)
4, 2;
( ) ; ( ) ( )
( )
Checking the regularity condition
4 ( /
Cas
2) ( )
4 / 8
/ 2
3/ 4
e 3:
1
T n T n n
a b n n
f n n f n n
T n n
f n cf n
n cn
n cn
c
30