![Page 1: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/1.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.1
Introduction to Algorithms6.046J/18.401J
LECTURE 3Divide and Conquer• Binary search• Powering a number• Fibonacci numbers• Matrix multiplication• Strassen’s algorithm• VLSI tree layout
Prof. Erik D. Demaine
![Page 2: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/2.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.2
The divide-and-conquer design paradigm
1. Divide the problem (instance) into subproblems.
2. Conquer the subproblems by solving them recursively.
3. Combine subproblem solutions.
![Page 3: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/3.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.3
Merge sort
1. Divide: Trivial.2. Conquer: Recursively sort 2 subarrays.3. Combine: Linear-time merge.
![Page 4: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/4.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.4
Merge sort
1. Divide: Trivial.2. Conquer: Recursively sort 2 subarrays.3. Combine: Linear-time merge.
T(n) = 2 T(n/2) + Θ(n)
# subproblemssubproblem size
work dividing and combining
![Page 5: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/5.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.5
Master theorem (reprise)T(n) = a T(n/b) + f (n)
CASE 1: f (n) = O(nlogba – ε), constant ε > 0⇒ T(n) = Θ(nlogba) .
CASE 2: f (n) = Θ(nlogba lgkn), constant k ≥ 0⇒ T(n) = Θ(nlogba lgk+1n) .
CASE 3: f (n) = Ω(nlogba + ε ), constant ε > 0, and regularity condition
⇒ T(n) = Θ( f (n)) .
![Page 6: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/6.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.6
Master theorem (reprise)T(n) = a T(n/b) + f (n)
CASE 1: f (n) = O(nlogba – ε), constant ε > 0⇒ T(n) = Θ(nlogba) .
CASE 2: f (n) = Θ(nlogba lgkn), constant k ≥ 0⇒ T(n) = Θ(nlogba lgk+1n) .
CASE 3: f (n) = Ω(nlogba + ε ), constant ε > 0, and regularity condition
⇒ T(n) = Θ( f (n)) .Merge sort: a = 2, b = 2 ⇒ nlogba = nlog22 = n
⇒ CASE 2 (k = 0) ⇒ T(n) = Θ(n lg n) .
![Page 7: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/7.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.7
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
![Page 8: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/8.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.8
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
Example: Find 9
3 5 7 8 9 12 15
![Page 9: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/9.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.9
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
Example: Find 9
3 5 7 8 9 12 15
![Page 10: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/10.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.10
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
Example: Find 9
3 5 7 8 9 12 15
![Page 11: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/11.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.11
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
Example: Find 9
3 5 7 8 9 12 15
![Page 12: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/12.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.12
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
Example: Find 9
3 5 7 8 9 12 15
![Page 13: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/13.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.13
Binary search
Find an element in a sorted array:1. Divide: Check middle element.2. Conquer: Recursively search 1 subarray.3. Combine: Trivial.
Example: Find 9
3 5 7 8 9 12 15
![Page 14: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/14.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.14
Recurrence for binary search
T(n) = 1 T(n/2) + Θ(1)
# subproblemssubproblem size
work dividing and combining
![Page 15: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/15.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.15
Recurrence for binary search
T(n) = 1 T(n/2) + Θ(1)
# subproblemssubproblem size
work dividing and combining
nlogba = nlog21 = n0 = 1 ⇒ CASE 2 (k = 0)⇒ T(n) = Θ(lg n) .
![Page 16: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/16.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.16
Powering a number
Problem: Compute a n, where n ∈ N.
Naive algorithm: Θ(n).
![Page 17: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/17.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.17
Powering a number
Problem: Compute a n, where n ∈ N.
Naive algorithm: Θ(n).
a n =a n/2 ⋅ a n/2 if n is even;a (n–1)/2 ⋅ a (n–1)/2 ⋅ a if n is odd.
Divide-and-conquer algorithm:
![Page 18: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/18.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.18
Powering a number
Problem: Compute a n, where n ∈ N.
Naive algorithm: Θ(n).
a n =a n/2 ⋅ a n/2 if n is even;a (n–1)/2 ⋅ a (n–1)/2 ⋅ a if n is odd.
Divide-and-conquer algorithm:
T(n) = T(n/2) + Θ(1) ⇒ T(n) = Θ(lg n) .
![Page 19: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/19.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.19
Fibonacci numbersRecursive definition:
Fn =0 if n = 0;
Fn–1 + Fn–2 if n ≥ 2.1 if n = 1;
0 1 1 2 3 5 8 13 21 34 L
![Page 20: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/20.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.20
Fibonacci numbersRecursive definition:
Fn =0 if n = 0;
Fn–1 + Fn–2 if n ≥ 2.1 if n = 1;
0 1 1 2 3 5 8 13 21 34 L
Naive recursive algorithm: Ω(φ n)(exponential time), where φ =is the golden ratio.
2/)51( +
![Page 21: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/21.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.21
Computing Fibonacci numbers
Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming
each number by summing the two previous.• Running time: Θ(n).
![Page 22: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/22.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.22
Computing Fibonacci numbers
Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming
each number by summing the two previous.• Running time: Θ(n). Naive recursive squaring:
Fn = φ n/ rounded to the nearest integer.5• Recursive squaring: Θ(lg n) time. • This method is unreliable, since floating-point
arithmetic is prone to round-off errors.
![Page 23: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/23.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.23
Recursive squaringn
FFFF
nn
nn⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡
−
+
0111
1
1Theorem: .
![Page 24: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/24.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.24
Recursive squaringn
FFFF
nn
nn⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡
−
+
0111
1
1Theorem: .
Algorithm: Recursive squaring.Time = Θ(lg n) .
![Page 25: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/25.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.25
Recursive squaringn
FFFF
nn
nn⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡
−
+
0111
1
1Theorem: .
Algorithm: Recursive squaring.Time = Θ(lg n) .
Proof of theorem. (Induction on n.)
Base (n = 1): .1
0111
01
12⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡FFFF
![Page 26: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/26.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.26
Recursive squaring
.
.
Inductive step (n ≥ 2):
n
nFFFF
FFFF
nn
nn
nn
nn
⎥⎦
⎤⎢⎣
⎡=
⎥⎦
⎤⎢⎣
⎡⋅−
⎥⎦
⎤⎢⎣
⎡=
⎥⎦
⎤⎢⎣
⎡⋅⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡
−−
−
−
+
0111
01111
0111
0111
21
1
1
1
![Page 27: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/27.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.27
Matrix multiplication
Input: A = [aij], B = [bij].Output: C = [cij] = A⋅ B. i, j = 1, 2,… , n.
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⋅
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
nnnn
n
n
nnnn
n
n
nnnn
n
n
bbb
bbbbbb
aaa
aaaaaa
ccc
cccccc
L
MOMM
L
L
L
MOMM
L
L
L
MOMM
L
L
21
22221
11211
21
22221
11211
21
22221
11211
∑=
⋅=n
kkjikij bac
1
![Page 28: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/28.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.28
Standard algorithm
for i ← 1 to ndo for j ← 1 to n
do cij ← 0for k ← 1 to n
do cij ← cij + aik⋅ bkj
![Page 29: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/29.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.29
Standard algorithm
for i ← 1 to ndo for j ← 1 to n
do cij ← 0for k ← 1 to n
do cij ← cij + aik⋅ bkj
Running time = Θ(n3)
![Page 30: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/30.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.30
Divide-and-conquer algorithm
n×n matrix = 2×2 matrix of (n/2)×(n/2) submatrices:IDEA:
⎥⎦
⎤⎢⎣
⎡⋅⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡hgfe
dcba
utsr
C = A ⋅ Br = ae + bgs = af + bht = ce + dgu = cf + dh
8 mults of (n/2)×(n/2) submatrices4 adds of (n/2)×(n/2) submatrices
![Page 31: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/31.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.31
Divide-and-conquer algorithm
n×n matrix = 2×2 matrix of (n/2)×(n/2) submatrices:IDEA:
⎥⎦
⎤⎢⎣
⎡⋅⎥⎦
⎤⎢⎣
⎡=⎥⎦
⎤⎢⎣
⎡hgfe
dcba
utsr
C = A ⋅ Br = ae + bgs = af + bht = ce + dhu = cf + dg
8 mults of (n/2)×(n/2) submatrices4 adds of (n/2)×(n/2) submatrices^
recursive
![Page 32: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/32.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.32
Analysis of D&C algorithm
# submatricessubmatrix size
work adding submatrices
T(n) = 8 T(n/2) + Θ(n2)
![Page 33: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/33.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.33
Analysis of D&C algorithm
# submatricessubmatrix size
work adding submatrices
T(n) = 8 T(n/2) + Θ(n2)
nlogba = nlog28 = n3 ⇒ CASE 1 ⇒ T(n) = Θ(n3).
![Page 34: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/34.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.34
Analysis of D&C algorithm
# submatricessubmatrix size
work adding submatrices
T(n) = 8 T(n/2) + Θ(n2)
nlogba = nlog28 = n3 ⇒ CASE 1 ⇒ T(n) = Θ(n3).
No better than the ordinary algorithm.
![Page 35: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/35.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.35
Strassen’s idea• Multiply 2×2 matrices with only 7 recursive mults.
![Page 36: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/36.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.36
Strassen’s idea• Multiply 2×2 matrices with only 7 recursive mults.
P1 = a ⋅ ( f – h)P2 = (a + b) ⋅ hP3 = (c + d) ⋅ eP4 = d ⋅ (g – e)P5 = (a + d) ⋅ (e + h)P6 = (b – d) ⋅ (g + h)P7 = (a – c) ⋅ (e + f )
![Page 37: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/37.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.37
Strassen’s idea• Multiply 2×2 matrices with only 7 recursive mults.
r = P5 + P4 – P2 + P6s = P1 + P2t = P3 + P4u = P5 + P1 – P3 – P7
P1 = a ⋅ ( f – h)P2 = (a + b) ⋅ hP3 = (c + d) ⋅ eP4 = d ⋅ (g – e)P5 = (a + d) ⋅ (e + h)P6 = (b – d) ⋅ (g + h)P7 = (a – c) ⋅ (e + f )
![Page 38: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/38.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.38
Strassen’s idea• Multiply 2×2 matrices with only 7 recursive mults.
r = P5 + P4 – P2 + P6s = P1 + P2t = P3 + P4u = P5 + P1 – P3 – P7
P1 = a ⋅ ( f – h)P2 = (a + b) ⋅ hP3 = (c + d) ⋅ eP4 = d ⋅ (g – e)P5 = (a + d) ⋅ (e + h)P6 = (b – d) ⋅ (g + h)P7 = (a – c) ⋅ (e + f )
7 mults, 18 adds/subs.Note: No reliance on commutativity of mult!
7 mults, 18 adds/subs.Note: No reliance on commutativity of mult!
![Page 39: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/39.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.39
Strassen’s idea• Multiply 2×2 matrices with only 7 recursive mults.
r = P5 + P4 – P2 + P6= (a + d) (e + h)
+ d (g – e) – (a + b) h+ (b – d) (g + h)
= ae + ah + de + dh + dg –de – ah – bh+ bg + bh – dg – dh
= ae + bg
P1 = a ⋅ ( f – h)P2 = (a + b) ⋅ hP3 = (c + d) ⋅ eP4 = d ⋅ (g – e)P5 = (a + d) ⋅ (e + h)P6 = (b – d) ⋅ (g + h)P7 = (a – c) ⋅ (e + f )
![Page 40: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/40.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.40
Strassen’s algorithm1. Divide: Partition A and B into
(n/2)×(n/2) submatrices. Form terms to be multiplied using + and – .
2. Conquer: Perform 7 multiplications of (n/2)×(n/2) submatrices recursively.
3. Combine: Form C using + and – on (n/2)×(n/2) submatrices.
![Page 41: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/41.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.41
Strassen’s algorithm1. Divide: Partition A and B into
(n/2)×(n/2) submatrices. Form terms to be multiplied using + and – .
2. Conquer: Perform 7 multiplications of (n/2)×(n/2) submatrices recursively.
3. Combine: Form C using + and – on (n/2)×(n/2) submatrices.
T(n) = 7 T(n/2) + Θ(n2)
![Page 42: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/42.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.42
Analysis of StrassenT(n) = 7 T(n/2) + Θ(n2)
![Page 43: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/43.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.43
Analysis of StrassenT(n) = 7 T(n/2) + Θ(n2)
nlogba = nlog27 ≈ n2.81 ⇒ CASE 1 ⇒ T(n) = Θ(nlg 7).
![Page 44: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/44.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.44
Analysis of StrassenT(n) = 7 T(n/2) + Θ(n2)
nlogba = nlog27 ≈ n2.81 ⇒ CASE 1 ⇒ T(n) = Θ(nlg 7).
The number 2.81 may not seem much smaller than 3, but because the difference is in the exponent, the impact on running time is significant. In fact, Strassen’s algorithm beats the ordinary algorithm on today’s machines for n ≥ 32 or so.
![Page 45: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/45.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.45
Analysis of StrassenT(n) = 7 T(n/2) + Θ(n2)
nlogba = nlog27 ≈ n2.81 ⇒ CASE 1 ⇒ T(n) = Θ(nlg 7).
The number 2.81 may not seem much smaller than 3, but because the difference is in the exponent, the impact on running time is significant. In fact, Strassen’s algorithm beats the ordinary algorithm on today’s machines for n ≥ 32 or so.
Best to date (of theoretical interest only): Θ(n2.376L).
![Page 46: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/46.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.46
VLSI layoutProblem: Embed a complete binary tree with n leaves in a grid using minimal area.
![Page 47: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/47.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.47
VLSI layoutProblem: Embed a complete binary tree with n leaves in a grid using minimal area.
H(n)
W(n)
![Page 48: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/48.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.48
VLSI layoutProblem: Embed a complete binary tree with n leaves in a grid using minimal area.
H(n)
W(n)
H(n) = H(n/2) + Θ(1)= Θ(lg n)
![Page 49: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/49.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.49
VLSI layoutProblem: Embed a complete binary tree with n leaves in a grid using minimal area.
H(n)
W(n)
H(n) = H(n/2) + Θ(1)= Θ(lg n)
W(n) = 2W(n/2) + Θ(1)= Θ(n)
![Page 50: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/50.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.50
VLSI layoutProblem: Embed a complete binary tree with n leaves in a grid using minimal area.
H(n)
W(n)
H(n) = H(n/2) + Θ(1)= Θ(lg n)
W(n) = 2W(n/2) + Θ(1)= Θ(n)
Area = Θ(n lg n)
![Page 51: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/51.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.51
H-tree embeddingL(n)
L(n)
![Page 52: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/52.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.52
H-tree embeddingL(n)
L(n)
L(n/4) L(n/4)Θ(1)
![Page 53: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/53.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.53
H-tree embeddingL(n)
L(n)
L(n) = 2L(n/4) + Θ(1)= Θ( )n
Area = Θ(n)
L(n/4) L(n/4)Θ(1)
![Page 54: Introduction to Algorithms - MIT OpenCourseWare Fibonacci numbers Bottom-up: • Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. ... Introduction](https://reader030.vdocuments.us/reader030/viewer/2022020204/5b1b594b7f8b9a2d258e74a7/html5/thumbnails/54.jpg)
September 14, 2005 Copyright © 2001-5 Erik D. Demaine and Charles E. Leiserson L2.54
Conclusion
• Divide and conquer is just one of several powerful techniques for algorithm design.
• Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math).
• The divide-and-conquer strategy often leads to efficient algorithms.