divide-and-conquergsyoung/cs3310/cs3310notes/divideandconquer.pdftopic: divide and conquer 23 3....
TRANSCRIPT
CS 331 Dr. Young
Divide and Conquer0
1
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 1
Divide-and-Conquer
General idea:
Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 2
List List 1 List 2n elements n/2 elements n/2 elements
min, max min1, max1 min2, max2
min = MIN ( min1, min2 )max = MAX ( max1, max2)
CS 331 Dr. Young
Divide and Conquer0
2
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 3
The Divide-and-Conquer algorithm:procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax,
begin // fmin are output parameters case:
i = j: fmax ← fmin ← A (i);i = j –1: if A (i) < A (j) then fmax ← A (j);
fmin ← A (i); else fmax ← A (i);
fmin ← A (j);else: mid ← ;
call Rmaxmin (i, mid, gmax, gmin);call Rmaxmin (mid+1, j, hmax, hmin);fmax ← MAX (gmax, hmax);fmin ← MIN (gmin, hmin);
endend;
2
ji
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 4
Example: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )
Index: 1 2 3 4 5 6 7 8 9Array: 22 13 -5 -8 15 60 17 31 47
Rmaxmin(1, 9, 60, -8)
1, 5, 22, -8 6, 9, 60, 17
1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31
1,2, 22, 13 3, 3,-5, -5
(1)
(2)
(3) (4)
(5)
(6)
(7) (8)
CS 331 Dr. Young
Divide and Conquer0
3
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 5
The straightforward algorithm:
max ← min ← A (1);for i ← 2 to n do
if A (i) > max, max ← A (i);if A (i) < min, min ← A (i);
Key comparisons: 2(n – 1)
1. Find the maximum and minimum
The problem: Given a list of unordered n elements, find max and min
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 6
Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity
T(n) - # of comparisons needed for RmaxminRecurrence relation:
otherwisen
TnT
nnT
nnT
2)2
(2)(
21)(
10)(
222)
2(222)2)
8(2(2
22)2
(22)2)4
(2(2
2)2
(2)(
233
322
22
2
nT
nT
nT
nT
nTnT
CS 331 Dr. Young
Divide and Conquer0
4
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 7
Assume n = 2k for some integer k
25.1
212
)22()2(2
)222()2
(2
1
1211
1
n
nn
T
nT
kk
kkk
k
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 8
Theorem:
Claim:
1)(
1)(
nbnc
naT
nbnT where are a, b, c constants
They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result.
canO
cannLogO
canO
nTaLogc
c
)(
)(
)(
)(
CS 331 Dr. Young
Divide and Conquer0
5
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 9
Proof: Assume n = ck
bnc
nTanT )()(
) ) ( (
) ) ( ) ( ) (( ) (
) ) ( ) ( ) ((
) ) ( ) ( ) ((
) ( ) (
) 1 ( ) (
) ) ( (
) ( ) ) ( ( ) (
0
021
021
021
0
0
2
2
1
1
2
2
33
232
22
2
ik
i
kkk
kkk
k
kkk
k
k
k
k
kk
c
a bn
c
a
c
a
c
a bn
c
a bn
c
a
c
a
c
a bn
c
n b a
c
a
c
a
c
a bn b a
c
a
c
a
c
a bn
c
n T a
c
a
c
a bn
c
n T a
bn c
n ab
c
n b
c
n T a a
bnc
n ab
c
n T a bn
c
n b
c
n T a a n T
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 10
If a < c, is a constant
T(n) = bn · a constant T(n) = O(n)
if a = c,
ik
i c
a)(
0
1)(0
kc
a ik
i
)(
)1(
)1(1)(0
nLognO
nLogbn
kbnbnnT
c
k
i
CS 331 Dr. Young
Divide and Conquer0
6
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 11
If a > c, 1)(
1)()(
1
0
c
ac
a
c
ak
ik
i
)(
)(
1)(
1)()()(
1
0
aLog
aLog
nLogkk
k
ik
i
c
c
c
nO
nb
ababc
abn
c
ac
a
bnc
abnnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 12
2. Matrix multiplication
The problem:
Multiply two matrices A and B, each of size nn
nnnnnn
CBA
CS 331 Dr. Young
Divide and Conquer0
7
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 13
The traditional way:
use three for-loop
kj
n
kikij BAC
1
)()( 3nOnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 14
The Divide-and-Conquer way:
2221
1211
2221
1211
2221
1211
BB
B
BB
AA
A
AA
CC
C
CC
2222122122
2122112121
2212121112
2112111111
BABAC
BABAC
BABAC
BABAC
transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size
22
nn
CS 331 Dr. Young
Divide and Conquer0
8
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 15
)(
)2
(8)(
3
2
nO
ann
TnT
which an2 is for addition
so, it is no improvement compared with the traditional way
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 16
111
111
111
111
111
111Example:
use Divide-and-Conquer way to solve it as following:
0000
0333
0333
0333
0000
0111
0111
0111
0000
0111
0111
0111
00
03
00
01
00
01
01
01
00
11
00
33
00
11
00
01
11
11
00
11
03
03
00
01
01
01
01
01
11
11
33
33
00
11
01
01
11
11
11
11
22
21
12
11
C
C
C
C
CS 331 Dr. Young
Divide and Conquer0
9
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 17
Strassen’s matrix multiplication: Discover a way to compute the Cij’s using 7 multiplications
and 18 additions or subtractions
))((
)(
)(
)(
)(
))((
12111121
221211
112122
221211
112221
22112211
BBAAU
BAAT
BBAS
BBAR
BAAQ
BBAAP
UQRPC
SQC
TRC
VTSPC
BBAAV
22
21
12
11
22212212 ))((
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 18
•Algorithm :
procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix
beginif n = 2,
else(cont.)
;
;
;
;
2222122122
2122112121
2212121112
2112111111
babaC
babaC
babaC
babaC
CS 331 Dr. Young
Divide and Conquer0
10
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 19
elsePartition A into 4 submatrices: ;Partition B into 4 submatrices: ;call Strassen ( ;
call Strassen (
call Strassen ( ;
call Strassen ( ;
call Strassen ( ;
22211211 ,,, AAAA22211211 ,,, BBBB
),,,2 22112211 PBBAAn
);,,,2 112221 QBAAn
),,,2 221211 RBBAn
),,,2 112122 SBBAn
),,,2 221211 TBAAn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 20
call Strassen ( ;
call Strassen ( ;
end;
),,,2 12111121 UBBAAn
;
;
;
;
22
21
12
11
UQRPC
SQC
TRC
VTSPC
),,,2 22212212 VBBAAn
CS 331 Dr. Young
Divide and Conquer0
11
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 21
Analysis:
2
2)2
(7)(2
nb
nann
TnT
22223
3
222
2
2
)4
7()
4
7()
2(7
)4
7()
2(7
)2
(7)(
ananann
T
anann
T
ann
TnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 22
Assume n = 2k for some integer k
)()(
)(
)(7)4
7(7
)4
7(7
14
7
1)4
7(
7
1)4
7()
2(7
81.27
777
4
722
2
1
21
221
1
nOnO
ncbcnnb
ncnbcnb
ncb
anb
ann
T
Lg
LgLgLg
LgLgnLgnLgn
kk
k
k
kk
k
CS 331 Dr. Young
Divide and Conquer0
12
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 23
3. Integer multiplication
The problem:
Multiply two large integers (n digits)
The traditional way:
Use two loops, it takes O(n2) operations
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 24
The Divide-and-Conquer way:
Suppose x and y are large integers, divide x into two parts (a and b), and divide y into c and d.
x:
y:
a b baxn
210
c d dcyn
210
)(1010
)10)(10(
2
22
bcadbdac
dcbayxn
n
nn
So, transform the problem of multiplying two n-digit integersinto four subproblems of multiplying two -digit integers.
2
n
CS 331 Dr. Young
Divide and Conquer0
13
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 25
Worst-Case is:
However, it is same as the traditional way.
Instead, we write:
Worst-Case is:
)()(
)2
(4)(
242 nOnO
bnn
TnT
Log
)()(
)2
(3)(
58.132 nOnO
bnn
TnT
Log
)))(((1010
)(1010
2
2
bdacdcbabdac
bcadbdacyxn
n
nn
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 26
The algorithm by Divide-and-Conquer: function multiplication (x,y)
beginn = MAX ( # of digits in x, # of digits in y);if (x = 0) or (y = 0), return 0;else if (n = 1), return x*y in the usual way;
elsem = ;a = x divide 10m;b = x rem 10m;c = y divide 10m;d = y rem 10m;p1= MULTIPLICATION (a, c);p2= MULTIPLICATION (b, d);p3 = MULTIPLICATION (a+b, c+d);return ;
end;
2
n
)(1010 21322
1 ppppp mm
CS 331 Dr. Young
Divide and Conquer0
14
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 27
4. Merge SortThe problem:
Given a list of n numbers, sort them in non-decreasing order
idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the two sorted sub-lists into one sorted list.
The algorithm:procedure MergeSort (low, high)
beginif then
call MergeSort (low, mid);call MergeSort (mid+1, high);call Merge (low, mid, high);
end;
highlow
;2
highlow
mid
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 28
procedure Merge (low, mid, high)begin
while (i mid and j high) if A(i) < A(j), then U(k) A(i);
i++;else U(k) A(j);
j++;k++;
if (i > mid)move A(j) through A(high) to U(k) through U(high);
elsemove A(i) through A(mid) to U(k) through U(high);
for p low to highA(p) U(p);
end;
;,1, lowkmidjlowi
CS 331 Dr. Young
Divide and Conquer0
15
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 29
Analysis:Worst-Case for MergeSort is:
Best-Case & Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging.
Therefore Best-Case & Average-Case are the same as the Worst-Case. O(nLogn)
)(
)2
(2)(
nLognO
bnn
TnT
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 30
5. Quick Sort
The problem:
Same as Merge Sort
Compared with Merge Sort:
• Quick Sort also use Divide-and-Conquer strategy
• Quick Sort eliminates merging operations
CS 331 Dr. Young
Divide and Conquer0
16
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 31
How it works?
Use partition
Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 4565 70 75 80 85 60 55 50 45
60 55 50 45 70 75 80 85
55 50 45
50 45
45
75 80 85
80 85
85
Pivot item
65
60 65 70
55 60 65 70 75
55 60 65 70 75 80
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 32
The algorithm:procedure QuickSort (p, q)
beginif p < q,then call Partition (p, q, pivotposition);
call QuickSort (p, pivotposition –1);call QuickSort (pivotposition+1, q);
end;
procedure Partition (low, high, pivotposition)begin
v A(low);j low;for i (low + 1) to high
if A(i) < v,j++;A(i) A(j);
pivotposition j;A(low) A(pivotposition);
end;
CS 331 Dr. Young
Divide and Conquer0
17
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 33
Analysis:
Worst-Case:Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.
Best-Case: Split the list evenly each time.
)(nLognO
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 34
Average-Case:Assume pivot is selected randomly, so the probability of pivot being the kth element is equal, k.
C(n) = # of key comparison for n items
• average it over all k, (CA(n) is average performance)
• multiply both side by n
kn
kpivotprob ,1
)(
)1())1()1()0((2)1(
)(
nCCCnn
nCn
AAA
A
)()1()1( knCkCn
))()1((1
)1()(1
knCkCn
nnC A
n
kAA
CS 331 Dr. Young
Divide and Conquer0
18
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 35
• replace n by n-1
• subtract (2) from (1)
)2())2()0((2)2)(1(
)1()1(
nCCnn
nCn
AA
A
)1()1()1(2)(
)1(2)1(2)1()1()(
nCnnnCn
nCnnCnnCn
AA
AAA
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 36
• divide n(n + 1) for both sides
Best-Case = Average-Case
)()(
)(
)1(21
21
2
)1(
120
))1(
1
43
2
32
1(2
2
)1(
0)1(:
)1(
)1(2
)1(
)2(2
)1()2(
)3(2
2
)3(
)1(
)1(2
)1(
)2(2
1
)2(
)1(
)1(2)1(
1
)(
12
2
nLognOnC
LognO
LognLogdkkk
kk
k
nn
nC
CNote
nn
n
nn
n
nn
n
n
nC
nn
n
nn
n
n
nC
nn
n
n
nC
n
nC
A
ee
nn
k
n
k
A
A
A
A
AA
CS 331 Dr. Young
Divide and Conquer0
19
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 37
6. Selection (Project #2)The problem:
Given a list of n elements find the kth smallest oneNote: special cases: when k = 1, min
when k = n, max
The solving strategy:
Algorithm #1
Use sorting strategy like MergeSort, sort the whole list and then return the kth element of the sorted list
Both Best-Case and Worst-Case take O(n Log n)
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 38
Algorithm #2idea: use “partition” (from Quick Sort) procedure repeatedly until we find the kth smallest element.
For each iteration:If pivot position i = k Done!If pivot position i < k to select (k-i)th element in the 2nd sub-list.If pivot position i > k to select kth element in the 1st sub-list.
Best-Case is O(n) (when pivot is the kth smallest one)Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n2).
pivot
i = pivot position
1st sub-list 2nd sub-list
CS 331 Dr. Young
Divide and Conquer0
20
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 39
procedure Algorththm#2 (A, n, k) // A is array, n is # of array, k is key
beginm 1;j n;loop
call Partition (m, j, pivotposition);case:
k = pivotposition:return A(k);
k < pivotposition:j pivotposition – 1;
else:m pivotposition + 1;k k - pivotposition;
end loop;end;
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 40
Algorithm #3idea: Same as Algorithm#2
except it always find a special element of the list called MM (Median of Medians),and use MM as the pivot.
In class KLA activity to explain MM.
We can show that both Best-Case and Worst-Case take O(n)
CS 331 Dr. Young
Divide and Conquer0
21
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 41
By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)
How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v.
mm (median of medians)Non-decreasingOrder(all columns) The middle row is in
non-decreasing order
Elements mm
Elements mm
Example: 35 (n) elements divide into 7 (n/r) subsets of size 5 (r)
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 42
By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)
Use mm (median of medians) rule to choose pivot
procedure Select2 (A, n, k)begin
if n r, then sort A and return the kth element;
divide A into subset of size r each, ignore excess
elements, and let be the set of
medians of the subsets;
r
n
r
nmmmM ,,, 21
r
n
CS 331 Dr. Young
Divide and Conquer0
22
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 43
Select2 ;
use “Partition” to partition A using v as the pivot;// Assume v is at pivotposition
case:k = pivotposition: return (v);
k < pivotposition: let S be the set of elementsA (1,…, pivotposition –1), return Select2 (S, pivotposition –1, k);
else: let R be the set of element A (pivotposition+1,…, n), return Select2 (R, n- pivotposition, k-pivotposition);
end case;end;
v )2,,(
r
n
r
nM
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 44
Analysis:• How many Mi’s (or ) mm?
At least
• How many elements (or ) mm?
At least
• How many elements > mm (or < mm)? At most
Assume r = 5
2
r
n
22 r
nr
)22
(
r
nrn
)24(75.0
2.17.05
45.1
55.1)2
53(
nn
nn
n
nn
nn
CS 331 Dr. Young
Divide and Conquer0
23
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 45
• Therefore, procedure Select2 the Worst-Case complexity is:
where c is chosen sufficiently large so that T(n) cn for n < 24.
Proof:
Use induction to show T(n) 20 cn (n 24)
IB (induction base):
cnnTn
TnT )4
3()
5()(
cnccncn
cTTnTn
2024
24)244
3()
5
24()(,24
YoungCS 331 D&A of Algo.
Topic: Divide and Conquer 46
IH (induction hypothesis):
Suppose
IS (induction solution):
When n = m;
T(n) = O(n) complexity of Select2 of n elements
mncnnT 2420)(
cn
cmmcm
c
cmmTm
TmT
204
320
520
)4
3()
5()(