divide-and-conquer - cpp.edugsyoung/cs5300/cs530 hybrid notes/cs530...divide and conquer 2 young cs...

Divide and Conquer 1

YoungCS 530 Adv. Algo.

Topic: Divide and Conquer 1

Divide-and-Conquer

General idea:

Divide a problem into subprograms of the same kind; solve subprograms using the same approach, and combine partial solution (if necessary).



The straightforward algorithm:

max ← min ← A (1);for i ← 2 to n do

if A (i) > max, max ← A (i);if A (i) < min, min ← A (i);

Key comparisons: 2(n – 1)

1. Find the maximum and minimum

The problem: Given a list of unordered n elements, find max and min




List List 1 List 2n elements n/2 elements n/2 elements

min, max min1, max1 min2, max2

min = MIN ( min1, min2 )max = MAX ( max1, max2)



The Divide-and-Conquer algorithm:procedure Rmaxmin (i, j, fmax, fmin); // i, j are index #, fmax,

begin // fmin are output parameters case:

i = j: fmax ← fmin ← A (i);i = j –1: if A (i) < A (j) then fmax ← A (j);

fmin ← A (i); else fmax ← A (i);

fmin ← A (j);

else: mid ← ;call Rmaxmin (i, mid, gmax, gmin);call Rmaxmin (mid+1, j, hmax, hmin);fmax ← MAX (gmax, hmax);fmin ← MIN (gmin, hmin);

endend;

+

2ji




Eg: find max and min in the array: 22, 13, -5, -8, 15, 60, 17, 31, 47 ( n = 9 )

Index: 1 2 3 4 5 6 7 8 9Array: 22 13 -5 -8 15 60 17 31 47

Rmaxmin(1, 9, 60, -8)

1, 5, 22, -8 6, 9, 60, 17

1, 3, 22, -5 4, 5, 15, -8 6,7, 60, 17 8, 9, 47, 31

1,2, 22, 13 3, 3,-5, -5

(1)

(2)

(3) (4)

(5)

(6)

(7) (8)



Analysis: For algorithm containing recursive calls, we can use recurrence relation to find its complexity

T(n) - # of comparisons needed for RmaxminRecurrence relation:

+=

====

otherwisenTnT

nnTnnT

2)2

(2)(

21)(10)(

L222)

2(222)2)

8(2(2

22)2

(22)2)4

(2(2

2)2

(2)(

233

322

22

2

+++⋅=+++⋅=

++⋅=++⋅=

+=

nTnT

nTnT

nTnT




Assume n = 2k for some integer k

25.1

212

)22()2(2

)222()2

(2

1

1211

1

−=

−+⋅=−+⋅=

++++=

−

−−−

−

n

nnT

nT

kk

kkk

k L



Theorem:

Claim:

>+

==

1)(

1)(

nbncnaT

nbnT where are a, b, c constants

They don’t have to be the same constant. We make them the same here for simplicity. It will not affect the overall result.

>

=<

=

canO

cannLogOcanO

nTaLogc

c

)(

)()(

)(




Proof: Assume n = ck

bncnTanT +⋅= )()(

) ) ( (

) ) ( ) ( ) (( ) (

) ) ( ) ( ) ((

) ) ( ) ( ) ((

) ( ) (

) 1 ( ) (

) ) ( (

) ( ) ) ( ( ) (

0

021

021

021

0

0

2

2

1

1

2

2

33

232

22

2

ik

i

kkk

kkk

k

kkk

k

k

k

k

kk

c

a bn

c

a

c

a

c

a bn c

a bn

c

a

c

a

c

a bn c

n b a

c

a

c

a

c

a bn b a

c

a

c

a

c

a bn c

n T a

c

a

c

a bn c

n T a

bn c

n ab c

n b c

n T a a

bnc

n ab c

n T a bn c

n b c

n T a a n T

∑ =

− −

− −

− −

−

−

−

−

⋅ =

+ + + ⋅ + ⋅ =

+ + + ⋅ + ⋅ ⋅ =

+ + + ⋅ + ⋅ =

+ + + ⋅ + ⋅ =

+ + ⋅ + ⋅ =

+ ⋅ + ⋅ + ⋅ ⋅ =

+ ⋅ + ⋅ = + ⋅ + ⋅ ⋅ =

L

L

L

L

L



If a < c, is a constant

T(n) = bn · a constant∴ T(n) = O(n)

if a = c,

ik

i ca )(

0∑=

1)(0

+=∑=

kca i

k

i

)()1(

)1(1)(0

nLognOnLogbn

kbnbnnT

c

k

i

=+⋅=

+⋅== ∑=




If a > c, 1)(

1)()(

1

0 −

−=

+

=∑

ca

ca

ca

k

ik

i

)(

)(

1)(

1)()()(

1

0

aLog

aLog

nLogkk

ki

k

i

c

c

c

nO

nb

ababcabn

ca

ca

bncabnnT

=

⋅=

⋅=⋅=⋅≤

−

−⋅=⋅=

+

=∑



2. Integer multiplication

The problem:

Multiply two large integers (n digits)

The traditional way:

Use two for loops, it takes operations




The Divide-and-Conquer way:

Suppose large integers, , divide into two part a and b, same as into c and d.

x:

y:

ba baxn

+⋅=∴ 210

dc dcyn

+⋅=∴ 210

)(1010

)10)(10(

2

22

bcadbdac

dcbayxn

n

nn

+++⋅=

+⋅+⋅=⋅

So, transform the problem of multiply two integers of n-digit into four subproblems of multiply two integers of -digit

2n



Worst-Case is:

however, it is same as the traditional way.

Therefore, we need to improve equation show before:

Worst-Case is:

)()(

)2

(4)(

242 nOnO

bnnTnT

Log ==

+⋅=

)()(

)2

(3)(

58.132 nOnO

bnnTnT

Log ≈=

+⋅=

)))(((1010

)(1010

2

2

bdacdcbabdac

bcadbdacn

n

nn

−−++⋅++⋅=

+⋅++⋅=

K




The algorithm by Divide-and-Conquer: Large-int function multiplication (x,y)

beginn = MAX ( # of digits in x, #of digits in y);if (x = 0) or (y = 0), return 0;else if (n = 1), return x*y in the usual way;

elsem = ;a = x divide 10m;b = x rem 10m;c = y divide 10m;d = y rem 10m;p1= MULTIPLICATION (a, c);p2= MULTIPLICATION (b, d);p3 = MULTIPLICATION (a+b, c+d);return ;

end;

2n

)(1010 21322

1 ppppp mm −−++⋅



example: x = 143, y = 256

P’’=527p1 = 3p2 = 7p3 = 32

a = 1b = 7c = 3d = 1

m = 1n = 2x = 17y = 31

P’’

P’=350p1 = 2p2 = 20p3 = 35

a = 1b = 4c = 2d = 5

m = 1n = 2x = 14y = 25

P’

P=36608p1= p’p2= 18p3= p’’

a = 14b = 3c = 25d = 6

m = 1n = 3x = 143y = 256

P




3. Merge SortThe problem:

Given a list of n numbers, sort them in non-decreasing order

idea: Split each part into 2 equal parts and sort each part using Mergesort, then merge the tow sorted sub-lists into one sorted list.

The algorithm:procedure MergeSort (low, high)

beginif then

call MergeSort (low, mid);call MergeSort (mid+1, high);call Merge (low, mid, high);

end;

highlow <

+

←2

highlowmid



procedure Merge (low, mid, high)

begin

while (i ≤ mid and j ≤ high) if A(i) < A(j), then U(k) ←A(i);

i++;else U(k) ←A(j);

j++;k++;if (i > mid)

move A(j) through A(high) to U(k) through U(high);

elsemove A(i) through A(mid) to U(k) through U(high);

end;

lowkmidjlowi ←+←← ,1,




Analysis:

Worst-Case for MergeSort is:

Average-Case? Merge sort pays no attention to the original order of the list, it will keep divide the list into half until sub-lists of length 1, then start merging.

Therefore Average-Case is the same as the Worst-Case.

)(

)2

(2)(

nLognO

bnnTnT

=

+⋅=



4. Quick Sort

The problem:

Same as Merge Sort

Compared with Merge Sort:

• Quick Sort also use Divide-and-Conquer strategy

• Quick Sort eliminates merging operations




How it works?

Use partition

Eg: Sort array 65, 70, 75, 80, 85, 60, 55, 50, 4565 70 75 80 85 60 55 50 45

60 55 50 45 70 75 80 85

55 50 45

50 45

45

75 80 85

80 85

85

Pivot item

65

60 65 70

55 60 65 70 75

55 60 65 70 75 80



The algorithm:procedure QuickSort (p, q)

beginif p < q,then call Partition (p, q, pivotposition);

call QuickSort (p, pivotposition –1);call QuickSort (pivotposition+1, q);

end;

procedure Partition (low, high, pivotposition)begin

v ← A(low);j ← low;for i ← (low + 1) to high

if A(i) < v,j++;A(i) ↔ A(j);

pivotposition ← j;A(low) ↔ A(pivotposition);

end;




Analysis:

Worst-Case:Call Partition O(n) times, each time takes O(n) steps. So O(n2), and it is worse than Merge Sort in the Worst-Case.

Best-Case: Split the list evenly each time.

)(nLognO



Average-Case:Assume pivot is selected randomly, so the probability of pivot being the kth element is equal, ∀k.

C(n) = # of key comparison for n items

• average it over all k, (CA(n) is average performance)

• multiply both side by n

kn

kpivotprob ∀=← ,1)(

)()1()1( knCkCn −⋅+−⋅+−=

))2()1((1)1()(1

−+−+−= ∑=

kCkCn

nnC A

n

kAA

)1())1()1()0((2)1()(

−+++⋅+−=⋅

nCCCnnnCn

AAA

A

L




• replace n by n-1

• subtract (2) from (1)

)2())2()0((2)2)(1()1()1(

−++⋅+−−=−⋅−

nCCnnnCn

AA

A

L

)1()1()1(2)()1(2)1(2)1()1()(

−⋅++−⋅=⋅⇒−⋅+−⋅=−−−⋅

nCnnnCnnCnnCnnCn

AA

AAA



• divide n(n + 1) for both sides

∴ Best-Case = Average-Case

)()()(

)1(21212

)1(120

))1(

143

232

1(22

)1(0)1(

:

)1()1(2

)1()2(2

)1()2()3(2

2)3(

)1()1(2

)1()2(2

1)2(

)1()1(2)1(

1)(

12

2

nLognOnCLognO

LognLogdkkk

kkk

nnnC

CNote

nnn

nnn

nnn

nnC

nnn

nnn

nnC

nnn

nnC

nnC

A

ee

nn

k

n

k

A

A

A

A

AA

=⇒=

−⋅=⋅≤⋅≤

+−

⋅+=

+−

++⋅

+⋅

⋅+=

=

+−⋅

+⋅−

−⋅+

−⋅−−⋅

+−−

=

+−⋅

+⋅−

−⋅+

−−

=

+−⋅

+−

=+

∫∑

∑

=

=

L

L




5. Selection

The problem:

Given a list of n elements find the kth smallest one

Note: special cases: when k = 1, minwhen k = n, max

The solving strategy:

If use sorting strategy like MergeSort, it takes

If use Quick Sort, Best-Case is O(n), Worst-Case will call partition O(n) times, each time takes O(n), so Worst-Case is O(n2).

)(nLognO



Select1 – the first algorithm

idea: use “partition” (from Quick Sort) repeatedly until we find the kth element.

For each iteration:

If pivot position = k ⇒ Done!

If pivot poaition < k ⇒ to select (k-i)th element in the 2nd sub-list.

If pivot poaition > k ⇒ to select kth element in the 1st sub-list.

pivot

i = pivot position

1st sub-list 2nd sub-list




procedure Select1 (A, n, k) // A is array, n is # of array, k is keybegin

m ← 1;j ← n;loop

call Partition (m, j, pivotposition);case:

k = pivotposition:return A(k);

k < pivotposition:j ← pivotposition – 1;

else:m ← pivotposition + 1;

end loop;end;



3) By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)

How: Make sure pivot v is chosen s.t. at least some fraction of the elements will be smaller than v and at least some other fraction of elements will be greater than v.

mm(median of medians)Non-decreasingOrder(all columns)

The middle row is in non-decreasing order

Elements ≥ mm

Elements ≤ mm




3) By choosing pivot more carefully, we can obtain a selection algorithm with Worst-Case complexity O(n)

Use mm (median of medians) rule to choose pivot

procedure Select2 (A, n, k)begin

if n ≤ r, then sort A and return the kth element;

divide A into subset of size r each, ignore excess

elements, and let be the set of

medians of the subsets;

rn

=

rnmmmM ,,, 21 L

rn



Select2 ;

use “Partition” to partition A using v as the pivot;// Assume v is at pivotposition

case:k = pivotposition: return (v);

k < pivotposition: let S be the set of elementsA (1,…, pivotposition –1), return Select2 (S, pivotposition –1, k);

else: let R be the set of element A (pivotposition+1,…, n), return Select2 (R, n- pivotposition, k-pivotposition);

end case;end;

←v )2,,(

rn

rnM




Analysis:• How many Mi’s ≤ or ≥ mm?

At least

• How many elements ≤ or ≥ mm?

At least

• How many elements in or ?

At most

Assume r = 5

2

rn

⋅

22 r

nr

mmR > mmS <

)22

(

⋅

−

rnrn

)24(75.0

2.17.05

45.1

55.1)2

53(

≥≤

+=−

⋅−≤

−≤

⋅−∴=

nn

nnn

nnnn



• Therefore, procedure Select2 the Worst-Case complexity is:

where c is chosen sufficiently large, such that T(n) ≤ cn for n < 24.

Proof:

Use induction to show T(n) ≤ 20 • cn (n ≥ 24)

IB (induction base):

cnnTnTnT ++= )43()

5()(

cnccncn

cTTnTn

⋅≤++≤

⋅+⋅+==

2024

24)2443()

524()(,24




IH (induction hypothesis):

Suppose

IS (induction solution):

When n = m;

∴ T(n) = O(n) complexity of Select2 of n elements

mncnnT <≤∀⋅≤ 2420)(

cn

cmmcmc

cmmTmTmT

⋅≤

+⋅⋅⋅+⋅⋅≤

++=

204320

520

)43()

5()(



6. Matrix multiplication

The problem:

Multiply two matrices A and B, each of size [ ]nn×

nnnnnn

CBA

×××

=

⋅




The traditional way:

use three for-loop

kj

n

kikij BAC ×= ∑

=1

)()( 3nOnT =∴



The Divide-and-Conquer way:

⋅

=

2221

1211

2221

1211

2221

1211

BBB

BB

AAA

AA

CCC

CC

2222122122

2122112121

2212121112

2112111111

BABAC

BABAC

BABAC

BABAC

⋅+⋅=

⋅+⋅=

⋅+⋅=

⋅+⋅=

transform the problem of multiplying A and B, each of size [n×n] into 8 subproblems, each of size

×

22nn




)(

)2

(8)(

3

2

nO

annTnT

=

+⋅=∴

which an2 is for addition

so, it is no improvement compared with the traditional way



⋅

111111111

111111111Eg:

use Divide-and-Conquer way to solve it as following:

=

⋅

0000033303330333

0000011101110111

0000011101110111

=

⋅

+

⋅

=

=

+

⋅

=

=

⋅

+

⋅

=

=

⋅

+

⋅

=

0003

0001

0001

0101

0011

0033

0011

0001

1111

0011

0303

0001

0101

0101

1111

3333

0011

0101

1111

1111

22

21

12

11

C

C

C

C




Strassen’s matrix multiplication:• Discover a way to compute the Cij’s using 7 multiplications

and 18 additions or subtractions

))(()(

)()(

)())((

12111121

221211

112122

221211

112221

22112211

BBAAUBAATBBASBBAR

BAAQBBAAP

+−=+=

−=−=

+=++=

UQRPCSQCTRC

VTSPCBBAAV

+−+=+=+=

+−+=+−=

22

21

12

11

22212212 ))((



•Algorithm :

procedure Strassen (n, A, B, C) // n is size, A,B the input matrices, C output matrix

beginif n = 2,

else(cont.)

;;;;

2222121222

2122111221

2212121112

2112111111

babaCbabaCbabaCbabaC

⋅+⋅=⋅+⋅=⋅+⋅=⋅+⋅=




elsePartition A into 4 submatrices: ;Partition B into 4 submatrices: ;call Strassen ( ;

call Strassen (

call Strassen ( ;

call Strassen ( ;

call Strassen ( ;

22211211 ,,, AAAA22211211 ,,, BBBB

),,,2 22112211 PBBAAn

++

);,,,2 112221 QBAAn

+

),,,2 221211 RBBAn

−

),,,2 112122 SBBAn

−

),,,2 221211 TBAAn

+



call Strassen ( ;

call Strassen ( ;

end;

),,,2 12111121 UBBAAn

+−

),,,2 12111121 UBBAAn

+−

;;;

;

22

21

12

11

UQRPCSQCTRC

VTSPC

+−+=+=+=

+−+=




Analysis:

≤

>+⋅=2

2)2

(7)(2

nb

nannTnT

L

22223

3

222

2

2

)47()

47()

2(7

)47()

2(7

)2

(7)(

ananannT

anannT

annTnT

+⋅+⋅+⋅=

++⋅=

+⋅=



Assume n = 2k for some integer k

)()()(

)(7)47(7

)47(7

147

1)47(

7

1)47()

2(7

81.27

777

47

22

2

1

21

221

1

nOnOncbcnnb

ncnbcnb

ncb

anb

annT

Lg

LgLgLg

LgLgnLgnLgn

kk

k

k

kk

k

==

⋅+=+⋅=

+⋅=⋅+⋅=

⋅⋅+⋅≤

−

−+⋅=

++⋅+⋅=

−

−

−−

− L

divide-and-conquer - cpp.edugsyoung/cs5300/cs530 hybrid notes/cs530...divide and conquer 2 young cs...

Documents