Chapter 13

Multiplication of MatricesSolving n unknown equations, matrix inversion,

Gaussian elimination requires 2/3 n^3AX=b, A;b, reduced raw echelon form,

X;c

Multiplication of Matrices

A = (aij) and B = (bij) 1 i,j n

where A and B are n x n matrices

AB = C where C = (cij) and

n

cij = aik bkj

k=1

Multiplication of Matrices

Classic method of multiplying 2x2 matrices: 8 multiplication, 4 additions

AB =

a11 a12

a21 a22

b11b12

b21b22

a11b11 + a12 b21 a11b12 + a12 b22

a21b11 + a22 b21 a21b12 + a22 b22

=

Multiplication of Matrices

In 1969, Strassen discovered a way to do same product AB using only 7 multiplications and 18 additions: m1 = (a11 + a22)(b11 + b22) m2 = (a21 + a22)b11 m3 = a11(b12 - b22) m4 = a22(b21 – b11) m5 = (a11 + a12)b22 m6 = (a21 - a11)(b11 + b12) m7 = (a12 - a22)(b21 + b22)

Multiplication of Matrices

AB is given by

AB = m1 + m4 – m5 + m7

m1 + m4

m3 + m5

m1 + m3 – m2 + m6

Multiplication of Matrices

Consider 2 nxn matrices, where n=2k

Partition A and B into 4 (n/2)x(n/2) submatrices;

A =A11 A12

A21 A22

B =B11 B12

B21 B22

AB =A11B11 + A12B21 A11B12 + A12B22

A21B11 + A22B21 A21B12 + A22B22

Multiplication of Matrices

M1 = (A11 + A22)(B11 + B22) M2 = (A21 + A22)B11

M3 = A11(B12 - B22) M4 = A22(B21 – B11) M5 = (A11 + A12)B22

M6 = (A21 - A11)(B11 + B12) M7 = (A12 - A22)(B21 + B22)

M1 + M4 – M5 + M7

M1 + M4

M3 + M5

M1 + M3 – M2 + M6 AB =

Multiplication of Matrices

The complexity of Strassen’s algorithm Satisfies the recurrence relation, T(n) = 7 T( ), n > 1, T(1)

= 1

T(n) (n ) *n^2.807

n-

2

log27

Multiplication of MatricesIn 1980, Winograd discovered an algorithm with 15 additions and still 7 multiplications- check!!!

m1=(a21 + a22 - a11)(b22 - b12 + b11)m2=a11b11m3=a12b21m4=(a11 - a21)(b22 – b12)m5=(a21 + a22)(b12 - b11)m6=(a12 - a21 + a11 - a22)b22m7=a22(b11 - b22 - b12 + b21)

Multiplication of Matrices

AB = m2 + m3

m1 + m2 + m4 – m7

m1 + m2 + m5 + m6

m1 + m2 + m4 + m5

Multiplication of Matrices

As of 2008, Coppersmith-Winograd algorithm for nxn matrix multiplication is the fastest known algorithm with n^2.376… (1987)Restrictions due to huge constants factors hidden in the big O notation… Search for algorithm details in the net…. n^(2+epsilon) is possible… But how???

Numbers Mathematics is the queen of the sciences and number theory is the queen of mathematics." — Gauss

It is also impossible to cover all aspects of number theory in this class…

Many public-key cryptography schemes use number theory e.g. RSA Cryptosystem, Elliptic curve cryptography.    

Numbers-Prime Number Theorem-1the prime number theorem (PNT) describes the asymptotic distribution of the prime numbers.

The prime number theorem gives a rough description of how the primes are distributed.

Roughly speaking, the prime number theorem states that if a random number nearby some large number N is selected, the chance of it being prime is about 1 / ln(N), where ln(N) denotes the natural logarithm of N. For example, near N = 10,000, about one in nine numbers is prime, whereas near N = 1,000,000,000, only one in every 21 numbers is prime. In other words, the average gap between prime numbers near N is roughly ln(N).[1]

Numbers-Prime Number Theorem-2Roughly speaking, the prime number theorem states that if a random number nearby some large number N is selected, the chance of it being prime is about 1 / ln(N), where ln(N) denotes the natural logarithm of N.

For example, near N = 10,000, about one in nine numbers is prime, whereas near N = 1,000,000,000, only one in every 21 numbers is prime. In other words, the average gap between prime numbers near N is roughly ln(N).

Numbers-Prime Number Theorem-3

As a consequence of the prime number theorem, one gets an asymptotic expression for the nth prime number, denoted by pn = N(lnN)

An even better approximation is pn = N(lnN)+Nln(lnN)+…

Yet we quest for newer approximations.

Numbers-square free integers In mathematics, a square-free integer is one divisible by no perfect square, except 1.

For example, 10 is square-free but 18 is not, as it is divisible by 9 = 32.

The first few square-free numbers are 1, 2, 3, 5, 6, 7, 10, 11, 13, 14, 15, 17, 19, 21, 22, 23, 26, 29, 30, 31, 33, 34, 35, 37, 38, 39,…       

Numbers-composite numbers

A composite number is a positive integer which has a positive divisor other than one or itself. In other words, if n > 0 is an integer and there are integers 1 < a, b < n such that n = a × b then n is composite. By definition, every integer greater than one is either a prime number or a composite number.

        

Numbers-composite numbers-2The number one is a unit - it is neither prime nor composite. For example, the integer 14 is a composite number because it can be factored as 2 × 7. Likewise, the integers 2 and 3 are not composite numbers because each of them can only be divided by one and itself.The first few composite numbers (sequence A002808 in OEIS) are4, 6, 8, 9, 10, 12, 14, 15, 16, 18, 20, 21, 22, 24, 25, 26, 27, 28, 30, 32, 33, 34, 35, 36, 38,       

Numbers-Carmichaelnumbers-1

Carmichael numbers are very important because they pass the Fermat primality test but are not actually prime. Carmichael number is a composite number n (n ≥ 2) and for any c with GCD(c, n) = 1, we have cn-1= 1 mod n where GCD denotes the Greatest Common Denominator.           

Korselt Theorem: A positive composite integer n is a Carmichael number if and only if n is square-free, and for all prime divisors p of n, it is true that (p – 1) / (n – 1) where the notation a / b indicates that a divides b.     

Numbers-Carmichaelnumbers-2

Quiz Determine the three square-free integers that provide the smallest Carmichael number ?            

Numbers-Carmichaelnumbers-3

Quiz Determine the three square-free integers that provide the smallest Carmichael number ?  

Answer: Smallest Carmichael number: 561 = 3 * 11 * 17          

Numbers-Carmichaelnumbers-3

Chapter 18

18.3 Monte Carlo Algorithms18.3.1 Biased MC Algorithms18.3.2 A MC algorithm for Testing Polynomial Equality18.3.3 Primality Testing

Eg. http://www.brighton-webs.co.uk/montecarlo/concept.asp

Monte Carlo Algorithms

A MC algorithm is a probabilisticalgorithm that has a certain probability of

returningthe correct answer whatever the input isconsidered.

A p-correct MC algorithm is aprobabilistic alg. that returns the correctanswer with the probability not less than

p,independent of the input.

Biased MC Algorithms

A MC algorithm for a decision problem is false-biased if it always correct when it returns the value false and only has some (small) probability of making a mistake when returning the value true.

Biased MC Algorithms(2)Idea is that the probability of returning correct output increases with repeated

trials…Example:function MCRepeat(k)for i:=1 to k do

if MC returns .false. thenreturn (.false.)

endifendforreturn(.true.) …….>>>> if all k execution returns true, then we return true!!!

end MCRepeat

Proposition:

Suppose that we have a p-correct false-biased Monte Carlo algorithm. Then the algorithm MCRepeat(k) is a

(1-power((1-p),k)) correct false biased MC algorithm.

Biased MC Algorithms(3)

Proof: For any given input I, Let q: probability that MCRepeat(k)

outputs .false. p: probability that MCRepeat(k) outputs the

correct answer for the given input I.

p = q(I)* Prob (MCRepeat(k) is correct | MCRepeat(k) outputs .false.) + (1-q(I))* Prob(MCRepeat(k) is correct | MCRepeat(k) outputs. true.)>=q (I) 1 + (1-q(I)) (1-power((1-p),k))>=q(I) (1-power((1-p),k)) + (1-q(I)) (1-power((1-p),k))= (1-power((1-p),k)) (q(I)+1-q(I))= (1-power((1-p),k))

Assume ¾- correct MC algorithm for primality testing. If MC ¾ correct, then for k=8, MCRepeat(k) is 0.9999847-correct. That is, MCRepeat(8) returns .true. with a strong confidence.

A MC algorithm for testing Polynomial Equality

Given f(x) , g(x) ,h(x) with degree 2n,n and n respectivelywe want to test if f(x) = g(x) * h(x)

Assume false biased 1/3-correct MC algorithm, test f(x) = g(x) * h(x) Input: f(x) , g(x), h(x) Output: .true. 100% of the time if f(x) is equal to g(x)*h(x) \ and at

most 2/3 of the time f(x) is not equal to g(x)*h(x)function TestPolyEqual (f(x),g(x),h(x))call Random({1,2…,3n},j)if f(j)=g(j)*h(j) then

return(.true.)else

return(.false.)endif end TestPolyEqualThis algorithm has O(n) complexity since the evaluations of f(j),g(j).h(j)can be done with 2n,n and n multiplications, respectively, using the

algorithmHornerEval.We can have f(j)=g(j)*h(j) at most 2n points for j : {1,2…,3n} Hence

TestPolyEqual is 1/3 correct. Also, Prob. that TestPolyEqualRepeat(20) makes an error is less than Power((2/3),20) < 0.0003 …

Observe: Power((2/3),20) close to unity!

Primality TestingThe primality testing problem and the related factoring problem of

finding the prime factors of a composite number, play important roles in cryptography.

Assume n is a m-digit number. Is this a prime number or not?

There are deterministic and probabilistic approaches for the answer…

The basic structure of randomized primality tests is as follows:1. Randomly pick a number a.2. Check some equality (corresponding to the chosen test) involving a and the given number n. If the equality fails to hold true, then n is a composite number, a is known as a witness for the compositeness, and the test stops.3. Repeat from step 1 until the required certainty is achieved.

After several iterations, if n is not found to be a composite number, then it can be declared probably prime.

Primality Testing- earlier approach

Assume n is a m-digit number. Is this a prime number or not?

A simple approach : Divide n to each integer between 2 and √n (assume lower limit sqrt(n))

Input: n positive integerOutput: .true. If n is a prime .false. otherwise

function TestPrimality(n)for i:=2 to √n do (assume lower limit sqrt(n))

if n mod i =0 then return(.false.) endif

endfor return(.true.)

end TestPrimality

Primality Testing- earlier approach

If n has m digits in base 10, then the number of digits in the decimal representation of n is given by the upper limit of m= log10 (n+1)

Thus, the worse case occurs when n is prime, at which the TestPrimality

executes the for loop √n times. W(n) = √n = power (10, m/2), where n is the number of digits. Thus,

TestPrimality is exponential with respect to input size, m. !!!!

eg. for an input having more than 40 digits, it would take millions ofyears to execute in worst case.!!!!

Primality Testing(another approach)Another approach would be to check whether

n=p1.p2 where p1 and p2 are two primes.function MCTestPrimality(n)call Random({2,...√n},j)if n mod j =0 then

return(.false.)else

return(.true.)endifend MCTestPrimality

If n=p1*p2, where p1<p2 are primes, then the probability to return correct answer .false. is less than 1/p1.We need an algorithm with a fixed positive probability for any size of input.

Primality Testing (some rules!!! )

A simple, but very inefficient primality test uses Wilson's theorem, which states that n is prime if and only if:

(n-1)!= -1 modnAlthough this method requires about n modular multiplications, rendering it impractical!!!.

Primality Testing(Fermats primality tests, a well known probabilistic approach)

If n is prime, then for all positive integers a<n, an-1 ≡1 (mod n)

If the result is different from 1, then n is composite. If it is 1, then n may or may not be prime…..

The Fermat primality test is only a heuristic test; some composite numbers (Carmichael numbers) will be declared "probably prime”.

Nevertheless, it is sometimes used if a rapid screening of numbers is needed, for instance in the key generation phase of the RSA public key cryptographic algorithm.

Primality Testing (5)By utilizing Fermats theorem, RSSTestPrimality algorithm is

designed, with a 0.75 correct false-biased property (see your book for details).

Many other probabilistic primality tests includes:The Miller–Rabin primality test and Solovay–Strassen primality

test

Also, deterministic primality tests includesManindra Agrawal, Neeraj Kayal and Nitin Saxena invented a new

deterministic primality test, the AKS primality test, which they proved runs in Õ((log n)12), later improved to Õ((log n)6) in 2004.

The elliptic curve primality test, which actually proves that the given number is prime, can be proven to run in O((log n)6)

the cyclotomy test; its runtime can be proven to be O((log n)c log log

log n), where n is the number to test for primality and c is a constant independent of n.

Another Prob. Algorithm: Las Vegas

An algorithm that always returns a correct answer, but it may run arbitrarily long without producing an answer, or it might make a random decision from which there is no recovery (a dead end)

HWLA

Discrete Fourier TransformLet x0, ...., xN-1 be complex numbers. The DFT and Inverse DFT is defined by the formula

The Discrete Fourier Transform DFTw of a n’th order polynomial is defined relative to a primitive root of unity w . A primitive n’th root of unity is a complex number w such that

wn = 1 and wk 1 for 1< k < n

DFTw(P(x)) = P(wn-1)xn-1 + ...+ P(w)x + P(1)

where P(x) = an-1xn-1 + .... +a1x + a0

Discrete Fourier Transform

DFTw can be seen as transforming the coefficient array a=[a0,a1,..,an-1] of the polnomial P(x) into coefficient array b=[b0,b1,...,bn-1] of the polynomial DFTw(P(x)).

Then a and b are related by; b0

b1

.

.

.bn-1

=

1 1 1 ... 11 w w2 ... wn-1

. . . ... .

. . . ... .

. . . ... .1 wn-1 (wn-1)2 ... (wn-1)n-1

a0

a1

.

.

.an-1

The Fast Fourier Transform

Choose the n points to evaluate the P(x) as;z0, z1, ..., zn/2-1 , -z0, -z1, ..., -zn/2-1 (1)

Let P(x) = Even(x2) + xOdd(x2) (2)

where Even(x) = an-2xn/2-1 + ... + a2x + a0

and Odd(x) = an-1xn/2-1 + ... + a3x + a1

now if we replace x by –x in (2), we obtain

P(-x) = Even(x2) - xOdd(x2) (2.1)Thus, to evaluate P(x) at the n points (0), we need only evaluate the polynomials Even and Odd at the n/2 points(z0)2,(z1)2,...,(zn/2-1)2 and then perform n/2 additions, n/2 subtractionsi and n multiplications as described in (2)

The Fast Fourier Transform

T(n) = 2T(n/2) + O(n) O(nlogn)

The same strategy can be applied to evaluate Even and Odd at (z0)2 ,(z1)2,..,(zn/2-1)2 if choose z0,z1 ,.., zn/2-1 to satisfy

(zn/4+j)2 = -(zj)2 , j = 0,1,...,n/4-1 (3)

zn/4+j = izj , j = 0,1,...,n/4-1, i = -1 (4)If we choose zn/8+j , j = 0,1...,n/8-1

(zn/8+j)4 = -(zj)4 , j = 0,1,...,n/8-1

The Fast Fourier Transform

Formula (4) can be satisfied if we set zn/8+j = i (zj) , j = 0,1,...,n/8-1

Table of special points for evaluating polynomials of degree 7z0 z1 z2 z3 z4 z5 z6 z7

1 i i i i -1 - i -i -i i

The Fast Fourier Transform

procedure FFTRec(a[0:n-1],n,w,b[0:n-1]) recursiveInput: a[0:n-1](an array of coefficients of the polynomial P(x)=an-1xn-1 + ... + a1x + a0

) n (a power of two) {n=2k} w (a primitive nth root of unity) Output: b[0:n-1](an array of values b[i]=P(wj), i=0,..,n-1) if n=1 then b[0] := a[0]

The Fast Fourier Transform

else {divide into even-indexed and odd indexed coefficients} for i:=0 to n/2-1 do Even[i] := a[2i] {[a0,a2,....,an-2]} Odd[i] := a[2i+1] {[a1,a3,....,an-1]} end for call FFTRec(Even[0:n/2-1],n/2,w2,e[0:n/2-1]) call FFTRec(Odd[0:n/2-1],n/2,w2,d[0:n/2-1])

The Fast Fourier Transform

[now combine according to (2) and (2.1)] for i:=0 to n/2-1 do b[i] := e[j] + wi d[i] b[i+n/2] := e[i] – wi d[i] end for end if end FFTRec

The Fast Fourier Transform

Tree recursive calls to FFTRec with n=8

P(x) n w [a0,a1,a2,a3,a4,a5,a6,a7] , 8, i

[a0,a2,a4,a6],4,i = w2 [a1,a3,a5,a7],4,i

[a0,a4],2,-1= w4 [a2,a6],2,-1 [a1,a5],2,-1 [a3,a7],2,-1

[a0],1,1 [a4],1,1 [a2],1,1 [a6],1,1 [a1],1,1 [a5],1,1 [a3],1,1 [a7],1,1

The Fast Fourier Transform

Suppose n = 2k , and :[0,...,n-1] [0,...,n-1] is the permutation such that aj is the coefficient to the i’th leaf node T in the tree of recursive calls of FFTRec with input coefficient array a[0:n-1]. Then the k-digit binary representation of k (i) is obtained from the k-digit binary representation of i by siply reversing the digits, where lading zeros are included (if neccassary)

The Fast Fourier Transform

Level by level FFT computations for n = 8P1,0(x)=a0+a1x P1,2(x)=a2+a0x P1,4(x)=a1+a5x P1,6(x)=a3+a7xP1,0(1) P1,0(-1) P1,2(1) P1,2(-1) P1,4(1) P1,4(-1) P1,6(1) P1,6(-1)

index 0 1 2 3 4 5 6 7

P0,0(1) P0,1(-1) P0,2(1) P0,3(-1) P0,4(1) P0,5(-1) P0,6(1) P0,7(-1)

a0 a4 a2 a6 a1 a5 a3 a7

a0+a4 a4 - a0 a2+ a6 a2 – a6 a1 + a5 a1 – a5 a3 + a7 a3 – a7

FFT calculations performed and stored in the array b[0:7] at level 2(j=1)

b[0:7]

The Fast Fourier Transform

P1,0(1) P1,0(-1) P1,2(1) P1,2(-1) P1,4(1) P1,4(-1) P1,6(1) P1,6(-1)

a0+a4 a4 - a0 a2+ a6 a2 – a6 a1 + a5 a1 – a5 a3 + a7 a3 – a7

P2,0(1) P2,0(i) P2,0(1) P1,2(-i) P2,4(1) P2,4(i) P2,4(-1) P2,4(-i)

index 0 1 2 3 4 5 6 7

FFT calculations performed and stored in the array b[0:7] at level 1( j=2)

b[0:7]

The Fast Fourier Transform

P2,0(1) P2,0(i) P2,0(i) P2,0(-i) P2,4(1) P2,4(i) P2,4(-1) P2,4(-i)

P3,0(1) P3,0(w) P3,0(i) P3,0(w3) P3,0(-1) P3,0(-w) P3,0(-i) P3,0(-w3)

index 0 1 2 3 4 5 6 7

FFT calculations performed in the array b[0:7] at level 0 (j=3)

b[0:7]

The Fast Fourier Transform

procedure FFT(a[0:n-1],n,w,b[0:n-1])input: a[0:n-1] (an array of coefficients of the polynomial P(x)=an-1xn-1 + ... + a1x + a0

) n (a positive integer) {n=2k} w (a primitive n’th root of unity)output: b[0:n-1] (an array of values b[i] = P(wi),i=0,...,n-1) call ReverseBinPerm(R[0:n-1]) for i:= 0 to n-1 b[i] = a[R[i]]end for

The Fast Fourier Transform

for j:= 1 to k do for s:= 0 to n-2J by 2j do for i := 0 to 2j-1 –1

Temp[s+i] := b[s+i] + w(wn/2j)i b[s+i+2j-1]

Temp[s+i+2j-1] := b[s+i] - w(wn/2j)i b[s+i+2j-1]

end for for i:= to n-1 do b[i] := Temp[i] end for end forend FFT

Transformation Techniques

multiplication

a,b

addition log10a + log10blog10a,log10b

log10x

ab

10x

Commutative diagram for transforming multiplication via logarithms

The Inverse Fourier Transforman & Fast Polynomial Multiplication

problem in

original domainprocedure 1 solution in

original domain

inversetransformation

transformation

problem in transformed domain procedure 2

solution in transformed domain

Generic commutative diagram for transforming problems

The Inverse Fourier Transforman & Fast Polynomial Multiplication

polynomials P and Qof input size at most m-1 2m-1 n=2k

product polynomialPQ

inverse of Discrete Fourier Transform

(DFTw)-1Discrete FourierTransform DFTw

[P(1),P(w),...,P(wn-1)][Q(1),Q(w),...,Q(wn-1)]

scalar multiplication

[P(1)Q(1),P(w),Q(w)...P(wn-1)Q(wn-1)]

Commutative diagram for computing product of two polynomials using DFTw

The Inverse Fourier Transforman & Fast Polynomial Multiplication

(DFTw)-1 = (1/n)DFTw-1

Fw-1 = 1/n

1 1 1 ... 11 w-1 w-2 ... w-(n-1)

. . . ... .

. . . ... .

. . . ... .1 w-(n-1) (w-2)n-1 ... (w-(n-1))n-1

The Inverse Fourier Transforman & Fast Polynomial Multiplication

polynomials P(x) and Q(x)of input size at most m-1 2m-1 n=2k

product polynomialPQ

Fast Fourier Transform (FFTw)

[P(1),P(w),...,P(wn-1)][Q(1),Q(w),...,Q(wn-1)]

O(nlogn)

scalar multiplication

[P(1)Q(1),P(w),Q(w)...P(wn-1)Q(wn-1)]

O(n)

use FFTw-1 to compute

(DFTw)-1 = (1/n)FFTw-1O(nlogn)

PARALEL SORTINGwith REGULAR SAMPLING

(MIMD)

The PSRS Algorithm

PSRS is a combination of a sequential sort, a

Load balancing phase, a data exchange and a

Parallel merge. Although any sequential sorting

and merge algorithm can be used, PSRS isdemonstrated using quicksort and

successive 2 way merge.

The PSRS Algorithm

Given n data items (indices 1, 2, 3, ..., n) and

p processors (1, 2, 3, ..., p), PSRS consists of

four phases.For example; with n = 36, p = 3

and the keys 0, 1, 2, ..., 36. For brevity, let ρ= p/2 and w = n/ p².

Phase 1:Sort Local Data

Each processor is assigned a contiguous block of n/p items. The blocks assigned to different processors are disjoint.Each processor, in parallel, sorts their local block of items using sequential quicksort.Begin the regular sampling load balancing heuristic. All p processors, in parallel, select the data items at local indices 1, w+1, 2w +1, ...., (p-1)w +1 to form a representative , sample of the locally sorted block .

Phase 1:Sort Local Data (cont.)

The p² selected data items, p from each of p processors, are regular sample of the entire data array.The local regular samples represent the keys and their value distribution at each processor. In Figure 1, each processor is assigned n/ p = 12 contiguous keys to sort.Each processor takes three samples, at the 1st, 5th and 9th indices since w = n / p² = 4, to form the local regular sample. Note that the distance between the indices of the samples is of fixed size.

One designated processor gathers and sorts the local regular samples.p-1 pivots are selected from the sorted regular sample, at indices p +ρ , 2p + ρ , 3p + ρ,..,(p-1) + ρ.Each processor receives a copy of the pivots and forms p partitions from their sorted local blocks. A partition is contiguous internally and is disjoint from the other partitions.In Figure 1, the 9 samples are collected together and sorted. From this list p-1 = 2 pivots are selected.The pivots are 23 and 51 because ρ = p/2 = 1. All processors then create three partitions.

Phase 2: Find Pivots then Partition

Phase 3: Exchange Partitions

In parallel, each processor i keeps the ith partition for itself and assigns the jth partition to the jth processor.For example, processor 1 receives partition 1 from all of the processors. Therefore, each processor keeps one partition, and reassigns p-1 partitions.For example, processor 3 sends the list (1,3,7,13,23) to processor 1, sends (39,43,45) to processor 2, and keeps (57,59,65,73) for itself.

Phase 4: Merge Partition

Each processor, in parallel, merges its p partitions into a single list that is disjoint from the merged lists of the other processors. The concatenation of all the lists is the final sortedlist. Note that, the keys in the final merged partitions at each processor are also partitioned by the pivots 23 and 51. The final sorted list is distributed over the 3processors.

Example: Phase1

Unsorted Initial List p=3, n=36

Processor1 Processor3

Processor2

-------------------------------------------------------------------------------

p1 Sorted Local Blocks p3

p2

Local Regular

Samples:

Phase2

Gathered Regular Samples: Processor1

Sorted Regular Samples:

Pivots:

Phase3

p1 p3

p2Formed Partitions

Phase4Re-assigned Partitions

From self:

From p2:

From p3:

From p1:

From self:

From p3:

From p1:

From p2:

From self:

Final merged partitions

Final Sorted List