1

CS 201Compiler Construction

Array Dependence Analysis &Loop Parallelization

Loop Parallelization

Goal: Identify loops whose iterations can be executed in parallel on different processors of a shared-memory multiprocessor system.

Matrix Multiplicationfor I = 1 to n do -- parallel

for J = 1 to n do -- parallelfor K = 1 to n do –- not parallel

C[I,J] = C[I,J] + A[I,K]*B[K,J]2

Data Dependences

Flow Dependence:S1: X = ….S2: … = X

Anti Dependence:S1: … = XS2: X = …

Output Dependence:S1: X = …S2: X = …

3

S1 δf S2

S1 δa S2

S1 δo S2

S1S1 S2

S2

δf

S1S1 S2

S2

δa

S1S1 S2

S2

δo

Example: Data Dependences

do I = 1, 40S1: A(I+1) = ….S2: … = A(I-1)

enddodo I = 1, 40

S1: A(I-1) = …S2: … = A(I+1)

enddodo I = 1, 40

S1: A(I+1) = …S2: A(I-1) = …

enddo4

S1S1 S2

S2

δf

S2S2 S1

S1

δa

S1S1 S2

S2

δo

Sets of Dependences

do I = 1, 100S: A(I) = B(I+2) + 1T: B(I) = A(I-1) - 1

enddo

5

SS TTδf

S(1): A(1) = B(3) + 1T(1): B(1) = A(0) - 1S(2): A(2) = B(4) + 1T(2): B(2) = A(1) - 1S(3): A(3) = B(5) + 1T(3): B(3) = A(2) - 1…………..S(100): A(100) = B(102) + 1T(100): B(100) = A(99) - 1

Due to A()

Set of iteration pairs associated with this dependence:{(i,j): j=i+1, 1<=i<=99}Dependence distance:j-i=1 constant in this case.

Nested Loops

level 1: do I1 = 1, 100level 2: do I2 = 1, 50

S: A(I1,I2) = A(I1,I2-1) + B(I1,I2) enddo

enddo

6

Value computed by S in an iteration (i1,i2) is same as the value used in an iteration (j1,j2):

A(i1,i2)A(j1,j2-1)iff i1=j1 and i2=j2-1

• S is flow dependent on itself at level 2 (corresponds to inner loop) for fixed value of I1 dependence exists between different iterations of second loop (inner loop).• iteration pairs:

{((i1,i2),(j1,j2)): j1=i1, j2=i2+1, 1<=i1<=100, 1<=i2<=49}

Nested Loops Contd..

7

• Iteration pairs:{((i1,i2),(j1,j2)): j1=i1, j2=i2+1,

1<=i1<=100, 1<=i2<=49}• Dependence distance vector:

(j1-i1,j2-i2) = (0,1)

• There is no dependence at level 1.

Computing Dependences - Formulation

8

Can we find iterations (i1,i2) & (j1,j2) such that

i1+1=j1; i2+1=j2

and1<=i1<=8 1<=j1<=8i1-3<=i2<=i1j1-3<=j2<=j1i1-3<=i2<=5 j1-3<=j2<=51<=i2<=i1 1<=j2<=j11<=i2<=5 1<=j2<=5

and i1, i2, j1, j2 are integers.

level 1: do I1 = 1, 8level 2: do I2 = max(I1-3,1), min(I1,5)

A(I1+1,I2+1) = A(I1,I2) + B(I1,I2) enddo

enddo

Computing Dependences Contd..

9

• Dependence testing is an integer programming problem NP-complete• Solutions trade-off Speed and Accuracy of the solver.• False positives: imprecise tests may report dependences that actually do not exist – conservative is to report false positives but never miss a dependence.

DependenceTests: extreme value test; GCD test; Generalized GCD test; Lambda test; Delta test; Power test; Omega test etc…

Extreme Value Test

10

Approximate test which guarantees that a solution exists but it may be a real solution not an integer solution.Let f: Rn R

st f is bounded in a set S contained in Rn

Let b be a lower bound of f in S and B be an upper bound of f in S:

b ≤ f(ā) ≤ B for any ā ε S contained in Rn

For a real number c, the equation f(ā) = c will

have solutions iff b ≤ c ≤ B.

Extreme Value Test Contd..

11

Example: f: R2 R; f(x,y) = 2x + 3y S = {(x,y): 0<=x<=1 & 0<=y<=1} contained in R2

lower bound, b=0; upper bound, B=51. Is there a real solution to the equation 2x + 3y = 4 ? Yes. (0.5,1) is a solution, there are many others.2. Is there an integer solution in S ? No. f(0,0)=0; f(0,1)=3; f(1,0)=2; & f(1,1)=5. For none of the integer points in S, f(x,y)=4.If there are no real solutions, there are no integer solutions. If there are real solutions, then integer solutions may or may not exist.

Extreme Value Test Contd..

12

Example: DO I = 1, 10

DO J = 1, 10A[10*I+J-5] = ….A[10*I+J-10]….

10*I1+J1-5 = 10*I2+J2-1010*I1-10*I2+J1-J2 = -5f: R4 R; f(I1,I2,J1,J2) = 10*I1-10*I2+J1-J2 1<=I1,I2,J1,J2<=10; lower bound, b=-99; upper bound, B=+99

since -99 <= -5 <= +99 there is a dependence.

Extreme Value Test Contd..

13

Extreme Value Test Contd..

14

Extreme Value Test Contd..

15

Extreme Value Test Contd..

16

Nested Loops – Multidimensional Arrays

17

Nested Loops Contd..

18