towards auto‐tuning for the finite difference method in era of 200+ thread parallelisms

Towards Auto‐tuning for the Finite Difference Method

in Era of 200+ Thread Parallelisms

Takahiro Katagiri, Satoshi Ohshima, Masaharu Matsumoto

(Information Technology Center, The University of Tokyo)

1

ACSI2015, Tsukuba Application Session, January 28th (Wed) 9:45 ‐ 10:15

Outline1. Background and ppOpen‐AT Functions2. Code Optimization Strategy3. Performance Evaluation4. Conclusion

Background• High‐Thread Parallelism (HTP)

– Multi‐core and many‐core processors are pervasive.

• Multicore CPUs: 16‐32 cores, 32‐64 Threads with Hyper Threading (HT) or Simultaneous Multithreading (SMT)

• Many Core CPU: Xeon Phi – 60 cores, 240 Threads with HT.

– Utilizing parallelism with full‐threads is important.

4

Performance Portability (PP)◦ Keeping high performance in multiple computer environments. Not only multiple CPUs, but also multiple compilers. Run‐time information, such as loop length and number of threads, is important.

◦ Auto‐tuning (AT) is one of candidates technologies to establish PP in multiple computer environments.

ppOpen‐HPC Project• Middleware for HPC and Its AT Technology

– Supported by JST, CREST, from FY2011 to FY2015.– PI: Professor Kengo Nakajima (U. Tokyo)

• ppOpen‐HPC – An open source infrastructure for reliable simulation codes on post‐peta (pp) scale parallel computers.

– Consists of various types of libraries, which covers 5 kinds of discretization methods for scientific computations.

• ppOpen‐AT – An auto‐tuning language for ppOpen‐HPC codes – Using knowledge of previous project： ABCLibScript Project.– Auto‐tuning language based on directives for AT.

5

Software Architecture of ppOpen‐HPC

6

FVM DEMFDMFEM

Many‐core CPUs GPUsMulti‐core

CPUs

MG

COMM

Auto‐Tuning FacilityCode Generation for Optimization CandidatesSearch for the best candidateAutomatic Execution for the optimization

ppOpen‐APPL

ppOpen‐MATH

BEM

ppOpen‐AT

User Program

GRAPH VIS MP

STATIC DYNAMIC

ppOpen‐SYS FT

Optimize memory accesses

Design Philosophy of ppOpen‐AT • A Directive‐base AT Language

– Multiple regions can be specified.– Low cost description with directives.– Do not prevent original code execution.

• AT Framework Without Script Languages and OS Daemons (Static Code Generation)– Simple: Minimum software stack is required. – Useful: Our targets are supercomputers in operation, or just after development.

– Low Overhead: We DO NOT use code generator with run‐time feedback.

– Reasons: • Since supercomputer centers do not accept any OS kernel modification, and any daemons in login or compute nodes.

• Need to reduce additional loads of the nodes.• Environmental issues, such as batch job script, etc. 7

ppOpen‐AT SystemppOpen‐APPL /*

ppOpen‐ATDirectives

User KnowledgeLibrary

Developer

① Before Release‐time

Candidate1

Candidate2

Candidate3

CandidatenppOpen‐AT

Auto‐Tuner

ppOpen‐APPL / *

AutomaticCodeGeneration②

:Target Computers

Execution Time④

Library User

③

Library Call

Selection

⑤

⑥

Auto‐tunedKernelExecution

Run‐time

AT Timings of ppOpen‐AT (FIBER Framework)

OAT_ATexec()…do i=1, MAX_ITERTarget_kernel_k()…Target_kernel_m()…

enddo

Read the best parameter

Is this first call?

Yes

Read the best parameter

Is this first call?

Yes

One time execution (except for varying problem size and number of MPI processes )

AT for Before Execute‐time

Execute Target_Kernel_k() with varying parameters

Execute Target_Kernel_m() with varying parameters

parameter Store the best parameter

…

Target Application• Seism_3D: Simulation for seismic wave analysis.

• Developed by Professor Furumura at the University of Tokyo.– The code is re‐constructed as ppOpen‐APPL/FDM.

• Finite Differential Method (FDM) • 3D simulation

–3D arrays are allocated.• Data type: Single Precision (real*4) 11

Flow Diagram of ppOpen‐APPL/FDM

12

),,,(

}],,,)21({},,)

21({[1

),,(

2/

1

zyxqp

zyxmxzyxmxcx

zyxdxd

pqpq

M

mm

pq

Space difference by FDM.

),,(,121

21

zyxptfzyx

uu np

nzp

nyp

nxpn

p

n

p

Explicit time expansion by central difference.

Initialization

Velocity Derivative (def_vel)

Velocity Update (update_vel)

Stress Derivative (def_stress)

Stress Update (update_stress)

Stop Iteration?NO

YES

End

Velocity PML condition (update_vel_sponge)Velocity Passing (MPI) (passing_vel)

Stress PML condition (update_stress_sponge)Stress Passing (MPI) (passing_stress)

Target Loop Characteristics• Triple‐nested loops

!$omp parallel do do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01<Codes from FDM>

end doend do

end do!$omp end parallel do

OpenMP directive to the outer loop (Z‐axis)

Loop lengths are varied according to problem size, the number of MPI processes and OpenMP threads.

The codes can be separable by loop split.

What is separable codes?Variable definitions and references are separated.There is a flow‐dependency, but no data dependency between each other.

d1 = …d2 = ……dk = ……

… = … d1 …… = … d2 ……… = … dk …

Variable definitions

Variable references

d1 = …d2 = …… = … d1 …… = … d2 …

……dk = ……… = … dk …

Split toTwo Parts

ppOpen‐AT Directives: Loop Split & Fusion with data‐flow dependence

15

!oat$ install LoopFusionSplit region start!$omp parallel do private(k,j,i,STMP1,STMP2,STMP3,STMP4,RL,RM,RM2,RMAXY,RMAXZ,RMAYZ,RLTHETA,QG)

DO K = 1, NZDO J = 1, NYDO I = 1, NX

RL = LAM (I,J,K); RM = RIG (I,J,K); RM2 = RM + RMRLTHETA = (DXVX(I,J,K)+DYVY(I,J,K)+DZVZ(I,J,K))*RL

!oat$ SplitPointCopyDef region start QG = ABSX(I)*ABSY(J)*ABSZ(K)*Q(I,J,K)

!oat$ SplitPointCopyDef region endSXX (I,J,K) = ( SXX (I,J,K) + (RLTHETA + RM2*DXVX(I,J,K))*DT )*QGSYY (I,J,K) = ( SYY (I,J,K) + (RLTHETA + RM2*DYVY(I,J,K))*DT )*QGSZZ (I,J,K) = ( SZZ (I,J,K) + (RLTHETA + RM2*DZVZ(I,J,K))*DT )*QG

!oat$ SplitPoint (K, J, I)STMP1 = 1.0/RIG(I,J,K); STMP2 = 1.0/RIG(I+1,J,K); STMP4 = 1.0/RIG(I,J,K+1)STMP3 = STMP1 + STMP2RMAXY = 4.0/(STMP3 + 1.0/RIG(I,J+1,K) + 1.0/RIG(I+1,J+1,K))RMAXZ = 4.0/(STMP3 + STMP4 + 1.0/RIG(I+1,J,K+1))RMAYZ = 4.0/(STMP3 + STMP4 + 1.0/RIG(I,J+1,K+1))

!oat$ SplitPointCopyInsertSXY (I,J,K) = ( SXY (I,J,K) + (RMAXY*(DXVY(I,J,K)+DYVX(I,J,K)))*DT )*QGSXZ (I,J,K) = ( SXZ (I,J,K) + (RMAXZ*(DXVZ(I,J,K)+DZVX(I,J,K)))*DT )*QGSYZ (I,J,K) = ( SYZ (I,J,K) + (RMAYZ*(DYVZ(I,J,K)+DZVY(I,J,K)))*DT )*QG

END DO; END DO; END DO!$omp end parallel do!oat$ install LoopFusionSplit region end

Re‐calculation is defined.

Using the re‐calculation is defined.

Loop Split Point

Specify Loop Split and Loop Fusion

Re‐ordering of Statements: increase a chance to optimize register allocation by compiler

16

!OAT$ RotationOrder sub region startSentence iSentence ii

!OAT$ RotationOrder sub region end!OAT$ RotationOrder sub region start

Sentence 1Sentence 2

!OAT$ RotationOrder sub region end

Sentence iSentence 1Sentence iiSentence 2

Generated Code

The Kernel 1 (update_stress)• m_stress.f90（ppohFDM_update_stress）!OAT$ call OAT_BPset("NZ01")!OAT$ install LoopFusionSplit region start!OAT$ name ppohFDMupdate_stress!OAT$ debug (pp)!$omp parallel do private(k,j,i,RL1,RM1,RM2,RLRM2,DXVX1,DYVY1,DZVZ1,D3V3,DXVYDYVX1,

DXVZDZVX1,DYVZDZV1)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01RL1 = LAM (I,J,K)

!OAT$ SplitPointCopyDef sub region startRM1 = RIG (I,J,K)

!OAT$ SplitPointCopyDef sub region endRM2 = RM1 + RM1; RLRM2 = RL1+RM2; DXVX1 = DXVX(I,J,K); DYVY1 = DYVY(I,J,K); DZVZ1 = DZVZ(I,J,K)D3V3 = DXVX1 + DYVY1 + DZVZ1SXX (I,J,K) = SXX (I,J,K) + (RLRM2*(D3V3)‐RM2*(DZVZ1+DYVY1) ) * DTSYY (I,J,K) = SYY (I,J,K) + (RLRM2*(D3V3)‐RM2*(DXVX1+DZVZ1) ) * DTSZZ (I,J,K) = SZZ (I,J,K) + (RLRM2*(D3V3)‐RM2*(DXVX1+DYVY1) ) * DT

Measured B/F=3.2

The Kernel 1 (update_stress)• m_stress.f90（ppohFDM_update_stress）!OAT$ SplitPoint (K,J,I)!OAT$ SplitPointCopyInsert

DXVYDYVX1 = DXVY(I,J,K)+DYVX(I,J,K)DXVZDZVX1 = DXVZ(I,J,K)+DZVX(I,J,K)DYVZDZVY1 = DYVZ(I,J,K)+DZVY(I,J,K)SXY (I,J,K) = SXY (I,J,K) + RM1 * DXVYDYVX1 * DTSXZ (I,J,K) = SXZ (I,J,K) + RM1 * DXVZDZVX1 * DTSYZ (I,J,K) = SYZ (I,J,K) + RM1 * DYVZDZVY1 * DT

end doend do

end do!$omp end parallel do!OAT$ install LoopFusionSplit region end

Automatic Generated Codes for the kernel 1ppohFDM_update_stress #1 [Baseline]: Original 3-nested Loop #2 [Split]: Loop Splitting with K-loop

(Separated, two 3-nested loops) #3 [Split]: Loop Splitting with J-loop #4 [Split]: Loop Splitting with I-loop #5 [Split&Fusion]: Loop Fusion to #1 for K and J-loops

(2-nested loop) #6 [Split&Fusion]: Loop Fusion to #2 for K and J-Loops

(2-nested loop) #7 [Fusion]: Loop Fusion to #1

(loop collapse) #8 [Split&Fusion]: Loop Fusion to #2

(loop collapse, two one-nest loop)

The Kernel 2 (update_vel)• m_velocity.f90（ppohFDM_update_vel）!OAT$ install LoopFusion region start!OAT$ name ppohFDMupdate_vel!OAT$ debug (pp)!$omp parallel do private(k,j,i,ROX,ROY,ROZ)do k = NZ00, NZ01do j = NY00, NY01do i = NX00, NX01! Effective Density

!OAT$ RotationOrder sub region startROX = 2.0_PN/( DEN(I,J,K) + DEN(I+1,J,K) )ROY = 2.0_PN/( DEN(I,J,K) + DEN(I,J+1,K) )ROZ = 2.0_PN/( DEN(I,J,K) + DEN(I,J,K+1) )

!OAT$ RotationOrder sub region end!OAT$ RotationOrder sub region start

VX(I,J,K) = VX(I,J,K) + ( DXSXX(I,J,K)+DYSXY(I,J,K)+DZSXZ(I,J,K) )*ROX*DTVY(I,J,K) = VY(I,J,K) + ( DXSXY(I,J,K)+DYSYY(I,J,K)+DZSYZ(I,J,K) )*ROY*DTVZ(I,J,K) = VZ(I,J,K) + ( DXSXZ(I,J,K)+DYSYZ(I,J,K)+DZSZZ(I,J,K) )*ROZ*DT

!OAT$ RotationOrder sub region endend do; end do; end do

!$omp end parallel do!OAT$ install LoopFusion region end

Measured B/F=1.7

Automatic Generated Codes for the kernel 2ppohFDM_update_vel• #1 [Baseline]: Original 3‐nested Loop.• #2 [Fusion]: Loop Fusion for K and J‐Loops.

(2‐nested loop)• #3 [Fusion]: Loop Split for K, J, and I‐Loops.

(Loop Collapse)• #4 [Fusion&Re‐order]:

Re‐ordering of sentences to #1. • #5 [Fusion&Re‐order]:

Re‐ordering of sentences to #2. • #6 [Fusion&Re‐order]:

Re‐ordering of sentences to #3.

The Kernel 3 (update_stress_sponge)

• m_stress.f90（ppohFDM_update_stress_sponge）!OAT$ call OAT_BPset("NZ01")!OAT$ install LoopFusion region start!OAT$ name ppohFDMupdate_sponge!OAT$ debug (pp)!$omp parallel do private(k,gg_z,i,gg_y,gg_yz,i,gg_x,gg_xyz)do k = NZ00, NZ01gg_z = gz(k)do j = NY00, NY01gg_y = gy(j); gg_yz = gg_y * gg_z; do i = NX00, NX01gg_x = gx(i); gg_xyz = gg_x * gg_yz; SXX(I,J,K) = SXX(I,J,K) * gg_xyz; SYY(I,J,K) = SYY(I,J,K) * gg_xyzSZZ(I,J,K) = SZZ(I,J,K) * gg_xyz; SXY(I,J,K) = SXY(I,J,K) * gg_xyzSXZ(I,J,K) = SXZ(I,J,K) * gg_xyz; SYZ(I,J,K) = SYZ(I,J,K) * gg_xyz

end doend do

end do!$omp end parallel do!OAT$ install LoopFusion region end

Measured B/F=6.8

Automatic Generated Codes forthe kernel 3ppohFDM_update_sponge• #1 [Baseline]: Original 3‐nested loop• #2 [Fusion]: Loop Fusion for K and J‐loops

(2‐nested loop)• #3 [Fusion]: Loop Split for K, J, and I‐loops

(Loop Collapse)

An Example of Seism_3D Simulation West part earthquake in Tottori prefecture in Japan

at year 2000. ([1], pp.14) The region of 820km x 410km x 128 km is discretized with 0.4km. NX x NY x NZ = 2050 x 1025 x 320 ≒ 6.4 : 3.2 : 1.

[1] T. Furumura, “Large‐scale Parallel FDM Simulation for Seismic Waves and Strong Shaking”, Supercomputing News, Information Technology Center, The University of Tokyo, Vol.11, Special Edition 1, 2009. In Japanese.

Figure : Seismic wave translations in west part earthquake in Tottori prefecture in Japan. (a) Measured waves; (b) Simulation results; (Reference : [1] in pp.13)

AT Candidates in This Experiment

1. Kernel update_stress– 8 Kinds of Candidates with Loop fusion and Loop Split.

2. Kernel update_vel– 6 Kinds of Candidates with Loop Fusion and Re‐ordering of

Statements. 3 Kinds of Candidates with Loop Fusion.3. Kernel update_stress_sponge4. Kernel update_vel_sponge5. Kernel ppohFDM_pdiffx3_p46. Kernel ppohFDM_pdiffx3_m47. Kernel ppohFDM_pdiffy3_p48. Kernel ppohFDM_pdiffy3_m49. Kernel ppohFDM_pdiffz3_p410. Kernel ppohFDM_pdiffz3_m4 26

3 Kinds of Candidates with Loop Fusion for Data Packing and Data Unpacking.

11. Kernel ppohFDM_ps_pack12. Kernel ppohFDM_ps_unpack13. Kernel ppohFDM_pv_pack14. Kernel ppohFDM_pv_unpack

27

AT Candidates in This Experiment(Cont’d)

ppohFDM_ps_pack (ppOpen‐APPL/FDM)• m_stress.f90（ppohFDM_ps_pack）!OAT$ call OAT_BPset("NZ01")!OAT$ install LoopFusion region start!OAT$ name ppohFDMupdate_sponge!OAT$ debug (pp)!$omp parallel do private(k,j,iptr,i)

do k=1, NZPiptr = (k‐1)*3*NYP*NL2do j=1, NYPdo i=1, NL2i1_sbuff(iptr+1) = SXX(NXP‐NL2+i,j,k)i1_sbuff(iptr+2) = SXY(NXP‐NL2+i,j,k)i1_sbuff(iptr+3) = SXZ(NXP‐NL2+i,j,k)i2_sbuff(iptr+1) = SXX(i,j,k)i2_sbuff(iptr+2) = SXY(i,j,k)i2_sbuff(iptr+3) = SXZ(i,j,k)iptr = iptr + 3

end doend do

end do!$omp end parallel do

ppohFDM_ps_pack (ppOpen‐APPL/FDM)• m_stress.f90（ppohFDM_ps_pack）!$omp parallel do private(k,j,iptr,i)do k=1, NZPiptr = (k‐1)*3*NL2*NXPdo j=1, NL2do i=1, NXPj1_sbuff(iptr+1) = SXY(i,NYP‐NL2+j,k); j1_sbuff(iptr+2) = SYY(i,NYP‐NL2+j,k)j1_sbuff(iptr+3) = SYZ(i,NYP‐NL2+j,k)j2_sbuff(iptr+1) = SXY(i,j,k); j2_sbuff(iptr+2) = SYY(i,j,k); j2_sbuff(iptr+3) = SYZ(i,j,k)iptr = iptr + 3

end do; end do; end do!$omp end parallel do!$omp parallel do private(k,j,iptr,i)do k=1, NL2iptr = (k‐1)*3*NYP*NXPdo j=1, NYPdo i=1, NXPk1_sbuff(iptr+1) = SXZ(i,j,NZP‐NL2+k); k1_sbuff(iptr+2) = SYZ(i,j,NZP‐NL2+k)k1_sbuff(iptr+3) = SZZ(i,j,NZP‐NL2+k)k2_sbuff(iptr+1) = SXZ(i,j,k); k2_sbuff(iptr+2) = SYZ(i,j,k); k2_sbuff(iptr+3) = SZZ(i,j,k)iptr = iptr + 3

end do; end do; end do!$omp end parallel do!OAT$ install LoopFusion region end

Machine Environment (8 nodes of the Xeon Phi) The Intel Xeon Phi Xeon Phi 5110P (1.053 GHz), 60 cores Memory Amount：8 GB (GDDR5) Theoretical Peak Performance：1.01 TFLOPS One board per node of the Xeon phi cluster InfiniBand FDR x 2 Ports

Mellanox Connect‐IB PCI‐E Gen3 x16 56Gbps x 2 Theoretical Peak bandwidth 13.6GB/s Full‐Bisection

Intel MPI Based on MPICH2, MVAPICH2 4.1 Update 3 (build 048)

Compiler：Intel Fortran version 14.0.0.080 Build 20130728 Compiler Options：

‐ipo20 ‐O3 ‐warn all ‐openmp ‐mcmodel=medium ‐shared‐intel –mmic‐align array64byte

KMP_AFFINITY=granularity=fine, balanced (Uniform Distribution of threads between sockets)

Execution Details• ppOpen‐APPL/FDM ver.0.2• ppOpen‐AT ver.0.2• The number of time step: 2000 steps• The number of nodes: 8 node• Native Mode Execution• Target Problem Size (Almost maximum size with 8 GB/node)– NX * NY * NZ = 1536 x 768 x 240 / 8 Node– NX * NY * NZ = 768 * 384 * 120 / node(!= per MPI Process)

• The number of iterations for kernels to do auto‐tuning: 100

Execution Details of Hybrid MPI/OpenMP

• Target MPI Processes and OMP Threads on the Xeon Phi– The Xeon Phi with 4 HT (Hyper Threading) – PX TY: XMPI Processes and Y Threads per process– P8T240 : Minimum Hybrid MPI/OpenMP execution for ppOpen‐APPL/FDM, since it needs minimum 8 MPI Processes.

– P16T120– P32T60– P64T30– P128T15 – P240T8– P480T4

– Less than P960T2 cause an MPI error in this environment.

#0

#1

#2

#3

#4

#5

#6

#7

#8

#9

#10

#11

#12

#13

#14

#15

P2T8

#0

#1

#2

#3

#4

#5

#6

#7

#8

#9

#10

#11

#12

#13

#14

#15

P4T4 Target of cores for one MPI Process

Loop Length per thread (Z‐axis)(8 Nodes, 1536x768x240 / 8 Node )

0.5 1 2 24

6

12

Loop length per threadMPI processes for Z‐axis are different between each MPI/OpenMP execution

due to software restriction.

PERFORMANCE ON WHOLE TIME

Maximum Speedups by AT(Xeon Phi, 8 Nodes)

558

200 17130 20 51

Speedup [%]

Kinds of Kernels

Speedup = max ( Execution time of original code / Execution time with AT )

for all combination of Hybrid MPI/OpenMP Executions (PXTY)

NX*NY*NZ = 1536x768x240/ 8 Node

BREAK DOWN OF TIMINGS

HYPER THREATING (HT) EFFECT

COMPARISON WITH DIFFERENT CPU ARCHITECTURES

AUTO‐TUNING TIME ANDTHE BEST IMPLEMENTATION

RELATED WORK

Originality (AT Languages)AT Language

/ Items#1

#2

#3

#4

#5

#6

#7

ppOpen‐AT OAT Directives ✔ ✔ ✔ ✔ NoneVendor Compilers Out of Target Limited ‐Transformation

Recipes Recipe

Descriptions✔ ✔ ChiLL

POET Xform Description ✔ ✔ POET translator, ROSE

X language Xlang Pragmas ✔ ✔ X Translation,‘C and tcc

SPL SPL Expressions ✔ ✔ ✔ A Script Language

ADAPT

ADAPT Language

✔ ✔ PolarisCompiler

Infrastructure, Remote Procedure

Call (RPC)

Atune‐IL Atune Pragmas ✔ A Monitoring Daemon

PEPPHER PEPPHER Pragmas (interface)

✔ ✔ ✔ PEPPHER task graph and run-time

Xevolver Directive Extension(Recipe Descriptions)

(✔) (✔) (✔) ROSE,XSLT Translator

#1: Method for supporting multi-computer environments. #2: Obtaining loop length in run-time.#3: Loop split with increase of computations, and loop fusion to the split loop.#4: Re-ordering of inner-loop sentences. #5: Algorithm selection.#6: Code generation with execution feedback. #7: Software requirement.

(Users need to define rules. )

Conclusion Remarks• Loop transformation with loop fusion and loop split is a key technology for FDM codes to obtain high performance on many‐core architecture.

• AT with static code generation (with dynamic code selection) is a key technology for AT with minimum software stack for supercomputers in operation.

To obtain free code (MIT Licensing):http://ppopenhpc.cc.u‐tokyo.ac.jp/

Future Work• Improving Search Algorithm

– We use a brute‐force search in current implementation.• But it is enough by applying knowledge of application.

– We have implemented a new search algorithm.• d‐Spline based search algorithm (a performance model) ,collaborated with Prof. Tanaka (Kogakuin U.)

• Code Selection Between Different Architectures– Selection of codes between vector machine implementation and scalar machine implementation.

– Whole code selection on main routine is needed.• Off‐loading Implementation Selection (for the Xeon Phi)– If problem size is too small to do off‐loading, the target execution is performed on CPU automatically.

Thank you for your attention!Questions?

http://ppopenhpc.cc.u‐tokyo.ac.jp/

towards auto‐tuning for the finite difference method in era of 200+ thread parallelisms

Technology

satoshi ohshima

university of tokyo1acsi2015

january28th wed9

orjustafterdevelop lowoverhead