GPU accelerated RBF-FD solution of Poisson’sequation

Mitja Jancic1,2, Jure Slak1,3, Gregor Kosec11 “Jožef Stefan” Institute, Parallel and Distributed Systems Laboratory, Ljubljana, Slovenia

2 “Jožef Stefan” International Postgraduate School, Ljubljana, Slovenia3 Faculty of Mathematics and Physics, University of Ljubljana, Ljubljana, Slovenia

mitja.jancic@ijs.si, jure.slak@ijs.si, gregor.kosec@ijs.si

Abstract – The Radial Basis Function-generated finitedifferences became a popular variant of local meshlessstrong form methods due to its robustness regarding the po-sition of nodes and its controllable order of accuracy. In thispaper, we present a GPU accelerated numerical solution ofPoisson’s equation on scattered nodes in 2D for orders from2 up to 6. We specifically study the effect of using differentorders on GPU acceleration efficiency.

I. INTRODUCTION

In contrast to the traditional numerical methodsfor solving partial differential equations (PDE) that re-quire connectivity between discretization nodes, mesh-less methods can operate on scattered nodes. Althoughseemingly minimal difference, this feature made mesh-less methods popular [1] in various fields of scienceand engineering ranging from computational fluid dynam-ics [2, 3, 4] to option pricing [5]. The key to becom-ing so popular is that discretization of arbitrary domainwith scattered nodes is considered much easier problemin comparison with meshing. To some degree it can alsobe automated in dimensionless sense [6].

From a historical point of view, meshless methodswere introduced in 1990s. Since then, many meshlessmethods have been developed, e.g. the Element FreeGalerkin Method [7], the Diffuse Element Method [8], thePartition of Unity Method [9], the Local Petrov-GalerkinMethods [10], the h-p Cloud Methods [11], etc.

The radial basis function-generated finite differences(RBF-FD) was first mentioned in [12] as a local strongform meshless method for solving PDEs. The methodapproximates differential operators only using scatterednodes. However, often used RBFs, e.g. Gaussians, in-clude a shape parameter that can be crucial to the over-all method stability [13]. Stability issue has been recentlyaddressed by using Polyharmonic splines (PHS) and addi-tionally augmenting them with polynomials [14].

The generality of the meshless methods comes withthe price, namely higher complexity. First, the shape func-tions of stencil weights have to be computed every timefrom scratch. Second, the support sizes are typically muchbigger than in mesh-based methods, especially in high or-der RBF-FD methods. The natural way to accelerate mostcomputationally demanding parts of the solution proce-dure is to employ parallel computing. There have beenseveral reports on parallel meshless solution procedures

in the past, including distributed computing as well asshared memory approaches [4, 15, 16, 17, 18]. In this pa-per we analyze the graphics processing unit (GPU) accel-eration of RBF-FD explicit solution of Poisson problemwith Dirichlet boundary condition. We are especially in-terested in the effect of using different polynomial orders.

The rest of the paper is organized as follows: in sec-tion II a short presentation of local meshless methods isgiven, in section III we present our workflow and howGPU was included in our computations, in section IVproblem is explained, in section V results are presented,and in section VI final conclusions are given.

II. LOCAL STRONG FORM MESHLESS METHODS

In general, the idea of meshless methods is the use oflocal discretization points to construct an approximationof the considered field and later use it for manipulationwith differential operators. Discretization points are re-ferred to as computational nodes or just nodes. The nodesare placed within the domain and on its boundary, andtheir distribution can be scattered, as presented in Fig. 1.

−1.00 −0.50 0.00 0.50 1.00

x

−1.00

−0.50

0.00

0.50

1.00

y

Solution

−1.00

−0.75

−0.50

−0.25

0.00

0.25

0.50

0.75

1.00

Figure 1. Solution of Poisson’s problem with Dirichlet boundaryconditions on scattered 2D nodes. Chosen highest polynomial degree

m = 2, support size n = 15 and number of nodes N = 1027.

Local strong form meshless methods approximatederivatives in the form

(Lu)(x) ≈∑

p∈N(x)

wLi (x)u(p), (1)

230 MIPRO 2020/DSBE

where N(x) is a set of neighboring nodes of x and L isa differential operator. Different ways of computing theweights w exists. We will use the RBF-FD method [19].

We have implemented the RBF-FD based solutionprocedure using object oriented approach and C++’sstrong template system. Node positioning, support se-lection, differential operator approximation and PDE dis-cretization and other modules are all available in onepackage, the Medusa library [20]. Please refer to our opensource Medusa library for more features and examples.

III. GPU IMPLEMENTATION

Compared to a CPU-based system, the memory band-width in GPU based systems is often an order of magni-tude higher, however at the price of higher memory la-tency. A single access to on board GPU memory takes ap-proximately 400-600 cycles compared to a floating pointoperation which takes approximately 1-2 cycles. CurrentGPUs support large numbers of processing elements andmust be programmed in a high data-parallel way in orderto observe the advantages of GPU programming and tohelp hide the memory latency. Large data is also neededto keep the processing units busy and good knowledge ofGPU systems is required to reduce the memory commu-nication and optimize the kernel functions [17].

In this work only part of the problem was dumped tothe GPU. The illustration of the GPU implementation isin Fig. 2. Green boxes are specific to GPU programmingwhile the yellow box presents the actual calculations doneon the GPU. Shapes, node positions and support nodeswere still tasks executed by the CPU, only the explicittime loop (yellow box in Fig. 2) was ported to the GPU.Time loop execution was also timed using high resolutiontimer, enabling us to estimate the speedups and thus eval-uate the performance of CPU and GPU.

Initialization

Domain discretization

Find support domain

Compute operatorshapes

Set initial coniditions

Allocate memory onGPU

Copy data to GPU

Time loop execution

Copy data from GPU

Save to file

End

Start

Figure 2. The implementation scheme.

A. Time loop implementationOnce the shapes, node positions and support nodes

computation was carried out on the host (CPU), memorywas allocated on the device (GPU) using cudaMallocmemory allocation function. As necessary data struc-tures for numerically feasible problem sizes on the on-board GPU memory, we simply copied relevant datato the host using the cudaMemcpy function withcudaMemcpyHostToDevice argument and left thedata there during time loop execution.

Time loop computation on the GPU was executed byintroducing a solveOnGpu kernel function as shown incode listing 1.

int tpb = 32;int bpg = (N + tpb − 1) / tpb;

for (int step = 0; step < timeSteps; ++step) solveOnGpu <<<bpg, tpb>>> (d_u1, d_u2, ...);cudaMemcpy(d_u1, d_u2, sizeof(double) * N,

cudaMemcpyDeviceToDevice);

Listing 1: Sample code of time loop execution on GPU.

In Compute Unified Device Architecture (CUDA) theactual use of multi-processors and processing units is con-trolled by block size [21]. This reflects in the number ofthreads which act on each processor. In order to fully ex-ploit the benefits of a GPU, proper definition of block sizeis of great importance. When chosen to small, processingunits tend to stall, if chosen to large other effects (e.g. reg-ister spilling) might interfere with top performance [17].

Several tests of block size of multiples of 32 weremade. Best time performance showed at Ntpb = 32threads per block andNbpg = (N+Ntpb−1)/Ntpb blocksper grid with N nodes in the domain.

Once the time loop execution on the GPU is fin-ished, cudaDeviceSynchronize function is usedfor synchronization and results are copied from thedevice to host using cudaMemcpy function withthe cudaMemcpyDeviceToHost argument. Theresults are saved to a file and post-processing isdone with python 3. To prevent memory leakage,cudaFree(varibaleName) was used before theshutdown.

IV. PROBLEM DESCRIPTION

With the aim to analyze the performance of CPU andGPU in terms of support size and highest augmented poly-nomial degree, proposed solution procedure and its imple-mentation is studied on a Poisson problem with Dirichletboundary conditions:

∂

∂tu(p)−∇2u(p) = f(p) in Ω, (2)

u(p) =d∏

i=1

sin(πi) on ∂Ω, (3)

where f(p) = dπ2∏d

i=1 sin(πpi) and domain space Ω isa d = 2 dimensional unit disk with boundary ∂Ω.

MIPRO 2020/DSBE 231

The problem is discretized in both time and spatial do-main space

u2(p) = u1(p) + dt(f(p) +∇2u1(p)), (4)

where dt is an infinitesimal time step and ∇2 is approxi-mated as described in section II.

The closed form solution of the above problem isu(p) =

∏di=1 sin(πpi) allowing us to validate the numer-

ically obtained solution.

V. RESULTS

Numerical results are computed using RBF-FD withPHS radial basis functions Φ(r) = r3 and monomialaugmentation. Radial function was kept the same for allcases. The convergence order is directly controlled bythe highest augmented polynomial degree m ∈ 2, 4, 6,however the larger the polynomial degree, the larger therecommended support size n =

(m+dm

). Recommended

support size in terms of domain space dimensionality andhighest polynomial degree was first given by Bayona [14],however larger supports can also be used [22]. In our testn ∈ 12, 15, 20, 30, 45, 60.

All computations were performed on a single core of acomputer with Intel(R) Xeon(R) CPU E5-2620v3 @ 2.40GHz processor and 64 GB of DDR4 mem-ory and graphical processing unit NVIDIA GeForceRTX 2080 Ti, 11GB GDDR6 with 4352 CUDAcores. Code was compiled using g++ (GCC) 8.1.0on Linux with -O3 -DNDEBUG -std=c++11 flagswhile GPU code was compiled using CUDA releaseV9.1.85.

An example of numerical solution is shown in Fig. 1.

A. Execution timesThe discretized Poisson problem (4) was solved for

105 time steps with time step dt = 10−6 seconds, result-ing in total simulation time t = 0.1 seconds.

The polynomial degree takes an important role in theshape computation stage. After shapes are stored the dif-ferential operator only depends on the support. As illus-trated on implementation scheme from Fig. 2, the shapesand supports are in our case calculated on the CPU. Dif-ferent time loop execution times for various polynomialdegrees m are therefore not expected neither are they ob-served.

In Fig. 3 we observe how the performance from CPU-only code is directly proportional to number of nodes,while multiple regimes can be seen in the single-GPU im-plementation.

102 103 104 105

N

10−1

100

101

102

103

tim

e[s

]

Time loop execution time

CPU

GPU

n = 12

n = 15

n = 20

n = 30

n = 45

n = 60

Figure 3. Time loop execution times on single CPU and GPU. Chosenhighest polynomial degree m = 2.

In the first regime it makes no significant differencewhat portion of the available L2 cache is occupied by thedata set. There is just not enough data dumped to the GPUto fully exploit the advantages of parallelization as mostof the time is spent on communication and data distribu-tion rather than on computation itself. For larger data sets(N > 103), the computation part prevails and faster com-putation times are observed. Increasing data set size evenfurther again leads to computation times proportional tothe number of nodes - similar to the CPU. As expectedgenerally higher execution times are observed for largersupport sizes.

Speedup is defined as ratio between the execution ofthe time loop on a single CPU tCPU and single GPUtGPU . Speedups are shown on Fig. 4. Similar regimesas in Fig. 3 can be seen starting with low or zero benefitfrom GPU implementation. Increasing the data exploitsthe parallelization advantages which peak at certain dataset size dumped to the GPU.

102 103 104 105

N

0

10

20

30

40

50

60

70

80

t CPU/tGPU

Speedup

n = 12

n = 15

n = 20

n = 30

n = 45

n = 60

Figure 4. Observed speedups.

To explain the notable drop of speedup in Fig. 4, wehave to understand hardware structure in more detail. TheL2 cache size of the NVIDIA GeForce RTX 2080Ti graphics processing unit is 5.5 MB [23]. Therefore in-creasing the data set size exploits the advantages of GPUimplementation but only until the L2 cache of the GPU is

232 MIPRO 2020/DSBE

almost, but not yet, full. Once the L2 cache is filled, morememory communication with higher latency is needed.

A rough estimation of how much data is copied to theGPU can be made. In our implementation we used dou-bles and integers,N(4+n) double-precision andNi+Nninteger numbers, where Ni is number of nodes in inte-rior. Knowing that size of double and integer is 8 and 4bytes respectively, we can estimate when the L2 cache isfull and approximately estimate the speedup peak posi-tion. The calculations are gathered in table 1. Note thatthe estimated N5.5 MB is calculated by enforcing Ni = N .The larger the number of nodes, the more acceptable thisenforcement is. Estimated data size copied on GPU is alsoshown in Fig. 5.

Support size Estimated N5.5 MB N speedup peak

12 30556 3108515 25463 2660120 19928 1952730 13889 1431345 9549 899160 7275 7707

TABLE 1. ESTIMATED AND OBSERVED PEAK PERFORMANCE ASFUNCTION OF NUMBER OF NODES N .

102 103 104 105

N

0

2

4

6

8

10

12

14

mem

ory

[MB

]

5.5 MB

Estimated data set size on GPU

n = 12

n = 15

n = 20

n = 30

n = 45

n = 60

Figure 5. Estimated data set size copied to GPU.

Table 1 shows how observed speedup peaks are closeto the estimated calculations. We therefore conclude thatthe drop of speedup performance is closely related to thesize of L2 cache.

VI. CONCLUSIONS

The execution performance of solution of a Poissonproblem on 2D scattered nodes with Dirichlet boundaryconditions was analyzed in this paper. We measured howchanging support sizes, a crucial parameter in using highorder RBF-FD, and total number of discretization nodesaffects the parallel execution performance. We observedthe speedup peaks, which can be explained by measuringthe data set size dumped to the GPU and comparing it withthe available L2 cache memory size.

In this paper only the execution of time loop wasconsidered as performance important, however some re-searchers already reported on using GPUs to calculate theshapes. The next step could therefore be to dump evenmore computation to the GPU. Additional extension couldalso be by solving the Poisson problem implicitly or bycomparing the computational performance using mulitpleCPU and GPU devices.

ACKNOWLEDGMENTS

The authors would like to acknowledge the financialsupport of the ARRS research core funding No. P2-0095and the Young Researcher program PR-08346.

REFERENCES

[1] Bengt Fornberg and Natasha Flyer. Solving pdeswith radial basis functions. Acta Numerica, 24:215–258, 2015.

[2] Gregor Kosec. A local numerical solution of a fluid-flow problem on an irregular domain. Adv. Eng. Soft-ware, 120:36–44, 2018.

[3] Eko Prasetya Budiana et al. Meshless numericalmodel based on radial basis function (rbf) method tosimulate the rayleigh–taylor instability (rti). Com-puters & Fluids, page 104472, 2020.

[4] Jia-Le Zhang, Zhi-Hua Ma, Hong-Quan Chen, andCheng Cao. A gpu-accelerated implicit meshlessmethod for compressible flows. J. Comput. Phys.,360:39–56, 2018.

[5] Jamal Amani Rad, Kourosh Parand, and Saeid Ab-basbandy. Pricing european and american optionsusing a very fast and accurate scheme: The mesh-less local petrov–galerkin method. Proceedings ofthe National Academy of Sciences, India Section A:Physical Sciences, 85(3):337–351, 2015.

[6] Jure Slak and Gregor Kosec. On generation ofnode distributions for meshless PDE discretiza-tions. SIAM Journal on Scientific Computing,41(5):A3202–A3229, jan 2019.

[7] T. Belytschko, Y. Y. Lu, and L. Gu. Element-freegalerkin methods. Int. J. Numer. Methods Eng.,37(2):229–256, 1994.

[8] B. Nayroles, G. Touzot, and P. Villon. Generalizingthe finite element method: Diffuse approximationand diffuse elements. Comput. Mech., 10(5):307–318, Sep 1992.

[9] J. M. Melenk and I. Babuška. The partition ofunity finite element method: Basic theory and ap-plications. Comput. Methods Appl. Mech. Eng.,139(1):289 – 314, 1996.

[10] Satya N. Atluri and Tulong Zhu. A new meshlesslocal petrov-galerkin (mlpg) approach in computa-tional mechanics. Comput. Mech., 22(2):117–127,1998.

MIPRO 2020/DSBE 233

[11] C. Armando Duarte and J. Tinsley Oden. H-pclouds—an h-p meshless method. Numerical Meth-ods for Partial Differential Equations: An Interna-tional Journal, 12(6):673–705, 1996.

[12] A. I. Tolstykh and D. A. Shirobokov. On using radialbasis functions in a “finite difference mode” with ap-plications to elasticity problems. Comput. Mech.,33(1):68–79, 2003.

[13] Natasha Flyer, Bengt Fornberg, Victor Bayona, andGregory A. Barnett. On the role of polynomials inRBF-FD approximations: I. Interpolation and accu-racy. J. Comput. Phys., 321:21–38, 2016.

[14] Victor Bayona, Natasha Flyer, Bengt Fornberg, andGregory A. Barnett. On the role of polynomials inRBF-FD approximations: II. Numerical solution ofelliptic PDEs. J. Comput. Phys., 332:257–273, 2017.

[15] Roman Trobec and Gregor Kosec. Parallel scientificcomputing: theory, algorithms, and applications ofmesh based and meshless methods. Springer, 2015.

[16] Jesse M. Kelly, Eduardo A. Divo, and Alain J.Kassab. Numerical solution of the two-phase in-compressible navier–stokes equations using a gpu-accelerated meshless method. Eng. Anal. BoundaryElem., 40:36–49, 2014.

[17] G. Kosec and P. Zinterhof. Local strong form mesh-less method on multiple Graphics Processing Units.CMES: Computer Modeling in Engineering and Sci-ences, 91(5):377–396, 2013.

[18] Evan F. Bollig, Natasha Flyer, and Gordon Er-lebacher. Solution to pdes using radial basis func-tion finite-differences (rbf-fd) on multiple gpus. J.Comput. Phys., 231(21):7133–7151, 2012.

[19] Jure Slak, Blaž Stojanovic, and Gregor Kosec. Highorder RBF-FD approximations with application toa scattering problem. In 2019 4th InternationalConference on Smart and Sustainable Technologies(SpliTech), pages 1–5. IEEE, 2019.

[20] Jure Slak and Gregor Kosec. Medusa: A C++ li-brary for solving pdes using strong form mesh-freemethods, 2019.

[21] John Nickolls, Ian Buck, Michael Garland, andKevin Skadron. Scalable parallel programming withcuda. Queue, 6(2):40–53, 2008.

[22] Mitja Jancic, Jure Slak, and Gregor Kosec. Anal-ysis of high order dimension independent RBF-FDsolution of Poisson’s equation, 2019.

[23] William Harmon. Nvidia geforce rtx 2080 ti graph-ics card review, 2019.

234 MIPRO 2020/DSBE