hpc-lab - tum · department of informatics v alex breuer 2 meetings date schedule 10/13/14 kickoff...

Department of Informatics V

Alex Breuer

Session 5: ProjectM. Bader, A. Breuer

HPC-Lab


Alex Breuer 2

MeetingsDate Schedule

10/13/14 Kickoff10/20/14 Q&A10/27/14 Presentation 111/03/14 H. Bast, Intel11/10/14 Presentation 212/01/14 Presentation 312/08/14 Q&A12/15/14 Presentation 412/22/14 Q&A, especially SWE01/12/15 Report: Project phase01/19/15 Presentation Project

Question w.r.t. to SWE?Go there!


Alex Breuer

SWEs: Differential Form

3

Prof. Dr. M. BaderDipl.-Math. A. BreuerS. Rettenberger, M. Sc.

Bachelor-Praktikum:Tsunami-Simulation April 2, 2013

In this assignment we are going to implement and test the most basic functionality of ourlab-course: The f-wave solver for the one-dimensional shallow water equations. The shallowwater equations are a system of nonlinear hyperbolic conservations laws with an optionalsource term:

h

hu

�

t

+

hu

hu2 + 1

2

gh2

�

x

= S(x, t). (1)

The quantities q = [h,hu]T are defined by h(x, t), the space-time dependent height of thewater column and hu(x, t), the space-time dependent momentum in spatial x-direction (uis the particle velocity). g the gravity constant (usually g := 9.81m/s2) and f := [hu,hu2 +1

2

gh2]T the flux function. As source term S(x, t) we will consider the e↵ect of space-dependentbathymetry only S(x) = [0,-ghB

x

]T , embedding of additional forces, such as friction or thecoriolis e↵ect is possible. Figure TODO illustrates the involved quantities.

To verify, that the most basic functionality of our program works as expected a proper(unit-) testing is required. We will do testing by a selection of standardized tests, for whicha solution is available.

Remark As units we use meters (m) and seconds (s) for all computations.

Literature

We discuss the basic ideas of numerics, software and strategies in our meetings, neverthelessmany important details can’t be covered in such a short time. We recommend a basic set ofliterature in each assignment as hint for your personal studies. In terms of this assignmentwe recommend the following list of books, papers and guides:

• Finite volume methods for hyperbolic problems, R. J. LeVeque, 2002

• Riemann solvers and numerical methods for fluid dynamics, E. F. Toro, 2009

• A wave propagation method for conservation laws and balance laws with spatially vary-

ing flux functions, D. S. Bale, 2003

• Thinking in C++: http://mindview.net/Books/TICPP/ThinkingInCPP2e.html

• git Documentation: http://git-scm.com/documentation

• Doxygen Manual : http://www.stack.nl/~dimitri/doxygen/manual

• CxxTest User Guide: http://cxxtest.com/guide.html

• SCons user Guide: http://www.scons.org/doc/production/HTML/scons-user.html

• Paraview Documentation: http://www.itk.org/Wiki/ParaView/Users_Guide/Table_Of_Contents


Alex Breuer

SWEs: Finite Volume Discretization

4


Alex Breuer

Homogenous SWEs: Exact Solution

5

2.4 lineare riemann-probleme 15

Abbildung 3: Anfangswerte des skalaren Riemann-Problems. Zum Zeit-punkt t = 0 sind die beiden konstanten Zustände q

l

undq

r

durch eine Unstetigkeit im Punkt x = 0 getrennt.

den Überlegungen in Kapitel 2.2 wissen wir bereits, dass sichjeder der Anfangswerte x

0

entlang einer Charakteristik X

0

(t) =

x

0

+ ut bewegt. Da die Charakteristiken im Fall der konstantenTransportgleichung parallele Geraden waren, folgt direkt, dass Die Charakteristik

der Unstetigkeit legtdie Lösung desskalarenRiemann-Problemsfest.

sich die Unstetigkeit entlang der Geraden X

dis

(t) = ut fortsetzt,wir erhalten als Lösung des skalaren Riemann-Problems

q(x, t) = q(x - ut) =

8<

:q

l

x - ut < 0

q

r

x - ut > 0.(2.24)

Graphisch nimmt q folglich in allen Punkten, die in Abbildung4 links bzw. rechts von X

dis

(t) liegen, den Wert q

l

bzw. q

r

an.Als Verallgemeinerung untersuchen wir im Folgenden noch das

Abbildung 4: Lösung des Riemann-Problems für die Transportglei-chung in der x-t-Ebene. Die Charakteristik der Unste-tigkeit X

dis

(t) bestimmt die Lösung.

262 13 Nonlinear Systems of Conservation Laws

13.4 A Two-Shock Riemann Solution

The shallow water equations are genuinely nonlinear, and so the Riemann problem alwaysconsists of two waves, each of which is a shock or rarefaction. In Example 13.4 the solutionconsists of one of each. The following example shows that other combinations are possible.

Example 13.5. Consider the Riemann data

h(x, 0) ≡ h0, u(x, 0) =!

ul if x < 0,

−ul if x > 0.(13.13)

If ul > 0, then this corresponds to two streams of water slamming into each other, withthe resulting solution shown in Figure 13.7 for the case h0 = 1 and ul = 1. The solutionis symmetric in x with h(−x, t) = h(x, t) and u(−x, t) = −u(x, t) at all times. A shockwave moves in each direction, bringing the fluid to rest, since the middle state must haveum = 0 by symmetry. The solution to this problem is computed in Section 13.7.1.

The characteristic structure of this solution is shown in Figure 13.8. Note again that 1-characteristics impinge on the 1-shock while crossing the 2-shock, whereas 2-characteristicsimpinge on the 2-shock.

Note that if we look at only half of the domain, say x < 0, then we obtain the solutionto the problem of shallow water flowing into a wall located at x = 0 with velocity ul . Ashock wave moves out from the wall, behind which the fluid is at rest. This is now exactlyanalogous to traffic approaching a red light, as shown in Figure 11.2.

−1 −0.5 0 0.5 1

1

1.1

1.2

1.3

1.4

1.5

1.6Height

−1 −0.5 0 0.5 1

−0.4

−0.2

0

0.2

0.4

0.6Velocity

−1 −0.5 0 0.5 1

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

Vertically integrated pressure

−1 −0.5 0 0.5 10

0.2

0.4

0.6

0.8

1Particle paths in x–t plane

Fig. 13.7. Structure of the similarity solution of the two-shock Riemann problem for the shallowwater equations with ul = − ur . The depth h, velocity u, and vertically integrated pressure aredisplayed as functions of x/t . The structure in the x–t plane is also shown with particle pathsindicated for a set of particles with the spacing between particles inversely proportional to the depth.[claw/book/chap13/twoshock]

Riemann problemShock-Shock Riemann solution

Source: R. LeVeque, Finite Volume Methods for Hyperbolic Problems, 2002


Alex Breuer

Homogenous SWEs: Exact Solution

6

2.4 lineare riemann-probleme 15

Abbildung 3: Anfangswerte des skalaren Riemann-Problems. Zum Zeit-punkt t = 0 sind die beiden konstanten Zustände q

l

undq

r

durch eine Unstetigkeit im Punkt x = 0 getrennt.

den Überlegungen in Kapitel 2.2 wissen wir bereits, dass sichjeder der Anfangswerte x

0

entlang einer Charakteristik X

0

(t) =

x

0

+ ut bewegt. Da die Charakteristiken im Fall der konstantenTransportgleichung parallele Geraden waren, folgt direkt, dass Die Charakteristik

der Unstetigkeit legtdie Lösung desskalarenRiemann-Problemsfest.

sich die Unstetigkeit entlang der Geraden X

dis

(t) = ut fortsetzt,wir erhalten als Lösung des skalaren Riemann-Problems

q(x, t) = q(x - ut) =

8<

:q

l

x - ut < 0

q

r

x - ut > 0.(2.24)

Graphisch nimmt q folglich in allen Punkten, die in Abbildung4 links bzw. rechts von X

dis

(t) liegen, den Wert q

l

bzw. q

r

an.Als Verallgemeinerung untersuchen wir im Folgenden noch das

Abbildung 4: Lösung des Riemann-Problems für die Transportglei-chung in der x-t-Ebene. Die Charakteristik der Unste-tigkeit X

dis

(t) bestimmt die Lösung.

13.2 Dam-Break and Riemann Problems 259

region is bounded by a hydraulic jump, where the depth of the water suddenly increasesand its speed abruptly decreases. By adjusting the flow rate or angle of the plate you shouldbe able to make the location of this shock wave move around. When the conditions arefixed, the shock is stationary and has zero propagation speed. This can be modeled by thetwo-dimensional shallow water equations, or in the radially symmetric case by the one-dimensional equations with additional source terms incorporated to model the geometriceffects as described in Section 18.9.

13.2 Dam-Break and Riemann Problems

Consider the shallow water equations (13.5) with the piecewise-constant initial data

h(x, 0) =!

hl if x < 0,

hr if x > 0,u(x, 0) = 0, (13.12)

where hl > hr ≥ 0. This is a special case of the Riemann problem in which ul = ur = 0,and is called the dam-break problem because it models what happens if a dam separatingtwo levels of water bursts at time t = 0. This is the shallow water equivalent of the shock-tube problem of gas dynamics (Section 14.13). We assume hr > 0.

Example 13.4. Figure 13.4 shows the evolution of the depth and fluid velocity for the dam-break problem with data hl = 3 and hr = 1. Figure 13.5 shows the structure of this solutionin the x–t plane. Water flows from left to right in a wedge that expands from the dam locationx = 0. At the right edge of this wedge, moving water with some intermediate depth hm andvelocity um > 0 slams into the stationary water with h = hr , accelerating it instantaneously

−5 −4 −3 −2 −1 0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5h at t = 0

−5 −4 −3 −2 −1 0 1 2 3 4 5−0.5

0

0.5

1

1.5

2hu at t = 0

−5 −4 −3 −2 −1 0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5h at t = 0.5

−5 −4 −3 −2 −1 0 1 2 3 4 5−0.5

0

0.5

1

1.5

2hu at t = 0.5

−5 −4 −3 −2 −1 0 1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5h at t = 2

−5 −4 −3 −2 −1 0 1 2 3 4 5−0.5

0

0.5

1

1.5

2hu at t = 2

Fig. 13.4. Solution of the dam-break Riemann problem for the shallow water equations withul = ur = 0. On the left is the depth h and on the right is the momentum hu. [claw/book/chap13/dambreak]

278 13 Nonlinear Systems of Conservation Laws

−5 −4 −3 −2 −1 0 1 2 3 4 50

0.5

1

1.5

2h at t = 0

−5 −4 −3 −2 −1 0 1 2 3 4 5−1

−0.5

0

0.5

1hu at t = 0

−5 −4 −3 −2 −1 0 1 2 3 4 50

0.5

1

1.5

2h at t = 0.75

−5 −4 −3 −2 −1 0 1 2 3 4 5−1

−0.5

0

0.5

1hu at t = 0.75

−5 −4 −3 −2 −1 0 1 2 3 4 50

0.5

1

1.5

2h at t = 3

−5 −4 −3 −2 −1 0 1 2 3 4 5−1

−0.5

0

0.5

1hu at t = 3

Fig. 13.14. Solution of the Riemann problem for the shallow water equations with ul = −ur < 0.[claw/book/chap13/tworaref]

two rarefaction waves as shown in Figure 13.14. (Looking at only half the domain gives thesolution to the boundary-value problem with water flowing away from a wall.)

To solve this Riemann problem, we can proceed in a manner similar to what we didin Section 13.7.1 for the all-shock solution. There is an integral curve of r1 through ql

consisting of all states that can be connected to ql by a 1-rarefaction, and an integral curveof r2 through qr consisting of all states that can be connected to qr by a 2-rarefaction. Theseare illustrated in Figure 13.15(a) for the Riemann data

ul = −0.5, ur = 0.5, and hl = hr = 1. (13.54)

(a)

−0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

h = depth

hu =

mom

entu

m

ql

qrqm

(b)

−0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5−2.5

−2

−1.5

−1

−0.5

0

0.5

1

1.5

2

2.5

h = depth

hu =

mom

entu

m

qlqr

qm

Fig. 13.15. (a) Construction of the all-rarefaction Riemann solution for the problem of Example 13.5.(b) The physically incorrect all-rarefaction Riemann solution for the dam-break problem of Exam-ple 13.4.

Rare-Rare Riemann solution Rare-Shock Riemann Solution

Source: R. LeVeque, Finite Volume Methods for Hyperbolic Problems, 2002


Alex Breuer

f-Wave Solver: Splitting the jump in fluxes

7

2.5 nichtlineare riemann-probleme 17

Abbildung 5: Lösung des Riemann-Problems für lineare Systeme inder x-t-Ebene (�1

< . . . < �

j

< 0 < . . . < �

n). An einemPunkt P(x, t) ergibt sich die Lösung durch seine Lage zuden Charakteristiken der Unstetigkeiten.

wieder aufgreifen werden. Dabei wird der Sprung q

r

- q

l

inEigenvektoren der Matrix A dargestellt. Durch Einsetzen derSummen aus den Gleichungen in (2.27) erhalten wir

q

r

- q

l

=nX

p=1

(wp

r

- w

p

l

)rp ⌘nX

p=1

↵

p

r

p =nX

p=1

r

p

↵

p. (2.31)

Der Sprung q

r

- q

l

wird also in Sprünge über die Unstetig-keiten zerlegt. Mit der Lösung ↵ = (↵1, . . . , ↵n)T des linearenGleichungssystems

R↵ = q

r

- q

l

(2.32)

und W

p ⌘ ↵

p

r

p hat die Zerlegung (2.30) in dieser Darstellungdie Form

q(x, t) = q

l

+X

p:�p

<x/t

W

p

= q

r

-X

p:�p

>x/t

W

p.(2.33)

2.5 nichtlineare riemann-probleme

Wir werden uns in diesem Kapitel mit den grundsätzlichen Ideenfür die exakte Lösung nichtlinearer Riemann-Probleme beschäfti-gen. Die hier präsentierte Theorie hat eher einen motivierendenCharakter (vgl. [37]), für eine ausführlichere Darstellung sei dahernoch einmal auf [30] verwiesen.

Aus Kapitel 2.2 wissen wir bereits, dass q in der sklaren Er-haltungsgleichung (2.7) eindeutig durch die Charakteristikengegeben ist, zumindest solange die Lösung glatt bleibt. Da die


Alex Breuer

1. Finite volume discretization at initial time

2. F-Wave solver: Edge-local Riemann solutions

3. Cells update to the next time step

Finite Volumes: Edge-local f-Waves

8

Q

n+1i = Q

ni -

�t

�x

⇣A

+�Q

ni-1/2 +A

-�Q

ni+1/2

⌘


Alex Breuer

f-Waves: Bathymetry

9


Alex Breuer

2D: Blocks with Ghost Layers

10

ghost layercomputational domain

10


Alex Breuer

2D: Discretization

11


11


Alex Breuer

2D: Edge Normal Net-updates

12


12


Alex Breuer

2D: Cell Updates

1313



Alex Breuer

2D: Update Formula

1414

Q

n+1i,j = Q

ni,j -

�t

�x

�A

+�Qi-1/2,j +A

-�Qi+1/2,j

�

-�t

�y

�B

+�Qi,j-1/2 + B

-�Qi,j+1/2

�

ghost layercomputational domain ghost layercomputational domain


Alex Breuer

Unsplit Scheme: Stability

1515

2.5 nichtlineare riemann-probleme 17

Abbildung 5: Lösung des Riemann-Problems für lineare Systeme inder x-t-Ebene (�1

< . . . < �

j

< 0 < . . . < �

n). An einemPunkt P(x, t) ergibt sich die Lösung durch seine Lage zuden Charakteristiken der Unstetigkeiten.

wieder aufgreifen werden. Dabei wird der Sprung q

r

- q

l

inEigenvektoren der Matrix A dargestellt. Durch Einsetzen derSummen aus den Gleichungen in (2.27) erhalten wir

q

r

- q

l

=nX

p=1

(wp

r

- w

p

l

)rp ⌘nX

p=1

↵

p

r

p =nX

p=1

r

p

↵

p. (2.31)

Der Sprung q

r

- q

l

wird also in Sprünge über die Unstetig-keiten zerlegt. Mit der Lösung ↵ = (↵1, . . . , ↵n)T des linearenGleichungssystems

R↵ = q

r

- q

l

(2.32)

und W

p ⌘ ↵

p

r

p hat die Zerlegung (2.30) in dieser Darstellungdie Form

q(x, t) = q

l

+X

p:�p

<x/t

W

p

= q

r

-X

p:�p

>x/t

W

p.(2.33)

2.5 nichtlineare riemann-probleme

Wir werden uns in diesem Kapitel mit den grundsätzlichen Ideenfür die exakte Lösung nichtlinearer Riemann-Probleme beschäfti-gen. Die hier präsentierte Theorie hat eher einen motivierendenCharakter (vgl. [37]), für eine ausführlichere Darstellung sei dahernoch einmal auf [30] verwiesen.

Aus Kapitel 2.2 wissen wir bereits, dass q in der sklaren Er-haltungsgleichung (2.7) eindeutig durch die Charakteristikengegeben ist, zumindest solange die Lösung glatt bleibt. Da die


�t <

1

2· min{�x,�y}

�

max


Alex Breuer

SWE: Example

16


Alex Breuer

SWE: Example

17


Alex Breuer

SWE: Hints• Optimize SWE with the following solver (solver=fwavevec):

https://github.com/TUM-I5/swe_solvers/blob/master/src/solver/FWaveVec.hpp

• The update scheme is located at: https://github.com/TUM-I5/SWE/blob/master/src/blocks/SWE_WavePropagationBlock.cpp

• Example implementations (implement your own!): https://github.com/TUM-I5/SWE/tree/master/src/examples

• There’s some documentation: https://github.com/TUM-I5/SWE/wiki

• Get in touch with me for Tsunami data (bathymetry and displacements)

18

https://github.com/TUM-I5/swe_solvers/blob/master/src/solver/FWaveVec.hpp

https://github.com/TUM-I5/SWE/blob/master/src/blocks/SWE_WavePropagationBlock.cpp

https://github.com/TUM-I5/SWE/tree/master/src/examples

https://github.com/TUM-I5/SWE/wiki


Alex Breuer

Scalasca: Hints• Use version 1.4.3 or 2.1+ (expect major

changes between 1.x and 2.x!)

• Use manual Instrumentation;Be aware of huge files!

• Compile your code with “scalasca-instrument -comp=none -user-mode=MPI”

• Use amd-Logins for compilation

• Always be aware of (and measure!) the instrumentation overhead

19

SCALASCA v2.1 Quick Reference

Manual source-code instrumentation• Region or phase annotations manually inserted in source files can augment or substitute automatic instrumentation,

and can improve the structure of analysis reports to make them more readily comprehensible.

• These annotations can be used to mark any sequence or block of statements, such as functions, phases, loop nests,etc., and can be nested, provided that every enter has a matching exit.

• If automatic compiler instrumentation is not used (or not available), it is typically desirable to manually instrumentat least the main function/program and perhaps its major phases (e.g., initialization, core/body, finalization).

SCORE-P user instrumentation APIC/C++:#include <scorep/SCOREP User.h>...void foo() {... // local declarationsSCOREP USER FUNC BEGIN();... // executable statementsif (...) {SCOREP USER FUNC END();return;} else {SCOREP USER REGION DEFINE(r name);SCOREP USER REGION BEGIN(r name, "bar",

SCOREP USER REGION TYPE COMMON);...SCOREP USER REGION END(r name);}... // executable statementsSCOREP USER FUNC END();

}

Fortran:#include <scorep/SCOREP User.inc>...subroutine foo... ! local declarationsSCOREP USER FUNC DEFINE()SCOREP USER REGION DEFINE(r name)SCOREP USER FUNC BEGIN("foo")... ! executable statementsif (...) thenSCOREP USER FUNC END()returnelseSCOREP USER REGION BEGIN(r name, "bar",

SCOREP USER REGION TYPE COMMON)...SCOREP USER REGION END(r name)end ifSCOREP USER FUNC END()

end subroutine foo

• SCOREP USER FUNC BEGIN and SCOREP USER FUNC END are provided explicitly to mark the entry and exit(s) offunctions/subroutines.

• Function names are automatically provided by C/C++, however, in annotated Fortran functions/subroutines anappropriate name should be registered with SCOREP USER FUNC BEGIN("func name").

• Region identifiers (e.g., r name) should be registered with SCOREP USER REGION DEFINE in each annotated pro-logue before use with SCOREP USER REGION BEGIN and SCOREP USER REGION END in the associated body.

• Every exit/break/continue/return/etc. out of each annotated region must have corresponding END() annotation(s).

• Source files annotated in this way need to be compiled with the --user flag given to the SCORE-P instrumenter,otherwise the annotations are ignored. Fortran source files need to be preprocessed (e.g., by FPP or CPP).

POMP user instrumentation API

POMP annotations provide a mechanism for preprocessors (such as OPARI2) to conditionally insert user instrumentation.C/C++:#pragma pomp inst init // once only, in main...#pragma pomp inst begin(name)

...[ #pragma pomp inst altend(name) ]...

#pragma pomp inst end(name)

Fortran:!POMP$ INST INIT ! once only, in main program...!POMP$ INST BEGIN(name)

...[ !POMP$ INST ALTEND(name) ]...

!POMP$ INST END(name)

• Every intermediate exit/break/return/etc. from each annotated region must have an altend or ALTEND annotation.

• Source files annotated in this way need to be processed with the --pomp flag given to the SCORE-P instrumenter,otherwise the annotations are ignored.

Project URL: http://www.scalasca.org 4 Contact/support: [email protected]

Example instrumentation: SCALASCA v2.1 Quick Reference


Alex Breuer

More Links• Git: http://git-scm.com

• Doxygen: http://www.stack.nl/~dimitri/doxygen/

• SCons: http://www.scons.org

• CxxTest: http://cxxtest.com

• ParaView: http://paraview.org, http://www.paraview.org/Wiki/ParaView/Git

20

http://www.git-scm.com

http://www.stack.nl/~dimitri/doxygen/

http://www.scons.org

http://cxxtest.com

http://paraview.org

http://www.paraview.org/Wiki/ParaView/Git

hpc-lab - tum · department of informatics v alex breuer 2 meetings date schedule 10/13/14 kickoff...

Documents