high performance computing for neutron diffusion and ... · for neutron diffusion and transport...

High performance computingfor neutron diffusion and transport

equations

Horizon Maths 2012

Fondation Science Mathématiques de Paris

A.-M. Baudron, C. Calvin, J. Dubois,E. Jamelot, J.-J. Lautard, O. Mula-Hernandez

Commissariat à l’Énergie Atomique etaux Énergies Alternatives

Centre de Saclay

CEA/DEN/DANS/DM2S/SERMA/LLPR

December 20, 2012

Physical background MINOS solver MINARET solver Conclusions

Outline

Physical background

MINOS solver

MINARET solver

Conclusions

CEA/DEN/DANS/DM2S/SERMA/LLPR – E. Jamelot – HPC for neutronics – 20/12/12 – 2 / 26


Top view of a nuclear reactor core (PWR 900MW)

After discretization: millions of unknowns. .

⇒ HPC to reduce volume of data and CPU time.



Fission chain reaction

Target nucleus

γ

(x, Ω, E)

NeutronFission product




Target nucleus

γ

(x, Ω, E)

NeutronFission product N : neutron density,

ψ = vN : neutron angular flux,

J = Ωψ: neutron angular current.∂N∂t

= Production rate − Loss rate,

∂N

∂t=




Target nucleus

γ

(x, Ω, E)





Σs: scattering cross section,

Σf : fission cross section,

Sext: external sources,

∂N

∂t=

∫

+∞

0

∫

S2Σs

(

x, (Ω,Ω′), E′→ E

)

ψ(x,Ω′, E

′)dΩ

′dE

′

+χ(E)

4π

(∫

+∞

0

ν(E′) Σf (x, E

′)

∫

S2ψ(x,Ω

′, E

′)dΩ

′dE

′

)

+Sext




Target nucleus

γ

(x, Ω, E)








div J: leakage and streaming,

Σt: total cross section.

∂N

∂t=

∫

+∞

0

∫

S2Σs

(

x, (Ω,Ω′), E′→ E

)

ψ(x,Ω′, E

′)dΩ

′dE

′

+χ(E)

4π

(∫

+∞

0

ν(E′) Σf (x, E

′)

∫

S2ψ(x,Ω

′, E

′)dΩ

′dE

′

)

+Sext

−div J(x,Ω, E)−Σt(x, E)ψ(x,Ω, E).




Target nucleus

γ

(x, Ω, E)








div J: leakage and streaming,

Σt: total cross section.

∫

+∞

0

∫

S2Σs

(

x, (Ω,Ω′), E′→ E

)

ψ(x,Ω′, E

′)dΩ

′dE

′

+χ(E)

4π

(∫

+∞

0

ν(E′) Σf (x, E

′)

∫

S2ψ(x,Ω

′, E

′)dΩ

′dE

′

)

=

div J(x,Ω, E)+Σt(x, E)ψ(x,Ω, E).



Criticality calculation: 6 variables

div J(x,Ω, E)+Σt(x, E)ψ(x,Ω, E)

=

∫ +∞

0

∫

S2

Σs

(

x, (Ω,Ω′), E′ → E)

ψ(x,Ω′, E′)dΩ′dE′

+1

λ

χ(E)

4π

(∫ +∞

0

ν(E′)Σf (x, E′)

∫

S2

ψ(x,Ω′, E′)dΩ′dE′

)

Physical solution: ψ ≥ 0, λ = keff | 1/keff = smaller EV.

Criticality:

keff < 1: production rate < loss rate, subcritical state,keff = 1: production rate = loss rate, critical state,keff > 1: production rate > loss rate, supercritical state.



Description of MINOS [Baudron, Lautard 2007]

• Criticality calculations, source problems, kinetics.• Cartesian or hexagonal mesh.• Energy E: multigroup theory.Eg ∈ [EG, EG−1] ∪ ... ∪ [E1, E0], EG < E0.

• Angular discretization Ω :Simplified spherical harmonics SPN (coupled diffusion equations).Diffusion.

• Space discretization x: Raviart-Thomas-Nédélec finite elements.Accurate computation of the neutron current and flux.

• C++, librairie boost, MPI.



SP1 multigroup equations

• φg(x) =∫ Eg−1

Eg

∫

S2ψ(x,Ω, E)dΩ dE,

• pg(x) =∫ Eg−1

Eg

∫

S2J(x,Ω, E)dΩ dE,

Solve in (pg, φg) |

σgr,1 p

g +grad φg =∑

g′ 6=g

σg′→gs,1 pg′

,

divpg +σgr,0 φ

g =∑

g′ 6=g

σg′→gs,0 φg

′

+1

λχg

G∑

g′=0

νg′σg′

f φg′

.

• σgr,0 and σg

r,1: Removal cross-sections of order 0 and 1,• σg

s,0 and σgs,1: Scattering cross-sections of order 0 and 1,

• νg: Nb of neutrons emitted by fission,

• σgf : Fission cross section, χg : Fission spectrum.



MINOS solver algorithm (criticality)

SPN solverEnergy sweeping

Outer iterations

of keffComputationSource

computation fission sourceUpdate of the

Acceleration of the outer iterations by means of Chebychev polynomials.The outer iterations lead the convergence.

More details in [Baudron, Lautard 2007].



Schwarz iterative method with optimized Robin interface conditions

[Lions (DD3) 1990, Guérin 2007, Nataf and Nier 1994].Let (p0

I , φ0I). ∀n ∈ N, solve in (pn+1

I , φn+1I ) such that :

σr,1 pn+1I +grad φn+1

I = QI in RI ,

divpn+1I +σr,0 φ

n+1I = SI in RI ,

φn+1I = 0 on ∂R∩ ∂RI ,

−pn+11 · n1 +α1φ

n+11 = pn

2 · n2 +α1φn2 on Γ,

−pn+12 · n2 +α2φ

n+12 = p

n(+1)1 · n1 +α2φ

n(+1)1 on Γ.

α1, α2 positive parameters. (pnI , φ

nI ) → (p, φ)|RI

with α1 = α2.

SPN case: more details in [Jamelot, Baudron, Lautard 2012].



Schwarz iterative method with optimized Robin interface conditions

[Lions (DD3) 1990, Guérin 2007, Nataf and Nier 1994].Let (p0

I , φ0I). ∀n ∈ N, solve in (pn+1

I , φn+1I ) such that :

σr,1 pn+1I +grad φn+1

I = QI in RI ,

divpn+1I +σr,0 φ

n+1I = SI in RI ,

φn+1I = 0 on ∂R∩ ∂RI ,

−pn+11 · n1 +α1φ

n+11 = pn

2 · n2 +α1φn2 on Γ,

−pn+12 · n2 +α2φ

n+12 = p

n(+1)1 · n1 +α2φ

n(+1)1 on Γ.

Other DD at EDF: Lagrange multiplier [Lathuilière 09].



MINOS solver algorithm with domain decomposition

Outer iterations

Subdomain j

Subdomain i

Exchange of the interfaceconditions if Γij 6= ∅

fission sourcecomputationSource

MPI calls

computationSource

Outer iterations

Exchange of the scalarproducts on the sources

Computationof keff

Computationof keff

Energy sweepingSPN solver

Energy sweepingSpatial solver

Update of thefission source

Update of the

The outer iterations lead the convergence. One DD iteration only.



Plate-fuel nuclear core.

SP3, 4 energy groups, RTN0,364× 364× 100 unit meshes: 425× 106 unknowns,εf = 5× 10−5.



Power maps

keff = 1.078065



Efficiency and accuracy tests (Titane, CCRT)

SP3, 4 energy groups, RTN0,364× 364× 100 unit meshes: 425× 106 unknowns.εf = 5× 10−5. Eff.= T1/(N × TN ) (%).

N (x, y, z) Nout Err. (10−5) CPU (s) Eff.1 (1, 1, 1) 644 0.0 7 165 100%2 (2, 1, 1) 651 0.0 4 175 86%4 (2, 2, 1) 642 0.0 2 290 78%8 (2, 2, 2) 654 0.0 1 285 70%16 (2, 2, 4) 650 0.0 639 70%32 (4, 4, 2) 657 0.4 319 70%64 (4, 4, 4) 659 0.4 170 66%128 (8, 8, 2) 654 0.2 68 82%

The method seems robust: our optimization choice is validated.



Numerical results with a PWR 900MW

Diffusion approximation, 2 energy groups, cell by cell, RTN0,289× 289× 60 unit meshes: 40× 106 unknowns,εf = 10−5.



Power maps

keff = 1.230157



GPU and DD (Titane, CCRT)

[Jamelot, Dubois, Lautard, Calvin, Baudron M&C 2011].



Kinetics diffusion equations

∀(x, t) ∈ R× [0, T ], ∀g ∈ 1, ..., G, ∀l ∈ 1, ...L :

Dggrad φg(x, t) + pg(x, t) = 0,

1

vg∂φg

∂t(x, t)− divpg(x, t) + σg

t (x)φg(x, t)

=G∑

g′=1

Sgg′

s (x)φg′

(x, t) +

G∑

g′=1

Sg′

f (x)φg′

(x, t) +

L∑

l=1

χgl (x)λlCl(x, t),

∂Cl

∂t(x, t) = −λlCl(x, t) +

G∑

g′=1

Fg′

l φg′

(x, t).

• Cl delayed neutron precursor concentrations.

• λl decay constant of precursor l.



Parareal algorithm (O. Mula-Hernandez)

∂ty +A(t; y) = 0, t ∈ [0, T ), y(0) = y0.

y(Tn) is approched by Y kn . Recurrence relation :

Y 0n+1 = G

Tn+1

Tn,

Y k+1n+1 = G

Tn+1

Tn(Y k+1

n ) + FTn+1

Tn(Y k

n )− GTn+1

Tn(Y k

n ),

0 10 20 30 40 500

2

4

6

8

Number of processors (N)

Spe

ed−

up

D, S−SMS, S−SD, W−SMS, W−S

More details in [Baudron, Lautard, Maday, Mula-Hernandez (DD21) 2012]



Description of MINARET (J.-J. Lautard, [Moller 2011])

• Criticality calculations, source problem and kinetics under progress.• Cylindrical mesh, unstructured in the (x, y) plane.• Energy E: multigroup theory.Eg ∈ [EG, EG−1] ∪ ... ∪ [E1, E0], EG < E0.

• Angular discretization Ω :SN transport (discrete ordinates method).SPN eqs and diffusion.

• Space discretization x : DGFEM.

• C++, librairie boost, MPI.



SN neutron transport equations

Ωd ∈ S2, d ∈ 1, ..., D. Solve in ψgd(x) such that:

Ωd .gradψgd(x)+ σg

t (x)ψgd(x) =

G∑

g′=0

∞∑

l=0

σg′→gs,l (x)

l∑

m=−l

Y ml (Ωd)φ

g′

lm(x)

+1

keff

χg

4π

G∑

g′=0

νg′

σg′

f (x)φg′

(x),

φgl,m(x) =

D∑

d=1

Y ml (Ωd)ψ

gd(x), φg(x) =

D∑

d=1

ωd ψgd(x).

Source iterationsAngular iterations d = 1 à D⇒ Jacobi

Spatial sweeping:

Ωd .gradψgd(x)+ σt(x)ψ

gd(x) = S(x, ψg′

d′ ,Ωd).



SN neutron transport equations

Ωd ∈ S2, d ∈ 1, ..., D. Solve in ψgd(x) such that:

Ωd .gradψgd(x)+ σg

t (x)ψgd(x) =

G∑

g′=0

∞∑

l=0

σg′→gs,l (x)

l∑

m=−l

Y ml (Ωd)φ

g′

lm(x)

+1

keff

χg

4π

G∑

g′=0

νg′

σg′

f (x)φg′

(x),

φgl,m(x) =

D∑

d=1

Y ml (Ωd)ψ

gd(x), φg(x) =

D∑

d=1

ωd ψgd(x).

Source iterationsAngular iterations d = 1 à D⇒ Distributed computing

Spatial sweeping:

Ωd .gradψgd(x)+ σt(x)ψ

gd(x) = S(x, ψg′

d′ ,Ωd).



MINARET solver algorithm (criticality)

SN solverEnergy sweeping

Outer iterations

of keffComputationSource

computation fission sourceUpdate of the

Acceleration of the outer iterations by means of Chebychev polynomials.DSA to accelerate the source iteration [Moller, Lautard, M&C 2011].

Parallelism: distributed computing.



J. Horowitz experimental core reactor

6 energy groups, 2D, 48× 103 cells, DG P1.



Numerical results.

6 energy groups, 2D, 48× 103 cells, DG P1, εf = 10−4.Opteron, 30 nodes, 4 proc. by nodes, 4 cores by proc., 2.6 Ghz.gcc 4.1.1 compiler, MPI Voltaire/Infiny Band 1.2.6 version.

keff = 1.31305S4, D = 12, 10.4M unknownsN Nd CPU (s) Eff.1 12 219 100%2 6 142 77%4 3 96 56%8 2 79 34%12 1 68 27%

keff = 1.31314S8, D = 40, 34.6M unknownsN Nd CPU (s) Eff.1 40 647 100%2 20 371 87%4 10 213 76%8 5 127 64%12 4 116 47%16 3 97 40%20 2 89 35%40 1 85 18%



CCRT Challenge: ESFR core reactor



Conclusions and perspectives

Numerical analysis:

• MINOS: DD for the source problem and kinetics (under work).

• MINOS kinetics: coupling DD and parareal [O. Mula-Hernandez].

• MINOS: Non conforming grids.

• MINARET kinetics: parareal [O. Mula-Hernandez].

• MINARET: unstructured 3D mesh.

• Coupling the solvers.

Computer engineering:

• MINOS: DD on hexagonal meshes.

• Data parallelism.



Thank you!


high performance computing for neutron diffusion and ... · for neutron diffusion and transport...

Documents