large-eddy simulation of the flow over a circular cylinder ... · pdf file7th openfoam®...

7th OpenFOAM® Workshop, Darmstadt, Germany, June 25-28, 2012

Large-eddy simulation of the flow over a circular cylinder at Reynolds number 3900

Dmitry A. Lysenko1

Ivar S. Ertesvåg1

Kjell Eric Rian2

1Norwegian University of Science and Technology, Trondheim, Norway2Computational Industry Technologies AS, Trondheim, Norway


Agenda

• Motivation & background• Test case description• Numerical aspects and HPC• Results for ReD = 3900

• Results for ReD = 20000• Concluding remarks


Motivation & background (I)

• Advancing the numerical modeling of interactions between turbulent flow and chemical kinetics for Large-eddy simulation, with special emphasis on biomass combustion, efficiency and pollutant formation• The Eddy Dissipation Concept (EDC) model of turbulent combustion has been developed by Bjørn F. Magnussen and coworkers at NTNU/SINTEF over nearly four decades• EDC has become the standard model for turbulent combustion almost in all commercial CFD codes • EDC is most widely-used in engineering due to its robustness, relatively easy implementation in CFD codes, tractable with complex chemistry and low computational cost • Although considerable success, there is still potential for further development ☺


• Excessive validation (I) - Turbulent 2D bluff-body test problems• Intensive comparison with ANSYS FLUENT• Good agreement between OpenFOAM, FLUENT, DNS and other experimental/numerical data • Lysenko, D.A., Ertesvåg, I.S. and Rian K.E., Modeling of turbulent separated flows using OpenFOAM. Comput Fluids (2012), doi:10.1016/j.compfluid.2012.01.015

Flow visualization of the vortex shedding behind a triangular rod: a – the experimental interferogram taken from Nakagawa 2005; b – numerical interferogram (OF)

The contours of instantaneous radiated density gradient field for the laminar flow over a circular cylinder at Re =140

Motivation & background (II)


Test case descriptionLES of the flow over a circular cylinder at Reynolds number ReD=3900


NRBC

Laminar

General scheme (a), description of the grid (b) and zoom into the cylinder region (c): θ is the the circumferential coordinate, x and y are the domain extents in stream-wise and transverse directions

Iso-thermal, non-slipping

• O-type grid• Lx = Ly = 50 × D

• Lz = π × D

Test case description (I)

Run Re M Mesh SGS model

1 3900 0.2 300 × 300 × 64 Conventional Smagorinsky

2 3900 0.2 300 × 300 × 64 Dynamic k-equation





Test case description (II)Experimental and DNS references

• PIV by Lourenco & Shih (1993)• PIV by Parnaudeau et al. (2008)• HWA by Ong & Wallace (1996)• HWA by Norberg (1994, 2001)• DNS by Ma et al. (2000)• DNS by Dong et al. (2006) • DNS by Wissink & Rodi (2008) for ReD = 3300 [1206 × 406 × 1024]

• & other LES studies as well


Numerical aspects& High performance computing


Numerical method (I)

• OpenFOAM 1.7/2.1• LES approach

• Dynamic turbulence kinetic energy equation SGS model (TKE hereafter) • Conventional Smagorinsky SGS model with default constants (Cs=0.053)

(SMAG hereafter)• Wall-resolving LES (Y+~1)

• Spatial discretization• rhoPisoFOAM• 2nd order central-differences (CDC2) for viscous terms• 4th order for inviscid terms and pressure gradient

• Temporal discretization• 2nd order implicit backward Euler method (BDF2)• Dynamic adjustable time stepping (CFL<1)

• Linear algebra & accuracy• ICCG (1x10-7)


Numerical method (II)• Thermodynamics

• Ideal gas (sub-grid scales incompressibility hypothesis)• Const for viscosity and other properties

• Boundary conditions• Laminar for inlet / NRBC for outlet• Isothermal non-slip for walls

• Visualization • λ2 (native Foam utility) • Numerical schlieren method [Hadjadj & Kudryavtsev 2005]

• Spectral analysis• MatLab wavelet toolbox• 1D continues wavelet analysis (Morlet wavelet)

[Farge 1992, Addison et al. 2001 … ]• Welch periodogram technique (adaptive-time stepping) [Welch 1967]


HPC (I)

• OpenFOAM is PC software that has been extended to work on supercomputers.• The source code is cluttered with non-MPI versions of libraries• MPI programs normally:

• Scatter input data from process 0 to other processes with MPI• Loop

• Calculate• Synchronize processes with MPI

• Gather partial results on process 0 with MPI• OpenFOAM does:

• decomposePar - creates files with the initial data for each process• rhoPisoFoam - loop, stores partial results in files• reconstructPar - reads the many files and creates end result files


HPC (II)OpenFOAM – scalability study done in PRACE(Thanks to HPC @NTNU team)

«RunTimeModifiable» set to «No»

Number of processes 64 128 256 512 1024

Execution time [s] 686 801 890 1161 2048

Time used for metadatahandling [s]

64 202 274 389 892

The share of time used on Meta datahandling

9.3% 25% 31% 34% 39%


Execution time [s] 542 381 343 411 N/A

Time used for metadatahandling [s]

0.99 2.62 6.03 14.4 N/A

The share of time used on Meta datahandling

0.2% 0.7% 1.8% 3.5% N/A


HPC (III)OpenFOAM – scalability study done in PRACE(Thanks to HPC @NTNU team)

Prohibits scaling Large difficulties for high quality visualization HPC is the next big issue after documentation (!?!)


Number of files created 512 1024 2048 4096 8192

Number of files read 1089 2177 4353 9729 17409

Average file size 597K 317K 163K 84K 47K

Number of stat()-calls 500 000 1000 000 2 000 000 4 400 000 8 500 000

‘RunTimeModifiable: No’

Number of stat()-calls 5000 10 000 20 000 44 000 N/A


HPC (IV)Next generation NTNU HPC facility

Strong scalability study N eff = 8 of 16 cores per node

Strong scalability per one nodeNumber of cores 1 2 4 8 16

Execution time [s] 188 96 83 46 35

Speedup 1 2 2.3 4.1 5.4

Ideal speedup 1 2 4 8 16

VILJE@NTNU SGI Altix 8600 cluster

Number of nodes 1440

Number of cores 23040

CPU type Intel Sandy Bridge

Cores/node 2x8

Memory/node 32 Gb

Network FDR Infiniband


Results for Re @ 3900


Instantaneous flow fieldTime history of the lift and drag coefficients for SMAG and TKE models for

the flow over a circular cylinder at Re = 3900

Sampled statistics (150 vortex shedding cycles)

The Smagorinsky model:

St = 0.19

<Cd> = 1.18

(Cl)rms = 0.44

The dynamic k-equation model:

St = 0.209

<Cd> = 0.97

(Cl)rms = 0.09


Mean flow features

Mean pressure coefficient (a) and mean, normalized vorticity magnitude (b) distributions on the cylinder surface (θsep = 0 is the stagnation point) for the flow over a circular cylinder at Re = 3900

Mean streamlines in the x-y plane by TKE (a) and SMAG (b) runs for the flow over a circular cylinder at Re = 3900

Mean streamlines in the x-y plane from DNS run by Wissink & Rodi (2008) for the flow over a circular cylinder at Re = 3300 (from private communication)

θ=89° (SMAG) θ=88° (TKE)

<Cp,b> = -0.8 (SMAG) <Cp,b> = -0.91 (TKE)

<Lr>/D = 1.588 (DNS) <Lr>/D = 1.67 (TKE) <Lr>/D = 0.97 (SMAG)

a

b


First order statistics

x/D = 1.06

x/D = 1.54

x/D = 2.02

x/D = 4.00

x/D = 7.00 x/D = 10.00

x/D = 1.06

x/D = 1.54

x/D = 2.02

x/D = 4.00

x/D = 7.00 x/D = 10.00

Mean stream-wise velocity and its fluctuations at different locations in the wake of a circular cylinder at Re = 3900

TKE solution agrees fairly well with PIV by Parnaudeau et al. 2008 and DNS by Wissink & Rodi 2008 (U shape)

SMAG solution agrees fairly well with HWA by Lourenco & Shih 1993 (V shape)


Spectral analysis (I)Vortex shedding and shear-layer instability

c d

The Iso-surfaces of λ2 and density gradient fields (numerical schlieren method) obtained by TKE (a,c) and SMAG (b,d) models for the flow over a circular cylinder at Re = 3900


Spectral analysis (II)One-dimensional spectra

x/D = 5; y/D = 0

x/D = 3; y/D = 0


Spectral analysis (III)Wavelet signatures Fundamental Strouhal frequency (fvs) in the wake at x/D=3, y/D = 0

Shear-layer (Kelvin–Helmholtz) frequency (fsl) at the point x/D = 0.69, y/D = 0.69

fsl/fvs = 7.6 (TKE & SMAG)

fsl/fvs = 7.83 (DNS by Dong et al. 2006)

fsl/fvs = 5.99 (Exp by Prasad and Williamson 1996)

LES-TKE LES-SMAG


Parameters and integral flow features


Results for Re @ 20000

Run Re M Mesh SGS model







Flow visualization

Shear-layer instability at Re = 20000λ2 Iso-surface for the flow over a circular cylinder at Re = 20000

Iso-surface of the fluctuating pressure at p’ = −0.1 for the flow over a circular cylinder at Re = 20000


Spectral analysis

Shear-layer frequency (fsl) in the shear layer

Strouhal frequency (fvs) in the wake at x/D=3, y/D = 0

fsl/ /fvs (Re) = C1 × Re0.67

Variation of normalized shear-layer frequency with Reynolds number. Data from Prasad & Williamson 1997

fsl/ /fvs (Re) = C2 × Re0.41

fsl/fvs = 15.08

LES-TKE

Strouhal frequency (fvs) in the wake at x/D=3, y/D = 0


Summary


Concluding remarks (I)Quality & Reliability (LES-IQ)

Index of resolution quality for LES (Celik et al. 2005)• Based on Richardson extrapolation• Sufficient LES resolution ~80% of the TKE (Pope, 2000)• Two sets of grids: 300×300×64 & 440×440×64


Concluding remarks (II)

• Excessive validation effort was carried out for LES of the flow over a circular cylinder at ReD = 3900• Good agreement with recent experimental data and DNS data was achieved as well as fulfilled physics entitlement • The main results were submitted to the Flow, Turbulence & Combustion

(DOI: 10.1007/s10494-012-9405-0)• The next effort will be extended for ReD = 20000 and ReD = 140000


Acknowledgements

• This work was conducted as a part of the CenBio Center for environmentally-friendly energy

• We are very thanked for the uninterrupted HPC computational resources and the useful technical support provided by the Norwegian Meta center for Computational Science (NOTUR).

• Special thanks to Bjørn Lindi & Henrik Nagel from HPC @NTNU team

large-eddy simulation of the flow over a circular cylinder ... · pdf file7th openfoam®...

Documents