source term parameterization for pca combustion modelingk chemical species. 2.2 pca scaling scaling...

Paper # 070RK-0252 Topic: Reaction Kinetics

8th US National Combustion MeetingOrganized by the Western States Section of the Combustion Institute

and hosted by the University of UtahMay 19-22, 2013.

Source Term Parameterization for PCA Combustion Modeling

Isaac, B.1,2

Parente, A.2

Sutherland, J.1

Smith, P.1

Fru, G.3

Thevenin D.3

1Chemical Engineering Department,

University of Utah, Salt Lake City, Utah

2Aero-Thermo-Mechanique,

Universite Libre de Bruxelles, Brussels, Belgium

3Institute of Fluid Dynamics and Thermodynamics,

Otto-von-Guericke University Magdeburg, Magdeburg, Germany

Modeling the physics of turbulent combustion systems remains a challenge due to large range of scales,which are important in these systems. Often, detailed chemical kinetic mechanisms are used to fullydescribe the chemistry involved in the combustion process, yielding highly coupled partial differentialequations for each of the chemical species used in the mechanism. Recently, Principal ComponentsAnalysis (PCA) has shown promise in its ability to identify a low dimensional manifold describing thereacting system [1]. Sutherland and Parente demonstrated the formulation of a PCA model [2] wherethe Principal Components (PCs) of the system are transported. Evaluation of the PCs source-term isexpensive (as all chemical species source-terms must be evaluated) and inherits error through the PCapproximation. Parameterization methods can be employed to quickly and accurately produce source-term values, allowing one to avoid the expensive calculation of the source-terms and avoid the additionalerror from the state space approximation. The present work demonstrates the ability to parameterizethe source-term space of a 2D Direct Numerical Simulation of a spherical premixed syn-gas (CO/H2)air flame using non-linear regression, comparing several non-linear regression methods. In addition theability to parameterize the source-terms while altering the scaling parameters used in PCA is interestingas the scaling greatly effects the behavior and shape of the low-dimensional space being modeled.

1 Introduction

The ability to accurately model a turbulent combustion system remains challenging due to thecomplex nature of combustion systems. A simple fuel such as CH4 has been accurately describedusing 53 species and 325 chemical reactions [3]. More complex fuels require increasingly complexchemical mechanisms. Each resolved chemical species requires a conservation equation which isa coupled, highly non-linear partial differential equation. Such systems are only possible to solveunder very limited situations at this time due to computational costs. This issue leads to the needof a reduced model, which can adequately describe the chemical reactions. Many methods suchas computational singular perturbation (CSP) [4], and Rate Controlled Constrained Equilibrium(RCCE) [5] attempt to reduce the complexity of the mechanism by using equilibrium assumptionsfor fast chemical processes, and using the computational resources on the more pertinent evolution

1

8th US Combustion Meeting – Paper # 070RK-0252 Topic: Reaction Kinetics

in the reaction process. Indeed in these complex combustion reaction mechanisms many of thespecies evolve at time scales much smaller than the time scales of interest, allowing for decouplingof fast and slow processes while maintaining accuracy. Low dimensional manifolds exists in thesesystems which describe well the governing characteristics of the flames. For example, the steadylaminar flamelet model [6] uses the mixture fraction and mixture fraction variance to describe theflame as an ensemble of steady laminar diffusion flames undergoing various strain rates, providinga very good representation of the entire system with low number of variables. Principal Compo-nents Analysis (PCA) has been shown to identify the low dimensional manifolds [7] for turbulentcombustion systems. PCA uses the eigenvalue decomposition of the covariance matrix of the ther-modynamic state space to identify the manifold. In previous work by Sutherland [7], a modelingapproach was presented which uses conservation equations for q PCs (η) which are calculated fromq independent linear combinations of the state space variables. The selection of q depends on thecomplexity of the system of interest, as well as the desired accuracy of the representation of thesystem. The transport equations for η are of the following form:

∂

∂t(ρη) +

∂

∂xi(ρuiη) = − ∂

∂xijη + sη (1)

where jη is the diffusive flux of η and sη is the source-term for η. A major challenge of this mod-eling strategy is in the evaluation of sη. Introduction of a small amount of error to the state spacevariables by using a PCA representation, can dramatically effect the chemical species source-term(ωk) calculation. The error in many cases is exponentially propagated due to the characteristics ofthe reaction rate equations. The present work investigates the ability to accurately model sη usinga high-fidelity data set containing exact or similar physics to the system of interest. Non-linearregression is used to create a model for sη as a function of η, where the training values for sη arecalculated from a training set of X which contain no error from PCA approximation. The benefitin this approach is that approximation error due to PCA on the system is not propagated into themodel for sη. Several well known non-linear regression techniques are investigated for estimationof sη, as well as a novel regression method based on Gaussian kernels filters. In addition the effectsof the scaling used in PCA is assessed in terms of the ability to create regressions models for sη.

2 Approach

2.1 Principal Component Analysis

The PCs (η) are calculated from the PCA analysis. PCA is performed on a data set consisting of n

observations with k variables organized as an n × k matrix (X). The data X is centered to zero byits corresponding means X, and scaled by the diagonal matrix γ containing the scaling value foreach of the k variables:

X = (X− X)γ−1 (2)

PCA identifies a basis matrix A which when multiplied with X creates an approximation to η. Theaccuracy of the approximation of η is dependent on the number of retained columns of A. A isobtained by the eigenvalue decomposition of the covariance matrix of X:

2


1

k − 1X−1X = A−1ΛA (3)

The PCs are then defined by the projection of the basis matrix onto the scaled and centered data

η = XA (4)

The amount of variance represented by basis matrix columns are ordered from highest to lowest.Accordingly a subset of q columns of the original k columns in A, where q � k may yield a goodapproximation of X. Accordingly a subset of η may be used to approximate the entire state spacewith minimal error based on the number of retained eigenvectors from A.

X ≈ ηATkq (5)

The present application of PCA to the turbulent combustion system uses PCA to approximate thek chemical species.

2.2 PCA Scaling

Scaling plays a key role in PCA as well as the non-linear regression that follows. After centeringX the data needs to be scaled so that the PCA will give equal weights to the independent variables(γ from Equation 2). The following scaling methods where adopted for this study [8]:

- auto scaling (STD), uses the standard deviation sk. Auto scaling leaves all columns of Xwith a standard deviation of one, and now the data is analyzed on the basis of correlationsinstead of covariances, γk = sk.

- range scaling (RAN), uses the difference between the minimum and the maximum variablevalue, γk = max

�Xk − Xk

�−min

�Xk − Xk

�.

- pareto scaling (PAR), adopts the square root of the standard deviation as scaling factor,γk =

√sk.

- variable stability scaling (VAST), gives an emphasis to variables which do not show strongvariation, by using the product between the standard deviation and the coefficient of varia-tion, γk = sk

skXk

.

- level scaling (LEV), uses the mean value of the variables γk = Xk.

- max scaling (MAX), uses the maximum variable value as the scaling factor, here γk =max

�Xk − Xk

�.

The scaling of the data set effects the shape of the low dimensional manifold calculated from PCA,which yields a significant impact on the ability of the non-linear regression (see Section 3).

3


2.3 Principal Components Conservation Equation

As is discussed in the work by Sutherland [7], the conservation equations for the PCs are derivedfrom the general species transport equation [9]:

∂

∂t(ρYk) +

∂

∂xi(ρuiYk) =

∂

∂xi

�ρDk

∂Yk

∂xi

�+ ωk (6)

PCA provides akq, the scaling vector γk, and the centering vector Yk. One derives the transportequations for η by first centering and scaling the species mass fractions:

∂

∂t

�ρYk − Yk

γk

�+

∂

∂xi

�ρui

Yk − Yk

γk

�=

∂

∂xi

�Dk

∂

∂xi

�Yk − Yk

γk

��+

ωk

ργk(7)

multiplying by akq and substituting Equation 4 leaves:

∂

∂t(ρη) +

∂

∂xi(ρuiη) =

∂

∂xi

�Dη

∂

∂xi(η)

�+ sη (8)

sη =ωk

ργkakq (9)

Substituting the diffusion term for Ficks law yields Equation 1.

The source-term for this Equation (sη) is a highly non-linear function of all of the state variables.Although the resolution of this source-term should be straight forward as it is a simply a functionof the species mass fractions and the temperature, issues arise due to the approximated state space.The error in the approximation of the state space propagates into sη. A non-linear regression modelcan be used to model this source-term as a function of η. By training the non-linear function onvalues of sη that are free from the PCA approximation errors, the regression will provide accuratevalues for sη even though the state space is approximated.

2.4 Regression Models

In this study Non-Linear Regression models are used to develop a function, f, which estimates thesource-terms as a function of the PCs with an associated estimation error.

sη = f(η) + error (10)

The function is created on a training data set where η is calculated from Equation 4 and sη is cal-culated from Equation 9 with ωk being calculated from the real values of X . The function f isthen tested on a distinct testing sample from X . The current study analyzes five unique non-linearregression models, a simple linear regression model, the general additive model, multivariate adap-tive regression splines, support vector regression, and the response manifold regression which is anew Gaussian-kernel based regression method. A brief mathematical description and explanationof these regression techniques follows.

4


- Linear Regression Model (LIN)

The linear model applied in multiple dimensions is of the form

sη = ηa+ v (11)

Where a is the regression coefficient vector and v is the intercept vector [10].

- General Additive Model (GAM)

The general additive model is a more rigorous concept of the linear model where insteadof fitting a regression coefficient vector, functions are fitted in attempt to more accuratelymodel the dependent variable. The general form of the model is

sη = fηη (12)

where fη are functions dependent on η [11].

- Multivariate Adaptive Regression Splines (MARS)

Multivariate adaptive regression splines use the concept of building up the model from prod-uct spline basis functions. This model creates a number of basis functions, and automaticallydetermines knot location and implements splines at knot boundaries. The model is of theform

sη =M�

m=1

amBm(η) (13)

where Bm are the basis functions and am are the expansion coefficients [12].

- Support Vector Regression (SVR)

Support vector regression is a subset of the support vector machine work. The idea behindSVR is again to create a model which predicts sη given η using learning machines whichimplement the structural risk minimization inductive principle. The basic model form is

sη =N�

i=1

(α∗i − αi)K (η0, ηi) (14)

where α∗i and αi are Lagrange multipliers, and K (η0, ηi) is the kernel operator [13].

- Response Manifold (RM)

The response manifold approach uses the concept of the Nadaraya-Watson kernel estimation[14, 15]:

sη =

N�i=1

K (η0, ηi) sη,i

N�i=1

K (η0, ηi)

(15)

where the kernel function K provides the highest weights to the neighboring data pointsgiving a local estimation, being dependent on the selected filter width. η0 is the current value

5


of the test PCs, ηi are the training PCs, sη,i are the training source-terms and K (η0, ηi) is thekernel operator evaluated at the current η0. The drawback to the Nadaraya-Watson kernelestimation is the expensive evaluation of the kernel function when dealing with large datasets, or domains. The response manifold approach tabulates a manifold for the dependentvariables ’the response’ based on a grid spanning the independent variable space. Duringrun-time a quick interpolation provides the dependent variables from the tabulated responsemanifold. A kernel filter width, and manifold grid point spacing is selected so as to avoidover-fitting and provide accurate estimation of source-terms.

2.5 Data Set

The training data set, used to create the non-linear regression models and for the PC analysis,must contain several important features. First, the data-set should contain realizations both vary-ing temporally and spatially. Second the training data-set must produce the same low dimensionalmanifold in PC space as the combustion problem of interest. It has been demonstrated [18] that thelow dimensional manifolds, may in fact be invariant under certain conditions, allowing a systemto be modeled using the PCA from a similar combustion case. For demonstration of the regres-sion process for use in the PC Transport mode, detailed in Section 2.3, a two-dimensional DirectNumerical Simulation (DNS) data set of a spherical premixed syn-gas air flame has been selected[16]. The detailed reaction scheme [17], contains 13 chemical species (CO, HCO, CH2O, CO2,H2O, O2, O, H , OH , HO2, H2O2, H2, N2), and uses 67 chemical reactions. The DNS is ini-tialized with a unity equivalence ratio, with the fuel consisting of 0.5 CO and 0.5 H2, air as anoxidizer, and a rms turbulent velocity fluctuation (u�) of 10m/s. The DNS assumes a unity Lewisnumber, and has grid consisting of 800 by 800 points spanning 0.02 by 0.02 meters. The data setconsists of 30 time-steps.

3 Results and Discussion

Results using the various regression models outlined in Section 2.4 were computed with the sta-tistical computing package ’R’, and Matlab. The R code implementations for LIN, GAM, MARS,and SVM were used. The authors RM model was implemented in Matlab. Sample training andtesting data sets were taken randomly from the data set, where each sample consisted of 10,000observations. The coefficient of determination (R2) and the normalized root mean square error(NRMSE) were used as a means of quantifying the error produced by the models:

R2 =

N�i=1

(xpredicted,i − x)2

N�i=1

(xi − x)2(16)

NRMSE =

�N�i=1

(xpredicted,i − xi)2

max(σ(xpredicted, x))(17)

6


0 5 10 150

0.2

0.4

0.6

0.8

1

!

R2

STD

RANGE

PARETO

VAST

LEVEL

MAX

Figure 1: Mean R2 from the reconstruction of the chemical species mass fractions, as a function ofthe number of retained η, using STD scaling.

First the ability of PCA to represent the chemical species mass fractions is shown in Figures 1 and2. Figure 1 shows R2 is greater than 0.95 when η ≥ 4 (for STD scaling), and Figure 2 showsa NRMSE value less than 0.185. As mentioned in Section 2.3 the propagation of error from theapproximation of the state space is much more severe in the calculation of sη. Figures 3 and 4show the error in the calculation of sη using Equation 9 directly. It is observed that when η ≥ 9the coefficient of determination is great than 0.95 and a significant reduction in mean NRMSE isobserved.

In order to avoid the calculation of the chemical species source-terms and to prohibit the largedegree of error in the calculation of sη which is seen when using fewer η, the regression methodsare now tested. Table 1 shows the resultant error for the estimation of sη when four PCs areretained, regressing the value of sη on only three PCs.

Table 1: R2 for estimation of sη using STD scaling with various Regression MethodsRegression Method sη,1 sη,2 sη,3 sη,4

LIN 0.896 0.907 0.826 0.837GAM 0.985 0.982 0.980 0.980MARS 0.984 0.989 0.982 0.980SVM 0.989 0.995 0.989 0.990RM 0.993 0.994 0.991 0.992

A dramatic improvement in the estimation of sη is observed. This dramatic improvement is con-sistent with the work by Biglari [18]. In particular the SVM, and RM regression methods showremarkable accuracy in estimating the source-terms. Another important factor in PCA is the scal-ing factor γ used in Equation 2. Figures 5 and 6 show the PCA manifold calculated using STDscaling and RANGE scaling respectively. The color in the Figures represent the value of the firstPC source term sη,1. Here one observes the difference in scales, shapes, and gradients the different

7


0 5 10 150

0.2

0.4

0.6

0.8

1

!

NR

MS

E

STD

RANGE

PARETO

VAST

LEVEL

MAX

Figure 2: Mean NRMSE from the reconstruction of the chemical species mass fractions, as a func-tion of the number of retained η, using STD scaling.

0 5 10 150

0.2

0.4

0.6

0.8

1

!

R2

STD

RANGE

PARETO

VAST

LEVEL

MAX

Figure 3: Mean R2 from the reconstruction of the source-terms for the species mass fractions, as afunction of the number of retained η, using STD scaling.

8


0 5 10 150

0.5

1

1.5

2

!

NR

MS

E

STD

RANGE

PARETO

VAST

LEVEL

MAX

Figure 4: Mean NRMSE from the reconstruction of sη, as a function of the number of retained η,using STD scaling.

!6!4

!20

!20

!10

0

!20

!10

0

!1

!2

!3

0

0.5

1

1.5

2x 10

6

Figure 5: PCA manifold created using STD scaling, here the independent variables are η1, η2, andη3 representing the cartesian coordinates, and the dependent variable sη mapped in color to themanifold.

scaling methods produce. Tables 2 and 3 show R2 and NRMSE error metrics using the variousscaling methods presented in Section 2.2 while using the RM regression method.

Table 2: R2 for the estimation of sη using the RM regression method with various scalingmethods

Scaling Method sη,1 sη,2 sη,3 sη,4STD 0.993 0.994 0.991 0.992

RANGE 0.970 0.975 0.902 0.870PARETO 0.970 0.749 0.854 0.904

VAST 0.893 0.915 0.935 0.667LEVEL 0.988 0.984 0.984 0.979

MAX 0.976 0.981 0.873 0.976

9


!1.5!1

!0.50

!1

!0.5

0

!0.5

0

0.5

!1

!2

!3

!1

0

1

2

x 104

Figure 6: PCA manifold created using RANGE scaling, here the independent variables are η1, η2,and η3 representing the cartesian coordinates, and the dependent variable sη mapped in color to themanifold.

Table 3: NRMSE for the estimation of sη using the RM regression method with various scal-ing methods

Scaling Method sη,1 sη,2 sη,3 sη,4STD 0.082 0.075 0.092 0.088

RANGE 0.170 0.157 0.298 0.341PARETO 0.171 0.157 0.298 0.341

VAST 0.306 0.272 0.240 0.487LEVEL 0.108 0.127 0.125 0.145

MAX 0.152 0.136 0.337 0.153

A clear benefit is seen while using the RM regression method with STD or LEVEL scaling. This isdue to that the fact that these manifolds contain smooth gradients, and a more simplified manifoldshape allowing for a more accurate regression.

4 Conclusion

The current work has addressed the ability to use non-linear regression methods to estimate source-terms for a PCA based combustion model. Various non-linear regression methods have been ana-lyzed showing the ability to produce accurate estimation even when using a lower number of η. Inparticular the SVM and RM methods have shown desired accuracy in estimation of sη. In additionthe effect of various PCA scaling methods on the non-linear regression models have been assessed,with excellent results using STD and LEVEL scaling. The current work outlines an example ofan apriori analysis which provides the best regression and scaling method for a given turbulentcombustion data set. Additional work may include an analysis on the invariance of the PC basedmanifold with respect to flow conditions, specifically by increasing the turbulence intensity, and ademonstration of the non-linear regression methods in conjunction with a simple perfectly stirredreactor system.

10


Acknowledgments

We are grateful to our sponsor for which part of the present research was funded: The NationalNuclear Security Administration under the Accelerating Development of Retrofittable CO2 Cap-ture Technologies through Predictivity program through DOE Cooperative Agreement DE NA 0000 740.

References

[1] Alessandro Parente, James C. Sutherland, L. Tognotti, and Philip J. Smith. Proceedings of the Combustion

Institute, (2009) journal.

[2] J. C. Sutherland and A. Parente. Proceedings of the Combustion Institute, 32 (2009) 1563–1570.

[3] G. P. Smith, D. M. Golden, M. Frenklach, N. W. Moriarty, B. Eiteneer, M. Goldenberg, C. T. Bowman, R. K.Hanson, S. Song, W. C. Gardiner, V. V. Lissianski, and Z. Qin journal.

[4] R. Fox. Computational Models for Turbulent Reacting Flows. Cambridge University Press, 2003.

[5] W.P. Jones and Rigopoulos Stelios. Combustion and Flame, 142 (2005) 223–234.

[6] N. Peters. Progress in Energy and Combustion Science, 10 (1984) 319–339.

[7] James C. Sutherland and Alessandro Parente. Proceedings of the Combustion Institute, 32 (2009) 1563–1570.

[8] I. T. Jolliffe. Principal Component Analysis. Springer, New York, NY, 1986.

[9] T. Poinsot and D. Veynante. Theoretical and Numerical Combustion. R.T. Edwards, Inc., 2001.

[10] William S Cleveland, Eric Grosse, and William M Shyu. Statistical models in S, (1992) 309–376.

[11] Simon Wood. Generalized additive models: an introduction with R, Vol. 66. Chapman & Hall/CRC, 2006.

[12] Jerome H Friedman. The annals of statistics, (1991) 1–67.

[13] Alex J Smola and Bernhard Schölkopf. Statistics and computing, 14 (2004) 199–222.

[14] Elizbar A Nadaraya. Theory of Probability & Its Applications, 9 (1964) 141–142.

[15] Geoffrey S Watson. Sankhya: The Indian Journal of Statistics, Series A, (1964) 359–372.

[16] Gordon Fru, Gábor Janiga, and Dominique Thévenin. Flow, turbulence and combustion, 88 (2012) 451–478.

[17] Ulrich Maas and Stephen B Pope. Combustion and Flame, 88 (1992) 239–264.

[18] Amir Biglari and James C Sutherland. Combustion and Flame, (2012) journal.

11

source term parameterization for pca combustion modelingk chemical species. 2.2 pca scaling scaling...

Documents