an automated method for sensitivity analysis using complex...

AN AUTOMATED METHODFOR SENSITIVITY ANALYSISUSING COMPLEX VARIABLES

Joaquim R. R. A. MartinsIlan M. Kroo

Juan J. Alonso

Department of Aeronautics and AstronauticsStanford University

January 12, 2000

– Joaquim Martins – January 12, 2000 –

Outline

• Background on complex-step and other sensitivityanalysis methods

• Theory

– First derivative approximations– Higher derivative approximations

• Implementation of the complex-step in algorithms

– Fortran functions and operators– Automatic implementation

• Results

– Structural sensitivities– Aerodynamic sensitivities

• Conclusions

– Joaquim Martins – January 12, 2000 – 1

Sensitivity Analysis Methods

• Finite-Difference Methods

• ADIFOR

• Analytic Methods

• Complex-Step:

– Lyness & Moler, 1967:nth derivative approximation by integration in thecomplex plane

– Squire & Trapp, 1998:Simple formula for first derivative,f ′ ≈ Im[f(x + ih)]/h

– Newman, Anderson & Whitfield, 1998:Aero-structural code

– Anderson, Whitfield & Nielsen, 1999:3D Navier-Stokes with turbulence model


Finite-Difference DerivativeApproximations

From Taylor series expansion.

• Forward-difference:

f ′(x) ≈ f(x + h)− f(x)h

+ O(h)

• Central-difference:

f ′(x) ≈ f(x + h)− f(x− h)2h

+ O(h2)

Any finite-difference formula subject to subtractivecancellation.


Cauchy-Riemann Equations

• Extension of rational numbers to solve x2 = 2:

Define w = a +√

2b where a, b are rational.

Derivative of f(w):

∂f

∂w=

∂f

∂a=√

2∂f

∂b.

• Extension of real numbers to solve x2 = −1:

Define z = x + iy, where x, y are real.

Derivative of f(z):

∂f

∂z=

∂f

∂y= i

∂f

∂x

. Define f(z) = u(z) + iv(z),

∂u

∂x=

∂v

∂y,

∂u

∂y= −∂v

∂x.

Complex number really just one number.


Complex-Step Derivative Approximation

• From Cauchy-Riemann:

∂u

∂x= lim

h→0

v(x + i(y + h))− v(x + iy)h

.

For a real functions of a real variable, y = 0,u(x) = f(x) and v(x) = 0, then,

∂f

∂x= lim

h→0

Im [f (x + ih)]h

.

• From Taylor series expansion, step ih:

f(x+ih) = f(x)+ihf ′(x)−h2f′′(x)2!

−ih3f′′′(x)3!

+. . .

⇒ f ′(x) =Im [f(x + ih)]

h+ h2f

′′′(x)3!

+ . . .

No subtraction!


Simple Numerical Example

• Estimate derivative at x = 1.5 of the function,

f(x) =ex

√sin3x + cos3x

��

�� e

"!$#&%('*),+.-0/213)$%465$798;:$7=<(>@?BADCEC=F$7GF�H,IJFKBL$MONQP=R$SUT@VBWDXEX=L$P=L$M,YJL

ε =∣∣f ′ − f ′ref

∣∣ /∣∣f ′ref

∣∣


Higher Derivative Approximations

• General form of Cauchy’s Integral,

f (n)(z) =n!2πi

∫

Γ

f(ξ)(ξ − z)n+1

dξ.

• Discretize using a trapezoidal-rule, integrate aroundcircle z + rei,

f (n)(z) ≈ n!mr

m−1∑

j=0

f(z + rei2πj

m

)

ei2πjnm

.

• Fixed r, use more points for better approximation.Bound on error.

• With finite-difference, have to decrease h for betteraccuracy.

• First derivative formula can be derived from this......but it is the only one that does not involvesubtraction.


Complex Functions and Operators inFortran

• Relational operators

– Used with if statements to direct the executionthread.

– Complex algorithm must follow same thread.– Therefore, compare only the real parts.– Also, max, min, etc.

• Arithmetic functions and operators:

– Most of these have a mathematical standarddefinition that is analytic.

– Some of them are implemented in Fortran.– Exception: abs

∂u

∂x=

∂v

∂y=

{−1 ⇐ x < 0+1 ⇐ x > 0

abs(x + iy) =

{−x− iy ⇐ x < 0+x + iy ⇐ x ≥ 0

.


Overloaded Functions

abs ✘ abs(z) =

{−z ⇐ x < 0+z ⇐ x ≥ 0

tan ✔ tan(z) = e−iz−eiz

e−iz+eiz

asin ✔ arcsin(z) = −i log[iz + (1− z2)12]

acos ✔ arccos(z) = −i log[z + (z2 − 1)12]

atan ✔ arctan(z) = i2 log

(1−iz1+iz

)

sinh ✔ sinh(z) = ez−e−z

2

cosh ✔ cosh(z) = ez+e−z

2

tanh ✔ tanh(z) = ez−e−z

ez+e−z

dim ✘ dim(z1, z2) =

{z1 − z2 ⇐ x1 > x2

0 ⇐ x1 ≤ x2

sign ✘ sign(z1, z2) =

{+|x1| ⇐ x2 ≥ 0−|x1| ⇐ x2 < 0

max ✘ max(z1, z2) =

{z1 ⇐ x1 ≥ x2

z2 ⇐ x1 < x2

min ✘ min(z1, z2) =

{z1 ⇐ x1 ≤ x2

z2 ⇐ x1 > x2


Automatic Implementation

• Cookbook procedure:

– Substitute all real type variable declarations withcomplex declarations.

– Define all functions and operators that are notdefined for complex arguments.

– A complex-step can then be added to the desiredvariable and the derivative can be estimated byf ′ ≈ Im[f(x + ih)]/h.

• Fortran 77: write new subroutines, substitute someof the intrinsic function calls by the subroutinenames, e.g. abs by c abs. But ... need to knowvariable types in original code.

• Fortran 90: can overload intrinsic functionsand operators, including comparison operators.Compiler knows variable types and chooses correctversion of the function or operator.

• Complexify.py: Python script processes code.complexify.f90: overloaded definitions.


Structural Sensitivities:Finite Element Solver

• FESMEH, finite element solver used in MDO.

• Triangular plates and truss elements.

• Wing modeled with spars, ribs and skin.

• Spars and ribs modeled with plates and trusses.


Sensitivity Estimates vs. Step Size

• Sensitivity of truss stress w.r.t. truss area.

��

�� e

"!$#&%('*),+.-0/$12)$%3$465(487:9(;=<"4?>@>:9$A:9$5,BC9

• Finite-difference: reasonable estimate, if you’relucky...

• Complex-step: practically insensitive to step size.


Computational Accuracy and Cost

Method Sample Sensitivity

Complex –39.049760045804646ADIFOR –39.049760045809059Analytic –39.049760045805281FD –39.049724352820375

• Finite-difference is worst.

• Other methods achieve the solver’s precision.

Method Time Memory

Complex 1.00 1.00ADIFOR 2.33 8.09Analytic 0.58 2.42FD 0.88 0.72

• Analytic: best, but not easy to implement.

• ADIFOR: costly.

• Complex-step: good compromise.

• Caveat: ratios depend on problem.


Aerodynamic Sensitivities: FLO82

RAE 2822 Airfoilα = 3.0o, M∞ = 0.70

1.

20

0.80

0.

40

0.00

-0.

40 -

0.80

-1.

20 -

1.60

-2.

00

Cp

+++++++++++++++++++++++++

++

+++++++++++++++++++++++++++++++++++++++++++

+

+

+

+

+

+

+

+

+

+

+

++++++++++++++++++++++++++++++++++++++++++++++++++

+

++

+++++++++++++++++++++++++++

• 2D, cell-centered, finite volume Euler solver.

• Multigrid.

• Implicit residual smoothing.

• Artificial dissipation options: JST, CUSP, ECUSPand HCUSP.

• Same numerical schemes as FLO87 and FLO107.


Sensitivity Estimates vs. Step Size

∂CD/∂M∞

10−30

10−20

10−10

10−8

10−6

10−4

10−2

100

Step Size, h

Nor

mal

ized

Err

or, ε

Complex−Step Finite−difference


Convergence of Sensitivity Estimates

0 50 100 15010

−10

10−5

100

105

1010

Number of Iterations

Res

idua

l

Complex−Step Finite−difference Average density residual

• Both estimates converge at same rate as solver.


Aerodynamic Sensitivity Results

Sensitivity Complex FD

∂CL/∂α 12.37054751691092 12.370726665267277∂CD/∂α 0.8602380269441042 0.86024234610682058∂CM/∂α –0.5026301652982372 –0.5026313670830616∂CL/∂M∞ 3.2499722985150532 3.2499447577549727∂CD/∂M∞ 0.43978671505102005 0.43978998888818932∂CM/∂M∞ –0.99037388394690016 –0.9903747405504145

• Imaginary part of result converged to the same number of digits as thereal part.


Drag Coefficient Sensitivity toHicks-Henne “Bump” Functions

10 20 30 40 500

1

2

3

4

5

6

7

8

9

10

11

Design Points

Dra

g C

oeffi

cien

t Sen

sitiv

ity

Complex−Step Finite−difference

Method Time Memory

Complex 1.00 1.00FD 0.31 0.55


Conclusions

• The theory behind the application of the complex-step method to real-world numerical algorithms wasintroduced.

• The implementation of the complex-step methodwas successfully automated.

• The sensitivity results given by this method havebeen further validated and shown to be veryaccurate.

• Unlike finite-differencing, the complex-step methodis relatively insensitive to the step size.

• The complex-step method is now an attractivealternative to ADIFOR.


Further Work

• Continue to work on Complexify.py.Need feedback from users.

• Apply the method to other solvers.

• More detailed stability and convergence analysis.

• Compare with CFD adjoint results.


an automated method for sensitivity analysis using complex...

Documents