David Rumpf, Statistician GE Aircraft Engines Page 1

Computer Experiments to Predict Propagation of VariationAn Aircraft Engine Blade Assembly Case Study

David Rumpf and William J. Welch GE Aircraft Engines Department of Statistics and Actuarial Science (781) 594-5508 (519) 888-4567 x5545 fax (781) 594-0954 fax (519) 746-1875 1000 Western Ave. University of Waterloo Lynn, MA 01910 Waterloo, Ontario N2L 3G1 USA Canada David.Rumpf@ae.ge.com wjwelch@uwaterloo.ca

Presented at the Fall Technical ConferenceKing of Prussia, PA

October 2002

By

Special thanks to Robert Shankland, GEAE Engineering for his patience and expertise in running the analytic computer stress model.

David Rumpf, Statistician GE Aircraft Engines Page 2

Blade Assembly Stress Study

Goals: 1) A DFSS (design for six sigma) design

which meets LCF life requirements2) Tolerance requirements for X1, X2, X3What was available:• A computer model which evaluates

stress for any specific set of X1, X2 and X3 values. Run time ~ 1 hour

• Two stress points, Y1 and Y2 as shown.

• Two outcomes for each location, mean stress and alternating stress. • Alternating is bigger driver for part

low cycle fatigue life.What was needed:• Non-linear transfer functions for Y1a,b

and Y2a,b versus X1, X2 and X3 which could be used for multiple Monte Carlo models, ~1000 iterations each, for stress versus tolerances on X1, X2 and X3.

X1=interference fit

Y1a,b= notch mean and alternatingstress

Y2a,b = rabbet fillet mean andalternatingstress

X2=CP rabbetinterference fit

X3=CP dropa function of three drops

David Rumpf, Statistician GE Aircraft Engines Page 3

Six Sigma, Producibility and Robust Design

Quality plan relies on inspection. Expect to have rework, scrap and MRB activity.

6

xbar +/- 3

+/- 6 will fit within tol

CombinedEngineering, Manufacturing6 Goal

ImprovedManufacturing Process

“sigma level”

Robust Design allowsWider Tolerance tomeet Customer Need

TypicalHistoricalSituation

Process“sigma level”

2

xbar +/- 3 only +/- 2 fit within tol

Tolerance meets Customer Need

Quality plan focus on parameter control and process monitoring. First time yield 100%.

Reaching six sigma goal requires combined Manufacturing/Design Effort

David Rumpf, Statistician GE Aircraft Engines Page 4

Statistical/Design of Experiment Opportunities

Manufacturing Process Improvement:• Screening designs• Factorial designs• Leveraging• EVOP• Quality Improvement metrics

• Review by Vice-president Engineering, meeting Customer Needs and improving producibility:• Quality Function Deployment• Voice of the Customer • Robust Design:

• Screening• Factorial • Response Surface Designs

• Producibility Scorecards• Review by Vice-president

Focus of this presentation

David Rumpf, Statistician GE Aircraft Engines Page 5

Robust Design

Y = Stress or Useful Life

X’s are parameters which impact Y which could include:• Part Key Characteristic values• Environment• Mating part Key Characteristic values• Customer usage pattern

X’s are typically a combination of controllable and noise (uncontrollable) factors

Y = f(XC , XN)

Goals:• Target Y

• Minimize Y, that is, variation in Y

David Rumpf, Statistician GE Aircraft Engines Page 6

Why Robust Design

Statistical/Engineering method for product/process improvement (Taguchi’s idea)

Two types of factor, control (Xc) and hard to control (Xn or noise)• Control factor levels can change target• Hard to control factors have variation during normal process or usage

Robust design: Set Xc to take advantage of non-linearity in Y = f(Xn)• Design space is typically non-linear

Non-linear Response

0

2

4

6

8

10

12

14

1 2 3 4 5 6 7

Res

pons

e

X n

David Rumpf, Statistician GE Aircraft Engines Page 7

Wu and Hamada recommend a two step process

Experiments: Planning, Analysis and Parameter Design Optimization, CFJ Wu and M Hamada, Wiley 2000

Obtain Transfer Functions: Ybar = f1(XC) Y = f2 (XC)Typically one finds different sets of X’s in the two transfer functions

If Target is goal:• Minimize variation in Y, the harder objective• Minimize Ybar distance from target

If maximum or minimum is the goal:• Optimize Ybar• Minimize variation in Y

An alternative approach is non-linear optimization of Z where

Z = |Target – Ybar|/ Y

David Rumpf, Statistician GE Aircraft Engines Page 8

Statistical Issues with Analytic Models

Designing a new part:• Typically done analytically• Often a complex, time consuming process to obtain a result for a single set of parameter values

• Examples include finite element analysis models, system models, etc• Leads to serious optimization and simulation issues

Recommended approach:• Run designed experiment, typically Response Surface, to capture non-linear effects• Use RSM transfer function for

• Optimization• Simulation to estimate effect of variation in X’s on Y

Statistical Problem:• Analytic models have no random variation, always the same answer for a set of X values• RSM assumes normally distributed error in residuals from model fit• Residuals from analytic model are entirely lack of fit.

David Rumpf, Statistician GE Aircraft Engines Page 9

0-0.01-0.003

-0.0070.0045

-0.0055

30

20

10

30

20

10

30

20

10

RetArm

CP Rab

CP Drop

0

-0.01

-0.003

-0.007

0.0045

-0.0055

Interaction Plot (data means) for notchCenterpoint

Case Study was a Learning Process for the GEAE Author

Initial approach: (Note: All results coded for proprietary reasons)• Full Factorial with center-point, 9 computer runs, ~ 9 hours run time

• Y’s = life required log transformation• Choose to use Y’s = stress• Interactions and curvature were significant, see Y1a graph below

Results led team to an RSM design in 3 factors.

Largest interaction and curvature effects

David Rumpf, Statistician GE Aircraft Engines Page 10

RSM Design

• Face centered central composite design, illustrated for two factors below• We ran 15 runs, no repeated center-points since computer model has no random variation

Factor A

Fact

or B

RSM analysis requires 9+ runs for a 2 factor design, 15+ runs for a 3 factor design.

David Rumpf, Statistician GE Aircraft Engines Page 11

RSM Results

Analysis plan and results:• Chose most parsimonious model via backwards selection based on p values

Response Main Two-way Quadratic R-sq adjusted Y1a 3 1 1 98.2Y1b 3 1 2 95.0Y2a 3 0 1 98.7Y2b 3 1 0 94.3

Y1a

Y1b

Y2a

Y2b

151050

5

4

3

2

1

0

-1

-2

-3

Run-Num

Re

sid

ual

Residual versus Standard Order Run NumberConcerns: 1) High residuals, especially for

Rabbet stress, cause concern.2) R-sq not as high as desired

Questions:1) Does this transfer function fit well

enough for engineering need?2) Is there a better way to fit

analytic/computer model results

David Rumpf, Statistician GE Aircraft Engines Page 12

Enter Professor William Welch and a Space Filling Design

GEAE author looked for help. Jeff Wu suggested Professor W. Welch of the University of Waterloo

New Experimental Plan:• Space filling design instead of DOE or RSM design

• Recommended for computer/analytic experiments• Multiple levels to provide better estimate for non-linear and interaction effects

• 33 runs for 3 factors• Doubled the number of runs• Spaces levels for each factor at 1/32nd of the range

• Plots below show experimental grid, 2 factors at a time • Computer experiment was run for the 33 sets of conditions (~30 hours of run time)

0.0050.000-0.005

0.000

-0.005

-0.010

RetArmC

P D

rop

0.000-0.005-0.010

-0.003

-0.004

-0.005

-0.006

-0.007

CP Drop

CP

Rab

be

t

-0.003-0.004-0.005-0.006-0.007

0.005

0.000

-0.005

CP Rabbet

Re

tArm

David Rumpf, Statistician GE Aircraft Engines Page 13

Approximating Random Variable Function Model

Treat Deterministic output Y(x) as a realization of a random function(stochastic process)

Y(x) = Ybar(x) + Z(x) Sacks et al, Statistical Science, 1989

Intuition: • Model correlation between Z(x) and Z(x’) for any two input vectors x and x’• x close to x’ – correlation large• x far from x’ – correlation small• Leads to a distribution of Y(x) at any x given the Y’s at the design points

Perform diagnostic tests on model• Accuracy of prediction? Standard error of prediction?

David Rumpf, Statistician GE Aircraft Engines Page 14

Accuracy comparison (e.g., Notch-Alt or Y1b)

--------------------------------------------------

Approximating Cross-Validated

Model RMSE

---------------------------------------------------

Polynomial – 2nd degree 0.71

Polynomial – 3rd degree 1.04

Random function 0.48

3rd degree polynomial fits even worse than 2nd degree!

Error = Y – fitted Y

Fitted Y is leave-one-out cross validated (take observation out and predict it)

RMSE = 1/n sum (Y – fitted Y)2

David Rumpf, Statistician GE Aircraft Engines Page 15

Diagnostic Checking of Random-Function Model

Accuracy assessment: Plot Y versus fitted YStandard error (se) assessment: Plot (Y – fitted Y) / se(fitted Y) versus fitted Y

(Fitted Y is leave-one-out cross validation)

Ac t

u al

David Rumpf, Statistician GE Aircraft Engines Page 16

Visualization of Input-Output Relationships

e.g. Y1b as a function of RetArm

Other two inputs (CPRabbet and CPDrop) averaged out

David Rumpf, Statistician GE Aircraft Engines Page 17

Propagating Variation Through the Random Function Model

CPRabbet, CPDrop, RetArm have independent N(mu, sigma) distributions

e.g., set mu = center of range

Sample CPRabbet, CPDrop, RetArm and pass through model to get a distribution of e.g., Y1b values

David Rumpf, Statistician GE Aircraft Engines Page 18

Conclusions

RSM approach:• Good starting point

• Will work fine for Ybar and simple underlying Physics

Space Filling Design:• Allows us to model responses with very nonlinear underlying Physics

Random-function model:• Provides valid standard errors of prediction• Can adapt to nonlinearities in data• Fast, so can quickly propagate variation inputs => outputs via Monte Carlo