intro to response surface methods - cdn.statease.com · approximation of the true response surface...

1

Intro to Response Surface MethodsPart 1 – Central Composite Designs

By Shari Kraber, MS, Applied Stats.Stat-Ease, Inc., Minneapolis, MN

[email protected]

*Presentation is posted at www.statease.com/webinar.html

Please use the raise hand feature on GotoWebinar, which I will watch for during my presentation. To avoid disrupting the Voice over Internet Protocol (VoIP) system, I will mute all. If I do not get to you, please accept my apology in advance. Then I’d appreciate you sending me an email after the talk so we can discuss your issue(s) ‘off-line.’ -- Shari

mailto:[email protected]

http://www.statease.com/webinar.html

http://www.statease.com/sharkrab.html

Introduction to Response Surface Methods

1. Response Surface Methodology

Response surface designs

Central composite designs

Whey protein case study

2. Multiple Response Optimization


2

Agenda Transition

Response Surface Methodology:




(design and analysis) yes

Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

yes

Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

3

4

Subject Matter

Knowledge

Factors

Process

Responses

Empirical Models

(polynomials)

ANOVA

Contour Plots

Optimization

Design of Experiments

• Region of Operability

• Region of Interest

Response Surface Methodology

5

Region of Interestversus Region of Operability

Region of Operability

Region of InterestUse factorial design to

get close to the peak.

Then RSM to climb it.

Polynomial Approximations

A decent approximation of any mathematical function can be made via an

infinite series of powers of X, such as that proposed by Taylor. For RSM,

this takes the form:

1. The higher the degree of the polynomial, the more closely the Taylor

series can approximate the truth.

2. The smaller the region of interest, the better the approximation. It

often suffices to go only to quadratic level (x to the power of 2).

3. If you need higher than quadratic, think about:

A transformation

Restricting the region of interest

Looking for an outlier(s)

Using a higher order polynomial6

2 2

0 1 1 2 2 12 1 2 11 1 22 2

2 2 3 3

112 1 2 122 1 2 111 1 222 2

y x x x x x x

x x x x x x ...

7

Simple Maximum (or Minimum)

2 2y 83.57 9.39A 7.12B 7.44A 3.71B 5.80AB

-4.00

-2.00

0.00

2.00

4.00

-4.00

-2.00

0.00

2.00

4.00

65

75

85

95

M

ax

imu

m

A B -4.00 -2.00 0.00 2.00 4.00

-4.00

-2.00

0.00

2.00

4.00Maximum

AB

65

70

75

80

85

8

Rising Ridge

2 2y 77.57 8.80A 8.19B 6.95A 2.07B 7.59AB

-4.00 -2.00 0.00 2.00 4.00

-4.00

-2.00

0.00

2.00

4.00Rising Ridge

AB 65

65

70

75

80

85

90

-4.00

-2.00

0.00

2.00

4.00

-4.00

-2.00

0.00

2.00

4.00

65

75

85

95

R

isin

g R

idg

e

A B

9

Stationary Ridge

2 2y 83.93 10.23A 5.59B 6.95A 2.07B 7.59AB

-4.00 -2.00 0.00 2.00 4.00

-4.00

-2.00

0.00

2.00

4.00Stationary Ridge

AB

65

65

70

7075 7580

8085 85

-4.00

-2.00

0.00

2.00

4.00

-4.00

-2.00

0.00

2.00

4.00

65

75

85

95

S

ta

tio

na

ry

R

idg

e

A B

10

Saddle, or MiniMax

2 2y 84.29 11.06A 4.05B 6.46A 0.43B 9.38AB

-4.00 -2.00 0.00 2.00 4.00

-4.00

-2.00

0.00

2.00

4.00Saddle

AB

65

65

75 75

85

85

95

95

105

105

115

115

125

135

145

-4.00

-2.00

0.00

2.00

4.00

-4.00

-2.00

0.00

2.00

4.00

65

80

95

110

125

140

155

S

ad

dle

A B

11

Requires a quantitative response affected by continuous

factors.

Works best with only a handful of critical factors, those that

survive the screening phases of the experimental program.

Produces an empirical polynomial model which gives an

approximation of the true response surface over a factor

region.

Seeks the optimal settings for process factors so you can

maximize, minimize, or stabilize the responses of interest.

By overlaying contour maps from multiple responses, RSM

can be used to find the ideal "window" of operability.

Response Surface MethodologyConsiderations

Response Surface MethodologyTypes of Designs

12

• Central Composite Design

• Classic 5-level design

• Great statistical properties

• Box Behnken Design

• 3-level design

• Also great statistical properties

• Optimal (Custom) Design

• Customizable for nearly any situation

• Categoric factors, constrained design space

Agenda Transition






Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

yes

Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

13

14

Response Surface MethodologyCentral Composite Design

Central Composite DesignElements

Two-level full/fractional factorial (Res V or higher).

Estimate first-order and two factor interactions.

Center points

Estimate pure error and tie blocks together.

Star (or axial) points

Estimate pure quadratic effects.

15

CCDs are good designs for fitting

second order (quadratic) polynomials

Central Composite DesignTemplate for 3 Factors

16

A B C

Factorial −1 −1 −1

points: 1 −1 −1

−1 1 −1

1 1 −1

−1 −1 1

1 −1 1

−1 1 1

1 1 1

Axial (star) − 0 0

points: 0 0

0 − 0

0 0

0 0 −

0 0

Center 0 0 0

points: 0 0 0

0 0 0

0 0 0

0 0 0

0 0 0

Structuring a CCDRegion of Interest

Stay within the

box* when you

use your model

for making

predictions!

*region of

interest

Keep axial

(star) runs

within the

circle.

This is the

region of

operability.

17

Agenda Transition






Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

yes

Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

18

Whey Protein ConcentratesCase Study (design and analysis)

Richert et. al.* (1974) used a central composite design to study the effects of five

factors on whey protein concentrates. The factors, with ranges noted in terms of

alpha (star levels), are:

A. Heating temperature, °C/30 min. 65 85

B. pH level 4 8

C. Redox potential, volts -0.025 0.375

D. Sodium oxalate, molar 0 0.05

E. Sodium lauryl sulfate, % of solids 0 0.2

The experimenters chose a CCD based on a one-half fraction for the cube portion

(25-1). This rotatable design (with α = ±2) has six center points.

19

1. The experimenters chose a CCD based on a ½ fraction for the

cube portion (25-1): choose the “½ Fraction”.

(Be sure to choose the half fraction before clicking on “Enter

factor ranges in terms of alpha”.)

2. Then choose “Enter factor ranges in terms of alpha”.

Rsm section 3 20

Whey Protein ConcentratesInstructions (1 of 4)

2

1

Rsm section 3 21


3. The experimenters used a rotatable design (α = ±2) and six

center points.

Rsm section 3 22


4. Enter the factor ranges as the alpha values:

5. Enter the one response we will investigate, undenatured

protein, in abbreviated form such as “Unde Pro”. The units of

measure are percent (%).

Whey Protein Case StudyData – Factorial portion of CCD

Factor Factor Factor Factor Factor Response

Std Run A:Heat B:pH C:Redox D:Na ox E:Na lau Unde Pro

C / 30 min volt Molar % of soli %

1 9 70.0 5.0 0.075 0.0125 0.15 80.6

2 25 80.0 5.0 0.075 0.0125 0.05 67.9

3 3 70.0 7.0 0.075 0.0125 0.05 83.1

4 19 80.0 7.0 0.075 0.0125 0.15 38.1

5 4 70.0 5.0 0.275 0.0125 0.05 79.7

6 29 80.0 5.0 0.275 0.0125 0.15 74.7

7 22 70.0 7.0 0.275 0.0125 0.15 71.2

8 18 80.0 7.0 0.275 0.0125 0.05 36.8

9 11 70.0 5.0 0.075 0.0375 0.05 81.7

10 31 80.0 5.0 0.075 0.0375 0.15 66.8

11 2 70.0 7.0 0.075 0.0375 0.15 73.0

12 23 80.0 7.0 0.075 0.0375 0.05 40.5

13 13 70.0 5.0 0.275 0.0375 0.15 74.9

14 30 80.0 5.0 0.275 0.0375 0.05 74.2

15 7 70.0 7.0 0.275 0.0375 0.05 63.5

16 12 80.0 7.0 0.275 0.0375 0.15 42.8

23

Whey Protein Case StudyData – Star and center points

Factor Factor Factor Factor Factor Response

Std Run A:Heat B:pH C:Redox D:Na ox E:Na lau Unde Pro

C / 30 min volt Molar % of soli %

17 8 65.0 6.0 0.175 0.0250 0.10 80.9

18 27 85.0 6.0 0.175 0.0250 0.10 42.4

19 16 75.0 4.0 0.175 0.0250 0.10 73.4

20 24 75.0 8.0 0.175 0.0250 0.10 45.0

21 10 75.0 6.0 -0.025 0.0250 0.10 66.0

22 17 75.0 6.0 0.375 0.0250 0.10 71.7

23 15 75.0 6.0 0.175 0.0000 0.10 77.5

24 28 75.0 6.0 0.175 0.0500 0.10 76.3

25 32 75.0 6.0 0.175 0.0250 0.00 67.4

26 21 75.0 6.0 0.175 0.0250 0.20 86.5

27 20 75.0 6.0 0.175 0.0250 0.10 77.4

28 5 75.0 6.0 0.175 0.0250 0.10 74.6

29 6 75.0 6.0 0.175 0.0250 0.10 79.8

30 26 75.0 6.0 0.175 0.0250 0.10 78.3

31 1 75.0 6.0 0.175 0.0250 0.10 74.8

32 14 75.0 6.0 0.175 0.0250 0.10 80.9

24

Case Study

Whey Protein Concentrates

There were nine responses, let’s look at three key ones:

• Y1 – Undenatured protein, %.

• Y2 – Whipping time, min.

• Y3 – Time at first drop, min.

25

Whey Protein Case StudySequential Model Sum of Squares

26

Sequential Model Sum of Squares

Sum of Mean F

Source Squares DF Square Value Prob > F

Mean 1.516E+005 1 1.516E+005

Linear 4323.77 5 864.75 9.77 < 0.0001

2FI 883.30 10 88.33 1.00 0.4848

Quadratic 1179.84 5 235.97 10.88 0.0006 Suggested

Cubic 202.04 5 40.41 6.64 0.0196 Aliased

Residual 36.51 6 6.09

Total 1.582E+005 32 4943.93

"Sequential Model Sum of Squares": Select the highest order polynomial where the

additional terms are significant.

Whey Protein Case StudyLack of Fit Tests

Do you want significant lack of fit?

27

Lack of Fit Tests

Sum of Mean F


Linear 2268.60 21 108.03 16.32 0.0029

2FI 1385.30 11 125.94 19.03 0.0022

Quadratic 205.46 6 34.24 5.17 0.0459 Suggested

Cubic 3.42 1 3.42 0.52 0.5044 Aliased

Pure Error 33.09 5 6.62

"Lack of Fit Tests": Want the selected model to have insignificant lack-of-fit.

Whey Protein Case StudyModel Summary Statistics

What’s wrong with these statistics?

28

Model Summary Statistics

Std. Adjusted Predicted

Source Dev. R-Squared R-Squared R-Squared PRESS

Linear 9.41 0.6526 0.5858 0.4673 3529.61

2FI 9.42 0.7859 0.5852 -1.1703 14379.16

Quadratic 4.66 0.9640 0.8985 0.1632 5544.20 Suggested

Cubic 2.47 0.9945 0.9715 0.4325 3759.87 Aliased

"Model Summary Statistics": Focus on the model minimizing the "PRESS", or

equivalently maximizing the "PRED R-SQR".

Whey Protein Case StudySignificance (?) of Quadratic Terms

Let’s try reducing this model to only significant terms.

29

Source SS DF MS F Prob > F

A 2458.35 1 2458.35 113.36 < 0.0001

B 1807.87 1 1807.87 83.36 < 0.0001

C 0.26 1 0.26 0.012 0.9147

D 12.18 1 12.18 0.56 0.4693

E 45.10 1 45.10 2.08 0.1771

A2 506.85 1 506.85 23.37 0.0005

B2 667.23 1 667.23 30.77 0.0002

C2 162.93 1 162.93 7.51 0.0192

D2 3.48 1 3.48 0.16 0.6965

E2 3.23 1 3.23 0.15 0.7069

AB 616.28 1 616.28 28.42 0.0002

AC 122.66 1 122.66 5.66 0.0366

AD 50.06 1 50.06 2.31 0.1569

AE 7.98 1 7.98 0.37 0.5564

BC 45.23 1 45.23 2.09 0.1766

BD 1.05 1 1.05 0.048 0.8298

BE 3.71 1 3.71 0.17 0.6873

CD 0.031 1 0.031 1.412E-003 0.9707

CE 36.30 1 36.30 1.67 0.2222

DE 0.016 1 0.016 7.205E-004 0.9791

Algorithmic Model Reduction

Backward Selection

1. Begin with the full model.

2. Remove from the model the factor with the smallest F value.

3. Stop when the p-value of the next factor out satisfies the

specified alpha value criterion.

We put this first on the list because it gives every term a

chance to get into the model.

30

Whey Protein Case Study

Model Reduction (Instructor-led)

1. Return to model selection by

pressing the Model button

2. Reduce the model by changing the

“Selection” method from “Manual” to

“Backward”.

3. Choose ANOVA for this reduced

model.

31

Hierarchical Models*

32

YES!


Full vs Reduced Quadratic Model (1 of 2)

33

ANOVA for Response Surface Reduced Quadratic Model

Sum of Mean F


Model 6331.63 12 527.64 34.12 < 0.0001

Residual 293.83 19 15.46

Lack of Fit 260.73 14 18.62 2.81 0.1297

Pure Error 33.09 5 6.62

Cor. Total 6625.46 31

ANOVA for Response Surface Quadratic Model (Full)

Sum of Mean F


Model 6386.91 20 319.35 14.73 < 0.0001

Residual 238.55 11 21.69

Lack of Fit 205.46 6 34.24 5.17 0.0459

Pure Error 33.09 5 6.62

Cor. Total 6625.46 31


Full vs Reduced Quadratic Model (2 of 2)

Benefits are clear for using the reduced model for this response.

34

Full Quadratic ModelStd. Dev. 4.66 R-Squared 0.9640

Dep Mean 68.83 Adj R-Squared 0.8985

C.V. 6.77 Pred R-Squared 0.1632

PRESS 5544.20 Adeq. Precision 11.789

Reduced Quadratic ModelStd. Dev. 3.93 R-Squared 0.9557

Mean 68.83 Adj R-Squared 0.9276

C.V. % 5.71 Pred R-Squared 0.8589

PRESS 934.73 Adeq Precision 17.589

Case Study






35


Y2 – Whipping time

Response: Whip time Transform: Base 10 log Constant: 0.000


Analysis of variance table [Partial sum of squares]

Sum of Mean F


Model 0.47 7 0.067 15.43 < 0.0001

A 0.23 1 0.23 53.78 < 0.0001

B 5.528E-004 1 5.528E-004 0.13 0.7241

C 0.037 1 0.037 8.53 0.0075

D 4.136E-003 1 4.136E-003 0.95 0.3384

A2 0.100 1 0.100 23.03 < 0.0001

AB 0.069 1 0.069 16.02 0.0005

BD 0.024 1 0.024 5.56 0.0269

Residual 0.10 24 4.333E-003

Lack of Fit 0.093 19 4.905E-003 2.27 0.1854

Pure Error 0.011 5 2.161E-003

Cor Total 0.57 31

36


Y2 – Whipping time


Std. Dev. 0.066 R-Squared 0.8182




37

Case Study






38


Y3 – Time at first drop

Response: Time at first drop Transform: Base 10 log Constant: 0.000

ANOVA for Response Surface Reduced 2FI Model

Analysis of variance table [Partial sum of squares]

Sum of Mean F

Source Squares DFSquare Value Prob > F

Model 0.85 7 0.12 8.01 < 0.0001

A 0.019 1 0.019 1.26 0.2735

B 0.53 1 0.53 35.41 < 0.0001

C 0.089 1 0.089 5.89 0.0231

D 2.095E-004 1 2.095E-004 0.014 0.9072

E 0.025 1 0.025 1.68 0.2075

AD 0.11 1 0.11 7.55 0.0112

AE 0.064 1 0.064 4.24 0.0504

Residual 0.36 24 0.015

Lack of Fit 0.35 19 0.019 11.72 0.0063

Pure Error 7.950E-003 5 1.590E-003

Cor Total 1.21 31

39


Y3 – Time at first drop






40

Whey Protein ConcentratesOptimization

Next Step:

Use the three response models we just fit to find the best

tradeoff in properties to give the “optimum” operating

conditions.

41

Introduction to Design of Experiments

1. Response Surface Methodology




2. Multiple Response Optimization


42

Agenda Transition

Multiple Response Optimization:


(optimization)

yes

Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

yes

Factor effects

and interactions

Response

Surface

Methods

Curvature?

Confirm?

Known

Factors

Unknown

Factors

Screening

Backup

Celebrate!

no

no

yes

Trivialmany

Vital few

Screening

Characterization

Optimization

Verification

43

Simultaneous Optimization

of Multiple Responses

1. Analyze each response separately and establish an appropriate

transformation and model for each.

2. Optimize using the models to search the independent factor

space for a region that simultaneously satisfies the

requirements placed on the responses.

Useful models are essential!

Design of experiments is critical!

44

First Step: Develop Good ModelsDon’t Over Interpret the Statistics!

Be sure the fitted surface adequately represents your process

before you use it for optimization. Check for:

1. A significant model: Large F-value with p<0.05.

2. Insignificant lack-of-fit: F-value with p>0.10.

3. Adequate precision >4.

4. Well behaved residuals: Check diagnostic plots!

45

Whey Protein ConcentratesY1 – Undenatured protein

Response: Undenatured Protein


Sum of Mean F


Model 6331.63 12 527.64 34.12 < 0.0001

Residual 293.83 19 15.46

Lack of Fit 260.73 14 18.62 2.81 0.1297

Pure Error 33.09 5 6.62

Cor. Total 6625.46 31



C.V. % 5.71 Pred R-Squared 0.8589


46

Whey Protein ConcentratesY2 – Whipping time



Sum of Mean F


Model 0.47 7 0.067 15.43 < 0.0001

Residual 0.10 24 4.333E-003

Lack of Fit 0.093 19 4.905E-003 2.27 0.1854

Pure Error 0.011 5 2.161E-003

Cor Total 0.57 31





47

Whey Protein ConcentratesY3 – Time at first drop



Sum of Mean F


Model 0.85 7 0.12 8.01 < 0.0001

Residual 0.36 24 0.015

Lack of Fit 0.35 19 0.019 11.72 0.0063

Pure Error 0.00795 5 1.590E-003

Cor Total 1.21 31





48

Response Surface Numeric Optimization

Desirability as an Objective Function (1/2)

To determine a best combination of responses, we use an

objective function, D(X), that involves the use of a geometric

mean:

The di, which range from 0 to 1 (least to most desirable

respectively), represents the desirability of each individual

(i) response.

n is the number of responses being optimized.

49

1n1 n

n1 2 n i

i 1

D d d ... d d

Response Surface Numeric Optimization

Desirability as an Objective Function (2/2)

Now you can search for the greatest overall desirability (D) for

responses and/or factors (for example, if time is a factor, you may

want to keep it to a minimum):

D = 1 indicates that all the goals are satisfied.

(If this happens, you’re probably not asking

for enough!)

D = 0 when one or more responses fall outside acceptable

limits. (Hopefully this will not happen, but if so, try relaxing

some of your criteria!)

50

Desirability as an Objective Function

Assigning Optimization Parameters (1/2)

The crucial phase of numerical optimization is assignment of

various parameters that define the application of individual

desirabilities (di’s). The most important are:

Goal (none, maximum, minimum, target or range)

Limits (lower and upper).

In this case:

Want to maximize undenatured protein.

Want to minimize whip time.

Want to maximize time at first drop.

51

Desirability as an Objective Function

Assigning Optimization Parameters (2/2)

Of lesser importance are the parameters:

Weight (0.1 to 10) (We’ll leave them all = 1)

Importance (5-point scale displayed + to +++++)

In this case:

Undenatured protein is most important, + + + + +.

Whip time is least important, + +.

Time at first drop, this is of intermediate importance, + + +.

52


Want to maximize

undenatured protein,

this is the most important

response:

+ + + + +

53


Want to minimize whip

time, this is the least

important response:

+ +

54


Want to maximize time at

first drop, this is of

intermediate importance:

+ + +

55

Whey Protein ConcentratesNumeric Optimization

Solutions

# A B C D E Y1 Y2 Y3 “D”

1 70.00 6.23 0.15 0.04 0.15 82.948 3.3895 12.7 0.217

2 70.00 6.24 0.15 0.04 0.15 82.917 3.3842 12.7 0.217

3 70.00 6.26 0.14 0.04 0.15 82.924 3.3902 12.6 0.214

4 70.00 6.15 0.17 0.04 0.15 82.852 3.3890 12.7 0.214

5 70.00 6.17 0.16 0.04 0.14 82.921 3.3925 12.5 0.212

56

Factor Name

A Heating

B pH

C Redox pot

D Na oxalate

E Na lauryl

Response Name

Y1 Undenatured Protein

Y2 Whip time

Y3 Time at first drop

Whey Protein ConcentratesNumeric Optimization

57

Summary – Response Surface Methods

Goal – Optimization of process

Tools

• Central Composite design

(when it fits the problem)

• Optimal (Custom) design if needed

(Watch for webinar - Part 2!)

• Numerical Optimization

58

User Review of DOE Simplified:

As an engineer (just beginning self study on the topic of DOE) I found this book

very useful. The authors provide practical insight that I was unable to find in other

DOE or statistics books. This is not a book for advanced statisticians, however, it is

a great book for someone trying to understand and apply the principles of DOE.

* Published by Productivity Press, New York.

Practical Paperbacks on DOE*by Mark Anderson and Pat Whitcomb

59

Statistics Made Easy®

Best of luck for your

experimenting!

Thanks for listening!

-- Shari

60

Shari Kraber, MS, Applied StatsStat-Ease, Inc.

[email protected]

For all the new features in v8 of Design-Expert software, see

www.statease.com/dx8descr.html

*Pdf of this Powerpoint presentation posted at www.statease.com/webinar.html.For future webinars, subscribe to DOE FAQ Alert at www.statease.com/doealert.html.


http://www.statease.com/dx8descr.html

http://www.statease.com/webinar.htm

http://www.statease.com/doealert.html

How to get help

Search publications posted at www.statease.com.

In Stat-Ease software press for Screen Tips, view

reports in annotated mode, look for context-sensitive Help

(right-click) or search the main Help system.

Explore Experiment Design Forum http://forum.statease.com

and post your question (if not previously answered).

E-mail [email protected] for answers from Stat-Ease’s

staff of statistical consultants.

Call 612.378.9449 and ask for “statistical help.”

61

http://www.statease.com/

http://forum.statease.com/


intro to response surface methods - cdn.statease.com · approximation of the true response surface...

Documents