2012-01-rsm part 2 - optimal design (posted to website) · 1. mark anderson and pat whitcomb...

1

©2012

Stat

-Eas

e, Inc

.

Now we get into a number of complications that come up in real-life that cannot be accommodated by the standard textbook designs. Special tools must be employed to set up a design that can deal with the nature of the factors (for example, categoric) operating constraints (such as limits in equipment) and expected response behavior (as a specific ( q p ) p p ( pfunction of the factors).

1. Mark Anderson and Pat Whitcomb (2005), RSM Simplified, Productivity Press, chapters 7 and 10.

2. Raymond H. Myers, Douglas C. Montgomery and Christine M. Anderson-Cook (2009), Response Surface Methodology, 3rd edition John Wiley, section 8.2 - 8.4.(2009), Response Surface Methodology, 3 edition John Wiley, section 8.2 8.4.

2

©2012

Stat

-Eas

e, Inc

.

3

©2012

Stat

-Eas

e, Inc

.

The key to successful use of RSM is to remember that you will be trying to approximate the shape of the surface with a polynomial, and therefore the more you can focus your search the better. Factorial designs, specifically two level, get you in the neighborhood most effectively. These techniques rely on simple semi-linear models, in some cases y q y pafter appropriate transformation of the response. Sequential application of these statistical tools may gain us sufficient performance.

However, we may find that the simple factorial models prove inadequate to model the response surface. We then need to fit higher order polynomials to approximate the curved surfaces found near optimal regions. Maps of these surfaces now become very interesting visual aids to the experimenter and his "customers", the people to whom he or h i blshe is accountable.

We could just plot the data and use a French curve to draw curves. However, as you will, see statistical techniques provide a much more scientific approach to RSM.

4

©2012

Stat

-Eas

e, Inc

.

Somewhat different considerations than for a factorial design. Remember that the purpose of factorial designs was to determine which factors affect the response, while the purpose of response surface designs is to determine how the factors affect the response.

5

©2012

Stat

-Eas

e, Inc

.

6

©2012

Stat

-Eas

e, Inc

.


1. Mark Anderson and Pat Whitcomb (2005), RSM Simplified, Productivity Press, chapters 7 and 10.

2. Raymond H. Myers, Douglas C. Montgomery and Christine M. Anderson-Cook (2009), Response Surface Methodology, 3rd edition John Wiley, section 8.2 - 8.4.(2009), Response Surface Methodology, 3 edition John Wiley, section 8.2 8.4.

7

©2012

Stat

-Eas

e, Inc

.


8

©2012

Stat

-Eas

e, Inc

.

My mom’s recipe for corn pudding calls for baking it for one hour at 400 degrees Fahrenheit. I wanted to design an experiment to see if could be baked longer at a lower temperature. I knew from experience that at 400 any time longer than 60 minutes would lead to burnt pudding. I guessed that at 90 minutes that a temperature greater 350 would p g g p galso produce burnt pudding. We need to define a linear constraint exclude design points from the burnt region.

9

©2012

Stat

-Eas

e, Inc

.

CPA is highest level factor A can be set to when factor B is at its high level, in this case CPA = 60. CPB is highest level factor B can be set to when factor A is at its high level, in this case CPB = 350.

10

©2012

Stat

-Eas

e, Inc

.

There is a tool for calculating a multi linear constraint (MLC).

11

©2012

Stat

-Eas

e, Inc

.

The constraint tool writes the linear inequality defining the constraint plane (in this two factor example, a constraint line).

12

©2012

Stat

-Eas

e, Inc

.

Obviously you won’t be able to achieve rotatability with a design like this. You cannot go in all directions since one of the corners is missing. But, you do get a fairly level standard error surface.

The standard error will be reduced at points that get replicated You could add more butThe standard error will be reduced at points that get replicated. You could add more, but it would be a case of diminishing returns.

13

©2012

Stat

-Eas

e, Inc

.

Goals:

Complete solution of the elements.

Avoid losses and contamination.

Reduce handling and processing times.

14

©2012

Stat

-Eas

e, Inc

.

A representative sample is extracted and/or dissolved in concentrated nitric acid, or alternatively, a concentrated nitric acid and concentrated hydrochloric acid mix using microwave heating with a suitable laboratory microwave unit. The sample and acid(s) are placed in a fluorocarbon polymer microwave vessel. The vessel is sealed and heated p p yin the microwave unit for a specified period of time. After cooling, the vessel contents are filtered, centrifuged, or allowed to settle and then diluted to volume and analyzed by ICP-AES.

15

©2012

Stat

-Eas

e, Inc

.

2/22/2011

A representative sample of the waste stream dissolved using microwave digestion:

• The sample and nitric acid are placed in a fluorocarbon polymer microwave vessel. The vessel is sealed and heated in the microwave unit for a specified period of time.

• Hydrochloric acid is added to the fluorocarbon polymer microwave vessel. The vessel is sealed and heated in the microwave unit for a specified period of time.

• A 4% boric acid solution is added to the fluorocarbon polymer microwave vessel. The vessel is sealed and heated in the microwave unit for a specified period of time.

• After cooling, the vessel contents are filtered, centrifuged, or allowed to settle and then diluted to volume and analyzed.y

16

©2012

Stat

-Eas

e, Inc

.

Cd – Cadmium

Cr – Chromium

Pb – Lead

Hydrochloric acid – HCl

Nitric acid – HNO3

Boric acid – H3BO3

17

©2012

Stat

-Eas

e, Inc

.

Build a four factor face centered design CCD (FCD).

18

©2012

Stat

-Eas

e, Inc

.

Enter the factors (name, units, low and high value), the responses (name and units) and complete the design.

19

©2012

Stat

-Eas

e, Inc

.

The design sorted by factor 1 (t1) and then by factor 3 (t2).

20

©2012

Stat

-Eas

e, Inc

.

There are two options at this point:

1. A bad option: Make the CCD factor ranges smaller so the corners don’t exceed the constraints.

2 A d ti R i th t i t i lti l li t i t2. A good option: Recognize the constraints using multiple linear constraints (MLCs) and use optimal design.

21

©2012

Stat

-Eas

e, Inc

.

Rebuilt the microwave digestion design as an optimal design – page 1 of 3.

22

©2012

Stat

-Eas

e, Inc

.


Enter the constraints to prevent digestion time from being either too short or too long.

23

©2012

Stat

-Eas

e, Inc

.


Given how many factors (k) you want to study and the number of coefficients (p) in the model you select, the design will be built as follows:

Model: p points using an optimality criteria

Lack-of-Fit: 5 points, based on distance – an approach that fills in the gaps(see notes below for detail on this criteria)

Replicates: 5 points, using the model optimality criteria(most influential)

IV-optimal designs seek to minimizes the integral of the prediction variance across the design space; i.e. minimize the area under the FDS curve. The desire for precise prediction in a response surface design aligns perfectly with IV point selection.

IV- is also known as I- or V- optimal.

24

©2012

Stat

-Eas

e, Inc

.

On the “Evaluation” node, “Graphs” click on the “Contour” button on the “Graphs Tool”. To change which factors on the axes of the graph right click a factor on the “Factors Tool” and set it to be either the “X1 axis” or “X2 axis”.

25

©2012

Stat

-Eas

e, Inc

.

Running “digestive.sim” generates data for all three responses.

26

©2012

Stat

-Eas

e, Inc

.

The suggested model is based on:

That adding higher order terms to the model add further explanation; i.e. their sequential SS is significant.

Th t th d l fit th d t i L k f Fit i i i ifi tThat the model fit the data; i.e. Lack of Fit is insignificant.

That the model has the highest adjusted and predicted R-squares.

If a polynomial model is aliased, the least squares coefficient estimates are not unique and thus the surface plots are likely to be misleading. If the aliased cubic model is needed (in this case study it is not) then consider augmenting the design to de-alias the cubic model terms.

27

©2012

Stat

-Eas

e, Inc

.

You should select the highest degree model that has a p-value (Prob > F) that is lower than your chosen level of significance (for example p = 0.05). The “2FI” refers to a two-factor interaction model which could be estimated by the core factorial.

For the exercise data shown in the output above you would obviously choose theFor the exercise data shown in the output above, you would obviously choose the quadratic model, for which the sequential tables shows a Prob > F (probability) of 0.0008. Remember, the full cubic model cannot be completely determined due to aliasing of coefficients. However, even if it weren’t aliased, there’s no advantage to cluttering up the model with 3rd order terms that insignificantly improve the explanation of variance. Stick with the most parsimonious model.

28

©2012

Stat

-Eas

e, Inc

.

Lack-of-fit requires:

Excess design points (beyond the number of parameters in the model) to estimate variation about the fitted surface.

R li t i t t ti t “ ”Replicate experiments to estimate “pure” error.

In the case of the Cd data shown above, the linear model shows a significant lack of fit, p = 0.0034. If you see significant lack of fit, then the model should not be used as a predictor of the response. In this case, the quadratic and cubic models do not show significant lack of fit.

29

©2012

Stat

-Eas

e, Inc

.

The standard deviation of the error in the design obviously should be small. Quadratic is best by this measure.

The raw R-Squared value, the related Adjusted R-Squared and the Predicted R-Squared statistics will show a decreasing trend but they will be more consistent for the beststatistics will show a decreasing trend, but they will be more consistent for the best model. A value of one represents the ideal case at which 100 percent of the variance can be explained by the chosen model. The PRESS statistic, predicted residual sum of squares, indicates how well the model fits the data. The PRESS for the chosen model should be small relative to the other models under consideration. Predicted R-Squared incorporates PRESS, so these two statistics are redundant.

In this case, by almost any measure, the quadratic model looks like best., y y , q

You may be wondering how Predicted R-Square value could be negative. It means that the model makes poorer predictions than just using the overall average: You’re better off just using the mean.

30

©2012

Stat

-Eas

e, Inc

.

Are you a bit queasy about your ability to digest all these model indicators and spit out the correct model? Don’t worry, be happy that Design-Expert software will do this for you by default using a scoring system.

31

©2012

Stat

-Eas

e, Inc

.

Notice that a few terms do not appear to be significant, so let’s remove them – see the next slide.

32

©2012

Stat

-Eas

e, Inc

.

There are three algorithms for selection, 1) Backward, 2) Forward and 3) Stepwise. For well-designed experiments (little or not collinearity) all three will lead to the same reduced model. When the factors are correlated with one another (i.e. they are collinear) the reduce model can be different depending on the selection method used. Backward p gselection has the advantage of starting with all terms in the model.

33

©2012

Stat

-Eas

e, Inc

.

The least significant terms are removed one at a time until the least significant term remaining has p-value less than Alpha Out; in this case 0.10.

34

©2012

Stat

-Eas

e, Inc

.

How do the full versus reduced models compare? Answer – higher Model F-value.

35

©2012

Stat

-Eas

e, Inc

.

How do the full versus reduced models compare? (Answer: reduction improves predicted R-squared and Adequate Precision.)

With the first model, we were trying to predict what happens when D and E changed, when nothing happened We were over-fitting the model The reduced model works muchwhen nothing happened. We were over fitting the model. The reduced model works much better.

Students should continue with problem by reviewing diagnostic graphs and choosing best factor settings.

36

©2012

Stat

-Eas

e, Inc

.

The 2D (contour) response surface plot is the working tool for exploration. Later we’ll see how you can overlay these plots from multiple responses, shading out the undesirable areas, leaving you a window of feasibility.

37

©2012

Stat

-Eas

e, Inc

.

The 3D response surface plot helps you visualize the results and dazzle your customers.

38

©2012

Stat

-Eas

e, Inc

.

Using the handy RSM “cheat sheet” in you “Handbook for Experimenters” fit models toCd and Pb.

39

©2012

Stat

-Eas

e, Inc

.

This is a very powerful tool for finding your sweet spot – where all specifications can be met. But the reliability of the results depends on the validity of your predictive models.

Each response may have a different model and/or a different subset of factors.

40

©2012

Stat

-Eas

e, Inc

.

A significant model F-value gives you confidence that you can explain what causes variation. However, some statisticians advise that for prediction purposes, you need a stronger fit, perhaps as much as 4 times the F value you’d normally accept as significant. A more straight-forward statistic for determining the strength of your model for prediction g g g y pis the adequate precision, which was defined mathematically in Section 2 in the explanation of outputs from the RSM tutorial. Recall that this statistic measures the signal by taking the range of predicted response (max to min of y) which you can read off the case statistic table in the ANOVA report. It ratios this signal to noise defined by a function of the Mean Square Error (MSE) which comes from the ANOVA table.

41

©2012

Stat

-Eas

e, Inc

.

Comparing these results to the criteria for good models. Cd recovery looks good and examination of the diagnostic plots does not uncover any problems.

42

©2012

Stat

-Eas

e, Inc

.

Comparing these results to the criteria for good models. Cr recovery looks good and examination of the diagnostic plots does not uncover any problems.

43

©2012

Stat

-Eas

e, Inc

.

Comparing these results to the criteria for good models. Pb recovery looks good and examination of the diagnostic plots does not uncover any problems.

44

©2012

Stat

-Eas

e, Inc

.

Ideally, all goals can be achieved simultaneously. Often there are trade offs to be made. As one goal is reached the others fail. The idea of optimization is to find the best set of trade offs. This is NOT the same as finding the best “solution”. Finding the best solution requires subject matter knowledge. q j g

45

©2012

Stat

-Eas

e, Inc

.

Target Cd at 100% recovery.

46

©2012

Stat

-Eas

e, Inc

.

Target Cr at 100% recovery.

47

©2012

Stat

-Eas

e, Inc

.

Since highest observed Pb was recovery was only 92.5%, maximize Pb recovery using a stretch goal of 100% recovery.

48

©2012

Stat

-Eas

e, Inc

.

The search algorithm uses the points in the design, in this case, 20 (the number of non-replicated runs) plus 30 more random factor combinations (for a total of 50) as the starting points for the search. The random start points can produce slightly different optimums.p

49

©2012

Stat

-Eas

e, Inc

.

Use the Ramps view to compare the solutions; clicking on the solution numbers one after another and see what changes on the ramps.

50

©2012

Stat

-Eas

e, Inc

.

51

©2012

Stat

-Eas

e, Inc

.

Here’s a response surface design with multiple linear constraints. Many real life response surface designs are constrained, however the constraints aren’t always recognized; or the design is “shrunken” to avoid them. Neither of these are good situations.

52

©2012

Stat

-Eas

e, Inc

.

Set the factor entry to vertical and enter the factor names levels:

Factor A: enter the name: Flow rate enter the units: ml/min enter the units: ml/min Type: Continuous Low = 10 High = 30

Factor B: enter the name: Passes enter the units: # Type: DiscreteType: Discrete enter 1 for the low “L[1]” value enter 10 for the high “L[10]” value Levels: 10 press the “Tab” key to calculate the intermediate values

53

©2012

Stat

-Eas

e, Inc

.

Define the “too much coating” region to exclude from the design space.

54

©2012

Stat

-Eas

e, Inc

.

1. Pick the vertex to exclude: A=30 and B=10.

2. Ensure the sign on each factor is properly set: want both A and B to be less than their constraint points.

3 E t th t i t i t f h f t i l d i th t i t CP i th hi h t3. Enter the constraint point for each factor involved in the constraint: CPA is the highest level factor A can be set to when factor B is at its high level, CPA = 20. CPB is the highest level factor B can be set to when factor A is at its high level, CPB = 5.

55

©2012

Stat

-Eas

e, Inc

.

Define the “not enough coating” region to exclude from the design space.

56

©2012

Stat

-Eas

e, Inc

.

1. Pick the vertex to exclude: A=10 and B=1.

2. Ensure the sign on each factor is properly set: want both A and B to be greater than their constraint points.

3 E t th t i t i t f h f t i l d i th t i t CP i th l t3. Enter the constraint point for each factor involved in the constraint: CPA is the lowest level factor A can be set to when factor B is at its low level, CPA = 15. CPB is the lowest level factor B can be set to when factor A is at its low level, CPB = 4.

57

©2012

Stat

-Eas

e, Inc

.

The constraint tool writes the linear equations defining the constraint planes (in this two factor example, constraint lines).

58

©2012

Stat

-Eas

e, Inc

.

Accept the defaults of Best, IV optimality, Quadratic, 1 block, 6 model points , 5 lack of fit points and 5 replicates; click on the “Continue >>” button.

59

©2012

Stat

-Eas

e, Inc

.

Enter the response and build a optimal design that fills the constrained space.

You will see a message warning you that some of the combinations of passes and flow rate on the candidate list are not within the constrained region. In this case 14 of the 60 points on the candidate list are invalid Just click “OK” to continue:points on the candidate list are invalid. Just click OK to continue:

60

©2012

Stat

-Eas

e, Inc

.

Your design space should be identical and the design points similar. Your build may only have 2 center points, but that is okay. Worry more about the FDS score rather than the actual runs in the experiment.

61

©2012

Stat

-Eas

e, Inc

.

Hint: Use graph preferences to set:

“X2 Axis” 1 to 10 with 10 “Ticks”,

“Contours” to “Incremental” with “Start” = 7.5, “Step” = 0.5 and “Levels” = 10,

“Graphs 1” choose “Show 2D graph grid lines”.

Note: The squares on this graph WILL NOT appear in Design-Expert – they were manually added in PowerPoint.

63

©2012

Stat

-Eas

e, Inc

.

What if you want to do an RSM and you have some factors that are categoric? You could do separate studies at each of the categoric levels and then compare the graphs. Better yet, use the features in Design-Expert to set up one design, do the statistics on the combined model, and make the plots. Here’s a typical example with 2 continuous factors p yp pand 1 categoric.

65

©2012

Stat

-Eas

e, Inc

.

Make sure that you are using the optimal design on the Response Surface tab. There is a optimal design option on every tab and each will build a different type of design.

66

©2012

Stat

-Eas

e, Inc

.

Numerical Optimization really gets in the way here – use the Model Graphs to figure out the answer!

67

©2012

Stat

-Eas

e, Inc

.

FYI: “Vinyl” refers to a polymer made up of individual vinyl monomers. These univalent chemical radicals are created by combining ethylene (CH2CH2), typically derived from hydrocarbons such as gas or oil, and chlorine (Cl), made from brine (salt water). This is probably already more than you’d like to know, but for those who want to learn more p y y yabout this fascinating (?) material, the Vinyl Institute would love to see you at their website: www.vinylinfo.org.

71

©2012

Stat

-Eas

e, Inc

.

WARNING: There will be problems with the model fitting! Do not proceed until we discuss these problems.

72

©2012

Stat

-Eas

e, Inc

.

Signs that a cubic model is required for the vinyl case are:

• Fit Summary: the “Sequential Model Sum of Squares”, “Lack of Fit Tests” and “Model Summary Statistics” all point to aliased cubic model.

• Quadratic model doesn’t fit well: it is only marginally significant, has significant lackQuadratic model doesn t fit well: it is only marginally significant, has significant lack of fit and has a negative predicted R-squared. Also it has many high “Cook’s Distance” values. (The many high Cook’s D values indicate the quadratic model is not very stable and help explain why there is a negative predicted R-squared.)

• Transformation, such as applying a log, doesn’t help.

In situations like this, sometimes you can use the information in hand to limit the range of the factors. If the region of interest is reduced enough, the quadratic model will be adequate However in the vinyl solubility study we cannot reduce the area of interest Itadequate. However, in the vinyl solubility study we cannot reduce the area of interest. It looks like a cubic model will be needed for an adequate fit of the data.

Details on how Design-Expert does augmentation:

• The software uses the optimal algorithm to fit the requested model (presumably beefed up).

• Two points (rather than one) are added for each additional coefficient to give adequate precision to the coefficient estimates

73

adequate precision to the coefficient estimates.

©2012

Stat

-Eas

e, Inc

.

It won’t hurt to do more than the minimum runs in the second block so you can generate more power for estimating the added coefficients.

74

©2012

Stat

-Eas

e, Inc

.

It won’t hurt to do more than the minimum runs in the second block so you can generate some power for estimating the added coefficients.

76

©2012

Stat

-Eas

e, Inc

.

Any of these designs are good choices for response surfaces. Choose the one that fits the problem the best.

79

©2012

Stat

-Eas

e, Inc

.

2012-01-rsm part 2 - optimal design (posted to website) · 1. mark anderson and pat whitcomb...

Documents