hypothesis testing 1

Sheela.V.K

2Analyze Phase

Introduction

Statistical inference involves two analysis methods: estimation and hypothesis testing,

Statistics often involve a comparison of two values when one or both values are associated with some uncertainty. The purpose of statistical inference is to aid the researcher, or administrator in reaching a conclusion concerning a population by examining a sample from that population.

Estimation can be carried out on the basis of sample values from a larger population (1). Point estimation involves the use of summary statistics, including the sample mean and SD. These values can be used to estimate intervals, such as the 95% confidence level

Hypothesis testing enables one to quantify the degree of uncertainty in sampling variation, which may account for the results that deviate from the hypothesized values in a particular study

http://radiology.rsna.org/content/226/3/609.full

3Analyze Phase

Our goal is to improve our process capability, this translates to the need to move the process mean (or proportion) and reduce the standard deviation.

Because it is too expensive or too impractical (not to mention theoretically impossible) to collect population data, we will make decisions based on sample data.Because we are dealing with sample data, there is some uncertainty about the true population parameters.

A hypothesis test converts the practical problem into a statistical problem.Since relatively small sample sizes are used to estimate population parameters, there is always a chance of collecting a non-representative sample.Inferential statistics allows us to estimate the probability of getting a non-representative sample.

Hypothesis Testing

4Analyze Phase

Statistical Hypotheses

A hypothesis is a predetermined theory about the nature of, or relationships between variables. Statistical tests can prove (with a certain degree of confidence), that a relationship exists.

A general procedure is that of calculating the probability ofobserving the difference between two values if they really are not different.

This probability is called the P value, and this condition is called the null hypothesis (H0).

On the basis of the P value and whether it is low enough, one can conclude that H0 is not true and that there really is a difference.

We have two alternatives for hypothesis.

– The “null hypothesis” Ho assumes that there are no differences or relationships. This is the default assumption of all statistical tests.

– The “alternative hypothesis” Ha states that there is a difference or relationship.

5Analyze Phase

Hypothesis Testing Risk

The alpha risk or Type 1 Error (generally called the “Producer’s Risk”) is the probability that we could be wrong in saying that something is “different.” It is an assessment of the likelihood that the observed difference could have occurred by random chance. Alpha is the primary decision-making tool of most statistical tests.

Type 1Error

Type IIError

CorrectDecision

CorrectDecision

Actual ConditionsNot Different Different

Not Different

StatisticalConclusions

(Ho is True) (Ho is False)

(Fail to Reject Ho)

Different(Reject Ho)

6Analyze Phase

Alpha Risk

Alpha ( ) risks are expressed relative to a reference distribution.

Distributions include:

– t-distribution

– z-distribution

– 2- distribution

– F-distribution

Region of DOUBT

Region of DOUBT

Accept as chance differences

The a-level is represented by the clouded areas.

Sample results in this area lead to rejection of H0.

The a-level is represented by the clouded areas.

Sample results in this area lead to rejection of H0.

7Analyze Phase

Hypothesis Testing Risk

The beta risk or Type 2 Error (also called the “Consumer’s Risk”) is the probability that we could be wrong in saying that two or more things are the same when, in fact, they are different.

Type 1Error

Type IIError

CorrectDecision

CorrectDecision

Actual ConditionsNot Different Different

Not Different

StatisticalConclusions

(Ho is True) (Ho is False)

(Fail to Reject Ho)

Different(Reject Ho)

8Analyze Phase

Beta Risk

Beta risk is the probability of failing to reject the null hypothesis when a difference exists.

Distribution if Ha is true

Critical value of test statistic

Critical value of test statistic

Reject H0

= Pr(Type 1 error)

Accept H0

= Pr(Type II error)

= 0.05

H0 value

Distribution if H0 is true

9Analyze Phase

Distinguishing between Two Samples

Recall from the Central Limit Theorem as the number of individual observations increase the standard error decreases.

In this example when n=2 we cannot distinguish the difference between the means (> 5% overlap, p-value > 0.05).

When n=30, we can distinguish between the means (< 5% overlap, p-value < 0.05) There is a significant difference.

Theoretical Distribution of MeansWhen n = 2d = 5S = 1

Theoretical Distribution of MeansWhen n = 30 d = 5

S = 1

10Analyze Phase

Delta Sigma—The Ratio between and S

Delta (d) is the size of the difference between two means or one mean and a target value.

Sigma (S) is the sample standard deviation of the distribution of individuals of one or both of the samples under question.

When S is large, we don’t need statistics because the differences are so large.

If the variance of the data is large, it is difficult to establish differences. We need larger sample sizes to reduce uncertainty.

Large Delta

Large S

11Analyze Phase

The Perfect Sample Size

The minimum sample size required to provide exactly 5% overlap (risk). In order to distinguish the Delta.

Note: If you are working with non-normal data, multiply your calculated sample size by 1.1

40 50 60 7040 50 60 70

40 60 7050

Population

12Analyze Phase

Hypothesis Testing Roadmap

Normal

Test of Equal Variance 1 Sample t-test1 Sample Variance

Variance Not EqualVariance Equal

2 Sample T One Way ANOVA 2 Sample T One Way ANOVA

Continuous

Data

13Analyze Phase


Non Normal

Test of Equal Variance Median Test

Mann-Whitney Several Median Tests

Continuous

Data

14Analyze Phase


Attribute Data

One Factor Two Factors

One Sample Proportion

Two Sample Proportion

Minitab:Stat - Basic Stats - 2 ProportionsIf P-value < 0.05 the proportions are different

Chi Square Test (Contingency Table)

Minitab:Stat - Tables - Chi-Square TestIf P-value < 0.05 the factors are not independent

Chi Square Test (Contingency Table)

Minitab:Stat - Tables - Chi-Square TestIf P-value < 0.05 at least one proportion is different

Two or More Samples

Two SamplesOne Sample

Attribute

Data

15Analyze Phase

Common Pitfalls to Avoid

While using hypothesis testing the following facts should be borne in mind at the conclusion stage:

– The decision is about Ho and NOT Ha.

– The conclusion statement is whether the contention of Ha was upheld.

– The null hypothesis (Ho) is on trial.

– When a decision has been made:• Nothing has been proved.• It is just a decision.• All decisions can lead to errors (Types I and II).

– If the decision is to “Reject Ho,” then the conclusion should read “There is sufficient evidence at the α level of significance to show that “state the alternative hypothesis Ha.”

– If the decision is to “Fail to Reject Ho,” then the conclusion should read “There isn’t sufficient evidence at the α level of significance to show that “state the alternative hypothesis.”

17Analyze Phase

Test of Means (t-tests)

t-tests are used:– To compare a mean against a target.

• ie. The team made improvements and wants to compare the mean against a target to see if they met the target.

– To compare means from two different samples. • ie. Machine one to machine two.• ie. Supplier one quality to supplier two quality.

– To compare paired data.• Comparing the same part before and after a given

process.

18Analyze Phase

1 Sample t

A 1-sample t-test is used to compare an expected population mean to a target.

Minitab performs a one sample t-test or t-confidence interval for the mean.

Use 1-sample t to compute a confidence interval and perform a hypothesis test of the mean when the population standard deviation, σ, is unknown. For a one or two-tailed 1-sample t:

– H0: μsample = μtarget If p-value > 0.05 fail to reject Ho

– Ha: μsample ≠, <, > μtarget If p-value < 0.05 reject Ho

Target μsample

19Analyze Phase

1 Sample t-test Sample Size

n=2

Population

n=30

Can not tell the differencebetween the sample and the target.

Can tell the differencebetween the sample and the target.

Target

n

SMean SE

X X

T

X

XX X X

X XXXXX

X

XXX

XX

XX X X

20Analyze Phase

1-Sample t-test Example

1. Practical Problem:• We are considering changing suppliers for a part that we currently

purchase from a supplier that charges us a premium for the hardening process.

• The proposed new supplier has provided us with a sample of their product. They have stated that they can maintain a given characteristic of 5 on their product.

• We want to test the samples and determine if their claim is accurate.

2. Statistical Problem:

Ho: μN.S. = 5

Ha: μN.S. ≠ 5

3. 1-sample t-test (population standard deviation unknown, comparing to target).

α = 0.05 β = 0.10

21Analyze Phase

Example

4. Sample size: • Open the Minitab worksheet: Exh_Stat.MTW• Use the C1 column: Values

– In this case, the new supplier sent 9 samples for evaluation.– How much of a difference can be detected with this sample?

22Analyze Phase


Power and Sample Size

1-Sample t Test

Testing mean = null (versus not = null)

Calculating power for mean = null + difference

Alpha = 0.05 Assumed standard deviation = 1

Sample

Size Power Difference

9 0.9 1.23748

Minitab Session Window

This means we will only be able to detect a difference of only 1.24 if

the population has a standard deviation of 1 unit.

23Analyze Phase

Example: Follow the Road Map

5. State statistical solution

Are the data in the values column normal?

Are the data in the values column normal?

Values

Perc

ent

5.45.25.04.84.64.44.2

99

95

90

80

70

605040

30

20

10

Mean

0.442

4.789StDev 0.2472N 9AD 0.327P-Value

Probability Plot of ValuesNormal

24Analyze Phase


Click “Graphs”

- Select all 3

Click “Options

- In CI enter 95

25Analyze Phase

One-Sample T: Values

Test of mu = 5 vs not = 5

Variable N Mean StDev SE Mean 95% CI T P

Values 9 4.78889 0.24721 0.08240 (4.59887, 4.97891) -2.56 0.034

Session Window

HoHa

N – sample sizeMean – calculate mathematic averageStDev – calculated individual standard deviation (classical method)SE Mean – calculated standard deviation of the distribution of the meansConfidence Interval that our population average will fall between 4.5989, 4.9789

n

SMean SE

n

1i

i

1n

)X(Xs

2

T-Calc = Observed – Expected over SE MeanT-Calc = X-bar – Target over Standard ErrorT-Calc = 4.7889 – 5 over .0824 = - 2.56

26Analyze Phase

Since the p-value of 0.034 is less than 0.05, reject the null hypothesis.

Based on the samples given there is a difference between the average of the sample and the desired target.

6. State practical conclusions.The new suppliers claim that they can meet the target

of 5 for the hardness is not correct.

Evaluating the Results

X Ho

27Analyze Phase

Manual Calculation of 1- Sample t

Let’s compare the manual calculations to what the computer calculates.– Calculate t-statistic from data:

– Determine critical t-value from t-table in reference section.• When the alternative hypothesis has a not equal sign, it

is a two-sided test.• Split the α in half and read from the 0.975 column in the

t-table for n -1 (9 - 1) degrees of freedom.

56.2

9

247.000.579.4

n

sTargetX

t

28Analyze Phase

Manual Calculation of 1- Sample t

If the calculated t-value lies anywhere

in the critical regions, reject the null hypothesis.– The data supports the alternative hypothesis that the

estimate for the mean of the population is not 5.0.

degrees of freedom

.600 .700 .800 .900 .950 .975 .990 .9951 0.325 0.727 1.376 3.078 6.314 12.706 31.821 63.6572 0.289 0.617 1.061 1.886 2.920 4.303 6.965 9.9253 0.277 0.584 0.978 1.638 2.353 3.182 4.541 5.8414 0.271 0.569 0.941 1.533 2.132 2.776 3.747 4.6045 0.267 0.559 0.920 1.476 2.015 2.571 3.365 4.032

6 0.265 0.553 0.906 1.440 1.943 2.447 3.143 3.7077 0.263 0.549 0.896 1.415 1.895 2.365 2.998 3.4998 0.262 0.546 0.889 1.397 1.860 2.306 2.896 3.3559 0.261 0.543 0.883 1.383 1.833 2.262 2.821 3.250

10 0.260 0.542 0.879 1.372 1.812 2.228 2.764 3.169

T - Distribution

α/2 =.025α/2=.025

m

0

-2.306 2.306

Critical Regions

-2.56

29Analyze Phase

Confidence Intervals for Two-Sided t-test

The formula for a two-sided t-test is:

4.9789 to4.5989

.0824 * 2.306 788.4SE t X

or n

stXμ

n

stX

meancrit

1nα/2,1nα/2,

Ho4.5989 4.97894.7889

X

30Analyze Phase

1-Sample t-test Exercise: Solution

Since we do not know the population standard deviation, we will use the 1 sample T test to determine if we are at target.

31Analyze Phase


After selecting column C1 and setting “Hypothesis mean” to 32.0, click graphs and select “Histogram of data” to get a good visualization of the analysis.

Depending on the test you are running you may need to select “Options” to set your desired confidence Interval and hypothesis. In this case the Minitab Defaults are what we want.

32Analyze Phase


Because we used the option of “Graphs”, we get a nice visualization of the data in a histogram AND a plot of the null hypothesis relative to the confidence level of the population mean.

Because the null hypothesis is within the confidence level, you know we will “fail to reject” the null hypothesis and accept the equipment is running at the target of 32.0.

ppm VOC

Frequency

50454035302520

10

8

6

4

2

0X_

Ho

Histogram of ppm VOC(with Ho and 95% t-confidence interval for the mean)

33Analyze Phase


In Minitab’s session Window (ctrl – M), you can see the P-value of 0.201. Because it is above 0.05, we “fail to reject” the null hypothesis so we accept the equipment is giving product at a target of 32.0 ppm VOC.

34Analyze Phase

2 Sample t-test

A 2-sample t-test is used to compare two means.

Minitab performs an independent two-sample t-test and generates a confidence interval.

Use 2-Sample t to perform a hypothesis test and compute a confidence interval of the difference between two population means when the population standard deviations, σ’s, are unknown.

Two tailed test:– H0: μ1 = μ2 If p-value > 0.05 fail to reject Ho

– Ha: μ1 ≠ μ2 If p-value < 0.05 reject Ho

One tailed test:– H0: μ1 = μ2

– Ha: μ1 > or < μ2

Stat > Basic Statistics > 2-Sample t

m1 m2

35Analyze Phase


1. Practical Problem:• We have conducted a study in order to determine the effectiveness

of a new heating system. We have installed two different types of dampers in home ( Damper = 1 and Damper = 2).

• We want to compare the BTU.In data from the two types of dampers to determine if there is any difference between the two products.


Ho:μ1 = μ2

Ha:μ1 ≠ μ2

3. 2-Sample t-test (population standard deviations unknown).

α = 0.05 β = 0.10

36Analyze Phase

2 Sample t-test Example

4. Sample size:• Open the Minitab worksheet: Furnace.MTW• Scroll through the data to see how the data is coded.• In order to work with the data in the BTU.In column, we will need

to unstack the data by damper type.

37Analyze Phase


38Analyze Phase


39Analyze Phase


Power and Sample Size 2-Sample t TestTesting mean 1 = mean 2 (versus not =)Calculating power for mean 1 = mean 2 + differenceAlpha = 0.05 Assumed standard deviation = 1Sample Size Power Difference 40 0.9 0.733919 50 0.9 0.654752The sample size is for each group.


40Analyze Phase

Test of equal Variance (Bartlett’s Test)

41Analyze Phase

Test of Equal Variance

Dam

per

95% Bonferroni Confidence Intervals for StDevs

2

1

4.03.53.02.52.0

Dam

per

BTU.In

2

1

2015105

F-Test

0.996

Test Statistic 1.19P-Value 0.558

Levene's Test

Test Statistic 0.00P-Value

Test for Equal Variances for BTU.In

Sample 1

Sample 2

42Analyze Phase

2 Sample t-test Equal Variance

43Analyze Phase

Box Plot

5. State statistical conclusions: Fail to reject the null hypothesis

6. State practical conclusions: There is no difference between the dampers for BTU’s in.

Damper

BTU

.In

21

20

15

10

5

Boxplot of BTU.In by Damper

44Analyze Phase


Number of Samples

Number of Samples

Calculated Average

n

1i

i

1n

)X(Xs

2

nSMean SE (N1 – 1) + (N2-1)

T-Calc = Observed d – Expected d divided by sT-Calc = Estimate for difference – Target for distance over sT-Calc = (9.91 – 10.14) / T-Calc = -0.235 / s

-1.450 0.980

-0.38

Two- Sample T-Test(Variances Equal)

Two- Sample T-Test(Variances Equal)

Ho: μ1 = μ2

Ha: μ1≠ or < or > μ2

Ho: μ1 = μ2

Ha: μ1≠ or < or > μ2

45Analyze Phase

Unequal Variance Example

Open Minitab worksheet: 2 sample unequal variance data

46Analyze Phase

Normality Test

Our data sets are normally distributed.

Our data sets are normally distributed.

Let’s compare the data in Sample one and Sample three columns.

Sample 3

Perc

ent

151050-5

99.9

99

95

90

80706050403020

10

5

1

0.1

Mean

0.658

4.852StDev 3.134N 100AD 0.274P-Value

Probability Plot of Sample 3Normal

Sample 1

Perc

ent

87654321

99.9

99

95

90

80706050403020

10

5

1

0.1

Mean

0.411

4.853StDev 1.020N 100AD 0.374P-Value

Probability Plot of Sample 1Normal

47Analyze Phase

Test for Equal Variance

1 2 3 4

95% Confidence Intervals for Sigmas

2

1

0 5 10 15

Boxplots of Raw Data

Stacked

F-Test

Test Statistic: 0.106

P-Value : 0.000

Levene's Test

Test Statistic: 67.073

P-Value : 0.000

Factor Levels

1

2

Test for Equal Variances for StackedStandard Deviationof SamplesStandard Deviationof Samples

Medians of SamplesMedians of Samples

We use F-Test Statistic because our data is normally distributed.P-Value is less than 0.05, our variances are not equal.

We use F-Test Statistic because our data is normally distributed.P-Value is less than 0.05, our variances are not equal.

Stat>ANOVA>Test of Equal Variance

48Analyze Phase

2-Sample t-test Unequal Variance

UNCHECK “assume equal variances” box.

49Analyze Phase


C4

Sta

cked

21

15

10

5

0

-5

Boxplot of Stacked by C4

Indicate Sample Means

50Analyze Phase


C4

Sta

cked

21

15

10

5

0

-5

Individual Value Plot of Stacked vs C4

Indicate Sample Means

51Analyze Phase


Two-Sample T-Test(Variances Not Equal) Two-Sample T-Test(Variances Not Equal)

Ho: μ1 = μ2 (P-Value > 0.05)Ha: μ1 ≠ or < or > μ2 (P-Value < 0.05)

Ho: μ1 = μ2 (P-Value > 0.05)Ha: μ1 ≠ or < or > μ2 (P-Value < 0.05)

Stat>Basic Stats> 2 sample T (Deselect Assume Equal Variance)

52Analyze Phase

• A paired t-test is used to compare the means of two measurements from the same samples generally used as a before and after test.

• Minitab performs a paired t-test. This is appropriate for testing the difference between two means when the data are paired and the paired differences follow a normal distribution.

• Use the Paired t command to compute a confidence interval and perform a hypothesis test of the difference between population means when observations are paired. A paired t-procedure matches responses that are dependent or related in a pair-wise manner. This matching allows you to account for variability between the pairs usually resulting in a smaller error term, thus increasing the sensitivity of the hypothesis test or confidence interval.– Ho: μδ = μo

– Ha: μδ ≠ μo

• Where μδ is the population mean of the differences and μ0 is the hypothesized mean of the differences, typically zero.

Paired t-test

Stat > Basic Statistics > Paired t

mbefore

delta(d)

mafter

53Analyze Phase

Example

1. Practical Problem:• We are interested in changing the sole material for a popular

brand of shoes for children.• In order to account for variation in activity of children wearing the

shoes, each child will wear one shoe of each type of sole material. The sole material will be randomly assigned to either the left or right shoe.


Ho: μδ = 0

Ha: μδ ≠ 0

3. Paired t-test (comparing data that must remain paired).

α = 0.05 β = 0.10

54Analyze Phase

Example

4. Sample size:• How much of a difference can be detected with 10 samples?

EXH_STAT DELTA.MTW

55Analyze Phase

Paired t-test Example


1-Sample t Test

Testing mean = null (versus not = null)

Calculating power for mean = null + difference

Alpha = 0.05 Assumed standard deviation = 1

Sample

Size Power Difference

10 0.9 1.15456


This means we will only be able to detect a difference of only 1.15 if the

standard deviation is equal to 1.

56Analyze Phase

Paired t-test Example

5. State statistical solution

We need to calculate the difference between the two distributions. We are concerned with the delta, is the Ho outside the t-calc (confidence interval).

Calc>Calculator

57Analyze Phase

Analyzing the Delta

Following the hypothesis test roadmap, we first test the AB-Delta distribution for normality.

AB Delta

Perc

ent

1.51.00.50.0-0.5

99

95

90

80

70

605040

30

20

10

5

1

Mean

0.622


Probability Plot of AB DeltaNormal

58Analyze Phase

1-Sample t-test

Stat > Basic Statistics > 1-Sample t-test…

Since there is only one column, AB Delta, we do not test for equal variance per the hypothesis testing roadmap.

Check this data for statistical significance in its departure from our expected value of zero.

59Analyze Phase

Box Plot

5. State statistical conclusions: Reject the null hypothesis

6. State practical conclusions: We are 95% confident that there is a difference in wear between the two materials.

Boxplot of AB Delta

One-Sample T: AB Delta

Test of mu = 0 vs not = 0

Variable N Mean StDev SE Mean AB Delta 10 0.410000 0.387155 0.122429

95% CI T P(0.133046, 0.686954) 3.35 0.009


60Analyze Phase

Paired t-Test

Another way to analyze this data is to use the paired t-test command.

Stat>Basic Statistics>Paired T-test

Click on Graphs and selectthe graphs you would like to generate.

Click on Graphs and selectthe graphs you would like to generate.

61Analyze Phase

Paired t-Test

Differences0.0-0.3-0.6-0.9-1.2

_X

Ho

Boxplot of Differences(with Ho and 95% t-confidence interval for the mean)

Paired T-Test and CI: Mat-A, Mat-B Paired T for Mat-A - Mat-B

N Mean StDev SE MeanMat-A 10 10.6300 2.4513 0.7752Mat-B 10 11.0400 2.5185 0.7964Difference 10 -0.410000 0.387155 0.122429

95% CI for mean difference: (-0.686954, -0.133046)T-Test of mean difference = 0 (vs not = 0): T-Value = -3.35 P-Value = 0.009

Paired T-Test and CI: Mat-A, Mat-B Paired T for Mat-A - Mat-B

N Mean StDev SE MeanMat-A 10 10.6300 2.4513 0.7752Mat-B 10 11.0400 2.5185 0.7964Difference 10 -0.410000 0.387155 0.122429

95% CI for mean difference: (-0.686954, -0.133046)T-Test of mean difference = 0 (vs not = 0): T-Value = -3.35 P-Value = 0.009

The P-Value of from this Paired T-Test tells us the difference in materials is statistically significant.

The P-Value of from this Paired T-Test tells us the difference in materials is statistically significant.

62Analyze Phase

Paired t-Test

The wrong way to analyze this data is to use a 2-sample t-test:


Two-sample T for Mat-A vs Mat-B

N Mean StDev SE Mean

Mat-A 10 10.63 2.45 0.78

Mat-B 10 11.04 2.52 0.80

Difference = mu (Mat-A) - mu (Mat-B)

Estimate for difference: -0.410000

95% CI for difference: (-2.744924, 1.924924)

T-Test of difference = 0 (vs not =): T-Value = -0.37 P-Value = 0.716 DF = 18

Both use Pooled StDev = 2.4851

63Analyze Phase

Paired t-test Exercise: Solution

Because the two labs ensured to exactly report measurement results for the same parts and the results were put in the correct corresponding row, we are able to do a paired t-test.

The first thing we must do is create a new column with the difference between the two test results.

Calc>Calculator

64Analyze Phase

We must confirm the differences (now in a new calculated column) are from a normal distribution. This was confirmed with the Anderson-Darling Normality Test by doing a graphical summary under basic statistic.


0.750.500.250.00-0.25-0.50

Median

Mean

0.60.40.20.0

Anderson-Darling Normality Test

Variance 0.14418Skewness -0.833133Kurtosis -0.233638N 11

Minimum -0.50000

A-Squared

1st Quartile -0.10000Median 0.400003rd Quartile 0.50000Maximum 0.70000

95% Confidence Interval for Mean

-0.02782

0.45

0.48237

95% Confidence Interval for Median

-0.11644 0.50822

95% Confidence Interval for StDev

0.26531 0.66637

P-Value 0.222

Mean 0.22727StDev 0.37971

95% Confidence Intervals

Summary for TX_MX-Diff

65Analyze Phase


As we’ve seen before, this 1 Sample T analysis is found with: Stat>Basic Stat>1-sample T

66Analyze Phase


Even though the mean difference is 0.23, we have a 95% confidence interval that includes zero so we know the 1-sample T test’s null hypothesis was “failed to be rejected”. We cannot conclude the two labs have a difference in lab results.

TX_MX-Diff

Frequency

0.750.500.250.00-0.25-0.50

5

4

3

2

1

0 _X

Ho

Histogram of TX_MX-Diff(with Ho and 95% t-confidence interval for the mean)

The P-value is greater than 0.05 so we do not have the 95% confidence we wanted to confirm a difference in the lab means. This confidence interval could be reduced with more samples taken next time and analyzed by both labs.

68Analyze Phase

Tests of Variance

Tests of variance are used for both normal and non-normal data.

Normal Data– 1 Sample to a target– 2 Samples – F-Test– 3 or More Samples Bartlett’s Test

Non-Normal Data– 2 or more samples Levene’s Test

The null hypothesis states there is no difference between the standard deviations or variances.

– Ho: σ1 = σ2 = σ3 … – Ha = at least on is different

69Analyze Phase

1-Sample Variance

A 1-sample variance test is used to compare an expected population variance to a target.

If the target variance lies inside the confidence interval, fail to reject the null hypothesis.

– Ho: σ2Sample = σ2

Target

– Ha: σ2Sample ≠ σ2

Target

Use the sample size calculations for a 1 sample t-test since they are rarely performed without performing a 1 sample t-test as well.

Stat > Basic Statistics > Graphical Summary

70Analyze Phase

1-Sample Variance

1. Practical Problem:• We are considering changing supplies for a part that we

currently purchase from a supplier that charges a premium for the hardening process and has a large variance in their process.

• The proposed new supplier has provided us with a sample of their product. They have stated they can maintain a variance of 0.10.


Ho: σ2 = 0.10 or Ho: σ = 0.31

Ha: σ2 ≠ 0.10 Ha: σ ≠ 0.31

3. 1-sample varianceα = 0.05 β = 0.10

71Analyze Phase

1-Sample Variance

4. Sample size• Open the Minitab worksheet: Exh_Stat.MTW• This is the same file used for the 1 Sample t example.

– We will assume the sample size is adequate.

5. State Statistical solution

Stat > Basic Statistics > Graphical Summary

72Analyze Phase

1-Sample Variance

Recall the target standard deviation is 0.31.

5.04.84.64.4

Median

Mean

5.15.04.94.84.74.6



Minimum 4.4000

A-Squared

1st Quartile 4.6000Median 4.70003rd Quartile 5.0500Maximum 5.1000

95% Confidence I nterval for Mean

4.5989

0.33

4.9789

95% Confidence I nterval for Median

4.6000 5.0772

95% Confidence I nterval for StDev

0.1670 0.4736

P-Value 0.442

Mean 4.7889StDev 0.2472


Summary for Values

73Analyze Phase

Test of Variance Example

1. Practical problem:

We want to determine the effect of two different storage methods on the rotting of potatoes. You study conditions conducive to potato rot by injecting potatoes with bacteria that cause rotting and subjecting them to different temperature and oxygen regimes. We can test the data to determine if there is a difference in the standard deviation of the rot time between the two different methods.

2. Statistical problem:

Ho: σ1 = σ2

Ha: σ1 ≠ σ2

3. Equal variance test (F-test since there are only 2 factors.)

74Analyze Phase

Test of Variance Example

4. Sample size:

α = 0.05 β = 0.10

Stat > Power and Sample Size > One-Way ANOVA…


EXH_AOV.MTW


One-way ANOVA

Alpha = 0.05 Assumed standard deviation = 1 Number of Levels = 2

Sample Maximum

Size Power SS Means Difference

50 0.9 0.214350 0.654752

The sample size is for each level.

75Analyze Phase

Normality Test – Follow the Roadmap

5. Statistical solution:

Stat>Basic Statistics>Normality Test

76Analyze Phase

Normality Test – Follow the Roadmap

Ho: Data is normalHa: Data is NOT normal

Rot 1

Perc

ent

8765432

99.9

99

95

90

80706050403020

10

5

1

0.1

Mean

0.559

4.871StDev 0.9670N 100AD 0.306P-Value

Probability Plot of Rot 1Normal

Stat>Basic Stats> Normality Test(Use Anderson Darling)

77Analyze Phase


Stat>ANOVA>Test for Equal Variance

78Analyze Phase


Fact

ors


2

1

1.41.31.21.11.00.90.80.7

Fact

ors

Rot 1

2

1

765432

F-Test

0.469


Levene's Test


Test for Equal Variances for Rot 1

Use F-Test for 2 samples normally distributed data.

P-Value >0.05 (.298) Assume Equal Variance

Use F-Test for 2 samples normally distributed data.

P-Value >0.05 (.298) Assume Equal Variance

6. Practical Solution:The difference between the standard deviations from the two samples is not significant.

79Analyze Phase

Normality Test

Perform another test using the column Rot.

Rot

Perc

ent

35302520151050-5

99

95

90

80

70

605040

30

20

10

5

1

Mean

0.586


Probability Plot of RotNormal

The P-Value is > 0.05We can assume our data is

normally distributed.

The P-Value is > 0.05We can assume our data is

normally distributed.

80Analyze Phase

Test for Equal Variance (Normal Data)

Test for equal variance using Temp as factor.

81Analyze Phase


Tem

p


16

10

12108642

Tem

p

Rot

16

10

2520151050

F-Test

0.824


Levene's Test


Test for Equal Variances for Rot

Ho: σ1 = σ2

Ha: σ1≠ σ2

P-Value > 0.05, There is no statistically significant difference.

Ho: σ1 = σ2

Ha: σ1≠ σ2

P-Value > 0.05, There is no statistically significant difference.

82Analyze Phase


Use F- Test for 2 samples of normally distributed

data.

Use F- Test for 2 samples of normally distributed

data.

83Analyze Phase

Continuous Data - Normal

84Analyze Phase

Test For Equal Variances

Stat>ANOVA>Test for Equal Variance

85Analyze Phase

Test For Equal Variances Graphical Analysis

p value > 0.05 shows insignificant difference between variance


Temp Oxygen

16

10

10

6

2

10

6

2

140120100806040200

Bartlett's Test

0.858


Levene's Test


Test for Equal Variances for Rot

86Analyze Phase

Test For Equal Variances Statistical Analysis

Test for Equal Variances: Rot versus Temp, Oxygen

95% Bonferroni confidence intervals for standard deviations

Temp Oxygen N Lower StDev `Upper 10 2 3 2.26029 5.29150 81.890 10 6 3 1.28146 3.00000 46.427 10 10 3 2.80104 6.55744 101.481 16 2 3 1.54013 3.60555 55.799 16 6 3 1.50012 3.51188 54.349 16 10 3 3.55677 8.32666 128.862

Bartlett's Test (normal distribution)Test statistic = 2.71, p-value = 0.744

Levene's Test (any continuous distribution)Test statistic = 0.37, p-value = 0.858

Use this if data is normal

and for Factors < 2

Use this if data is non normal

for factors > 2

87Analyze Phase

Tests for Variance Exercise: Solution

First we want to do a graphical summary of the two samples from the 2 suppliers.

88Analyze Phase


In “Variable” enter ppm VOC

In “By Variable” enter RM Supplier

We want to see if the 2 samples are from normal populations.

89Analyze Phase


The P-value is greater than 0.05 for both Anderson-Darling Normality Tests so we conclude the samples are from normally distributed populations because we “failed to reject” the null hypothesis that the data sets are from normal distributions.

50454035302520

Median

Mean

424038363432


Variance 50.265Skewness 0.261735Kurtosis -0.091503N 12

Minimum 25.000

A-Squared



33.079

0.33

42.088


33.263 42.000


5.022 12.038

P-Value 0.465

Mean 37.583StDev 7.090


Summary for ppm VOCRM Supplier = A

50454035302520

Median

Mean

37.535.032.530.027.525.0



Minimum 19.000

A-Squared


95% Confidence Interval for Mean

26.325

0.49

34.675

95% Confidence Interval for Median

25.000 37.000

95% Confidence Interval for StDev

4.655 11.157

P-Value 0.175



Summary for ppm VOCRM Supplier = B

Are both Data Sets are Normal?

90Analyze Phase


91Analyze Phase


For “Response” enter ppm VOC

For “Factors” enter RM Supplier

Note Minitab defaults to 95% confidence interval which is exactly the level we want to test for this problem.

92Analyze Phase


Because the 2 populations were considered to be normally distributed, the F-test is used to evaluate whether the variances (standard deviation squared) are equal.

The P-value of the F-test was greater than 0.05 so we “fail to reject” the null hypothesis.

So once again in English: The variances are equal between the results from the two suppliers on our product’s ppm VOC level.

RM

Supplier


B

A

141210864

RM

Supplier

ppm VOC

B

A

50454035302520

F-Test

0.890


Levene's Test


Test for Equal Variances for ppm VOC

93Analyze Phase

Purpose of ANOVA

Analysis of Variance (ANOVA) is used to investigate and model the relationship between a response variable and one or more independent variables.

Analysis of variance extends the two sample t-test for testing the equality of two population means to a more general null hypothesis of comparing the equality of more than two means, versus them not all being equal.

– The classification variable, or factor, usually has three or more levels (If there are only two levels, a t-test can be used).

– Allows you to examine differences among means using multiple comparisons.

– The ANOVA test statistic is:

withinS

between S

withinSS Avg

between SS Avg 2

2

94Analyze Phase

What do we want to know?

Is the between group variation large enough to be distinguished from the within group variation?

μ1 μ2

delta(δ)

(Between Group Variation)

Within Group Variation(level of supplier 1)

Total (Overall) Variation

X

X

XX

X

XX

X

X

95Analyze Phase

Calculating ANOVA

g

1j

nj

1i

2ij )X(X

Variation Total

g

1j

2nj )X(Xj

Variation GroupBetween

g

1j

nj

1i

2ij )X(X

Variation GroupWithin

Where:g- the number of groups (levels in the study)xij = the individual in the jth groupnj = the number of individuals in the jth group or level = the grand meanXj = the mean of the jth group or levelX

delta(δ)

(Between Group Variation)

Within Group Variation

Total (Overall) Variation

96Analyze Phase

Alpha Risk and Pair-Wise t-tests

The alpha risk increases as the number of means increases with a pair-wise t-test scheme. The formula for testing more than one pair of means using a t-test is:

risk alpha 30%or

30.00.05-1-1

:0.05αan and means of pairs 7for so,

means of pairs ofnumber k where

α11

7

k

97Analyze Phase

Three Samples

We have three potential suppliers that claim to have equal levels of quality. Supplier B provides a considerably lower purchase price than either of the other two vendors. We would like to choose the lowest cost supplier but we must ensure that we do not effect the quality of our raw material.

Supplier A Supplier B Supplier C

3.16 4.24 4.58

4.35 3.87 4.00

3.46 3.87 4.24

3.74 4.12 3.87

3.61 3.74 3.46

We would like test the data to determine whether there is a difference between the three suppliers.

We would like test the data to determine whether there is a difference between the three suppliers.

File>Open Worksheet > ANOVA.MTW

98Analyze Phase

Follow the Roadmap…Test for Normality

Supplier C

Perc

ent

5.04.54.03.53.0

99

95

90

80

70

605040

30

20

10

5

1

Mean

0.910


Probability Plot of Supplier CNormal

Supplier B

Perc

ent

4.504.254.003.753.50

99

95

90

80

70

605040

30

20

10

5

1

Mean

0.385


Probability Plot of Supplier BNormal

Supplier A

Perc

ent

4.54.03.53.02.5

99

95

90

80

70

605040

30

20

10

5

1

Mean

0.568


Probability Plot of Supplier ANormal

All three suppliers samples are normally distributed.

Supplier A P-Value 0.568Supplier B P-Value 0.385Supplier C P-Value 0.910

99Analyze Phase

Test for Equal Variance…

Test for equal variance (must stack data first):

Supplie

rs


Supplier C

Supplier B

Supplier A

1.81.61.41.21.00.80.60.40.20.0

Bartlett's Test

0.568


Levene's Test


Test for Equal Variances for Data

100Analyze Phase

ANOVA Minitab

Enter Stacked Supplier data in Response.

Click on Graph, Check Box plots

Stat>ANOVA>One-Way Unstacked

101Analyze Phase

ANOVA

What does this graph tell us?

Data

Supplier CSupplier BSupplier A

4.6

4.4

4.2

4.0

3.8

3.6

3.4

3.2

3.0

Boxplot of Supplier A, Supplier B, Supplier C

102Analyze Phase

ANOVA Session Window

Test for Equal Variances: Suppliers vs ID

One-way ANOVA: Suppliers versus ID

Analysis of Variance for Supplier

Source DF SS MS F P

ID 2 0.384 0.192 1.40 0.284

Error 12 1.641 0.137

Total 14 2.025

Individual 95% CIs For Mean

Based on Pooled StDev

Level N Mean StDev ----------+---------+---------+------

Supplier 5 3.6640 0.4401 (-----------*-----------)

Supplier 5 3.9680 0.2051 (-----------*-----------)

Supplier 5 4.0300 0.4177 (-----------*-----------)

----------+---------+---------+------

Pooled StDev = 0.3698 3.60 3.90 4.20

Normal data p-value > .05 No Difference

Normal data p-value > .05 No DifferenceStat>ANOVA>One Way

103Analyze Phase

ANOVA

Test for Equal Variances: Suppliers vs ID

One-way ANOVA: Suppliers versus ID

Analysis of Variance for Supplier

Source DF SS MS F P

ID 2 0.384 0.192 1.40 0.284

Error 12 1.641 0.137

Total 14 2.025

Individual 95% CIs For Mean

Based on Pooled StDev

Level N Mean StDev ----------+---------+---------+------

Supplier 5 3.6640 0.4401 (-----------*-----------)

Supplier 5 3.9680 0.2051 (-----------*-----------)

Supplier 5 4.0300 0.4177 (-----------*-----------)

----------+---------+---------+------

Pooled StDev = 0.3698 3.60 3.90 4.20

D/N 1 2 3 41 161.40 199.50 215.70 224.602 18.51 19.00 19.16 19.253 10.13 9.55 9.28 9.124 7.71 6.94 6.59 6.395 6.61 5.79 5.41 5.196 5.99 5.14 4.76 4.537 5.59 4.74 4.35 4.128 5.32 4.46 4.07 3.849 5.12 4.26 3.86 3.63

10 4.96 4.10 3.71 3.4811 4.84 3.98 3.59 3.3612 4.75 3.89 3.49 3.2613 4.67 3.81 3.41 3.1814 4.60 3.74 3.34 3.1115 4.54 3.68 3.29 3.06

F-CriticalF-Calc

104Analyze Phase

Sample Size

Let’s check and how much difference we can see with a sample of 5.


One-way ANOVA

Alpha = 0.05 Assumed standard deviation = 1 Number of Levels = 3

Sample Maximum

Size Power SS Means Difference

5 0.9 3.29659 2.56772

The sample size is for each level.

105Analyze Phase

ANOVA Assumptions

1. Observations are adequately described by the model.

2. Errors are normally and independently distributed.

3. Homogeneity of variance among factor levels.

In one-way ANOVA, model adequacy can be checked by either of the following:

4. Check the data for normality at each level and for homogeneity of variance across all levels.

5. Examine the residuals (a residual is the difference in what the model predicts and the true observation).

1. Normal plot of the residuals

2. Residuals versus fits

3. Residuals versus order

If the model is adequate, the residual plots will be structureless.

106Analyze Phase

Residual Plots

Stat>ANOVA>One-Way Unstacked>Graphs

107Analyze Phase

Histogram of Residuals

Residual

Frequency

0.60.40.20.0-0.2-0.4-0.6

5

4

3

2

1

0

Histogram of the Residuals(responses are Supplier A, Supplier B, Supplier C)

The histogram of residuals should show a bell shaped curve.

108Analyze Phase

Normal Probability Plot of Residuals

Residual

Perc

ent

1.00.50.0-0.5-1.0

99

95

90

80

70

605040

30

20

10

5

1

Normal Probability Plot of the Residuals(responses are Supplier A, Supplier B, Supplier C)

Normality plot of the residuals should follow a straight line.

Results of our example look good.

The normality assumption is satisfied.

109Analyze Phase

Residuals versus Fitted Values

Fitted Value

Resi

dual

4.054.003.953.903.853.803.753.703.65

0.75

0.50

0.25

0.00

-0.25

-0.50

Residuals Versus the Fitted Values(responses are Supplier A, Supplier B, Supplier C)

The plot of residuals versus fits examines constant variance.

The plot should be structureless with no outliers present.

Our example does not indicate a problem.

110Analyze Phase

ANOVA Exercise

Exercise objective: Utilize what you have learned to conduct and analyze a one way ANOVA using Minitab.

1. The quality manager was challenged by the plant director why the VOC levels in the product varied so much. The quality manager now wants to find if the product quality is different because of how the shifts work with the product.

2. The quality manager wants to know if the average is different for the ppm VOC of the product among the production shifts.

3. Use Data in columns “ppm VOC” and “Shift” to determine the answer for the quality manager at a 95% confidence level.

111Analyze Phase

ANOVA Exercise: Solution

First we need to do a graphical summary of the samples from the 3 shifts.

Stat>Basic Stat>Graphical Summary

112Analyze Phase


We want to see if the 3 samples are from normal populations.

In “Variables” enter ppm VOC

In “By Variables” enter Shift

113Analyze Phase


The P-value is greater than 0.05 for both Anderson-Darling Normality Tests so we conclude the samples are from normally distributed populations because we “failed to reject” the null hypothesis that the data sets are from normal distributions.

50454035302520

Median

Mean

35.032.530.027.525.022.520.0



Minimum 19.000

A-Squared



22.545

0.24

33.455


20.871 33.322


4.314 13.279

P-Value 0.658



Summary for ppm VOCShift = 3

50454035302520

Median

Mean

403836343230


Variance 25.411Skewness -0.74123Kurtosis 1.37039N 8

Minimum 25.000

A-Squared



30.411

0.37

38.839


30.614 37.322


3.333 10.260

P-Value 0.334



Summary for ppm VOCShift = 2

50454035302520

Median

Mean

50454035



Minimum 32.000

A-Squared



33.847

0.32

45.153


32.936 48.129


4.470 13.761

P-Value 0.446



Summary for ppm VOCShift = 1 P-Value 0.446

P-Value 0.658P-Value 0.334

114Analyze Phase


Now we need to test the variances

For “Response” enter ppm VOC”

For “Factors” enter Shift.

First we need to determine if our data has equal variances.

Stat > ANOVA > Test for Equal Variances…

115Analyze Phase


The P-value of the F-test was greater than 0.05 so we “fail to reject” the null hypothesis.

Shift


3

2

1

18161412108642

Bartlett's Test

0.440


Levene's Test


Test for Equal Variances for ppm VOC

Are the variances are equal…Yes!

116Analyze Phase


For “Response” enter ppm VOC

For “Factor” enter Shift

Also be sure to click “Graphs” to select “Four in one” under residual plots.

Also, remember to click “Assume equal variances” because we determined the variances were equal between the 2 samples.

We need to use the One-Way ANOVA to determine if the means are equal of product quality when being produced by the 3 shifts. Again, we want to put 95.0 for the confidence level.

Stat > ANOVA > One-Way…

117Analyze Phase


We must look at the residual plots to be sure our ANOVA analysis is valid.

Since our residuals look normally distributed and randomly patterned, we will assume our analysis is correct.

Residual

Perc

ent

100-10

99

90

50

10

1

N 24AD 0.255P-Value 0.698

Fitted Value

Resi

dual

403530

10

5

0

-5

-10

Residual

Fre

quency

1050-5-10

4.8

3.6

2.4

1.2

0.0

Observation Order

Resi

dual

24222018161412108642

10

5

0

-5

-10

Normal Probability Plot Residuals Versus the Fitted Values

Histogram of the Residuals Residuals Versus the Order of the Data

Residual Plots for ppm VOC

118Analyze Phase


Since the P-value of the ANOVA test is less than 0.05, we “reject” the null hypothesis that the mean product quality as measured in ppm VOC is the same from all shifts. We “accept” the alternate hypothesis that the mean product quality is different from at least one shift.

Since the confidence intervals of the means do not overlap between Shift 1 and Shift 3, we see one of the shifts is delivering a product quality with a higher level of ppm VOC.

hypothesis testing 1

Documents