HYPOTHESIS FORMATION, TYPES OF ERROR AND

ESTIMATION

WHAT IS A HYPOTHESIS?

• A proposition, tentative assumption, or educated conjecture about some aspect of the world around us that is testable.

• Usually derived from theoretical framework

in quantitative approach

WHY USE HYPOTHESES?

• If research were limited to gathering facts, knowledge would hardly advance. We need to determine what is relevant and what is not.

• Hypothesis testing distinguishes scientific reasoning from everyday speculation and old wives’ tales

HYPOTHESIS TESTING

• Hypothesis Testing. An inferential procedure that uses sample data to evaluate the validity of a hypothesis about a population.

• That is: hypotheses relate to populations but we usually test them with samples.

SEQUENCE IN HYPOTHESIS TESTING

Scientific thinking possesses three essential steps:

1. The proposal of a hypothesis to account for a phenomenon.

2. The deduction from the hypothesis that certain phenomena should be observed in given circumstances.

3. The checking of this deduction by observation and testing.

STATING THE HYPOTHESIS• Hypotheses may be stated in the form of

proposed relationships (associations) or in terms of differences (comparisons).

• A relationship hypothesis would exist if we propose ‘that changes in demand for a specified good are related to changes in price of the same good’.

• A difference hypothesis is ‘that female employees take more sick days than male employees’.

STATISTICAL TESTS ASSOCIATED WITH THE TWO

TYPES OF HYPOTHESES

• Relationship hypotheses use statistical tests of correlation or association, and regression when prediction is involved.

• Difference hypotheses use statistical tests of differences.

CRITERIA FOR JUDGING HYPOTHESES

• Hypotheses should be clear and precise• Hypotheses should be testable with

operationalized variables• Hypotheses should state expected

relationships between variables explicitly • Hypotheses should be limited in scope • Hypotheses should be grounded in past

knowledge or reasonably consistent with known facts

LOGIC OF HYPOTHESIS TESTING

• We set up two competing statements or hypotheses, the null hypothesis and the alternate hypothesis. These hypotheses are mutually exclusive and exhaustive.

• How do we test or compare these competing hypotheses? This is the counterintuitive part. We try to disprove the Null Hypothesis.

THE NULL AND ALTERNATIVE HYPOTHESES

• The Null Hypothesis – symbolized as Ho

– Ho: The finding was simply a chance (random) occurrence – nothing really occurred

• The Alternate Hypothesis – symbolized as H1 – H1: The finding did not occur by chance but is real

(alternative version) – something did occur• The null hypothesis is assumed to be true

unless we find evidence to the contrary which then allows us to assume the alternate hypothesis is more likely correct.

MAKING THE JUDGEMENT CALL

• On what basis can we decide between the two competing hypotheses?

• In ‘traditional statistics’ something occurring with a probability equal to or less than .05 (= 5% = 1 chance in 20) is conventionally considered the border line for ‘unlikely’ to be chance. The significance level is the risk of rejecting a null hypothesis when the latter is correct.

NULL HYPOTHESIS TESTING

• The phrase ‘No Significant Difference (or Relationship’) in the Ho statement does not imply that there is no relationship or no difference, i.e. it does not mean absolute equality.

• It implies that any differences or relationships are within the range of chance or sampling error bounded by a set significance level and do not reach that stated significance level.

SIGNIFICANCE LEVELS• The words ‘no statistically significant

difference/relationship’ in the Ho statement gives us a baseline to determine whether there has been any further effect beyond the chance level.

• We can thus set precise limits - our significance levels - for the rejection of a null hypothesis. This is the basic principle on which most statistical tests are based.

STATISTICALLY SIGNIFICANT DIFFERENCES

• Statistically significant results occur rarely and should be equal to or less than the 5% level ( p < .05). i.e. occur beyond the significance level cut-offs (outside the 95% confidence limits).

• Strongly significant results may reach the 1% level (p < .01 or 1 in 100 by chance ) or even the .01% level (p < .001 or 1 in 1000 by chance). Any significance less than .001 is quoted in SPSS results as .000

SIGNIFICANCE LEVEL (ALPHA LEVEL)

• The significance level is often termed the alpha level.

• The significance level offers a probability level for our evidence to be unreasonable as a chance result.

• A minimum decision criterion of p < 0.05 (5% level for two-tailed tests) is recommended.

• BUT a chance result at this level will occur 5% of the times.

Null Hypothesis Retention and Rejection Areas 5% level

Z

-3 -1.96 -1 0 +1 +1.96 +3

Significance or alpha level cut off for 5%. Probability of chance occurrence within this area is 95%. Null hypothesis zone

Test statistic in here, reject null hypothesis. Only 2.5% or less values here

Test statistic in here, reject null hypothesis. Only 2.5% or less values here

Sampling Distribution

Retain null hypothesis zone

HYPOTHESIS TESTING SEQUENCE1. Set up a testable hypothesis in its null and

alternate forms.2. Set a level of significance and conduct the

study.3. If the probability does not reach the set level of

significance, assume that the treatment did not work, that there is no real group difference or relationship.

4. Then we retain Ho and consequently cannot accept the alternate hypothesis (H1).

5. Alternatively, if we achieve or go beyond the level of significance previously set then we can reject Ho and accept H1.

PROBABILITY AND “PROOF"• Statistics can never ‘prove’ anything • Statistical tests only assign a probability value to

the results you have, indicating the likelihood (or probability) that they come from random fluctuations in sampling. We are looking for a low probability value that they are random, i.e. p> .05 in order to claim significance.

• Never ever talk about ‘proving’ a hypothesis – If we achieve significance we have simply

found support for it at a particular level of probability.

PROBABILITY AND PROOF• Because we are using probability levels (levels of

significance) we can never prove a hypothesis as even at these levels a result may occur by chance occasionally.

• These chances are 1 in 20 for p > .05, 1 in 100 for p > .01, and 1 in 1000 for p > .001. So if we employ the 1% significance level (p > .01), for example, a difference will actually occur at this level 1 in 100 occasions over the long run simply by chance.

• If our result is at or beyond that level, we are hoping and assuming that it is not the one in a hundred result. But we can never know, it may be!!

PROBABILITY AND PROOF

If our observations would be very unlikely to occur if the null hypothesis were true, it follows that the null hypothesis is probably false, and consequently the alternate hypothesis is probably true.

Notice the nagging use of the word ‘probably’. Unfortunately, that is as far as hypothesis testing can take us.

SELECTING A SIGNIFICANCE LEVEL• The researcher has complete control over

selecting the value of this significance level. • While we recommend a minimum decision

criterion of p < 0.05 (5% level for two-tailed tests), you should be cautious about the blind adoption of this level.

• There are research contexts (pharmaceutical research) in which one would want to be more conservative. In these cases, the significance level might be lowered to 0.001 (0.01%) where more extreme values of a statistic would be required before non-chance factors were suspected.

Reports refer to the significance level of a finding in many ways. All of the following are equivalent:

- The finding is significant at the .05 level- The confidence level is 95 percent- The type I error rate is .05- The alpha level is .05 = .05- There is a 95 percent certainty that the result

is not due to chance- There is a 1 in 20 chance of obtaining this

result by chance- The area of the region of rejection is .05- The p–value is .05- p = .05

REFERRING TO STATISTICAL SIGNIFICANCE

POSSIBILITY OF ERRONEOUS DECISION

• The aim of statistical testing is to correctly reject a false Ho and correctly retain a true Ho.

• Since there is some level of error in every study, the possibility that our results are erroneous is directly related to our acceptable level of error.

• If we set alpha at 0.05 we are saying that we will accept 5% error, which means that if the study were to be conducted 100 times, we would accept as significant results that were chance ones in 5 studies on average.

• How do we then know that our study doesn’t fall in the 5% error category? We don’t!

PROBABILITY IN SPSS OUTPUT• Examine SPSS tabular output usually under column

headings such as ‘Sig.’ or ‘Two-tailed Sig’, or ‘Prob.’ for the probability (or ‘p-value’) of your results.

• This is the probability you quote as being the ‘level of significance’ associated with your results.

• Normally this p-value should be less than or equal to 0.05 to make the claim of ‘significant’ in your discussion.

• Treat the p-value as a measure of the confidence or faith

you have in your results being real (and not being due to chance fluctuations in sampling). But remember that by chance a result at the 5% level will occur 5% of the times.

DIRECTIONS AND TAILS• Hypotheses can be stated in a NON-

DIRECTIONAL or DIRECTIONAL form • Non-directional hypotheses

– states that one group differs from another on some characteristic, i.e., it does NOT specify the DIRECTION of the difference

– Example: • H0 – ‘that there is no statistically significant

difference between the number of mistakes made by male and female bank tellers’

• H1 – ‘that there is a statistically significant difference between the number of mistakes made by male and female bank tellers’

DIRECTIONS AND TAILS• Non-directional hypotheses use two-tailed tests.

– Do not specify the direction of difference. – Tail refers to the ends of the normal distribution.

• Thus the regions of rejection lie in both tails of the normal distribution

• Assuming an alpha level (significance level) of .05: – the rejection region to the right is marked by the

critical value of +1.96 and contains .025 of the cases – that to the left is at -1.96 and also contains .025 of

cases

DIRECTIONS AND TAILS• Directional hypotheses

– specifies the DIRECTION of the difference or deviation from the null value, i.e., that one group is higher, or lower, than another group on some attribute

– Example: • H0 - ‘that female bank tellers do not make

significantly fewer mistakes than male bank tellers’ • H1 - ‘that female bank tellers make fewer mistakes

than male bank tellers’

DIRECTIONS AND TAILS• Directional hypotheses use one-tailed tests.

– Assuming an alpha level of .05: the region of rejection is fixed entirely in the predicted tail of the distribution

– that tail alone must now contain .05 of the cases – the critical value now drops to a z-score of +1.65

• The probability remains the same (.05), but because it lies only in one tail of the distribution, the cut-off is a smaller critical value: 1.65 < 1.96.

• one-tailed tests offer a better chance to reject your null hypothesis

-1.96Z 0 +1.96Z

Significance (alpha) level cut offs for 5%.Probability of chance occurrence withinthis area is 95%. Null hypothesis zone.

ACCEPT NULL HYPOTHESIS

Test statistic in here, reject null hypothesis

Test statistic in here, reject null hypothesis

Sampling Distribution2.5% of values above +1.96Z

2.5% of values below -1.96Z

+1.81Z here

Testing the null hypothesis with a two-tailed test

Testing the null hypothesis with a one-tailed test

-2Z -1Z 0Z + 1Z

Sampling Distribution 5% of values above +1.65Z

1% of values above 2.33Z

95% of values in this area Retain Null Hypothesis

1.81Z here. Value exceeds 5% significance level. Reject null hypothesis

Directional TestsDirectional tests should be used with caution

because 1. they may allow the rejection of H0 when

the experimental evidence is weak, or 2. they may predict in the wrong tail! This

has consequences, of not only non-significant findings, but acute embarrassment!

THE CRITICAL REGION

• The Critical Region. The area of extreme sample values that are very unlikely to be obtained if the null hypothesis is true.

• The size of the critical region for rejection is determined both by the significance level (alpha level) chosen and whether we have a directional or non-directional hypothesis. Sample data that fall in the critical region will warrant rejection of the null hypothesis.

STATISTICAL ERRORS - TYPE I AND TYPE II

Type I and Type II Errors Type I error:

• You might say things are significantly different when they are not, i.e. rejecting the null when it is true. The probability of this is equal to the alpha or significance level.

– Type II error:• You may say that things are not

significantly different when they are and miss a significant relationship that really exists, i.e. accepting the null when it is false

CONFIDENCE LEVELS AND INTERVALS

• Confidence Interval. The range within which we believe the true population estimate to lie

• Confidence Level. The probability that the population parameter falls within the confidence interval

CONFIDENCE LEVELS AND INTERVALS • levels of significance may also be termed

confidence levels as we may be confident that a particular value falls within their boundary

• For example, the 95% confidence interval (CI) contains values that have a 95% or 0.95 probability, or 95 in 100, or 19 in 20 chances of occurring there. Remember SPSS provides this 95% CI under Descriptives

• That is, a CI is a range within which a certain value like a mean will fall at a certain level of probability. It is the converse of the Critical Region which is the area outside the CI range

If we want a 95% confidence interval, then we want 95% (.95) around the mean – which leaves 2.5% (.025) at each tail.

EXAMPLE OF CI

Critical area

CI

Critical area