Hypothesis Testing

Hypothesis Testing (Ht): Introduction

• After discussing procedures for data preparation and preliminary analysis, the next step for many studies is hypothesis testing.

• In this context, induction and deduction is fundamental to hypothesis testing.

HT Introduction Contd.

• Induction and deduction are used together in research reasoning.

• Researchers describe this process as the double movement of reflective thought.

• Induction occurs when we observe a fact and ask, :Why is this?’

• In answer to this we advance a tentative explanation (hypothesis).

HT Introduction Contd.

• The hypothesis is plausible if it explains the event or condition (fact) that prompted the question.

• Deduction is the process by which we test whether the hypothesis is capable of explaining the fact

HT Introduction Contd.

• Example:– You promote a product but sales don’t

increase. (Fact1)

– You ask the question “Why didn’t sales increase?. (Induction)

– You infer a conclusion (hypothesis) to answer the question: The promotion was poorly executed. (Hypothesis)

HT Introduction Contd.

– You use this hypothesis to conclude (deduce) that sales will not increase during a poorly executed promotion. You know from experience that ineffective promotion will not increase sales. (Deduction1)

– We deduce that a well-executed promotion will result in increased sales. (Deduction2)

– We run an effective promotion, and sales increase. (Fact2).

HT Introduction Contd.

• In most research, the process may be more complicated than this specific example suggests.

• For instance, – we often develop multiple hypotheses by

which to explain the problem in question;– Then we design a study to test all hypotheses

at once;– This is not only more efficient but also is a

good way to reduce the attachment (potential bias) of the researcher for any given hypothesis.

HT Introduction Contd.

• Inductive reasoning moves from specific facts to general, but tentative, conclusions.

• With the help of probability estimates, conclusions can be refined and results discussed with a degree of confidence.

• Statistical inference is an application of inductive reasoning.

• It allows us to reason from evidence found in the sample to conclusions we wish to make about the population.

HT Introduction Contd.

• Two major categories of statistical procedures are:– Inferential statistics and– Descriptive statistics (describing distributions)

• Under inferential statistics, two topics are discussed:– estimation of population values, and– testing statistical hypothesis

HT Introduction Contd.

• We evaluate the accuracy of hypotheses by determining the statistical likelihood that the data reveal true differences – not random sampling error;

• We evaluate the importance of a statistically significant difference by weighing the practical significance of any change that we measure.

Hypothesis: Definition

• A hypothesis is a hunch, assumption, suspicion, assertion, or an idea about a phenomenon, relationship or situation, the reality or truth of which we do not know;

• The above become the basis for an inquiry for a researcher.

• In most studies, the hypothesis will be based upon either previous studies, or on your own or some one else’s observation.

Definition contd.

• A hypothesis has at least three characteristics:

It is a tentative proposition; Its validity is unknown; and In most cases, it specifies a relationship

between two or more variables.

Functions of a Hypothesis

• The formulation of a hypothesis provides a study with focus. It tells you what specific aspects of a research problem to investigate;

• It tells you what data to collect and what not to collect, thereby providing focus to the study;

Functions contd.

• As it provides a focus, the construction of a hypothesis enhances objectivity in a study; and

• It may enable you to add to the formulation of theory. It helps you to specifically conclude what is true or what is false.

Characteristics of a Hypothesis

• A hypothesis should be simple, specific and conceptually clear:– To be able to develop a good hypothesis one

must be familiar with the subject area (the literature review is of immense help).

– The more insight one has into a problem, the easier it is to construct a hypothesis.

Characteristics contd.

• A hypothesis should be capable of verification:– Methods and techniques must be available for

data collection and analysis;– The researcher might, while doing research,

develop new techniques to verify it.

• A hypothesis should be related to the existing body of knowledge; and it must add to it.

Characteristics contd.

• A hypothesis should be operationalisable.

– It should be expressed in terms that can be measured. If it can not be measured, it cannot be tested and, hence no conclusions can be drawn.

Approaches to Hypothesis Testing

• Classical or sampling theory approach– This, the more established approach,

represents an objective view of probability in which the decision making rests totally on an analysis of available sampling data.

– A hypothesis is established; it is rejected or fails to be rejected, based on the sample data collected.

• Bayesian statistics approach

HT Approaches contd.

• Bayesian statistics are an extension of the classical approach.– They also use sampling data, but they go beyond to

consider all other available information which consists of subjective probability estimates stated in terms of degrees of belief.

– These subjective estimates are based on general experience rather than on specific collected data.

– Various decision rules are established, cost and other estimates can be introduced, and the expected outcome of combinations of these elements are used to judge decision alternatives.

Statistical Significance

• Following classical statistics approach, we accept or reject a hypothesis on the basis of sampling information alone.

• Since any sample will almost surely vary somewhat from its population, we must judge whether the differences are statistically significant or insignificant.

• A difference has statistical significance if there is good reason to believe the difference does not represent random sampling fluctuations only.

Statistical Significance: Example

• Honda, Toyota, and other auto companies produce hybrid vehicles using an advanced technology that combines a small gas engine with an electric motor.

• The vehicles run on an electric motor at slow speeds but shifts to both the gasoline motor and the electric motor at city and higher freeway speeds.

• Their advertising strategies focus on fuel economy.

SS Example contd.

• Let’s say that the hybrid Civic has maintained an average of about 50 miles per gallon (mpg) with a standard deviation o 10 mpg.

• Suppose researchers discover by analyzing all production vehicles that the mpg is now 51.

• Is this difference statistically significant from 50?

– Yes it is, because the difference is based on a census of the vehicles and there is no sampling involved.

– It has been demonstrated conclusively that the population average has moved from 50 to 51 mpg.

SS Example contd.

• Since it would be too expensive to analyze all of a manufacturer’s vehicles frequently, we resort to sampling.

• Assume a sample of 25 cars is randomly selected and the average mpg is 54.

• Is this statistically significant?– The answer is not obvious.– It is significant if there is good reason to

believe the average mpg of the total population has moved up from 50.

SS Example contd.

• Since the evidence consists only of a sample consider the second possibility: that this is only a random sampling error and thus is not significant.

• The task is to decide whether such a result from this sample is or is not statistically significant.

• To answer this question, one needs to consider further the logic of hypothesis testing.

The Logic of Hypothesis Testing

• In classical tests of significance, two kinds of hypotheses are used which are– The null hypothesis and – the alternative hypothesis

• The null hypothesis is used for testing.– It is a statement that no difference exists

between the parameter (a measure taken by a census of the population), and

– the statistic being compared to it (a measure from a recently drawn sample of the population).

The Logic of HT contd.

• Analysts usually test to determine whether there has been no change in the population of interest or whether a real difference exists.

• A null hypothesis is always stated in a negative form.– In the hybrid-car example, the null hypothesis

states that the population parameter of 50 mpg has not changed.

The Logic of HT contd.

• The alternative hypothesis holds that there has been a change in average mpg (i.e., the sample statistic of 54 indicates the population value probably is no longer 50).

• The alternative hypothesis is the logical opposite of the null hypothesis.– The hybrid-car example can be explored

further to show how these concepts are used to test for significance.

The Logic of HT contd.

• The null hypothesis (Ho): There has been no change from the 50 mpg average.

• The alternative (HA) may take several forms, depending on the objective of the researchers.

• The HA may be of the “not the same” or the “greater than” or “less than” form:– The average mpg has changed from 50.– The average mpg has increased (decreased)

from 50.

The Logic of HT contd.• These types of alternative hypotheses

correspond with two-tailed and one-tailed tests.

• A two-tailed test, or nondirectional test, considers two possibilities:– the average could be more than 50 mpg, or– It could be less than 50.

• To test this hypothesis, the region of rejection are divided into two tails of the distribution.

The Logic of HT contd.

• Such hypothesis can be expressed in the following form:– Null Ho:μ = 50 mpg

– Alternative HA:μ ≠ 50 mpg (not the same case)

(See figure on the next slide)

The Logic of HT contd.

• A one-tailed test, ore directional test, places the entire probability of an unlikely outcome into the tail specified by the alternative hypothesis. Such hypotheses can be expressed in the following form:– Null Ho:μ ≤ 50 mpg

– Alternative HA:μ > 50 mpg (greater than case)Or

– Null Ho:μ ≥ 50 mpg

– Alternative HA:μ < 50 mpg (less than case)

The Logic of HT contd.

• In testing these hypotheses, adopt this decision rule:

• Take no corrective action if the analysis shows that one cannot reject the null hypothesis.

• Note the language “cannot reject” rather than “accept” the null hypothesis.

• It is argued that a null hypothesis can never be proved and therefore cannot be “accepted”.

Type I and Type II Errors

• In the context of testing of Hypothesis, there are basically two types of errors we can make.

• We may reject Ho when Ho is true, and we may accept Ho when in fact Ho is not true.

• The former is known as Type I error and the latter as Type II error.

Errors contd.

• Type I error is denoted by α (alpha) known as α error, also called the level of significance of test;

• Type II error is denoted by β (beta) known as β error.

Errors contd.

• If type I error is fixed at 5%, it means that there are about 5 chances in 100 that we will reject Ho, when Ho is true.

• We can control Type I error just by fixing it at a lower level. – For instance, if we fix it at 1 percent, we will

say that the maximum probability of committing a type I error would only be 0.01.

Errors contd.

• With a fixed sample size n, when we try to reduce Type I error, the probability of committing Type II error increases. There is a trade off between the two

(β error = 1 – α error).

Statistical Testing Procedures

Testing for statistical significance involves

the following six-stage sequence:

1. State the null hypothesis (Ho):

Make a formal clear statement of the null hypothesis and also of the alternative hypothesis (Ha);

Stat. Testing Proc. contd.

2. Choose an appropriate statistical test:

Two types of tests are parametric (t-test) and nonparametric (Chi-square).

In choosing a test, one can consider how the sample is drawn, the nature of the population, and the type of measurement scale used.

Stat. Testing Proc. contd.

3. Select the desired level of significance:

The choice of the level of significance should be made before we collect the data. The most common level is .05, although .01 is also widely used. Other α levels such as .10, .25, or .001 are sometimes chosen.

Stat. Testing Proc. contd.

The exact level to choose is largely

determined by how much α risk one is willing to accept and the effect that this choice has on β risk. The larger the α, the lower is the β.

4. Compute the calculated difference value:

After the data are collected, use the formula for the appropriate significance test to obtain the calculated value.

Stat. Testing Proc. contd.

5. Obtain the critical test value:

After we compute the calculated t, Chi-square, or other measure, we must look up the critical value in the appropriate table for that distribution. The critical value is the criterion that defines the region of rejection from the region of acceptance of the null hypothesis.

Stat. Testing Proc. contd.

6. Interpret the test:For most tests, if the calculated value is larger than the critical value, we reject the null hypothesis and conclude that the alternative hypothesis is supported (although it is by no means proved). If the critical value is larger, we conclude we have failed to reject the null hypothesis.

Probability Values (p Values)

• According to the “interpret the test” step of the statistical test procedure, the conclusion is stated in terms of rejecting or not rejecting the null hypothesis based on a rejection region selected before the test is conducted.

• A second method of presenting the results of a statistical test reports the extent to which the test statistic disagrees with the null hypothesis.

p Values• This second method has become popular

because analysts want to know what percentage of the sampling distribution lies beyond the sample statistic on the curve, and most statistical computer programs report the results of statistical tests as probability values (p values).

• The p value is the probability of observing a sample value as extreme as, or more extreme than, the value actually observed, given that the null hypothesis is true.

p Values contd.

• The p value is compared to the significance level (α), and on this basis the null hypothesis is either rejected or not rejected.– If p value is less than the significance level,

the null hypothesis is rejected.– If p value is greater than or equal to the

significance level. The null hypothesis is not rejected.