Descriptive Studies

A descriptive study is concerned with conditions or relationships that exist, opinion that are help, processes that are going on, effect that are evident, or trends that are developing.

It is primarily concerned with present, although it often considers past events and influences as they relate to current conditions.

In carrying out a descriptive research project, the researcher does not manipulate the variable, decide who receive the treatment, or arrange an event to happen

The researcher has no control over the variables, the researcher cannot assign the participants to control and experimental groups, the researcher doesn’t manipulate the environment or teaching method

Types of Descriptive Methods

1. Survey 2. Interrelational It deals with the relationship between

variables, the testing of a hypothesis, and the development of generalization, principles, and theories that have universal validity.

Developmental

Survey method

The survey method gathers data from a relatively large number of cases at a particular time. It is not concerned with characteristics of individuals as individuals.

Ninety-nine percent of American have at least one television set .

99% of students entering a university remain to graduate.

In 1977, 99% of Iranians vote for The Islamic Republic.

68% of graduate students haven’t found a decent job yet.

Factors that should be taken into

consideration in a Survey method 1. The specification of the exact purpose of the study.

Teacher views about teaching English (too broad)

2. The type of information to be obtained: Fact (e.g., gender, ethnicity, race, income, years of

education) opinion( respondent’s preferences, feelings, likes and

dislikes) behavior (how many time has a person attempted a

particular activity?)3. The instrument to be used in data collectionQuestionnaireInterviews of different typesobservations

Types of Survey

1. School survey -- to gather detailed information for judging the effectiveness of instructional

facilities, curriculum, teaching and supervisory personnel, quality and quantity of services.

Setting for learning Characteristics of educational personnel The nature of students

2. Social Survey Public opinions, attitudes, and preferences The television viewing habits of school children Health services Employment Public opinion surveys People’s opinion about Foreign aid programs, the adequacy of the public schools, the

incumbent president

Interrelational Methods

Investigating the relationship among existing factors.

1. Case studyThe data are collected about the present status,

past events, and environmental factors which contribute to the identity, individuality, and the behavioral patters of the unit.

2. Field studyThe researcher directly observes a naturally

occurring event ( Direct observation3. Correlational StudiesCorrelation indicates the strength and direction of a

linear relationship between variables.

Correlation and Significance

Is there a relationship between two variables/data?

What is the direction of the relationship?

What is the magnitude?

Pearson’s product moment coefficient correlation: -1.0 to +1.0

Ex post facto Design‘from what is done afterwards’

You do not randomly assign participants to different groups.

You are purposely putting them in a particular group based on some prior thing they have

Studies that investigate possible cause and effect relationships by observing an existing condition or state of affairs and searching back in time for plausible causal factors.

Characteristics of Ex Post Facto

Researcher takes the effect/dependent variable and examines it retrospectively

Establishes causes, relationships or associations and their meanings.

Researcher has little to no control over independent variables.

Flexible by nature.

When to use this?

You can use this where more powerful experimental designs are not possible; when you are unable to select, control and manipulate the factors necessary to study cause and effect relationships directly, or when control variables except a single independent variable may be unrealistic and artificial.

Causal –Comparative Studies

The researcher can go further and determine the reason for or the cause of the current status of the phenomenon under investigation.

It enables researchers to investigate possible cause-effect relationships by observing existing consequences and searching back through the data for plausible causal factors.

Causal-comparative (ex post facto) research

The independent variable (IV) is not manipulated; it has already occurred

Independent variables sometimes called “attribute variables”

Less costly and time-consuming to conduct

Establishing cause-effect relationships is more difficult than in experiments

The Post Hoc Fallacy

The conclusion that because two factors go together, one must be the cause and the other the effect.

The numbers of years of education and earned income

Cigarette-cancer

Causal comparative vs. correlational research

Causal comparative– Attempts to identify

cause-effect relationships

– At least one independent variable

– Two or more groups

– Involves a comparison

Correlational– No attempt to

understand cause and effect

– Two or more variables

– Only one group

Comparison to experiments

Causal comparative

– Individuals already in groups before study begins

– Independent variable has already occurred

– Independent variable is not manipulated

Experiment– Individuals

randomly assigned to groups (e.g., treatment or control)

– Independent variable manipulated by the researcher

Examples of non-manipulated independent

variables Age Sex

Ethnicity “Learning style”

Socioeconomic status (SES) Parent educational level

Family environment Type of school attended

Follow- up Study

It investigates individual who have left an institution after having completed a program, a treatment, or a course of study.

The study concerns what has happened to them and what impact the institution and its program has had on them.

By examining their status or seeking their opinions, one may get some ideas of the adequacy or inadequacy of the institution's program.

Developmental Methods ( Chapter Four of Zoltan

Dornyei) They are concerned with changes that take place

over time. In such studies, researchers describe variables in

the course of their development over time. Language acquisition Language development

1. Longitudinal (the ongoing examination of people or phenomena over time)

2. Cross-sectional (a snap-shot-like analysis of the target phenomenon at one particular point in time, focusing on a singl time interval

Longitudinal Studies

1. Panel study (prospective longitudinal study; cohort study) It involves taking a group of people and following their

development by multiple investigations over a period of time It allows us to collect information about change at the micro level

as it really happens. Drawbacks Expensive and time consuming Attrition ( a number of participant may drop out of the panel in

the successive waves because of non-availability, changing address or telephone number,; panel members may become ill or simply unwilling to continue because of a lack of time or loss of interest

Panel conditioning altering the panel members’ behavior and responses ( Howthorne

effect)

2. Trend studies(Repeated cross-

sectional studies) Administering repeated questionnaire survey to

different samples of respondents. If the subsequent waves examine samples that

are representative of the same population, then the result can be seen to carry longitudinal information at the aggregate level (for the whole group rather than for the individuals.

They are usually cheaper and easier to arrange and conduct

They do not suffer attrition or conditionning

3. Retrospective longitudinal studies

They gather information during a single investigation in which respondents are asked to think back and answer questions about the past.

4. Simultaneous cross-sectional studies

It is conducted with different age groups

Experimental Studies

Research in which the investigator deliberately controls and manipulates the independent variable to observe the effect of that change on another the dependent variable.

How and when to useExperimental Research

Most commonly used in Educational research.

You vary the independent variable and look for the effect it has on the dependent variable.

Experimental Research

Purpose– To make causal inferences about the

relationship between the independent and dependent variables

Characteristics– Direct manipulation of the independent

variable

– Control of extraneous variables Eliminate the variable from the study Statistically adjust for the effect of the variable

Experimental Validity

Internal validity– The extent to which the independent variable,

and not other extraneous variables , produced the observed effect on the dependent variable

External validity– The extent to which the results are

generalizable

Threats to Internal Validity

History– Extraneous events have an effect on the

subjects’ performance on the dependent variable

– The crash of the stock market, 9-11, the invasion of Iraq, etc.

Selection– Groups that are initially not equal due to

differences in the subjects in those groups

– Positive and negative attitudes, high and low achievers, etc.

Threats to Internal Validity

Maturation– Changes experienced within the subject over time

Pretesting– The effect of having taken a pretest

Threats to Internal Validity Subject attrition

– Differential loss of subjects from groups

Statistical regression– The natural movement of extreme scores toward the

mean

Threats to Internal Validity

Subject effects– The effects of being aware that one is involved

in a study

Hawthorne effect

John Henry Effect

A threat to internal validity wherein research participants in the control group try harder just because they are in the control group

Experimenter Bias Effect

The intentional or unintentional influence that an experimenter (researcher) may exert on a study

External Validity

The extent to which results can be generalized from a sample to a particular population.

Question – Why would really good internal validity often result in poor external validity?

External Validity

Factors affecting external validity– Subjects

Representativeness of the sample in comparison to the population

Personal characteristics of the subjects

– Situations - characteristics of the setting Specific environment Special situation Particular school

Overview of Experimental Research

Purpose is to investigate cause-and-effect relationships among variables

– Experimental groups vs. control groups

– Each group of participants receives a different instruction

– Always involves manipulation of the independent variable

Answers the question “What will be?”

Experimental Design

Types of designs (Campbell & Stanley, 1963)

– Pre-experimental

– True experimental

– Quasi-experimental

Pre-experimental designs

Weak experimental designs in terms of control

No random sampling Threats to internal and external validity

are significant problems

Pre-Experimental Designs

One Shot Case Study To attempt to explain a consequence by an

antecedent.

X T

Pre-Experimental Designs

One Group Pretest-Posttest Design

To evaluate the influence of a variable.

T1 X T2

Pre-Experimental Designs

Intact Group

GROUP I T1 X T2

GROUP II T1 X T2

True Experimental Designs

Important terminology– Random assignment

Subjects placed into groups by random Ensures equivalency of the groups

– Random selection of subjects Subjects chosen from population by

random Ensures generalizability to the population

from which the subjects were selected (i.e. external validity)

True Experimental Designs

Pretest-Posttest Control Group Design

T1 X T2

RT3 O T4

True Experimental Designs

Solomon Four Group Design To minimize the pre-testing effect.

T1 X T2

T1 T2

R

X T

T

Quasi-Experimental Designs

Nonrandomized Control Group Pretest-Posttest Design

To investigate a situation where random selection and assignment are not possible.

T1 X T2 T1 O T2

Quasi-Experimental Designs

Time Series Experimental Design To determine the influence of a variable introduced

only after a series of initial of observations and only where one group is available.

T1 T2 X T3 T4 X T5 T6 X

Quasi-Experimental Designs

Equivalent Time Series Design

T1 T2 X T3 T4 O T5 T6 X T7 T8 O T9 T10

Useful Terms

Correlation- Indicates the strength and direction of a linear relationship between two random variables.

Ex post facto- from the Latin for "from something done afterward"

Quasi- the prefix 'quasi' denotes methods that are "almost" or "socially approximate"

Variable- a measurable factor, characteristic, or attribute of an individual or a system

Data Collection

Sampling

TARGET POPULATION

SAMPLE UNIT

SAMPLE

• A population can be defined as a group of people whom the study is about.

• Because there is very rarely enough time or money to gather information from everyone or everything in a population, the goal becomes finding a representative sample (or subset) of that population.

• The sample is the group of participants whom the researcher actually examines in an empirical investigation.

•Probability sample/Random sampling – a method of sampling that uses of random selection so that all units/ cases in the population have an equal probability of being chosen.

• Non-probability sample/non-random sampling – does not involve random selection and methods are not based on the rationale of probability theory.

Types of SamplingTypes of Sampling

Probability (Random) Samples Simple random sample Systematic random sample Stratified random sample Cluster sample

Probability (Random) Samples Judgmental sampling Quota sampling Convenience sampling Snowball sampling

SIMPLE RANDOM SAMPLING• Each member of the population has an equal chance of being

included in the sample.• Applicable when population is small, homogeneous & readily

available• Each element of the frame thus has an equal probability of

selection.• A table of random number or lottery system is used to

determine which units are to be selected.

Advantage Easy method to use No need of prior information of population Equal and independent chance of selection to every element

Disadvantages If sampling frame large, this method impracticable. Does not represent proportionate representation

Techniques of selecting a simple random sample:

1. Lottery or fish bowl technique: The name or identifying number of each item in the population is recorded on a slip of paper and placed in a box - shuffled – randomly choose required sample size from the box.i.The process is relatively easy for small population but relatively difficult and time consuming for a large population

2. Table of random number:

Each item is numbered and a table of random numbers is used to select the members of the sample. There are many software programs, such as MINITAB and Excel with routines that will randomly select a given number of items from the population.

Simple Random Sampling (SRS)

In order to get your sample, you;

a.Assign a number from 001 to 500 to each students,b.use a table of randomly generated numbers (RandomNumber Tables)

Suppose your college has 500 students (population) and you need to conduct a short survey on the quality of the food served in the cafeteria. You decide that a sample of 70 students (sample) should be sufficient for your purposes.

ii. Decide on the first number and also the nth number

iii. Afterwards, every Nth must be drawn until the

total sample has been drawn.

2. Systematic (Random) Sampling

• When the researcher has no means of identifying the participants in advance and thus their names cannot be “put in a hat”.

• The researcher selects every nth number from a determined point

Steps:

 •SYSTEMATIC RANDOM SAMPLING:

In which one or two items are selected randomly, but other items are selected by adding the average sampling interval to the item selected randomly.

•STRATIFIED RANDOM SAMPLING:•A population is divided into homogenous, mutually exclusive subgroups, called strata and a sample is selected from each stratum.

•Goal: To guarantee that all groups in the population are adequately represented.

The population is divided to two or more groups called strata, according to some criterion, such as geographic location, grade level, age, or income, and subsamples are randomly selected from each strata.

Sunil Kumar

• To reduce the cost of sampling a population scattered over a large geographic area.

• To gather data quickly and cheaply at the expense of possible over – or under representing certain groups of people.

4. Cluster (Random) Sampling

- By the luck of the draw you will wind up with respondents who come from all over the state

Cluster (Random) Sampling

Steps:

• divides the population into groups or clusters- Within cluster- differences (heterogeneous)

- Between cluster– uniformity (homogenous)

• select clusters at random - all units within selected clusters are included

in the sample- No units from non-selected clusters are included in the sample

Non-Probability SamplesConvenience samples (ease of access)

sample is selected from elements of a population that are easily accessible

Purposive sample (Judgmental Sampling)

You chose who you think should be in the studyQuota SamplingSnowball Sampling (friend of friend….etc.)

The population is first segmented into mutually exclusive sub-groups, just as in stratified sampling.

Then judgment used to select subjects or units from each segment based on a specified proportion.

For example, an interviewer may be told to sample 200 females and 300 males between the age of 45 and 60.

It is this second step which makes the technique one of non-probability sampling.

In quota sampling the selection of the sample is non-random. For example interviewers might be tempted to interview those

who look most helpful. The problem is that these samples may be biased because not everyone gets a chance of selection. This random element is its greatest weakness and quota versus probability has been a matter of controversy for many years

CONVENIENCE SAMPLING:

The process of including whoever happens to be available at the time.

It also called “accidental” or “haphazard” sampling

Purposive/Judgmental Sampling

The process whereby the researcher selects a sample based on experience or knowledge of the group to be sampled.

SNOWBALL SAMPLING:

The sampling procedure in which the initial respondents are chosen by probability or non-probability methods, and then additional respondents are obtained by information provided by the initial respondents

How large should the sample Be?

Rules of thumb Correlational research: at least 30 participants Experimental studies: at least 15 participants in each group Multivariate procedures: at leas 100 participants Statistical Consideration The sample should have a “normal distribution. According to

Hatch and Lazarton (1991) the sample needs to include 30 or more people to have a normal distribution.

Safety margins Because of the possibility of attrition, include more individuals Reverse approach First approximate the expected magnitude and then determine the

sample size

Data Collection Instruments

Questionnaire Multi-item scales: Likert scales; Semantic differential

scales Open-ended questions Close ended questions: True-false items; multiple-choice

items