Research Methods of Research Methods of Applied Linguistics Applied Linguistics

and Statisticsand Statistics(3)(3)

Types of researchTypes of research

Constructing Research Constructing Research DesignsDesigns

Research TypesResearch Types

Data sources:Data sources: primary, secondaryprimary, secondary Objectives:Objectives: descriptive, exploratory, correlational, descriptive, exploratory, correlational,

explanatoryexplanatory Time for data collection:Time for data collection: cross-sectional, longitudinal (panel, trend)cross-sectional, longitudinal (panel, trend) Data types:Data types: qualitative, quantitative (survey, qualitative, quantitative (survey,

experimental)experimental)

Primary: studies based on primary, or Primary: studies based on primary, or original, data sources, such as original, data sources, such as classroom observations or real classroom observations or real students, or their test scores, or their students, or their test scores, or their responses to a questionnaire.responses to a questionnaire.

Secondary: studies based on secondary Secondary: studies based on secondary sources such as other researherssources such as other researhers’’ books and articles, including library books and articles, including library research and literature reviews. research and literature reviews.

Descriptive: studies involving the Descriptive: studies involving the collection of data in order to test collection of data in order to test hypotheses or to answer questions hypotheses or to answer questions concerning the current status of the concerning the current status of the subjects of the study. subjects of the study.

Exploratory: studies conducted into an Exploratory: studies conducted into an issue or problem where there are few or issue or problem where there are few or no earlier studies to refer to. The focus no earlier studies to refer to. The focus is on gaining insights and familiarity for is on gaining insights and familiarity for later investigation. later investigation.

Correlational: The basic question for Correlational: The basic question for descriptive research is - "What are the values descriptive research is - "What are the values of a number of variables for a given sample of of a number of variables for a given sample of subjects. The basic research question for subjects. The basic research question for correlation research is - What is the correlation research is - What is the relationship between two or more variables relationship between two or more variables for a given set of subjects. Notice that we said for a given set of subjects. Notice that we said relationship between variables and not the relationship between variables and not the effect of one variable on another variable. effect of one variable on another variable.

In descriptive research we are just describing In descriptive research we are just describing our subjects in terms of one or more our subjects in terms of one or more variables, while in correlational research we variables, while in correlational research we are looking at the relationship between the are looking at the relationship between the variables.variables.

Explanatory: studies conducted in Explanatory: studies conducted in order to explain any behaviour in L2 order to explain any behaviour in L2 learning and teaching. It could be learning and teaching. It could be done by using questionnaires, group done by using questionnaires, group discussions, interviews, etc.discussions, interviews, etc.

Cross-sectional: studies in which Cross-sectional: studies in which groups of participants of different groups of participants of different ages are observed and compared at ages are observed and compared at a given time a given time

Longitudinal: It is research that studies a Longitudinal: It is research that studies a person or group over a set period of time, person or group over a set period of time, normally to track the effect of some normally to track the effect of some variable and as such permits causal variable and as such permits causal pathways to be determined . For example, pathways to be determined . For example, when trying to discover the effect of when trying to discover the effect of language aptitude on L2 proficiency, a language aptitude on L2 proficiency, a researcher may track 50 children over a researcher may track 50 children over a 30 year period in order to find a common 30 year period in order to find a common factor between those children who are factor between those children who are quicker and effective L2 learners. quicker and effective L2 learners.

Constructing Research Constructing Research DesignsDesigns

Determining the designDetermining the design

Classification of designsClassification of designs

Research Design 1: Research Design 1: Quantitative / QualitativeQuantitative / Qualitative

Quantitative research is empirical research Quantitative research is empirical research involving analysis of numerical data Its aim is involving analysis of numerical data Its aim is to determine the relationship between one to determine the relationship between one thing (an independent variable) and another thing (an independent variable) and another (a dependent or outcome variable) in a (a dependent or outcome variable) in a population. Quantitative research designs are population. Quantitative research designs are either descriptive (subjects usually measured either descriptive (subjects usually measured once) or experimental (subjects measured once) or experimental (subjects measured before and after a treatment). A descriptive before and after a treatment). A descriptive study establishes only associations between study establishes only associations between variables. An experiment establishes causality. variables. An experiment establishes causality.

Qualitative research is empirical research Qualitative research is empirical research involving analysis of data such as words involving analysis of data such as words (e.g., from interviews), pictures (e.g., (e.g., from interviews), pictures (e.g., video), or objects (e.g., an artifact). Its aim video), or objects (e.g., an artifact). Its aim is building theory. It is generally inductive is building theory. It is generally inductive in approach, is based originally on the in approach, is based originally on the naturalistic assumption that reality is mind naturalistic assumption that reality is mind dependent (i.e., can only be known as it is dependent (i.e., can only be known as it is interpreted and has "meaning" for the interpreted and has "meaning" for the observer), is usually of single-subject observer), is usually of single-subject design, and generally deals with nominal design, and generally deals with nominal data data

Philosophical basePhilosophical base

Phenomenology (quali.) Logical positivism (quan.)

Data from personal involvement

Data from experiment

Subjective Objective

Empathy Distance

Data in the form of string

Data in form of number

MethodologyMethodology

Synthetic Analytic

From part to whole From whole to part

Internal structure External structure

Process-oriented Product-oriented

Dynamic reality hypothesis

Static reality hypothesis

PurposePurpose

Induction Deduction

Hypothesis generating Hypothesis confirming

Exploratory, descriptive

Inferential, confirmatory, explanatory

Result of pattern or hypothesis

Formation of theory

Features for both Features for both designsdesigns

High validity, low reliability

Low validity, high reliability

Preference of content to form

Preference of form to content

Low generalizing capacity

High generalizing capacity

High explanatory power

Low explanatory power

Mixed design: BalancedMixed design: Balanced

Mixed design: Mixed design: UnbalancedUnbalanced

Literature review of quantitative and qualitative studies by Lazaraton (2000)

Current Trends in Research Methodology and Statistics in Applied Linguistics

Research Design: Research Design: experimental researchexperimental research

An experimental study is a study in An experimental study is a study in which the researcher manipulates which the researcher manipulates one or more independent variables one or more independent variables and measures their effect(s) on one and measures their effect(s) on one or more dependent variables while or more dependent variables while controlling the effect of extraneous controlling the effect of extraneous variables.variables.

Major components of Major components of experimental researchexperimental research

The populationThe population A comparison which involves at least 2 A comparison which involves at least 2

groups or 2 conditions. groups or 2 conditions. The treatmentThe treatment A treatment in which one or more A treatment in which one or more

independent variables are manipulated.independent variables are manipulated. The measurement of the treatmentThe measurement of the treatment The measurement of dependent The measurement of dependent

variable(s) as the result of the variable(s) as the result of the treatment. treatment.

Letters standing for Letters standing for componentscomponents

X: an experimental treatmentX: an experimental treatment (teaching method, exposure to a (teaching method, exposure to a materialmaterial……))

－－ : no treatment: no treatment O: measurement of the effects of the O: measurement of the effects of the

treatment or a testtreatment or a test R: randomization, or the random R: randomization, or the random

assignment of subjects to groups in assignment of subjects to groups in order to control for extraneous order to control for extraneous variables.variables.

Types of experimental Types of experimental studiesstudies

Pre-experimental Quasi-experimental True-experimentalOne-group posttest only study:

X O1

Posttest only nonequivalent groups:

EG: X O1

CG: O2

Posttest only equivalent groups

EG: R X O1

CG: R O2

One-group pre/post test study:

O1 X O2

Pre/post test nonequivalent groups:

EG: O1 X O2

CG: O3 O4

Pre/post test equivalent groups:

EG: R O1 X O2

CG: R O3 O4

No control group; no randomization

a control group; no randomization

a control group + randomization

Pre-experimental:Pre-experimental: single single group designgroup design

One-shot designOne-shot design One group pretest-posttest One group pretest-posttest

designdesign Time-sampling designTime-sampling design

Example for one-shot Example for one-shot designdesign

An English teacher used the An English teacher used the immersion teaching method for immersion teaching method for teaching English communicative teaching English communicative use. After 3 weeks, a test is use. After 3 weeks, a test is administered to the class. The class administered to the class. The class performs well on the test. performs well on the test.

X OX O It is also called a pilot study designIt is also called a pilot study design

One group pretest-One group pretest-posttest designposttest design

It is also called repeated It is also called repeated measures. measures. It uses the subjects as It uses the subjects as their own controls and eliminates their own controls and eliminates the need for a control group design.the need for a control group design.

StrengthStrength: It is efficient because : It is efficient because some extraneous variables, attrition some extraneous variables, attrition and loss of subjects controlled. and loss of subjects controlled.

WeaknessWeakness: validity might be affected : validity might be affected by history, attrition and maturation.by history, attrition and maturation.

Time-sampling designTime-sampling design

It is also referred to as time-series It is also referred to as time-series designs.designs.

OO11 O O22 O Onn X O X O44 O O55 O Onn

OO11 X O X O22 O O33 -- O -- O44 OO55 X O X O66 OO77 -- O -- O88

Quasi-experimental: posttest only

nonequivalent groups

• EG: X O1

CG: O2

• O1-O2

• The treatment effect could also be attributed to a series of extraneous variables.

Posttest/posttest nonequivalent groups

EG: O1 X O2

CG: O3 O4

(O(O22-O-O11) ) –– (O (O44-O-O33)) Without randomization, the Without randomization, the

confounding effects cannot be fully confounding effects cannot be fully controlled. controlled.

True-experimental: posttest only equivalent groups

EG: R X O1

CG: R O2

O1-O2

The design assumes that the 2 groups are identical on all relevant features, but it is still sensitive to selection bias, mortality, Hawthorn effects.

Pre/post test equivalent groups

EG: R O1 X O2

CG: R O3 O4

Factorial DesignsFactorial Designs

Factorial designs are similar to the Factorial designs are similar to the true experimental designs and true experimental designs and include all of the elements found in include all of the elements found in those designs, such as those designs, such as randomization, pre- and post-testing, randomization, pre- and post-testing, and treatments. The difference is and treatments. The difference is that the effects of independent that the effects of independent variables and a moderating variable variables and a moderating variable may be tested at the same time. may be tested at the same time.

Example for a factorial Example for a factorial designdesign

In a study to test for the effect of language In a study to test for the effect of language laboratory training on pronunciation, it is also laboratory training on pronunciation, it is also decided to measure for the effect of such decided to measure for the effect of such training on learners of different proficiency training on learners of different proficiency levels, as well as the effect of different kinds levels, as well as the effect of different kinds of language laboratory practice. of language laboratory practice.

Types of practice (contextualized / Types of practice (contextualized / decontextualized drills) , proficiency levels of decontextualized drills) , proficiency levels of the learner (beginners / advanced learners )the learner (beginners / advanced learners )

Exemplary Figure for Exemplary Figure for Factorial DesignFactorial Design

Experimental study by Experimental study by Robb, et al. (1986)Robb, et al. (1986)

Salience of feedback on error and its effect on EFL writing quality

Types of feedback: complete correction, Types of feedback: complete correction, coded feedback; yellow highlighting of coded feedback; yellow highlighting of errors, listing the number of errorserrors, listing the number of errors

The compositions were graded on 19 The compositions were graded on 19 features which were collapsed into 7 features which were collapsed into 7 categories. categories.