8/3/2019 02ResDsnBB

1/28

Topics: Research Design

Basic issues of research design

Role of statistics in behavioral research

Classification of variables Quantification of variables (scales of

measurement)

Validity of interpretations of researchstudies

Types of research designs

8/3/2019 02ResDsnBB

2/28

Research Design Decisions

What kinds of subjects/participants and howmany?

What will subjects be asked to do? How many comparison groups if any?

What dependent/independent variables to

focus on? How and when subjects will be measured?

Where study will be conducted?

8/3/2019 02ResDsnBB

3/28

Design Issues: Subjects

Where did subjects come from?

What kinds of samples?

How many of intended subjects actually

supplied data? Were in final analysis?

If comparison groups, how were they

formed?

How motivated were subjects?

8/3/2019 02ResDsnBB

4/28

Design Issues: Data

Instrument quality

Question/data match

Independence of observations

Person/people collecting data

8/3/2019 02ResDsnBB

5/28

Design Issues: Study Context

Physical setting

Pretest sensitization

Treatment conditions

Subjects thoughts about the study

8/3/2019 02ResDsnBB

6/28

Descriptive and Inferential

Statistics

Descriptive Statistics: Methods used to obtain

indices that characterize or summarize data collected

Inferential Statistics: Methods that allow the

researcher to make inferences from a set of data

collected from a sample to a larger population.

8/3/2019 02ResDsnBB

7/28

Review of Terms

Research: a systematic approach to finding

answers to questions.

Research Design: a plan for gathering datafor answering specific research questions.

Statistics: the methods used on the data

collected to answer the research questions athand.

8/3/2019 02ResDsnBB

8/28

Basic Elements: Hypotheses

Hypothesis: a tentative statement

(educated guess) about the expectedrelationship between two or more variables.

State expected relationship or difference

between 2 variables

Be worthy of being tested

Be testable

Be brief and clear

8/3/2019 02ResDsnBB

9/28

Basic Elements: Variables

Variable: what is measured or varied. An

attribute or characteristic of a person (or object)

that can change from person to person. Independent

Dependent

Control

Intervening (mediator)

Moderator

8/3/2019 02ResDsnBB

10/28

Classification of Variables

Independent Variable: a variable that is manipulated,

measured or selected by the researcher in order to observe

its relation to the subject's "response on another variable.

An antecedent condition.

Dependent Variable: the variable that is observed and

measured in response to an independent variable.

8/3/2019 02ResDsnBB

11/28

Classification of Variables (Cont)

Control Variable: any variable that is held constant in a

research study by observing only one if its instances or

levels.

Intervening Variable: a hypothetical variable that is not

observed directly in the research study, but is inferred from

the relationship between the independent and dependent

variable.

8/3/2019 02ResDsnBB

12/28

Quantification of Variables

Measurement: the application of rules in

assigning numbers to cases so as to

represent the presence or absence ofquantity of an attribute possessed by each

case.

Four (4) scales of measurement

8/3/2019 02ResDsnBB

13/28

Scales of Measurement

Nominal Scale Measurement (Lowest)

Ordinal Scale Measurement

Interval Scale Measurement

Ratio Scale Measurement (Highest)

Variables measured at higher levels can be

scaled down to lower levels

8/3/2019 02ResDsnBB

14/28

Likert Scale: Disagree/Agree

(5 Question/Item Survey)Ann Billie Todd Jose Amita

1. I always like to find the most

unusual answer to a question.

2 3 4 5 3

2. I often spend time thinking about

alternative uses for a single object.

1 2 5 4 2

3. When I take a shower, I think

about all sorts of new ideas.

4 1 5 5 2

4. I am usually the one in groups to

think out side of the box

4 1 3 2 1

5. When in a problem solving group,

people always look to me for

creative ideas.

5 1 3 1 1

Total Score 16 8 20 17 9

8/3/2019 02ResDsnBB

15/28

Validity of the Study

Can you trust the conclusions of the study?

Internal Validity: The extent to which the outcomes ofthe study result from the variables manipulated,measuredor selected rather than from other variables not

systematically managed. External Validity: the extent to which the findings of a

particular study can be generalized to people and/orsituations other than those observed in the study.

8/3/2019 02ResDsnBB

16/28

Are Conclusions Trustworthy?

Latch-Key Study

Study of the effects of having young children spend part of their day without anadult. The issue of latch key or self-care children.

Two groups of 50 children each

One group who have never spent after school time without adult present(adult supervised)

One group who spend at least one hour a day without adult present (selfcare)

Dependent variables

Level of anxiety

Rate of delinquency

School achievement

8/3/2019 02ResDsnBB

17/28

Results Latch-Key Study

Mean Anxiety

Score

% Committing

Delinquent Acts

Mean Reading

Comprehension

Score

Self-CareChildren

27 9 20

Adult-

Supervised

Children

25 4 24

8/3/2019 02ResDsnBB

18/28

Conclusions of Latch-key Study

The researchers concluded:

The effects of self-care on anxiety are

negligibleSelf-care results in increased rates of

delinquency among young children

Self-care results in a small average loss inreading comprehension of young children

Do you agree? Are the results trustworthy?

8/3/2019 02ResDsnBB

19/28

Potential Threats to the Internal Validity:

Counter-Interpretations

History--Occurrence of events that take place in the course of the study

that might affect the dependent variable.

Maturation--Developmental (physical or mental) changes in the

participants which account for results

Testing--If participants given pre-test, they may learn from pre-test and

hence do better on the post-test.

Instrumentation--If measuring instruments are not reliable or valid, then

their scores could be inaccurate.

Selection--Initial differences among groups being compared might account

for results and lead to misinterpretations. Statistical Regression--If sample selected on basis of extreme scores, their

scores will move toward the mean on repeated testing

Mortality--Drop out of subjects who might share a common characteristic.

Stability--The possibility that the results are a fluke, a chance occurrence.

8/3/2019 02ResDsnBB

20/28

Potential Threats to the Internal

Validity: (Contd) Diffusion of treatments--comparison groups learn about

other treatment

Experimenter effects -- deliberate or unintentional

influence of researchers

Statistical conclusions--if statistical assumptions are

violated

Subject effects - changes in participating subjects

8/3/2019 02ResDsnBB

21/28

Counteracting Potential Threats

to Internal Validity

Control Group: a group of subjects whose selection and treatment are

exactly the same as those of the experimental group except that the control

group does not receive the experimental treatment. Note, that doesn't mean

"no treatment

Random Assignment: a method for assigning subjects to control and

experimental groups. Not to be confused with random selection (a method for

selecting a sample of subjects from a population).

Pretests:When random assignment is impossible or undesirable, pretests can

be used to examine the possibility or prior existing differences between

groups and to statistically adjust for these differences.

8/3/2019 02ResDsnBB

22/28

Potential Threats to External Validity:

Counter Interpretations

Reactive Effects of Subject Selection--If sample is not representative of thepopulation, the results can not be generalized to that population

Reactive Effects of Testing--If pretest is given and somehow affects theoutcomes of the study, results cant be generalized to population unless a pre-testwill also be given to that population.

Reactive Effects of Treatment Selection--If treatments of study can not bereplicated outside of study, then results cant be generalized to that population.

Multiple Treatment Interference--When subjects in one treatment are exposedto another treatment condition, then cant isolate results to a given treament, butonly to the interaction of the two treatments. So results cannot be generalized topopulation for which treatment interaction does not occur.

8/3/2019 02ResDsnBB

23/28

Major Types of Research Studies

Experimental: A type of research used to

establish cause-and-effect relationships by

manipulating variables/treatments Observational/Correlational: A type of

research that measures two or more

variables and looks to see how the variablesare related to each other.

8/3/2019 02ResDsnBB

24/28

Classes of Research Design

Pre-experimental

Experimental

Quasi-experimental

Ex Post Facto

8/3/2019 02ResDsnBB

25/28

Pre-Experimental Designs:No Control Group and/or Randomization

One-shot case study One-group pretest-posttest design

Intact-group comparison

8/3/2019 02ResDsnBB

26/28

True Experimental Designs:

Control Group & Randomization

Posttest-only control-group design Pretest-posttest control-group design

Factorial experimental design

8/3/2019 02ResDsnBB

27/28

Quasi-Experimental Designs:

Control Group But No Randomization

Non-equivalent control group design Time-series designs

Others

8/3/2019 02ResDsnBB

28/28

Ex-Post Facto Designs:

Researcher Arrives After Treatment Is Given

Correlational designs

-- Simple predictive

-- Causal modeling

Criterion-group designs