Ch 9 MethodologyCh 9 Methodology

Research DesignResearch Design

What is the research question? (What is the hypothesis?)Who are your population?Time and BudgetDefine types of the research

1.Quantitative Research2.Qualitative Research

Looking for the exist instrument or establishing a new instrument

Data analysisPlan the research procure

Research Design (Continued)Research Design (Continued)

Get the permission Do the surveyCollect dataAnalyze the dataFinal report

Quantitative Research Design Quantitative Research Design

Types of Quantitative Research

1. Survey Research Designs

-- Cross Sectional Research Designs

-- Longitudinal Studies

2. Experimental Research Designs

3. Quasi Experimental Research Designs

4. Causal-Comparative Research Designs

5. Correlational Research Designs

Survey ResearchSurvey Research

Major Characteristics

-- Information is collected from a group of people in order to describe some aspects or characteristics of the population of which the group is a part

-- Information is collected through asking questions; the answers to these questions by the members of the group constitute the data of the study

-- Information is collected from a sample rather than from every member of the population

Survey Research (Continued 1)Survey Research (Continued 1)

Purpose of Survey Research

-- To describe the characteristics of a population

-- To determine how a population distributes itself on one or more variables

Types of Surveys

1. Cross sectional

-- Information that is drawn from a predetermined population

-- Information is collected at one point in time

-- A census would be typical

Survey Research (Continued 2)Survey Research (Continued 2)

2. Longitudinal

-- Information is collected at different points in time to determine changes over a period of time

-- Trend studies would be typical-changing population surveyed at different times

-- Panel study-survey same population at different times during the course of the study

Survey Research (Continued 3)Survey Research (Continued 3)

Steps in Survey Research

1. Define the problem

-- What is it the researcher wants to find out

-- Establish research questions

-- Establish research hypothesis (es)

2. Identify the target population

-- Sampling will be discussed later

-- Sampling will be based on your research problem and what population can best attest to your problem

Survey Research (Continued 4)Survey Research (Continued 4)

3. Data gathering -- Direct administration: Need access to the entire sample or population

Need time and /or facilities a. Mail survey -- Mailed to participants -- Request to complete surveys -- Request to return surveys -- Consent forms -- Confidentiality -- Return rate

Survey Research (Continued 5)Survey Research (Continued 5)

b. E-Mail / Web surveys

c. Telephone survey

7. Personal interviews

Advantages and Disadvantages of Survey Data-Collection Advantages and Disadvantages of Survey Data-Collection MethodsMethods

Direct Administration Telephone Mail Interview

Comparative cost Lowest Same Same High

Facilities needed? Yes No No Yes

Require training of questioner? Yes Yes No Yes

Data collection time Shortest Short Longer Longest

Response rate Very high Good Poorest Very high

Group administration possible? Yes No No Yes

Allow for random sampling? Possibly Yes Yes Yes

Require literate sample? Yes No Yes No

Permit follow-up questions? No Yes No Yes

Encourage response to sensitive topics?

Somewhat Somewhat Easy Hardest

Standardization of responses Easy Somewhat Easy Hardest

Advantages and Disadvantages of Closed-Ended vs. Open-Advantages and Disadvantages of Closed-Ended vs. Open-Ended QuestionsEnded Questions

Advantages Enhances consistency of

response across respondents Easier and faster to tabulate More popular with

respondents Allows more freedom of

response Easier to construct Permits follow-up by

interviewer

Disadvantages May limit breadth of

responses Takes more time to construct Requires more questions to

cover the research topic Responses tend to be

inconsistent in length and content across respondents

Both questions and responses subject to misinterpretation

Hard to tabulate and synthesize

Preparing the InstrumentPreparing the Instrument

Questionnaire1. Appearance2. Questions clear --- Is this a question that can be asked exactly the way it written? --- Is this a question that will mean the same thing to

everybody? --- Is this a question that people can answer? --- Is this a question that people will be willing to answer, given the data collection procedures? --- Pilot test

Preparing the Instrument (Continued 1)Preparing the Instrument (Continued 1)

Types of questions

1. Open-ended

Used if you want an individuals true feelings

Gives people an opportunity to individualize responses

2. Close- ended

Respondents can select responses from a number of options

Questions need to be unambiguous

Keep the focus as simple as possible

Keep questions short

Preparing the Instrument (Continued 2)Preparing the Instrument (Continued 2)

2. Close- ended

Use language the respondents will be familiar with

Avoid using terms that might bias responsesAvoid leading questionsAvoid double negatives

3. Pilot Test4. Format and cover letter

There are several different sources that will assist the researcher to format their questionnaire and cover letter

Preparing the Instrument (Continued 3)Preparing the Instrument (Continued 3)

5. Nonresponse

Send reminders

Can create an additional sample until desired number is reached

May have to discard surveys if there are missed questions

6. Obtain demographics that will be pertinent to the study

Causal-Comparative ResearchCausal-Comparative Research

Definition

1. An attempt to determine the cause or consequence of differences that already exist between two or more groups of individuals

2. Sometimes called correlational research

--- Seeks to explore relationships among variables

--- Identifies variables for further research

--- Differs in that it compare groups rather than scores

--- Differs as it usually has group members rather than quantitative variables

Causal-Comparative Research (Continued Causal-Comparative Research (Continued 1)1)

3. Used to study differences between groups (male-female)4. Looks for noted differences between groups and causes or consequences of the difference5. Can be used as a substitute for experimental research the only difference would be that in experimental research the independent variable is manipulated by a treatment6. This is non-experimental research7. Describe current conditions (cancer)8. Identify or explore the past for causes of the a current condition (smoking)

Examples of the basic Causal-Comparative Examples of the basic Causal-Comparative DesignDesign(a) Group Independent Variable Dependent Variable

I C O

Giving a little gift Level of customer satisfaction

II (-C) O

Not giving a little gift Level of customer satisfaction

(b) Group Independent Variable Dependent Variable

I C1 O

Counselors Amount of job satisfaction

II C2 O

Teachers Amount of job satisfaction

Steps Involved in Causal-Comparative Steps Involved in Causal-Comparative ResearchResearch1. Problem

* Establish the phenomena and cause or consequences

* Once established become more specific

* Develop hypothesis (es) by comparing the variables with the causes that have been developed through your research problem

* How does one variable effect the other (independent variable’s effect on the dependent variables)

Steps Involved in Causal-Comparative Steps Involved in Causal-Comparative Research ( Continued 1)Research ( Continued 1)2. Sample

Define the characteristic to be studied with the idea of selecting a group that would be differing in regards to the characteristic but similar with one characteristic (male-female) (9th grade-11th grade)

3. Instrument

Open-end

4. Design

Select two or more groups that differ on at least one variable but may be similar in another

Steps Involved in Causal-Comparative Steps Involved in Causal-Comparative Research ( Continued 2)Research ( Continued 2)5. One group possesses a characteristic (criteria) the other

group don’

Investigate a categorical variable vs. a quantitative variable (group vs. score)

6. Data analysis

* Compare the differences in means using t-test or analysis of covariance

* Use frequency polygons

* Just report the cause and the effect do not try to prove causes or effects

Correlational ResearchCorrelational Research

Definition

1. The relationships among two or more variables are studied without any influence or treatment

2. It describes an existing relationship between variables

3. Scores within a certain range on one variable are associated with scores within a certain range on the other variable

Correlational Research (Continued 1)Correlational Research (Continued 1)Purposes

1. Explain important human behaviorsTo clarify our understanding of important phenomena by

identifying relations between variables

2. Prediction studies* Based on a high relationship between two variables it

becomes possible to predict a score on either variable if the score on the other variable is known

* Scattergram -- Ordinate-vertical axis -- Abscissa-horizontal axis -- Regression line-line that comes closest to all scores depicted

may reflect a perfect correlation

3. There are many complex correlational techniques

Correlational Research (Continued 2)Correlational Research (Continued 2)

Basic Steps in Correlational Research1. Problem Selection * The relationship two or more variables * The differences between two or more variables2. Sample * Represent the research variables * Should be selected randomly * Need 30 or more participants for meaningful results3. Instruments * Need to measure two or more variables * Need to yield qualitative data

Correlational Research (Continued 3)Correlational Research (Continued 3)

4. Design * Two scores are obtained from each participant

* One score for each variable

5. Data analysis * Correlation coefficient is produced

* A decimal between 0.00 and 1.0

* The closer to 1.0 the coefficient establishes a stronger relationship

* A correlation 3.5 show only a slight relationship between variables

* Correlations between .40 and .60 may have theoretical or practical value

* Correlations of .65 or higher one can make reasonable predictions

* Correlations over .85 have a strong relationship between variables

Experimental ResearchExperimental Research

What is it?* Research that directly attempts to influence a particular variable* Looks at the effect of an independent variable on one or more dependent variables* Independent variable is the experimental or treatment variable* Dependent variable is the criterion or outcome variable* Researchers manipulate the independent variable --- Researcher decide on the treatment

What is going to happen to the subject?

To whom is the treatment going to be applied

To what extent is the treatment going to be applied

Experimental Research (Continued 1)Experimental Research (Continued 1)

* Comparison of groupsExperimental group receives the treatment

Control group receives no treatmentComparison group receives a different treatment

Randomization* Random selection: every member of the population has an equal chance of being selected to be a member of the sample* Random assignment: any of the participants has an equal chance of being assigned to any of the groups

Takes place before the experiment takes place

It’s a process of assigning not just a distribution

Experimental Research (Continued 2)Experimental Research (Continued 2)

The groups should be equivalent (as much as human being can be)

Has the possibility of eliminating extraneous variables that might effect the outcome of the study by controlling for the characteristics that might have an impact

* Overcoming extraneous variables

--- The researcher has more control over the research than in other research designs

a. Researcher determines the treatment

b. Researcher selects the sample

Experimental Research (Continued 3)Experimental Research (Continued 3)

* Overcoming extraneous variables

--- The researcher determines the treatment

--- The researcher selects the sample

--- The researcher assigns individuals to groups

--- The researcher decides which group will get the treatment, controls and different treatment

--- The researcher controls for outside factors that might influence the study

a. The researcher observes or measures the effect of the treatment on the groups when the treatment is completed

Experimental Research (Continued 4)Experimental Research (Continued 4)

--- Ways the researcher can minimize threats due to subject characteristics

a. Randomization-assume individuals within a group will have different characteristics

b. Control variable by removing it from the study

c. Build needed variable into the study (gender, age, culture, etc.)

d. Match characteristics into the different groups

e. Control subjects by giving them each specific treatments

f. Control groups by grouping into certain variables based on pre-test, pilot test

Experimental Research (Continued 5)Experimental Research (Continued 5)* Weak experimental designs --- One shot study: single group is exposed to the treatment

and dependent variable is measured without any controls --- One group pretest-posttest: measured before and after treatment to determine if there is any change in the dependent variable --- Static group comparison: pre-formed groups compared

based on different treatments --- Example of a one-group pretest-posttest design

Pretest: Twenty-item attitude scale completed by employees (Dependent variable)

Treatment: Ten weeks of counselingPost-test: Twenty-item attitude scale completed by employees

(Dependent variable)

Experimental Research (Continued 6)Experimental Research (Continued 6)

--- Example of a one-shot case study design

--- Example of a static-group comparison design

X ○

New textbook Attitude scale to measure interest

(Dependent variable)

X1 ○

New rules Attitude scale to measure interest

X2 ○

Old rules Attitude scale to measure interest

Experimental Research (Continued 7)Experimental Research (Continued 7)

True experimental designs

* Subjects are randomly assigned to treatment groups

* Randomized posttest only control group design

--- One group received experimental treatment the other is the control group, both groups are post tested on a dependent variable

* Randomized pretest-posttest control group design

--- Measured at the pretest and measured again at posttest to determine whether there has been any change in the dependent variable

Experimental Research (Continued 8)Experimental Research (Continued 8)

Randomized Solomon four group design

--- Two groups are pretested and two are not

--- One of the pretested and one of the non-pretested groups are exposed to treatment

--- Two of the groups are the control groups

--- All four are posttested

Example of a Randomized Posttest-Only Control Group Example of a Randomized Posttest-Only Control Group DesignDesign

R X1 O

Random assignment of 50

employees to experimental

group

Treatment: sensitivity

training workshops

Posttest: Faculty morale

questionnaire

(Dependent variable)

100 employees randomly selected

R X2 O

Random assignment of 50

employees to control group

No treatment: Do not receive sensitivity

training

Posttest: Faculty morale

questionnaire

(Dependent variable)

Example of a Randomized Pretest-Posttest Control Example of a Randomized Pretest-Posttest Control Group DesignGroup Design

R O X1 O

Random assignment of 50

employees to experimental group

Pretest: Faculty morale

questionnaire

Treatment: sensitivity training

workshops

Posttest: Faculty morale

questionnaire

(Dependent variable)

(Dependent variable)

100 employees randomly selected

R O X2 O

Random assignment of 50

employees to control group

Pretest: Faculty morale

questionnaire

Treatment: Workshops that do

not included sensitivity training

Posttest: Faculty morale

questionnaire

(Dependent variable)

(Dependent variable)

Experimental Research (Continued 9)Experimental Research (Continued 9)

* Random assignment with matching --- Attempt to increase the likelihood that the groups of subjects in an experiment will be equivalent, participants would be matched on certain variables to insure equivalence on these variables --- The variables matched will be determined by the

researchera. Mechanical matching Pairing two persons based on similar scores on

a particular variable If there is no match some subjects could be

eliminated from the study

Experimental Research (Continued 10)Experimental Research (Continued 10)

b. Statistical matching

Based on a correlation a subject is given a predictable score on a dependent variable with the variable the subject is being matched

Matches could occur based on pretest scores

Quasi Experimental designsQuasi Experimental designs

Does not include the use of random assignment but other control techniques

Matching only design

* Subjects are matched in the experimental and control groups based on certain variables

* There are no assurances they are equivalent on other variables

* Correlations used need to be significant (above 0.04)

Counterbalanced designs

* Another way of equating experimental and control groups

* Each group is exposed to a treatment, but different groups will receive treatments at different times

Quasi Experimental designs (Continued 1)Quasi Experimental designs (Continued 1)

Time series designs

* Measurements or observations are at different times before and after treatment, whether pretest or posttest

Factor designs

* Extend the number of relationships

* Study the interaction of the independent variable with more than one moderator variable (treatment or characteristic variable)

* Uses a factor analysis statistic