Quantitative Research I

Survey Methods & Design

Survey Method• An approach to data COLLECTION

and ANALYSIS

• A method of collecting data that is

STRUCTURED and UNIFORM

• A method of analysing data involving

comparing responses located in a

DATA MATRIX.

A Classification of Survey Methods

TELEPHONE FACE TO FACE MAIL COMPUTER

SURVEY METHODS

TRADITIONAL TELEPHONE

CATI

IN HOME MALL INTERCEPT

CAPI

POSTAL SURVEY

POSTAL PANEL

FAX E-MAIL INTERNET

DIRECT COMPUTER INTERVIEW

In-home Interviews

Highest response rates

Target specific persons

Flexible question (naive) design

Sophisticated sequencing

Clustering of respondents

Day time not - at - homes

Supervision difficult

Costs of re - visits

Benefits Limitations

Probing answers

Clarifying ambiguity

Interviewer safety

Slow to complete

Use visual stimuli Interviewer bias

Record non-verbal reactions Very expensive

Mall Intercept Interviews

Many advantages of in - homes

Less expensive

Less time consuming

Ease of supervision

Short questionnaires

Sample representativeness

Length and frequency bias

Respondent co-operation

Benefits Limitations

Telephone Surveys

Low cost

Dispersed samples

Speed

Sophisticated routing systems

Domestic samples unrepresentative

Unlisted numbers

Call screening

No visual aids

Benefits Limitations

Direct data entry

Editing and validation by machine

Short questionnaires

Simple scales

Mail Surveys

Anonymity of responses

Sensitive & personal questions

Complete at own pace

No interviewer bias

Lack of control over respondent

Cannot control speed of response

Cannot probe / clarify response

Sequence bias

Benefits Limitations

Dispersed samples

Targeting specialist populations

Simple question (naive) design

Low response rates

Inexpensive Follow - ups & incentives

Which Survey Method ?• SAMPLING CONTROL

the researchers ability to direct the enquiry to a designated

respondent and secure the desired co-operation from the respondent.

• INFORMATION CONTROL

the kinds of questions that can be asked and the amount and

accuracy of information that can be obtained from respondents

• ADMINISTRATIVE CONTROL

the speed and cost and the control of replies afforded by the method of

administration.

(Churchill, 1996)

Questionnaire

A set of questions designed to generate

the data necessary for accomplishing the

objectives of the research.

Provides for standardisation and uniformity

in data collected from respondents.

The Questionnaire’s “Position” in the Research Process

A questionnaire matches the survey objectives with the respondent’s information

Respondent’s Information

Survey Objectives

Questionnaire

Data Analysis

Questionnaire Design

Preliminary ConsiderationsQuestion ContentResponse FormatQuestion SequenceQuestion WordingLayoutPilot Work

Preliminaries

Meeting Objectives?

Questions Relevant?

Content

Able?

Willing?

Format - Open

Respondents free to answer in their own words

Example :Question 10. What do you consider to be the benefits of this program PROBE ANY OTHERS.

Format - Closed

Respondents asked to select answer(s) that apply

Example :

Surroundings

Question 10. Here is a list of some of the benefits associated with this program. Which one or ones, if any, apply to you? TICK ALL THAT APPLY

PriceParticipantsContentTiming

MaterialsOther (specify)NoneNA / DK

Respondents indicate which of two alternative answers most closely corresponds to their position on a subject.

Example :

Question 10. Do you agree or disagree that the objectives of the program were met ? TICK ANSWER.

AgreeDisagreeOKNA

Format - Dichotomous

Format - Scale

Respondents indicate where their answer lies across a continuum.

Example :

Question 10. To what extent do you consider that the objective of the programme were met ? Cross (x) the number that corresponds with your answer.

NOT MET

MET

1 2 3 4 5 6 7 9 8 10

Primary Scales of Measurement

4 81 9

Nominal Numbers Assigned to Runners

Ordinal Rank Order of Winners

Third Place

Second Place

First Place

Interval Performance Rating on a 0 to 10 Scale

8.2 9.1 9.6

Ratio Time to Finish in Seconds 15.2 14.1 13.4

Types of Scaling Techniques

• COMPARATIVE SCALES

Involve the respondent directly comparing stimulus

objects. e.g. How does Pepsi compare with Coke on

sweetness.

• NONCOMPARATIVE SCALES

Respondent scales each stimulus object independently

of other objects e.g. How would you rate the sweetness

of Pepsi on a scale of 1 to 10

Paired Comparison

Rank Order

Constant Sum

Q-Sort and Other

Procedures

Comparative Scales

NonComparative Scales

Continuous Rating Scales

Itemised Rating Scales

StapelSemantic

DifferentialLikert

A Classification of Scaling Techniques

SCALING TECHNIQUES

Modern Store

Low prices

Unfriendly staff

Narrow product range

Sophisticated customers

Old- fashioned store

High prices

Friendly staff

Wide product range

Unsophisticated customers

Semantic Differential Scale

• Here are a number of statements that could be used to describe

K-Mart. For each statement tick ( X ) the box that best

describes your feelings about K-Mart.

Semantic Differential Scale - Snake Diagram

Modern Store

Low prices

Friendly staff

Wide product range

Sophisticated customers

Old- fashioned store

High prices

Unfriendly staff

Narrow product range

Unsophisticated customers

X

X

X

X

X

Key :Sears

X K-Mart

Scale Decisions

Number of Categories generally between 5 and 9

Balance preferably a balanced scale

Odd or Even if neutral responses likely, use odd number

Forced or Nonforced if no opinion likely, use nonforced scale

Verbal Description label and close to response categories

Physical Form should be piloted.

Balanced and Unbalanced Scales

Balanced Scale Unbalanced Scale

JOVAN MUSK FOR MEN IS JOVAN MUSK FOR MEN IS

Extremely good

Very good

Good

Bad

Very bad

Extremely bad

Extremely good

Very good

Good

Bad

Very bad

Extremely bad

Sequence Introductory StatementEasy to StartClassification at EndEmbarrassing at EndPresent Before PastBehaviour Before AttitudesLogic (conversation)Funnel Technique

WordingAvoid Technical LanguageAvoid Highbrow QuestionsAvoid AmbiguityAvoid BiasAvoid GeneralisationsAvoid Double QuestionsAvoid NegativesAvoid Hypothetical QuestionsAvoid Long-winded Questions

Layout

Identification DataRequest for Co-operationInstructionsInformation SoughtInformation SoughtClassification Data

Pilot Testing

Just Do It!

Just Do It Again!

Just Do It Again!!

Commonly Used Probes and Abbreviations

Standard Interviewer’s Probe Abbreviation (AO ?)

(Other ?)

(AE or Else ?)

(Tell more)

(How mean ?)

(RQ)

(What mean ?)

(Which closer ?)

(Why ?)

(What in mind ?)

Any other reason ?

Any others ?

Anything else ?

Could you tell me more about your thinking on that ?

How do you mean ?

Repeat question

What do you mean ?

Which would be closer to the way you feel ?

Why do you feel that way ?

Would you tell me what you have in mind ?

Dummy Table : Store Preference by Frequency of Visit

STORE PREFERRED

VISIT FREQUENCY Sears Wal-Mart K-Mart

Less than once a month

Once or twice a month

Three or four times a month

More than four times month

Screeners Qualifying questions Have you been snow

skiing in past 12 months?

First few questions

Warm - ups What brand of skis do you own ?

First third of questions

Transitions What features do you like best about the skis ?

Middle half to second third

Difficult and complicated

Here are ten characteristics of snow skis. Please rate your skis on each characteristic

using the scale below

Last Section Classification and demographic

What is the highest level of education you attained ?

LOCATION TYPE EXAMPLE

Summary: Questionnaire Organisation