lecture 3s

7/29/2019 Lecture 3s

1/73

DESCRIPTIVERESEARCH

1

MARKETING RESEARCH 103: LECTURE 3


2/73

2

OUTLINE

Systematic Observation

Sampling

Standardized measurement

Survey

Sampling

Questionnaire Design (if possible) Measurement and Scaling (next class)


3/73

Clearly-defined research question with theobjective of being descriptive

a structured research design guides thesampling and ensures control instandardizing what is being observed and

how it is recorded

3

I. SYSTEMATIC OBSERVATION


4/73

Time sampling

Situation sampling

Different locations (e.g., different places beer isconsumed: bar, student center, party etc.)

Sampling determines generalizability

4

SAMPLING

TECHNIQUE


5/73

STANDARDIZING OBSERVATION SHEETDate: 22 Aug 2007

Time Gender Type of Packaging P1 P2 P3 P4 P5 P6 P7 P8

1:00 PM

1:05 PM

1:10 PM

1:15 PM

Key

variables 1 2 3 4 5

P1approximate age

group

Below 12 12 to 18 19 to 25 26 to 35 Above 35

P2

attire uniform business casual sporty Dressy(for

mal)

P3 bag none Small medium large Extra large

P4 no. of companions none 1 to 2 3 to 5 6 to 10 Above 10P5 quantity 1 2 3 4 Above 4

P6 Other purchases none snacks meal Non food Others

P7Size of drink Extra small small medium large Extra large

P8

Type of drink Soft/fizzy fruit juice/

fruit-based

Tea/coffee Isotonic

drinks

Milk-based

Observer: GeorgeLocation: Borders

5


6/73

Nominal scale (checklist)

e.g., Brand of shoes, hair color, cellular phone or not,

with accompany or not.

Summarizing nominal data in terms of frequency,proportion, or percentage of instances in each of

the several categories.

6

MEASUREMENT SCALE


7/73

Physical actions

Verbal behavior

Expressive behavior

Spatial relations and locations

Temporal patterns

Physical objectsVerbal and pictorial records

7

WHATCANBEOBSERVED?


8/73

Cannot observe motives, attitudes or intentions

Inter-observer reliability

Can sometimes be time consuming

Ethical concern: violation of privacy?

8

LIMITATIONSOF OBSERVATIONAL

METHODS


9/73

II. SURVEY

Rationale: To enable quantification of opinions, preferences, etc.

Format: Open-ended or scaled questions, scenarios, ongoingversus one-shot, longitudinal vs. cross-sectional

Tools: mail, telephone, direct intercept, email, internet

Upsides/Downsides: (methodological and implementation)?


10/73

10

2.1 SAMPLING

A. Sampling jargons and design process

B. Sampling Methods

C. Sampling Size

D. Sampling errors vs. Non-sampling errors


11/73

11

A. SAMPLINGJARGON

Population

If the goal is to determine the average number of jobinterviewsafter graduation for the students who havetaken Marketing Research Class, then the entire body ofstudents who have taken 103 would be thepopulation

Sampling Frame:A list of students from which a sample isto be chosen (e.g., registration record)

Sampling: the selection of a fraction of the population

Census:all members of the population are measured


12/73

Sampling Frame

ElementOne student

Sample

100 students

Registrars list of currently enrolled students

Population

All students on SMU campus


13/73

SAMPLINGDESIGNPROCESS

Define the Population

Determine the Sampling Frame

Select Sampling Technique(s)

Determine the Sample Size

Execute the Sampling Process


14/73

14

B. SAMPLING METHODS

Probability sampling

Simple random

sampling

Proportionate

Disproportionate

Cluster

Non-probabilitysampling

Convenience sampling

Quota sampling

Judgment sampling

Snowball sampling


15/73

SIMPLE RANDOM SAMPLING

Each element in the population has a knownand equalprobability of selection, or

Each possible sample of a given size (n) has a knownand equalprobability of being the sample actuallyselected.

Randomly pickn(e.g., 5) units out ofN is statisticallyequivalent to enumerating all possible samples of size nandpicking one of them at random


16/73

16

SIMPLE RANDOM SAMPLING (SRS)

Procedure

Number everyone in the sampling frame

Picknelements randomly

Generated by computer or random number table


17/73

17

RANDOM NUMBERS GENERATORIN SPSS


18/73

18

ADVANCED PROBABILITYSAMPLING METHODS

Why do I need anything besides SRS?

Suppose our population of interest is

allmarketing students, lets say consisting of 700undergraduates (1/5 freshman, 2/5 sophomore,1/5 juniors and 1/5 seniors).

If we take a SRS of 140 students then we mayget a sample which looks like this: Seniors (28),

Juniors (28), Sophomores (56) and Freshman (28)


19/73

19

PROPORTIONATE STRATIFIEDRANDOM SAMPLING

The population is divided into strata on the basis of anappropriate variable (related to the outcome ofinterest)

Units within each stratum are alike and units acrossstrata are different

The sample size from each stratum is proportional tothe size of the stratum

E.g., N=6000, 2 strata with sizes 2000 and 4000.

In a sample of 120, we sample 40 and 80 respectively from the

2 strata


20/73

20

OTHERADVANCED METHODS

Disproportionate stratified random sampling

May want to over-sample small stratum or one that is

highly variable

Cluster Sampling

Population is naturally divided into clusters and clustersare selected at random.

All or some of the units within selected clusters aremeasured (e.g., selected blocks or counties)


21/73

21

NON-PROBABILISTIC SAMPLING METHODS

Conveniencesampling At the convenience of the experimenter (at the corner, at the

supermarket)

Quota sampling

Population is divided into segments on the basis of certain controlvariable

A quota of units to be selected from each segment is determined by theexperimenter

Sampling is done in each segment until the quota is filled

Better than convenience sampling in that you are at least guaranteedbalance in the population with respect to the control variable


22/73

22

NON-PROBABILISTIC SAMPLING METHODS

Judgment sampling

Snowball sampling


23/73

CLASSIFICATIONOF SAMPLING TECHNIQUES

Sampling Techniques

NonprobabilitySampling Techniques ProbabilitySampling Techniques

Convenience

Sampling

Judgmental

Sampling

Quota

Sampling

Snowball

Sampling

ProportionateStratified Sampling

ClusterSampling

Simple RandomSampling

DisproportionateStratified Sampling


24/73

24

C. DETERMINE SAMPLE SIZE

_

_

Common notation:


25/73

95% CONFIDENT

XL

_

XU

_

X

_

0.475 0.475

Estimation: Population vs. Sample


26/73

CONFIDENCE INTERVAL

Definitions:

Confidence interval: The confidence interval is the rangeinto which the true population parameter will fall, assuming agiven level of confidence.

Confidence level: The confidence level is the probability thata confidence interval will include the population parameter.

Precision level: When estimating a population parameter byusing a sample statistic, theprecision level is the desired widthof the estimating interval. This is the maximum permissibledifference between the sample statistic and the populationparameter.


27/73

27

CONFIDENCE INTERVAL APPROACHTOESTIMATE SAMPLE SIZE

1. Need to know how to construct ConfidenceInterval (CI)

We will see how it depends on n (your sample size)using SRS

2. Derive the sample size (n) that is necessary toconstruct a confidence interval with a desired

width (H)

Need to know the desired confidence level (?%) anddesired width (H)

E.g. want to be 95% confident that the market shareis +/- 3% around the true value (+/-3% is the width)


28/73

28

CONFIDENCE INTERVALFORUNKNOWNPARAMETERQ

CI = (Best Guess) +/- (Certain #) SE(Best Guess)

What makes a given statistic

T the best guess? And how

do I compute the SE of it?

The certain # depends on

how confident you want

to be

For the two examples we will

talk about today, i.e. unknown

is either a mean or an unknown

proportion p, SE = n/

This is precision level H. Notice, for a given level of confidence

and a given value of (SD in the population), we can

solve for the value ofn that gives you your desired H.


29/73

29

Best Guess for Population Mean

x ith

iIn general, let be the measurement on the person/household/firm etc.

Then,

m = 1N Xi

i=1

N

X=1

nX

i

i=1

n

Since we typically do not know we have to estimate it(best guess for population mean would be sample mean:


30/73

30

SE OF BEST GUESS

SE of X =

.

In the CI calculations, simply replace 2 with s2

.

s 2 = 1n - 1

Xi- X( )

2

i=1

n

n/

Since population variance is often unknown,

an unbiased estimate of the population variance is the sample variance definedas


31/73

31

Example 1: Confidence Interval for

We can construct a 100(1- q)% confidence interval for population mean by computing

where

= the sample mean

= the sample standard deviation

n = the sample size

= 1.645, 1.96, 2.575 for a 90%, 95% and 99% confidenceinterval

x

s

qz

What we will talk about later is the inverseproblem. What n is necessary so that zq*s/ =H?n

XZqs

n


32/73

32

APPLIED EXAMPLE 1

We are interested in the mean starting income of all SMU graduates sowe go out and obtain a SRS of 100 SMU graduates. The mean startingincome in our sample is $115,000 with standard deviation equal to$20,000. Construct a 95% confidence interval (CI) of the true

population mean starting income of all SMU graduates.

Answer:

)100

000,2096.1000,115,

100

000,2096.1000,115(

= (111080 , 118920)

X

s

qz n


33/73

33

EXAMPLE 2: CONFIDENCE INTERVALFORPROPORTIONP

A proportion is a mean where each observation is a 0 or 1, so we cancompute a CI for a population

We can construct a 100(1- q)% CI for population meanp by computing

where

= the sample proportion

n = the sample size

= 1.645, 1.96, 2.575 for a 90%, 95% and 99% confidenceinterval

p

qz

pZqp(1- p)

n


34/73

34

APPLIED EXAMPLE 2

We are interested in the proportion of students who havetried a new soup. We decide to go out and obtain an SRS of400 students and ask them if they have tried the new soup.

Suppose 20% of the 400 students say they have tried it.Construct a 90% confidence interval (CI) of the truepopulation market share.

Answer:


35/73

35

THE SIZEOFTHE SAMPLE (MEAN)

Question: What sample size do I need so that if I constructa 100(1-q)% confidence interval it will be an interval with

width H?

Compute Size of the Sample

Determine q, the desired CI you want, e.g., = 1.645 is a 90% CI

Determine the interval width you want, e.g.,

Compute an estimate of the standard deviation of the population of interest,denoted bys.

Compute the necessary sample size

0.10Z

5000$

n =Zq

2s

2

H2


36/73

36

Where do you get s from?

Historical data

Worst case scenario (largest possible s).

Pre-test


37/73

37

EXAMPLE

Suppose you want a 95% CI for mean graduatestarting salaries such that the interval will have

width $5000. Using the results of the previousexample where s= $20,000, how many graduatesdo I need to obtain in my SRS to obtain thisdesired width?

Answer: 2

22

5000

000,2096.1 n

61.5Note, you need to do this forevery question in your surveyand then take the maximum


38/73

38

THE SIZEOFTHE SAMPLE (PROPORTION)

Compute Size of the Sample

Determine qthe desired CI you want, e.g., = 1.645

is a 90% CI Determine the interval width you want H, e.g.,

Compute an estimate of population proportion

Compute the necessary sample size:

0.10Z

%3

p

n =Zq

2p(1- p)

H2


39/73

39

An approximate n for proportions (assuming 95% confidence)in a worst-case scenario is :

n = 1/H2

WHEREDOYOUGETANESTIMATEOFP?

The largest sampling error occurs when p =

0.51

SE

P0

p(1-p)


40/73

40

EXERCISE

Estimate the sample size for the following scenario:

95% CI

Interval width (H) = Assume 50% of people like the new ad copy

%3


41/73

41

D. SAMPLINGVS. NON-SAMPLING ERRORS

Total Survey Error

Non-Sampling

Error

Non-Response

Error

Response

Error

Random

Sampling Error


42/73

SOURCESOF ERRORSIN INFORMATIONFROM RESPONDENTS

POPULATION RESPONDENT INTERVIEWER

Inaccuracy in

response

Ambiguity

of answer

Ambiguity ofquestion

Interviewererror

sample Question

Answer

Samplingerror

Non-responsedue to refusalsor not-at-home

The method you select usually leadsto trade-offs in these areas.

Mail, Phone, Internet, Personal Interview.


43/73

RANDOM SAMPLING ERROR

Sample selected is not a perfect representation oftest population

Can be controlled by:

Random sampling procedure

Quota sampling

Increase sample size


44/73

44

NON-RESPONSE ERRORS

A non-responseerroroccurs when units selected aspart of the sampling procedure do not respond in

whole (called unit non-response) or in part (calleditem non-response)

If non-respondents are not different from those that didrespond, there is NOT a non-response error

It only happens when the existence of non-respondentscauses respondents to be poor representatives of totalsample


45/73

45

RESPONSE ERRORS

A Response or data erroris any systematic bias thatoccurs during data collection, analysis orinterpretation

Respondent error

Interviewer bias

Recording errors

Poorly designed questionnaires


46/73

DILEMMA: LARGEVS. SMALL

Large Sample Size: Reduce random sampling error

Small Sample Size: Better interviewer controls,higher response rate and higher accuracy


47/73

ATT STUDY

Purpose: compare a list of research procedures in terms of themagnitude of various sampling and non sampling errors.

4 methods of survey administration (same questionnaire):

mail telephone interviews

personal interviews

questionnaire drop-off

2 methods of prior notification:

mail and telephone

2 methods of follow up

mail and telephone


48/73

SPECIAL CHARACTERISTICSOFTHISSTUDY

Survey questions:

telephone usage

attitudes in telephone usage

organizational characteristics

Telephone usage for each company were obtained from ATT forboth respondents and non respondents (allowing for assessing nonresponses and response error)

Comparisons focused on the following three questions:

Company's average monthly telephone bill over the last 3 months.

Quantity of companys telephone lines.

Quantity of companys telephone sets (stations).


49/73

FINDING 1:

RandomSampling Error

~5% Non-SamplingError

~95%


50/73

RANDOM SAMPLING ERRORVS. NON-

SAMPLING ERROR

What does the 95% mean?Given random sampling in place, the need to get

large representative samples is likely to have been

exaggerated

What should we focus on instead?Obtain smaller samples so as to better spend

resources on: Interviewer control Performing pre-tests of questionnaires Trying to achieve greater reliability of survey

responses


51/73

FINDING 2:

>

>

>

Higher valuessuggest that

respondents arelarger companiesthan non-respondents

Respondentsmore economicallyupscale than non-respondents


52/73

DIRECTIONOF RESPONSE & NON-

RESPONSE BIASES


53/73

CONCLUSIONS

Given random sampling, focus on minimize non-sampling error:

Response Error

Control interviewer-respondent interface and/or questionnaireformat

Enough time

Non-response error

Ensure respondents are representative of the selected sample

Check whether respondents represent population


54/73

54

REVIEW:

Understand SRS and why you might depart from SRS

Understand the confidence approach to estimate sample

size Understand various errors in sampling

The proof of the pudding is in the eating. By a small

sample we may judge of the whole piece.

Miguel de Cervantes Saveedra


55/73

2.2 QUESTIONNAIRE DESIGN

55

Specify the Information Needed

Design the Question to Overcome the Respondents Inability andUnwillingness to Answer

Determine the Content of Individual Questions

Decide the Question Structure

Determine the Question Wording

Arrange the Questions in Proper Order

Reproduce the Questionnaire

Specify the Survey Administration Method

Identify the Form and Layout

Eliminate Bugs by Pre-testing

Process


56/73

STEP 0: MAKESURETHEDATAYOUGETWILLBEUSEFUL

Population has been defined correctly

Sample is representative of the population

Respondents selected to be interviewed are availableand willing to cooperate

Respondents understand the question and have

knowledge Motivate them to provide information (incentives)

response rates

56


57/73

STEP 1: WHATINFORMATIONTOCOLLECT?

Translating research objectives into information

requirements

Question relevancy

Use backward approach- What would you do with the

answer if you knew it?

57


58/73

STEP 2: COMPARISONOF SURVEYADMINISTRATION METHODS

Criteria Best Second Third

VersatilityNumber of Questions

Amount /Var of info

Presentation of stimuli

Personal

Personal

Internet

Mail/Internet

Phone/Internet

Personal

Phone

Mail

Mail/PhoneResponse Time Internet/Phone Personal Mail

Cost Internet/Mail

$1-$10

Phone

$15-$30

Personal

$50+

AccuracySampling Control

Supervisory control

Opport. for clarification

PersonalInternet/Mail

PersonalPhonePhone

Phone

Mail/InternetPersonal

Mail/Internet

Respondent Convenience Internet/Mail Telephone Personal


59/73

STEP 3: DETERMINETHE SPECIFICQUESTIONS

Is this question necessary?

Are several questions needed instead of one?

Do you think Coca-Cola is a tasty and refreshing soft drink?

Why do you shop at Nike Town?

59


60/73

STEP 4: OVERCOME INABILITYORUNWILLINGNESSTO ANSWER

Inability to answer

Filter/screening questions

Dont know option

Provide context to facilitate the memory

Unwillingness to answer

Reduce the effort (recall vs. recognition task, rating vs. rank order)

Justify purpose (legitimate reason)

Sensitive information

60


61/73

STEP 5: CHOOSE QUESTION STRUCTURE

Unstructured: Open-response questions

What do you think of people who patronize discount department

stores?

Structured: Closed-response questions

Multiple choice questions

Dichotomous questions

Scales

61


62/73

Response Formats Open-Ended

Advantages:

Respondents can give general (unbiased) reactions toquestions such as Why do you say that brand X isbetter?

Response given in real world terminology. i.e.,consumers own language

Can help interpret closed ended data.

e.g. Why was color the most important product attribute?

May suggest additional alternatives to be used in closed-ended questions.

62


63/73

Response Formats Open-Ended

Drawbacks:

Often not good for self-administered surveys

Answers depend on respondent ability toarticulate

Interviewer bias can be a problem

Post-coding is tedious

63


64/73

Response Formats Closed-Ended

The respondent is provided with pre-determineddescriptions and selects one or more of them

Advantages:

easy to use in field

less threatening for respondent

ability to reduce interviewer bias

simple to code and enter data listing responses may jog respondents memory

cheaper to administer

64


65/73

Response Formats Closed-Ended

Disadvantages:

Usually requires pre-testing

Presumes the list of responses is complete

Exploratory research using open-ended questions

can be used to suggest the items and scales in thequantitative survey

65


66/73

STEP 6: DETERMINETHE QUESTIONWORDING

Be specific (who, what, when, where etc.)

Which brand of shampoo do you use?

Use simple, conventional (ordinary) words Do you think the distribution of soft drinks is adequate?

Avoid ambiguity (usually, normally, often etc.)

In a typical month, how often do you shop in department stores?(Never, occasionally, sometimes, often, regularly)

Dual Statements: agreement and disagreement, like anddislike

66


67/73

Question wording-contd

Avoid leading and biasing questions

Do you think that we should buy imported goods when that would put

local labor out of work? Is Colgate your favorite toothpaste?

Avoid implicit alternatives

Do you like to fly when traveling short distances?

Avoid estimates

What is the annual per capita expenditure on groceries in yourhouseholds?

67


68/73

STEP 7: ARRANGEIN PROPERORDER

Responses to earlier questions may bias responses tolater questions

Funnel Approach

asking specific questions first may bias responses to general questionsasked later

What considerations are important to you in selecting a department store?

In selecting a department store, how important is convenience of location?


69/73

EXAMPLEOF ORDERBIAS:DOES THE QUESTION CREATE THE ANSWER?

69

1. No question asked 2.8

2. Asked only about advantages 16.7

3. Asked only about disadvantages 0.0

4. Asked about both advantages anddisadvantages

5.7

Percentage of Respondents

Very Much Interested inBuying New ProductQuestions Preceding Buying Interest Question


70/73

STEP 8: IDENTIFYTHE LAYOUT

Layout Guidelines

Open the survey with an easy and non-threatening question

Smooth and logical flow

From general to specific

Sensitive or difficult questions should not be placed at thebeginning of the questionnaire

Aesthetically pleasing: Font size, paragraph distance, thenumber of questions in one page, questions are numberedclearly

70


71/73

STEP 9-10: PRE-TESTINGANDCORRECTING PROBLEMS

Pre-testing specific questions

Variation

Meaning

Task difficulty

Respondent interest and attention

Response latencies

Question saliency

71


72/73

REVIEW:

Understand the advantages and disadvantages of eachtype of measurement scale

Conduct a mini

market research

when designing aquestionnaire

Search for and use proven questions

Pretest questionnaire

72


73/73

HOMEWORK

Assigned reading from course pack (Harvard SchoolSurvey, Causal Inference)

Textbook reading (chapter 8, 9, 10)

Group Assignment 1 due on Friday night, hardcopy, print,in my mailbox

lecture 3s

Documents