Sampling Fundamentals

Lecture 7

2

Chapter 10Sampling: Sample, design and sample size

Learning Objectives

• Distinguish between a census and a sample• Describe the steps in the sampling process• Define target population for a given research

problem• Identify sampling frames • Differentiate between probability & non-

probability sampling techniques• Assess non-response problems

Review• What should we ask? Relevance & accuracy• Open: free response• Closed: fixed, limited alternative / options

responses • Which is better & why?• Also depends on how you will ask eg mail,

telephone, personal

4

Example• How much do you depend on advertising

sources to get information about products that you are likely to buy?

• Very little 1 2 3 4 5 6 7 A lot

• What does, Infrequently, occasionally, frequently mean?

5

The Marketing Research Process

7

Sampling terminology• Sample: a subset, or some part, of a larger

population.• Population: any complete group of entities

that share some common set of characteristics.

• Population element: an individual member of a population.

• Census: an investigation of all the individual elements that make up a population.

8

Why sample?

• Pragmatic reasons: budget and time constraints

• Accurate and reliable results: most properly selected samples give sufficiently accurate results– Relationship between sample size and

accuracy• Destruction of test units.

9

How sampling works

10

Stages in the selection of a sample

Reasons for Using a Sample

• Time• Cost• Inability to study the whole population

Identify the Target Population

Define the Target Population• Precise statement of who should and should not be included in the sample

Sampling Elements– Object about which or from which the information is desired

e.g., males, females, over 18, buys the groceries etc.

Sampling Units– An element, or a unit containing the element that is available for selection at

some stage e.g., households, small businesses, schools etc.

Extent– Geographical boundaries

e.g., western metropolitan region of Melbourne, national, study of two countries

Time period under consideration

Identify the Target Population cont.

– Look to the research objectives– Consider the alternatives– Know your market– Consider the appropriate sampling unit– Specify clearly what is excluded– Don’t over-define– Should be reproducible i.e., consider

convenience

14

Defining the target population

• What is the relevant population?– Survey of students’ choice of university

might target university students, but should also include friends and parents who exert an influence in the decision.

– Operational definition required• Vital to carefully define the target population.

15

The sampling frame

• Sampling frame: a list of elements from which a sample may be drawn.– Example: student email list, membership

list • List brokers provide lists of specific

populations.• Sampling frame error occurs when certain

sample elements are not listed or accurately represented in a sampling frame.

16

The use of sampling frames in marketing research

• An accurate sampling frame will not always exist.– Potentially sacrifice accuracy by using

more practical methods like the shopping mall intercept to sample consumers.

– Telephone directories may be incomplete.• Using the wrong sample frame will result in

the inclusion of respondents who should not be part of the population.

17

Sampling units

• Does not have to be a person.– An airline may wish to select individual

passengers as the sampling unit or select certain flights as the sampling unit.

Determine the Sampling Frame• Representation of the target population• List or set of directions for identifying the target

population e.g., telephone book, association directory, mail list, map

• Sampling frame error– The list may omit some elements of the population or

include other elements which do not belong

• Dealing with population sampling frame differences– Subset problem– Superset problem– Intersection problem

Selecting a Sampling Technique: Non-Probability

• Personal judgement of the researcher is used rather than chance to select elements

• Difficult to generalise result to the population

Examples of the methods

used with this techniques will be

covered shortly

Selecting a Sampling Technique: Probability

• Sampling units are selected by chance• Pre-specifying every potential sample of a given

size that could be drawn from the population• Require precise definition of the target population

and sampling frame• Able to make inferences about the target

populationExamples of the methods

used with this techniques will be

covered shortly

How do you go about selecting a random sample?

William Burlace, Director, Media Services

Roy Morgan Research

Determine the Sample Size: Qualitative Factors

• Number of elements to be included in the study• Qualitative factors to consider:

– Importance of the decision– Number of variables– Nature of the analysis– Sample size used in similar studies– Incidence rates– Completion rates– Resource constraints

Quantitative Factors will be

covered later in this lecture

Execute the Sampling Process

• Detail specification of how the sampling, design decisions with respect to the population, sampling frame, sampling units, sampling techniques and sample size are to be implemented.

24

Random sampling and non–sampling errors

• Random sampling error: the difference between sample result and the result of a census conducted using identical procedures.– Statistical fluctuation due to chance

variations in elements selected for a sample.

– Example, student population has a true mean income of $20 000 but a sample shows a mean income of $10 000.

• Conceivably occurred by accident: random error.

25

Random sampling and non–sampling errors

• Beyond researcher’s ability to control random error.

• But, as sample size increases, random sampling error decreases.

• The more people we ask, the more likely the sample result is going to reflect the true result. – With statistical inference, we are able to

estimate the probability of random error.

26

Random sampling and non–sampling errors

• Non–sampling (systematic) error results from some imperfect aspect of the research design such as mistakes in sample selection, sampling frame error, or non–responses.– Errors not due to chance fluctuations, but

due to errors resulting from the researcher.

27

Less than perfectly representative samples

• Non–response error: statistical differences between a survey including only those who responded and a perfect survey that would also include those who failed to respond.– Sample is less likely to be perfectly

representative.– Example: consumers who are busier are

less likely to respond to a survey than those with more spare time.

28

Errors associated with sampling

29

Probability versus non–probability sampling

• Probability sampling: a sampling technique in which every member of the population has a known, non–zero probability of selection.

• Non–probability sampling: a sampling technique which units of the sample are selected on the basis of personal judgement or convenience.– The probability of any particular member of

the population being chosen is unknown.

30

Probability sampling

• Since this process is random, error related to researcher judgement is eliminated.

• There are various probability sampling methods:– Simple random sampling– Systematic sampling– Stratified sampling.

Probability Sampling: Simple Random Sampling (SRS)

• Each element in the population has a known and equal chance of selection

• A sample is drawn by a random procedure from a sampling frame

• Easily understood• Generalise to the population• Difficult to construct a sampling frame• May or may not result in representative sample• Accuracy-cost trade-off

32

Simple random sampling

• A sampling procedure that assures each element in the population of an equal chance of being included in the sample.– Example: drawing names from a hat

• Random number generator.

33

Systematic sampling

• A sampling procedure in which a starting point is selected by a random process and then every nth number on the list is selected.

• Requires sampling frame• Results appear to be random if there is no

other systematic pattern to the list.

34

Stratified sampling

• A sampling procedure in which simple random sub–samples that are more or less equal on some characteristic are drawn from within each stratum of the population.

• Involves dividing sampling frame into strata then randomly sampling within each strata.– Example: random sampling within the male

and female strata.

Probability Sampling: Stratified Sampling

• Population split into sub-populations• Strata are mutually exclusive and collectively

exhaustive• Then SRS from each stratum to select the elements

– Within stratum – homogeneous– Each(between) stratum - heterogeneous– Proportional stratified sampling– Disproportional stratified sampling

Refer to pp. 324-325 – Proportional stratified sampling

36

Probability sampling

37

Proportional versus disproportional sampling

• Proportional stratified sample: a stratified sample in which the number of sampling units drawn from each stratum is in proportion to the population size of that stratum.

• Disproportional stratified sample: a stratified sample in which the sample size for each stratum is allocated according to analytical considerations.

38

Cluster sampling

• An economically efficient sampling technique in which the primary sampling unit is not the individual element in the population but a large cluster of elements.

• Clusters are selected randomly.– Example: geographic cluster or area

sample• Unlike a strata, a cluster should be as

heterogeneous as the population itself.

Probability Sampling: Cluster Sampling

• Target population split into mutually exclusive and collectively exhaustive sub-populations

• Than random sample of clusters is selected based on SRS

• A sample from each cluster– Within cluster – heterogeneous– Each(Between) cluster - homogeneous

• Cost effective e.g., area sampling

40

Non–probability sampling

• Sometimes bias resulting from judgement in selection is necessary because marketers do not always have an accurate list available from which to select respondents.

• There are various non–probability sampling methods:– convenience sampling– judgement sampling– quota sampling– snowball sampling.

41

Convenience sampling

• The sampling procedure of obtaining those people or units that are most conveniently available.

• Convenient but may be unwilling or unrepresentative– Example: lecturer who uses students

• Best use for exploratory research but inappropriate for projecting results.

Non-probability Sampling: Judgmental Sampling

• Selection based on researcher judgement• Inexpensive, quick, can be used for exploratory

research, or pre-test questionnaire• Selection bias present, not representative, can not

generalise to target population

43

Judgement sampling

• A non–probability sampling technique in which an experienced individual selects the sample based on personal judgement about some appropriate characteristic of the sample member.

Non-probability Sampling: Snowball Sampling

• Initial group of respondents is selected• These respondents are asked to identify others

who belong to the target population of interest• Referrals will have demographic and

psychographic characteristics that are more similar to person referring than would be by chance e.g., minority groups, people involved in a specialist sport or hobby

Non-probability Sampling: Convenience Sampling

• Selection of sampling units is left to the interviewer• Inexpensive, quick, can be used for exploratory

research• Selection bias present, not representative, can not

generalise to the population e.g., students at uni, shopping centres without qualifying respondents, questionnaires in magazines or restaurants

Non-Probability Sampling: Quota Sampling

• Develop quotas of population elements e.g., gender, age

• Respondents are then selected based on convenience or judgement

• Not representative but could be relevant• Selection bias• Lower cost and greater convenience

Non-Response in Sampling• Refusals• Unwillingness / inability of people included in the

sample to participate• Reduce refusals by:

– Prior notification– Motivating the respondents– Incentives– Good questionnaire design and administration

Non-response in Sampling cont.

• Not at homes or inaccessible• Potential respondents are not at home when

contact is attempted• Use ‘call backs’

50

Quota sampling

• A non–probability sampling procedure that ensures various subgroups of a population will be represented to the extent that the investigator desires.

• Introduces bias because quota samples tend to include people who are easily found and willing to be interviewed.

51

Snowball sampling

• A sampling procedure in which initial respondents are selected by probability methods and additional respondents are obtained from information provided by the initial respondents.

• Locate members of rare populations by referrals.– Example: stamp collectors and adult

croquet players.

52

What is the appropriate sample design?

• There are a number of sampling criteria to evaluate each sample design:– Degree of accuracy

• Probability methods are better for conclusive projects that demand accuracy.

– Resources• Non–probability methods are better for projects

with financial and human resource constraints.

– Time• Simple sample design is better for projects with

time constraints.

53

What is the appropriate sample design?

• There are a number of sampling criteria to evaluate each sample design:– Advance knowledge of the population

• Lack of adequate lists rule out systematic and stratified sampling.

– National versus local project• Cluster sample better when population

elements are unequally distributed geographically.

– Need for statistical analysis• Non–probability methods do not allow statistical

analysis to project data beyond the sample.

54

What is the appropriate sample design?

55

What is the appropriate sample design?

56

Random error and sample size

• Random sampling error varies with samples of different sizes.– Increasing sample size increases accuracy.

• But since every project has a budget, we can ask a certain number of people and be confident of getting the same results had we asked many more people.

57

Factors in determining sample size

• Three factors affect sample size:– The variance or heterogeneity of the

population.• Only a small sample is required if the

population is homogenous.

– The magnitude of acceptable error.• Higher precision requires a larger sample.

– The confidence level.• Higher confidence requires a larger sample.

58

Determining sample size on the basis of judgement

• Sample size may also be determined on the basis of managerial judgements.– Rely on experience and sample size

similar to previous studies.• Researcher is governed by more practical

restrictions such as budget and time limitations.

Appropriate Sample Design

• Degree of accuracy• Resources• Time• Advance knowledge of the population• National vs. local project• Need for statistical analysis

Special Cases

• Web-based samples– Yields a high response rate because panel

members have agreed to cooperate with the research organisation and are compensated for their time.

• Shopping centre sampling– Shopping centre selection– Sample locations within a centre– Time sampling– Sampling people versus shopping visits

Sampling in an International Context

• Key Issues:– Absence of information on sampling frames– May need to consider regions rather than an

individual country– Selection of respondents– Generalisability of results across geographic

regions– High cost of multi-country research– Appropriateness of different sampling

techniques– Use of the same sampling technique in different

countries or regions– Sample size

62

Chapter 11Editing and coding: Transforming raw data into information

63

Stages of data analysis

64

Editing

• Editing is the process of checking the completeness, consistency and legibility of data and making the data ready for coding and transfer to storage.

• Coding is the process of assigning a numerical score or other character symbol to previously edited data.

65

Field editing

• Preliminary editing done by field supervisor on the day of interview.

• Purpose of field editing is to catch technical omissions, to check legibility of handwriting, and clarify responses that are logically or conceptually inconsistent.

66

In–house editing• In–house editing rigorously investigates the

results of data collection.• In–house editor’s tasks:

– Adjust inconsistent or contradictory responses so that the answers will not be a problem for coders and keyboard operators.

– Checks for adherence to the data collection framework.

– Checks for logically consistent responses.– Edit for completeness.

67

Facilitating the coding process

• Editing and tabulating ‘don’t know’ answers– An alternative is to record all ‘don’t knows’

as a separate category.• Write a code book.

– A systematic procedure for assessing the questionnaires should be developed.

– Clear and unambiguous instructions on how to deal with each type of response.

68

Coding

• Codes are rules for interpreting, classifying, and recording data in the coding process.

• A field is a collection of characters that represent a single type of data.

• A record is a collection of related fields.• A file is a collection of related records.

69

Pre– coding fixed–alternative questions

70

Pre–coding fixed–alternative questions

• The code for each response will be used by keyboard operator for data entry.– Example: a question with three possible

answers are pre–coded 1, 2, 3.• Pre–coding can be used if the researcher

knows what the answer categories will be prior to data collection.

71

Coding open–ended questions• Purpose of pre–coding open–ended

questions is to reduce the large number of individual responses to a few general categories of answers that can be assigned numerical codes.– Similar answers should be placed in a

general category and assigned the same code.

– Code building is based on thoughts, not just words.

– Test tabulation.

72

Coding open–ended questions

73

Computerised data processing

• Production coding is the process of transferring the data from the questionnaire to the storage medium.

• Optical scanning system and Intelligent Character Recognition are computerised methods to read questionnaire responses directly.

• Otherwise, data entry is used to transfer data from the research project to computers.