copyright © 2010 pearson education, inc. publishing as prentice hall conducting research and...

Copyright © 2010 Pearson Education, Inc.

Publishing as Prentice Hall

Conducting Research and Utilizing Tests to Measure Performance

Chapter 10

Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall

Please Note:


Opening Vignette

Omaha Steaks

International


Research Serves Direct Marketers

Fact-Finding Information Gathering Problem-Solving Decision-Making

Need for Marketing Research Managers must have good, accurate,

timely information with which to make decisions.

Marketing research helps to gather the needed information.

The results of research can be quantitative and/or qualitative.

Valid research measures results … not opinions.


Problem Structure How much advertising is needed? How will the direct marketing mix

be selected? How will our resources be utilized?


The Nature of ResearchSurveys vs. Experiments:

A survey looks at things the way they are to measure product preferences or determine future buying intentions

An experiment or test measures the effect of change

Which one is used more widely by direct marketers?


Databased Research and Analysis“My mind’s made up. Don’t confuse me with

facts.”

A characteristic of database-driven and directed marketing is measurement and accountability for actions. Decisions are based on facts, not opinions.

Direct marketers build databases from facts, relying not so much on responses derived from survey, but more on conclusions derived from experimentation.


Products and Services Media Offers/Propositions Copy Platforms Creative Formats Timing/Seasonality

What Do Direct Marketers Test?


TEST THE BIG THINGS


Sourcing & Collecting Information

Secondary Data

Primary Data


Secondary Data Sources The Organization’s Internal Records Government Sources: Federal, State & Local

U.S. Department of Commerce - Bureau of the Census U.D. Department of Labor - Bureau of Labor Statistics U.S. Department of Agriculture Other U.S. Government Sources: President’s Office,

Congress, Treasury Department, Interior Department, and Health and Human Services Department

State and Local Governments: economic surveys, license registrations, and tax records


Secondary Data Sources Trade, Technical, Professional and

Business Associations Private Research Organizations Foundations, Universities and Other

Nonprofits Libraries, Public and Private Advertising Media Financial Institutions and Utilities


Behavior

Intentions

Knowledge

Socioeconomic Status

Attitudes and Opinions

Motivations

Psychological Traits

Primary data, collected via survey, can yield information about:


PRIMARY DATA COLLECTION METHODSSurvey Method Advantages Disadvantages

Personal Interview More complete and accurate sample, more complete information, greater flexibility in structuring questions, high response rate

Cost, possibility of interview bias, and need for extensive interviewer supervision and control

Telephone Interview

Economy, speed, representative sampling, minimal nonresponse, simple callbacks, ability to make interview coincide with other activities

Limited availability of information at time of contact, excludes those without phone service, increasing public resistance to phone calls, inability to use questions requiring demonstration or visualization.

E-Mail Survey Ability to reach hard-to-reach consumer segments, speed, economy, respondent choice in when to reply

Representative sample issues, higher degrees of nonresponse bias


PRIMARY DATA COLLECTION METHODS

Survey Method Advantages Disadvantages

Mail Questionnaire Great versatility, low cost, no interviewer bias, no field staff needed, respondent choice in when to reply, respondent confidentiality

High rate of nonresponse, need for follow-up, response bias factors

On-line Surveys Ability to reach potential respondents at most opportune time, speed and convenience

Similar to e-mail surveys - Representative sample issues, higher degrees of nonresponse bias

Observation Removes respondent bias More costly and time consuming than some of the other methods


How to Design An Experiment

Control

Randomization

Statistically-valid sample size


How to Track Responses

Key Codes


Response rate & break-even analysis

Control vs. experimental packages

Direct marketers test or experiment with different offers and campaign themes to determine which one generates the greatest response rate


Promotion Cost

------------------------ = Break-even Number of Sales

Unit Profit per Sale

Break-even analysis


Samples & Estimations Random sample designs

Simple Random Samples Systematic Random Samples Stratified Random Samples Cluster Samples Replicated Samples Sequential Samples

Determination of sample size


Using Statistical Evaluation to Determine the Sample Size

Confidence level - the number of standard deviations from the mean in a normal distribution (for example a 95% confidence levels corresponds to 1.96 standard deviations from the mean)

Limit of error - the number of percentage points by which the researcher can miscalculate the actual response rate

Expected (actual) response rate - the number of positive responses expressed as a percentage of the total

Sample size - the number of observations in an experiment


Measurement of Differences

Hypothesis testing

Types of errors in hypothesis testing

Statistical evaluation of differences


Hypothesis TestingHypotheses are typically stated in negative terms; that

is, a null hypothesis (H0) versus an alternative hypothesis (Ha) in a form such as the following:

H0: Direct mail response from the test promotion is at or below direct mail response from the control promotion.

Ha: Direct mail response from the test promotion is above direct mad response from the control promotion.

The null hypothesis, then, states that direct mail response will not be better than the control. Our measurement sets out to disprove this null hypothesis.


Types of Errorin Hypothesis TestingType One: Results when the decision-maker rejects the

null hypothesis even though it is, in fact, true ... i.e., taking an action when one shouldn't

Type Two: Results when the decision-maker accepts the

null hypothesis when, in fact, not true ... i.e., not taking an action when one should.


Assume that a sample has been properly selected and is of an adequate size. Assume further that an experiment has been designed and implemented in a valid manner. It now remains for the direct marketer to be able to recognize the difference in the response rate from a test and that from a control, with some degree of confidence and within an acceptable limit of error.


Test Control Totals Response A C A + C Non-response B D B + D Total mailed A + B C + D A + B + C + D = N

The statistic 2 is computed as follows:2 = N [ (A x D) - (C x B) - N/2]2

(A+B) x (C+D) x (A+C) x (B+D) Here is a sample calculation:

Test Control Totals Response 200 100 300 Non-response 800 900 1700 Total mailed 1000 1000 2000

2 = 2,000 x [ 180,000 - 80,000 - 1,000]2

1,000 x 1,000 x 300 x 1,700

2 = 38.4 ... which is significant at the 99++% level since it exceeds the critical value in the X2 table for one degree of freedom for a significance level of 0.001, given as 10.83


STRUCTURING and EVALUATING AN EXPERIMENT

State the hypothesis Develop, by a priori analysis, the

assumptions required and compute the appropriate sample size

Structure and perform the experiment Develop, by a posteriori analysis,

statistics for judging hypothesis validity

Make the decision


CHAPTER CASE

THE AMERICAN HEART

ASSOCIATION


copyright © 2010 pearson education, inc. publishing as prentice hall conducting research and...

Documents

pearson education

prentice hallcopyright

prentice hallsourcing

prentice hallresearch

prentice hallproducts

prentice halltest

prentice hallneed

prentice hallplease