8Measuring the Effectiveness of the Promotional Program

© 2005 McGraw-Hill Ryerson Limited

Chapter Objectives

• To understand reasons for measuring promotional program effectiveness.

• To know the various measures used in assessing promotional program effectiveness.

• To evaluate alternative methods for measuring promotional program effectiveness.

• To understand the requirements of proper effectiveness research.

© 2005 McGraw-Hill Ryerson Limited

Measuring Advertising Effects

Product QualityProduct Quality

PromotionPromotion

DistributionDistribution

CompetitionCompetition

The EconomyThe Economy

Price PolicyPrice PolicySALES

SALES?

© 2005 McGraw-Hill Ryerson Limited

For and Against Measuring Effectiveness

Reasons to MeasureReasons to Measure Reasons Reasons NotNot to Measure to Measure

Problems With ResearchProblems With Research

Creative ObjectionsCreative Objections

Cost of MeasurementCost of Measurement

Disagreement About What to Test

Disagreement About What to Test

Avoid Costly MistakesAvoid Costly Mistakes

Evaluate StrategiesEvaluate Strategies

Increase Efficiency of Advertising in GeneralIncrease Efficiency of Advertising in General

Determine If Objectives Are Achieved

Determine If Objectives Are Achieved

Problems With ResearchProblems With Research

Cost of MeasurementCost of Measurement

Disagreement About What to Test

Disagreement About What to Test

Avoid Costly MistakesAvoid Costly Mistakes

Evaluate StrategiesEvaluate Strategies

Increase Efficiency of Advertising in GeneralIncrease Efficiency of Advertising in General

Determine If Objectives Are Achieved

Determine If Objectives Are Achieved

© 2005 McGraw-Hill Ryerson Limited

Measuring Advertising Effects

• PRETESTING– Concept tests

– Rough testing

• COMMERCIAL TESTING– Consumer juries

– Physiological measures

– Theatre tests

• MARKET TESTING– on-air (split run cable) tests

– Recall and Recognition tests (day after)

© 2005 McGraw-Hill Ryerson Limited

The Testing Process

3.Finished Art or Commercial Testing

3.Finished Art or Commercial Testing

3.Finished Art or Commercial Testing

3.Finished Art or Commercial Testing

1.Concept Testing1.Concept Testing

2.Rough Testing2.Rough Testing

4.Market Testing (Post-testing)

4.Market Testing (Post-testing)

2.Rough Testing2.Rough Testing

1.Concept Testing1.Concept Testing

Occurs at Various Stages

Occurs at Various Stages

8Concept Testing

© 2005 McGraw-Hill Ryerson Limited

Concept Generation and Testing

Qualitative And/or Quantitative Data Evaluating and Comparing Alternative ConceptsQualitative And/or Quantitative Data Evaluating and Comparing Alternative Concepts

Explores Consumers’ Responses to Ad Concepts Expressed in Words, Pictures, or SymbolsExplores Consumers’ Responses to Ad Concepts Expressed in Words, Pictures, or Symbols

Alternatives Are Exposed to Consumers Who Match the the Target AudienceAlternatives Are Exposed to Consumers Who Match the the Target Audience

Reactions and Evaluations Are Sought Through Focus Groups, Direct Questioning, Surveys, Etc.Reactions and Evaluations Are Sought Through Focus Groups, Direct Questioning, Surveys, Etc.

Sample Sizes Depend on the Number of Concepts and the Consensus of ResponsesSample Sizes Depend on the Number of Concepts and the Consensus of Responses

Explores Consumers’ Responses to Ad Concepts Expressed in Words, Pictures, or SymbolsExplores Consumers’ Responses to Ad Concepts Expressed in Words, Pictures, or Symbols

Alternatives Are Exposed to Consumers Who Match the the Target AudienceAlternatives Are Exposed to Consumers Who Match the the Target Audience

Reactions and Evaluations Are Sought Through Focus Groups, Direct Questioning, Surveys, Etc.Reactions and Evaluations Are Sought Through Focus Groups, Direct Questioning, Surveys, Etc.

Sample Sizes Depend on the Number of Concepts and the Consensus of ResponsesSample Sizes Depend on the Number of Concepts and the Consensus of Responses

ObjectiveObjective

MethodMethod

OutputOutput

© 2005 McGraw-Hill Ryerson Limited

Research House (Canada)

© 2005 McGraw-Hill Ryerson Limited

Concept Testing

• Limitations:

– Sample sizes are small (not quantifiable)

– Consumers become instant “experts”

– Members may not represent the target market

– Results may seem definitive than they are

• Company example

8Commercial Testing

© 2005 McGraw-Hill Ryerson Limited

Rough Art, Copy, and Commercial Testing

Number of Ads That Can Be Evaluated Is Limited

Number of Ads That Can Be Evaluated Is Limited

Preference for Ads Types May Overshadow ObjectivityPreference for Ads Types

May Overshadow Objectivity

Consumer May Become a Self-appointed Expert

Consumer May Become a Self-appointed Expert

A Halo Effect Is PossibleA Halo Effect Is Possible

Cost EffectivenessCost Effectiveness

Endorsements by Independent Third Parties

Endorsements by Independent Third Parties

Achievement of CredibilityAchievement of Credibility

Comprehension and Reaction TestsComprehension and Reaction Tests

Consumer JuriesConsumer Juries

ControlControl

AdvantagesAdvantages DisadvantagesDisadvantages

Number of Ads That Can Be Evaluated Is Limited

Number of Ads That Can Be Evaluated Is Limited

Consumer May Become a Self-appointed Expert

Consumer May Become a Self-appointed Expert

A Halo Effect Is PossibleA Halo Effect Is Possible

ControlControl

Cost EffectivenessCost Effectiveness

Endorsements by Independent Third Parties

Endorsements by Independent Third Parties

Achievement of CredibilityAchievement of Credibility

Comprehension and Reaction TestsComprehension and Reaction Tests

© 2005 McGraw-Hill Ryerson Limited

Consumer Juries

Potential viewers evaluate ads

• 50 to 100 participants

• Overall reaction to each ad

• And rank ordering of ads

• Examples of questions:

1. Which ads would you most likely read?

2. Which headlines interests you the most?

3. Which ad convinces you of the product’s quality?

4. Which ad did you like best?

5. Which ad did you find most interesting?

• Company example

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

Starch Test

Used for print Ads

• Asses what people saw

• Measures what they remember

• Used to compare ads

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

Physiological Tests of Finished Ads

1. Pupil dilation

2. Galvanic skin response

3. Eye tracking

4. Brain waves

© 2005 McGraw-Hill Ryerson Limited

Pupil Dilation

Size of Subjects pupils are measured

– Action Dilation

– Disinterest Constriction

Male

Female

NudeFemale

NudeMale

NudeInfant

© 2005 McGraw-Hill Ryerson Limited

Pupil Dilation - Camera

© 2005 McGraw-Hill Ryerson Limited

Galvanic Skin Response

Measures sweating

• GSR responds to:• feelings

• attention

• may indicate:• long-term recall

• effectiveness

© 2005 McGraw-Hill Ryerson Limited

GSR Devices

© 2005 McGraw-Hill Ryerson Limited

Brain wave activity (EEG)

Measures brain wave activity

1. Alpha activity is bad

• Associated with

Inactivity, resting & sleeping

• So: avoid ads with high alpha

2. Hemispheric lateralization

• Distinguishes between activity in the

different hemispheres of the brain

Tells you what people are

attending to

© 2005 McGraw-Hill Ryerson Limited

Cap

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

Eye Movements

Measures what is looked at • Method

•Eye camera and advanced electronics

• Output:

– Eye movements tell what is seen

– And what will be…•comprehended

•recalled

© 2005 McGraw-Hill Ryerson Limited

Device

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

© 2005 McGraw-Hill Ryerson Limited

Theater tests

– Product preferences

– Interest in the commercial

– Reactions (an adjective checklist)

– Recall of particular aspects

– Interest in the brand

– Continuous (frame-by-frame) reactions

© 2005 McGraw-Hill Ryerson Limited

Theater Test

© 2005 McGraw-Hill Ryerson Limited

Theater Test – Continuous Reaction Measures

2220181614121086420

1000

900

800

700

600

500

400

300

200

100

0Scores by Total Sample

Composite 48 676 8

© 2005 McGraw-Hill Ryerson Limited

Theater Test – Recall Measures

8Market (Post) Tests

© 2005 McGraw-Hill Ryerson Limited

Broadcast Tests

• On-air tests

– TV programs in specific markets

– Day-after recall

• Split Cable studies

– Ad run in one area, but not another

– One ad in one area, different ad in the other

– Day after recall

– SALES

© 2005 McGraw-Hill Ryerson Limited

1. Please look over these pictures and words from a TV commercial.

                                                                                                                                            

(Announcer) Remember Jared from Name? Turns out he has inspired a lot of people. (Singer) …

You’re still looking good. He’ll show you the way. His name is Jared, and he’ll lead you to Name.

2. Do you remember seeing that commercial on TV?

Yes No Not sure

3. How interested are you in what that TV commercial is trying to tell you or show you about what's

being advertised?

Very interested Somewhat interested Not interested

4. How does it make you feel about what's being advertised?

Good OK Bad Not sure

5. Please check any of the following if you feel they describe that commercial.

Amusing Appealing Familiar Fast moving Pointless Seen a lot

© 2005 McGraw-Hill Ryerson Limited

Summary

• 4 Basic Types of Ad Research:

– Concept tests

– Pilot tests

– Post tests

• Each has it’s own use

– Use the one appropriate for what you want to know

© 2005 McGraw-Hill Ryerson Limited

Quote of the day

Advertising people who ignore

research are as dangerous as

generals who ignore decodes of

enemy signals.

• David Ogilvy

(Ogilvy & Mather)