asme 2010 final - encsusers.encs.concordia.ca/~design/reference/asme_2010_assement.pdf · tangible...
TRANSCRIPT
1 Copyright © 2010 by ASME
Proceedings of the ASME 2010 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference
IDETC/CIE 2010 August 15-18, 2010, Montreal, Quebec, Canada
DETC2010-28197
ASSESSMENT OF ADVERTISING EFFECTIVENESS THROUGH AUDIENCE’S EYE MOVEMENTS
Shize Jin, Yong Zeng, Chun Wang Institution for Information System Engineering Faculty of Engineering and Computer Science
Concordia University {shi_jin, zeng, cwang}@ciise.concordia.ca
ABSTRACT The evaluation of advertisement effectiveness during the
advertisement design phase and pre-launch phase is critical for
the advertisement’s success in the targeted market. This
evaluation should predict advertisement’s final performance as
accurately as possible. In today’s advertisement business,
questionnaire-based evaluation methods, such as attitude and
opinion rating are widely used. To obtain good survey results,
high quality questionnaires and proper interviewing
procedures have to be developed with the support of the
competent execution and supervision. These activities are
usually costly even though some of them can be conducted
online. This paper proposes a novel method for assessing the
advertisement effectiveness through the automated capturing
and analyzing of audiences’ eye movements. This method is
based on the assumption that some attributes of audiences’ eye
movements are correlated to their visual attention defined in
the context of advertisement effectiveness. To validate our
research hypotheses, experiments were conducted. In the
experiments, subjects were required to watch several
advertisements in sequence and the subjects’ eye movement
data were collected simultaneously. By analyzing the data
patterns and comparing them with the effectiveness evaluation
obtained from questionnaire-based method, we found that the
proposed method produces similar evaluations to those
resulted from the traditional attitude and opinion rating
method.
1. INTRODUCTION Advertising effectiveness is concerned with making a
tangible contribution to a company or the brand by impacting
customers’ buying decisions through advertisements [1]. Based
on the objectives that an advertisement or advertising
campaign strives to achieve in markets, advertising
effectiveness can often be assessed by the effect on customer’s
short term and long term reactions [1]. While long term effects
can be gauged by the impact on customers’ structure of
decisions, attitude, preferences, beliefs and intentions as well
as the sales, short term effect is mainly represented by
customers’ attention to the advertising.
The effectiveness of advertising has significant financial
implications to the advertisers. Based on the advertising
objectives, the advertisers usually apply a variety of measures
to evaluate advertisement effectiveness before the final launch.
Commonly used measures include aided or unaided recall of
the brand name or advertisement and persuasion (beliefs,
attitude change, purchase intentions) [1]. However, obtaining
numerical values of these measures is not a trivial business.
Among many methods, such as opinion and attitude ratings,
recognition tests, objective methods, and laboratory testing and
analyses of content, opinion and attitude rating was the first
method widely applied in evaluating the effectiveness of
general consumer advertisements [2]. In the opinion and
attitude rating test, people are first provided with a scale and a
set of nouns or adjectives describing the advertisement; they
are asked to apply a scale or indicate their attitudes in relation
to the advertisement base on their feelings. The attitude rating
thus helps assess the advertisement effectiveness by examining
whether people are interested in, pay attention to, understand
and remember the information delivered by the advertisement.
Opinion and attitude rating test is usually conducted
through questionnaires after an advertisement is presented to
audiences. Many issues have to be addressed to guarantee a
good survey result. For example, only with a proper
development of questionnaires and interviewing procedure,
supported by the competent execution and survey supervision,
quality data can be collected and a good survey can be
achieved [2]. For a large number of customers (usually needed
to guarantee a quality survey result), opinion and attitude
rating test is a labor intensive, time consuming and costly
process. Furthermore, the results of this method sometimes are
inaccurate due to its subjectivity. People do not always say
2 Copyright © 2010 by ASME
what they really think and do; people may also forget and
change minds or make things up when they fill the
questionnaire.
While methods with the subjective nature, such as opinion
and attitude rating test and other questionnaire based methods,
are vulnerable to small influences coming from subjects’ inter
awareness and outer circumstances in different situations,
objective methods can be an effective alternative, which
provides experimental data for advertisement effectiveness
analysis. Among other effectiveness factors, customers’ visual
attention can be objectively assessed through their eye
movements. Customer’s attention is one of the significant
forms to represent the customer's short-term reaction [3],
which has considerable impacts on customers’ long term
reactions. Research in advertising effectiveness has concluded
that advertisements attracting the audience’s attention could
develop the potential preferences for the products or the
service in the future [4, 5, 6]. For example, Rossiter and Percy
pointed out that customers’ attention is capable of increasing
the customer’s product attitude and preference, which could
lead to the ultimate sales [4]. They believe that an
advertisement could guarantee a high memorability if it can
hold the customer’s attention for at least two seconds. The
primary concept of evaluating advertisement effectiveness
through customer’s attention is established by Miniard [5, 6].
They pointed out that it is important to know how and when
the final consumer would pay attention to the commercial
stimuli and to identify the critical factors that affect the
patterns and strategies related to customer’s attention. In
Advertising Response Modeling (ARM), which provides a
framework to assess advertising performance by means of
integrating several measures, it is clearly stated that gaining
the customer’s attention is the most important characteristic
that enables advertising to break through [1].
Although it is clear in the literature that customers’
attention has considerable impact on advertising effectiveness,
to our knowledge, it still remains to be an open question to
model the impact quantitatively. In this paper, we attempt to
quantify the relationship between customers’ visual attention
and advertising effectiveness through a set of controlled
experiments on audience’s eye movements. By analyzing the
experiment results, we conclude that the intensity of
customers’ visual attention correctly reflects the level of the
advertising effectiveness. This conclusion implies that eye-
tracking tools can be used for developing automated systems
to asses advertising effectiveness. This type of systems have
the potential of significantly reducing the labor and time costs
needed to evaluate advertising effectiveness. The rest of the
paper is organized as follows. Section 2 briefly describes the
eye-tracking method used to assess customers’ attention to the
advertisements. Section 3 introduces 2 experiments conducted
in this research. Finally, conclusions and future research
directions are presented in Section 4.
2. EYE-TRACKING METHOD AND EYE-DATA
Although there is not a complete one-to-one
correspondence between eye movement and attention, human
intention and interests can be revealed automatically by
tracking their eye movements [3].
Eye tracking can be described as a process to measure
where people look at or how eyes move relative to the head.
Different eye-trackers are invented to capture the eye
movements. Buswell built the first non-intrusive eye trackers
by recording on the film the beams of light that were reflected
on the eye [7]. Using the trackers he did systematic studies
into reading and picture viewing [8, 9]. In the 1950’s, Yarbus
conducted systematic research on the relationship between
human eye movements and thought processes [10]. It was
shown that there is a strong relationship between an observer’s
fixations and interest, which is reflected by the fact that the
observer's attention was usually focused on certain elements of
a picture. Since its invention, the eye tracking technologies
have been greatly improved. Many eye-tracking studies and
eye trackers are now used in cognitive, psychology, and
human-machine interface design [11]. Many publications have
revealed the close relationship between human eye movements
and psychological processes through tracking some critical
factors among the eye parameters. Experiments have
suggested that the attention played an important role in
voluntary eye movements [12].
Throughout the history of eye tracking research, several key
variables have emerged as significant indicators of ocular
behaviors, including fixation, saccade, pupil diameter, and
blinking frequency. By exploring the relationship between
attention and eye movement, researchers have found that
attention may affect saccade programming in different ways
[13]. Braun and Breitmeyer suggested that saccadic latencies
depend on the disengagement of attention from any location in
the visual field [14, 15]. O'Craven and his team found that eye
blink frequency becomes low during high attention conditions
[16]. Similar results were also obtained by Collins and Seeto
[17]. Furthermore, for eye blinking, there are 3 different types
of it. Two of which are regarded as blinking without external
stimuli (voluntary blinking and involuntary blinking). The
third type is reflex blinking, which is a rapid closure
movement of the eyelids. It is defined as a short duration
which responses to a variety of external stimuli, usually
auditory, cognitive, trigeminal or visual, including a
component of other motor behaviors. [18]
3. EXPERIMENT The objective of this study is to evaluate the advertisement
effectiveness through the assessment of audience’s attention
when they watch the advertisement. We focus on analyzing
one attribute of audience’s eye movements, blinking
frequency, because of two reasons. One is blinking frequency
is one of the most important indicators for visual attention, the
3 Copyright © 2010 by ASME
other is viewing advertising is a kind of external stimuli to eye
which causes eye blinking.
The hypothesis underlying our study is that the audience’s
attention can be assessed by studying audience’s eye
movements. The following two specific hypotheses were
tested in our study:
[H1]. Audience’s eye movement attributes are correlated
to their attention while watching TV advertisements.
[H2]. The audience’s eye data captured during TV
advertisement watching could quantitatively reflect the level of
audience’s attention, which is in line with the result obtained
by the traditional questionnaire-based method.
To verify the above hypotheses, two experiments were
devised and conducted. Note that, the acceptance of H1is the
precondition of conducting the H2 verification experiments.
In our experiments, we have used faceLAB4.5 as our eye-
tracking system. The faceLAB4.5, developed by “Seeing
Machines”, is a high accuracy vision-based eye-tracking
system. The system continuously monitors the head pose, gaze
direction and eyelid closure information in real-time manner.
No equipment needs to be worn during the testing. Figure 1a)
shows the hardware system of faceLab 4.5; Figure 1 b) shows
the systems’ real time eye-tracking user interface.
a) Hardware of faceLAB Eye-Tracking System
b) GUI of faceLAB Eye-Tracking System
Figure 1 faceLAB Eye-Tracking System
3.1. Experiment 1 This experiment was conducted to exam hypothesis H1.
We want to know that whether the audience’s eye movement
attributes are correlated to their visual attention while
watching TV advertisements. Here the main objective is to
obtain a qualitative conclusion.
3.1.1. Method Eight 30-second television commercials were selected and
divided into two groups as stimuli with each group containing
four TV commercials. In the first group, the four TV
commercials were chosen from a number of high-ranking TV
advertisements on www.youtube.com. In contrast, in the other
group, four TV advertisements with low-ranking were
selected. This experiment is designed as within-subjects,
where each subject was asked to watch two groups of TV
commercials and the subjects’ eye data were recorded
simultaneously by an eye-tracker (faceLAB system). The
collected eye data is analyzed to find out whether the eye
movements would change with the audience’s attention when
they watched the TV advertisements.
3.1.2. Subjects Five graduate students from Concordia University
voluntarily participated in the research. They were regular TV
audiences. English is their native or working language. Each
experiment for one subject lasted for approximately fifteen
minutes.
3.1.3. Procedure Subjects were invited respectively to come to the lab
where the experiment took place. After signing the consent
form, the subject was asked to watch the two groups of TV
commercials in sequence and their eye-data was recorded.
Subjects were asked to sit in front of the LCD at ease, and the
eye-tracker was placed lower than the LCD directly facing to
the subject. Experiment setting was shown in Figure 2. After
an explanation of the eye-tracking system, calibration of the
subject’s eye took place. Hence, when a subject was watching
the advertisements, his/her eye movements were observed
simultaneously by the eye-tracker (faceLAB system).
Figure 2 Experiment setting
4 Copyright © 2010 by ASME
3.1.4. Result As shown in Figure 3, the subject’s blinking frequency
(BF) was lower when they watched ads with the higher
ranking (blue bars) than when they watched ads with the lower
ranking (red bars). Hence, two groups of BF data were
captured while audiences watching randomly selected high
ranking and low ranking TV ads. Sample BF data were scatter
plotted in Figure 4. Mean values of BF associated with high
ranking TV ads and low ranking TV ads were presented in
Table 1.
Figure 3 Pattern of audience’s blinking frequency
when they watch low and high ranking TV ads
0.180.160.140.120.100.080.060.040.02
99
95
90
80
70
60
50
40
30
20
10
5
1
C1
Percent
Mean 0.1045
StDev 0.02185
N 14
AD 1.075
P-Value 0.005
Normal - 95% CI
Probability Plot of High Ranking ADs
a) Probability plot of high ranking ADs
0.300.250.200.150.100.050.00
99
95
90
80
70
60
50
40
30
20
10
5
1
C1
Percent
Mean 0.1432
StDev 0.04252
N 21
AD 0.734
P-Value 0.047
Normal - 95% CI
Probability Plot of Low Ranking Ads
b) Probability plot of low ranking Ads
Figure 4 Probability plot of high and low ranking Ads
In addition, we employed hypothesis T-testing to verify
whether the audience’ BF were lower when watching high
ranking TV ads. The T-testing construction is shown in the
following equations.
H0:μ1 − μ2 = 0 (1)
H1:μ1 − μ2 < 0 (2)
Sp2 = n1 − 1n1 + n2 − 2 S12 + n2 − 1
n1 + n2 − 2 S22
(3)
t0 = x�1 − x�2Sp� 1n1 + 1n2
(4)
In equations (1) and (2), μ� and μ� are the mean of the
audience’s BF when they watched high ranking TV ads and the
when they watched low ranking TV ads respectively.
Table 1 T-test: Subject’s blinking frequency while watching TV ads with high and low rankings
T-test
TV ADs with High Ranking TV ADs with Low Ranking
Subject Mean SD Sample1 size
Mean SD Sample2 size
t0
S1 0.32379 0.008 6844 0.36201 0.011 6843 (232.46)
S2 0.34276 0.017 6203 0.57997 0.03 6934 (548.88)
S3 0.20789 0.004 6782 0.24539 0.01 6964 (287.25)
S4 0.09711 0.003 6684 0.15567 0.011 6864 (420.28)
S5 0.08093 0.013 6963 0.22749 0.008 6984 (802.07)
0
0.5
1
S1 S2 S3 S4 S5
Blin
kin
g F
req
ue
ncy
Blinking Frequency
(High Ranking ADs VS Low Ranking ADs)
TV ADs with
high Ranking
TV Ads with
low Ranking
5 Copyright © 2010 by ASME
If t� < −t∝,���� 2 , we can reject H� (1) and accept
H� (2), which means the audience’ BF was lower for watching
high ranking TV ads than for watching low ranking TV ads.
The subject’s BF while watching high and low ranking TV ads
is represented by as sample 1 and sample 2, respectively, as
shown in Table 1. It was derived that subject’s BF were lower
while watching high ranking TV ads than while watching low
ranking TV ads. (α = 0.005, −t∝,���� − 2 = −4.576,t� < −4.576).
Experiment 1 provides convincing evidence in favor of our
Hypothesis H1. It suggests that the subjects blinked more
frequently when they watched unattractive TV ads. The main
finding of this experiment supports H1.
3.2. Experiment 2 Our first experiment supports H1. Experiment 2 goes one
step further and aims to verify the correlation pattern between
audiences’ eye movements and attention paid to the
advertisements.
3.2.1. Method Six 30-second TV commercials were selected as stimuli.
Subjects’ brand attitude and brand preference, as well as the
advertisements playing sequence which would affect the result.
Considering above facts, the stimuli contained several different
brands and different types of fast moving consumer goods
(FMCG). Furthermore, the stimuli were pre-edited in six
sequences. Each subject was asked to watch one sequence. The
intervals between each advertisement was inserted a 10–second
MTV, which enable the subject indentify each separate ad
clearly. The subjects were users of the advertising products.
Subjects were asked to watch TV advertisements in
sequence and the subjects’ eye-movements were recorded
simultaneously by an eye-tracker (faceLab system). After the
subjects finish watching the TV ads, the subjects were asked to
finish an attitude rating survey pre-designed to evaluate the
different advertisement’s attraction, which is the traditional
advertisement evaluation method. Finally, the results from two
approaches were analyzed to detect the correlation pattern.
As a traditional method, attitude rating survey is used to
evaluate the advertisement effectiveness through questionnaire.
In our attitude rating survey experiment, several items were
used to measure the advertisement effectiveness with an
emphasis on the visual attention aspect. Each item was
measured on a seven-point scale. Table 2 shows a sample
questionnaire. Relevant keywords, such as “appealing”, “eye-
catching”, “favorable” and “memorable” were selected to elicit
the level of attention that audiences pay to an advertisement.
Once the data collection from questionnaires was
completed, mean, standard deviation, min and max were
calculated for each item. A total score was generated by adding
the scores for all the items previously defined in the
questionnaire. The item-to-total result shows the attraction level
of each tested TV advertisement.
Furthermore, the data collected from eye-tracking device is
analyzed and compared to those obtained from the
questionnaire. Based on the analysis of the effectiveness results
from two methods, we attempt to verify that, for the same set of
advertisements, the attention patterns obtained through
questionnaire and eye-tracking are similar or identical.
Table 2 Sample questionnaire for attitude rating survey in evaluating the advertisement’s attraction
On each of the scale below, please check the space that you feel best describe the advertisement you have just watched
Unappealing 1 2 3 4 5 6 7 Appealing Not Eye Catching 1 2 3 4 5 6 7 Eye Catching
What is your overall reaction to the above advertisement?
Unfavorable 1 2 3 4 5 6 7 Favorable
How memorable did you find this ad
Unmemorable 1 2 3 4 5 6 7 Memorable *Seven-point scales used; “7” represents the most favorable (for the greatest amount of association) and “1”the least favorable (or the
smallest amount of association)
6 Copyright © 2010 by ASME
3.2.2. Subjects Six graduate students from Concordia University
voluntarily participated in the research. They were regular TV
audiences. English is their native or working language. Each
experiment of one subject lasted for approximately fifteen
minutes.
3.2.3. Procedure Subjects were invited respectively to come to the LAB
where the experiment took place. We conducted the Experiment
2 with the exactly same facility setting and facility calibration
procedure used in Experiment 1. However, different stimuli
materials were used.
Before the experiment, the potential subject was given the
following statement and instruction: “We are interested in
obtaining your opinions concerning particular test
advertisement. You will be shown 6 TV commercials
uninterrupted, and then you will be asked several questions
concerned with your attitude towards these advertisements.
Furthermore, we’ll record your eye movements simultaneously
when you watch the advertisements.”
After reading the statement above, if the potential subjects
agreed to participate in the experiment, they will sign the
“consent form to participate in research” before the experiment.
The subject was asked to sit in front of the LCD at ease for the
eye-tracking facility calibration. Generally, it takes the subject
around 4 minutes. Afterwards, the experiment starts, the stimuli
(6 TV advertisements) are shown to a subject in a monitor with
a comfortable size and distance.
After watching 6 TV commercials, the subject will be shown a
card with the seven semantic differential scales, which was first
applied by Mindak in the advertising research problems [1].
The card requires the subject to give the most negative ad a
score of 1 and the most positive ad a score of 7. This scale will
be used to answer questions such as those listed in Table 2.
3.2.4. Result All the questionnaire items in our analysis used a 7-point
scale to examine the TV ad’s performance in order to find out
which TV ad was the most attractive. Means and standard
deviations reflect the level of affection and attractiveness. The
final result through the attitude rating survey is reported in
Table 3. Furthermore, Figure 5 illustrates the ranking of stimuli.
It shows that, among 6 test TV ads, AD2 gained the highest
score (Mean Weight =5.25), which means that it was the most
attractive one. Accordingly, AD6 gained the lowest score (Mean
Weight = 2.84), which means that it was the most unattractive
one to the audiences.
Figure 5 Ranking list of attitude rating survey
Table 3 Response of Attitude Rating Survey
Attitude Rating SurveyAttitude Rating SurveyAttitude Rating SurveyAttitude Rating Survey
AD1AD1AD1AD1 AD2AD2AD2AD2 AD2AD2AD2AD2 AD4AD4AD4AD4 AD5AD5AD5AD5 AD6AD6AD6AD6
Weight Weight Weight Weight Mean Mean Mean Mean SDSDSDSD Mean Mean Mean Mean SDSDSDSD Mean Mean Mean Mean SDSDSDSD Mean Mean Mean Mean SDSDSDSD Mean Mean Mean Mean SDSDSDSD Mean Mean Mean Mean SDSDSDSD
Appealing Appealing Appealing Appealing 25% 4.33 1.03 5.00 0.63 4.67 0.52 4.17 0.41 3.83 0.41 3.00 0.89 Eye Eye Eye Eye catching catching catching catching
25% 4.17 0.75 5.33 0.82 4.33 0.82 3.67 0.52 3.50 0.55 2.50 0.55
Favorable Favorable Favorable Favorable 25% 4.33 0.82 5.50 0.55 4.00 0.89 4.17 0.41 4.00 0.63 2.67 0.82
Memorable Memorable Memorable Memorable 25% 4.17 0.75 5.17 0.75 3.83 0.75 3.83 0.41 3.67 0.52 3.17 0.98
Mean Mean Mean Mean Weight Weight Weight Weight
4.254.254.254.25 5.255.255.255.25 4.214.214.214.21 3.963.963.963.96 3.753.753.753.75 2.842.842.842.84
*Seven-point scales used; “7” represents the most favorable (for the greatest amount of association) and “1”the least favorable (or the
smallest amount of association)
0
2
4
6
AD1 AD2 AD3 AD4 AD5 AD6
Ranking List of Attitude Rating Survey
Attitude
Rate
Blinking Frequency (times/second)Blinking Frequency (times/second)Blinking Frequency (times/second)Blinking Frequency (times/second)
S1S1S1S1 S2S2S2S2 Mean SD Mean SD
AD1AD1AD1AD1 0.4084 0.006 0.1596 0.007
AD2AD2AD2AD2 0.3906 0.021 0.1524 2.776-16
AD3AD3AD3AD3 0.5902 0.013 0.1956 0.006
AD4AD4AD4AD4 0.4463 0.035 0.1573 0.014
AD5AD5AD5AD5 0.5282 0.019 0.1737 0.008
AD6AD6AD6AD6 0.6451 0.068 0.3034 0.080
The results of blinking frequency (BF) thro
tracking method are shown in Table 4, which includes the
means and standard deviations of each subject’s
watched 6 test TV ads. From Table 4, two main findings were
obtained: one was that all the subjects’ BF was the
they watched AD2 (data in frame) whereas the other was that
except S3, the subjects’ BF was the highest when they watched
AD6 (data in dashed frame). Base on the result of Experiment
1, that for the advertisements audiences, BF would
when they watched attractive ads than they watched
unattractive ads. Thus this indicates that, in Experiment 2, AD2
was the most attractive and AD6 was the most unattractive
among the 6 stimuli. This result complies with
the attitude rating survey. Table 5 shows the detailed
advertisement ranking results obtained from both eye tracking
method and the attitude rating survey. It is important to note
that the two methods give identical overall rankings
of 6 test TV ads.
ADADADAD Ranking result Ranking result Ranking result Ranking result (Attitude rating survey )(Attitude rating survey )(Attitude rating survey )(Attitude rating survey )
Ranking resultRanking resultRanking resultRanking result(Eye Tracking )(Eye Tracking )(Eye Tracking )(Eye Tracking )
AD1AD1AD1AD1 2222 2222 AD2AD2AD2AD2 1111 1111 AD3AD3AD3AD3 3333 3333 AD4AD4AD4AD4 4444 4444
AD5AD5AD5AD5 5555 5555 AD6AD6AD6AD6 6666 6666
Table 5 Advertisement ranking (Attitude rating survey vs BF)
7
Table 4 Blinking Frequency
Blinking Frequency (times/second)Blinking Frequency (times/second)Blinking Frequency (times/second)Blinking Frequency (times/second)
S3S3S3S3 S4S4S4S4 S5S5S5S5 mean SD mean SD mean SD
0.007 0.2559 0.008 0.2494 0.014 0.2685 0.013
16 0.2028 0.023 0.2188 0.008 0.2293 0.012
0.006 0.2180 0.016 0.2820 0.035 0.2384 0.009
0.014 0.4203 0.029 0.2681 0.006 0.2752 0.013
0.008 0.3148 0.018 0.3191 0.025 0.2853 0.011
0.080 0.380 0.032 0.3348 0.007 0.3457 0.055
through the eye-
which includes the
means and standard deviations of each subject’s BF when they
two main findings were
the lowest when
he other was that
when they watched
AD6 (data in dashed frame). Base on the result of Experiment
would be lower
n they watched
in Experiment 2, AD2
most unattractive
complies with the result from
shows the detailed
obtained from both eye tracking
method and the attitude rating survey. It is important to note
nkings for a group
Ranking resultRanking resultRanking resultRanking result (Eye Tracking )(Eye Tracking )(Eye Tracking )(Eye Tracking )
Attitude rating survey
a) Attitude rating survey vs BF (S1)
b) Attitude rating survey vs BF (S2)
Copyright © 2010 by ASME
S6S6S6S6 SD mean SD
0.013 0.2359 0.010
0.012 0.2058 0.026
0.009 0.2143 0.000
0.013 0.2147 0.011
0.011 0.2651 0.028
0.055 0.3438 0.050
survey vs BF (S1)
b) Attitude rating survey vs BF (S2)
c) Attitude rating survey vs BF (S3)
d) Attitude rating survey vs BF (S4)
e) Attitude rating survey vs BF (S5)
f) Attitude rating survey vs BF (S6)
Figure 6 Attitude rating survey vs BF
8
vs BF
The comparison of subjects’ eye movement
data and their attitude survey results shows
effectiveness ranking for subjects S1, S2, S5 are
Attitude survey results show that
regarded AD2 as the most attractive
unattractive among the stimuli. Also note that
plotted in Figure 6a,b,e in red)
highest BFs were generated respectively
watched AD2 and AD6. The eye movement data
subjects suggests that AD2 was
AD6 was the most unattractive.
discrepancies between the ranking results
both methods, the discrepancy is minor and
conclude that the results of eye
reflect the level of the advertisement effectiveness obtained
through traditional attitude rating surveys.
4. CONCLUSIONS AND FUTURE WORKExisting studies indicate that eye activities ha
relationship with attention and that eye
the human visual attention. In the present
objective approach to quantifying advertisement effectiveness
through capturing and analyzing audiences’ eye movement
attributes. We conducted two
hypotheses and to find a quantitative relationship between the
values of eye movement attributes and the levels of
advertisement effectiveness. Our experiments show that, while
watching TV advertisements, audiences’ eye
is correlated to the audience’s attention.
analyzing the patterns of audience’s
effectiveness ranking of a set of TV advertisements in terms of
audiences’ attention and the predicted ranking is identical to
those obtained from traditional attitude rating surveys.
results provide considerable evidence that
TV advertisement is able to be
patterns when they watch TV ad
raises the possibility of developing automated advertisement
effectiveness evaluation systems, which are significantly more
cost effective than traditional questionnaire
The work presented in this paper is still at
stage. Additional experiments are needed
insights of the relationship between eye movements and
advertisement attention. To this end
invited to participate in the future research. They will be of the
different ages and occupations. Furthermore,
systematic within-subjects and between
also plan to study the pattern of audiences’ attention changing
when they watch TV advertisements
ACKNOWLEDGEMENT This project is partially by an NSERC
(Grant number RGPIN 298255)
program.
Copyright © 2010 by ASME
eye movement experimental
and their attitude survey results shows that the
for subjects S1, S2, S5 are identical.
ttitude survey results show that all of these three subjects
most attractive and AD6 as the most
Also note that, the mean of BF (
) shows that the lowest and
were generated respectively while the audiences
eye movement data of these 3
2 was the most attractive and that
most unattractive. Although there were some
ranking results of S3, S4 and S6 in
the discrepancy is minor and it is reasonable to
of eye-tracking method correctly
advertisement effectiveness obtained
aditional attitude rating surveys.
AND FUTURE WORK that eye activities have a close
that eye-data would change with
In the present research, we study an
approach to quantifying advertisement effectiveness
through capturing and analyzing audiences’ eye movement
wo experiments to verify our
hypotheses and to find a quantitative relationship between the
nt attributes and the levels of
Our experiments show that, while
watching TV advertisements, audiences’ eye blinking frequency
the audience’s attention. In addition, by
patterns of audience’s BF, we can predict the
of a set of TV advertisements in terms of
audiences’ attention and the predicted ranking is identical to
those obtained from traditional attitude rating surveys. These
evidence that the effectiveness of
be measured by audience’s BF
TV advertisements. This, in turn,
raises the possibility of developing automated advertisement
effectiveness evaluation systems, which are significantly more
cost effective than traditional questionnaire-based methods.
sented in this paper is still at its preliminary
are needed to obtain more
insights of the relationship between eye movements and
attention. To this end, more subjects will be
invited to participate in the future research. They will be of the
Furthermore, we will conduct
subjects and between-subjects studies. We
pattern of audiences’ attention changing
when they watch TV advertisements.
This project is partially by an NSERC Discovery Grant
and Canada Research Chair
9 Copyright © 2010 by ASME
REFERENCES
[1]. Wells, W.D., Measuring advertising effectiveness.
1997, London: LAWRENCE ERLBAUM
ASSOCIATTES, PUBLISHERS.
[2]. Darrell Blaine Lucas, S.H.B., Measuring
Advertising Effectiveness. 1963: McGRAW-HILL
Book Company. INC.
[3]. Rick G.M. Pieters and M.W. Edward Rosbergen.
Visual attention to advertising: The impact of
motivation and repetition. in ACR- conference.
1995.
[4]. Rossiter, J.R. and L. Percy, Attitude Change
through Visual Imagery in Advertising. Journal of
Advertising, 1980. 9(2): p. 10-16.
[5]. Edward Rosbergen, R.P., Michel Wedel Visual
Attention to Advertising: A Segment A Level
Analysis. Chicago Journals, 1997. 24(December):
p. 305-314.
[6]. Ralph Radach, S.L., Christian Vorstius, Dieter
Heller, Karina Radach, Eye Movements in the
processing of print advertisements. 2003:
Hoyona, J., Rodach, R. & Deubel, H. (2003). The
Mind's Eye: Cognitive and Applied Aspects of
Eye Movement Research. Elsevvier Science:
Oxford 609-623.
[7]. Buswell, G., Fundamental reading habits: A
study of their development. 1922: The University
of Chicago Press.
[8]. Buswell, G.T., ed. How people look at pictures.
1935, Univ. Chicago Press 137–55.
[9]. Buswell, G.T., How adults read. 1937: University
of Chicago Press.
[10]. A.L.Yarbus, The motion of the eye in the process
of changing points of fixation, Biofizika. 1954: p.
76-78.
[11]. Minoru, N., T. Koji, and S. Yasutaka, The act of
task difficulty and eye-movement frequency for
the 'Oculo-motor indices', in Proceedings of the
2002 symposium on Eye tracking research \&
applications. 2002, ACM: New Orleans,
Louisiana.
[12]. Shepherd, M.F., John M.; Hockey, Robert J. , The
relationship between eye movements and spatial
attention. The Quarterly Journal of Experimental
Psychology 1986. 38: p. 475-491.
[13]. M.Findlay, J., Saccadic eye movement
programming: sensory and attentional
Psychological Research, 2008. 73(Number 2/
March 2009): p. 127-135.
[14]. D. Braun1, B.G.B., Relationship between directed
visual attention and saccadic reaction times
Experimental Brain Research 1988. 73(Number
3/December, 1988): p. 546-552.
[15]. James E.Hoffmam, B.S. The role of visual
attention in saccadi eye movements. in 32nd
Anuual Meeting of the Psychonomic Society.
1991. San Francisco, CA: University of
Delaware, Newark, Delaware.
[16]. Kathleen M.O'Craven, B.R.R., Kenneth K.
Kwong, Anne Treisman, Robert L. Savory,
Voluntary attention modulates fMRI activity in
human MT-MST. Neuron, 1997. 18 p. 591-598.
[17]. Michael Collins, R.S., Louella Campbell, Murray
Ross, Blinking and corneal sensitivity. Acta
Ophthalmologica, 1989. 67(5): p. 525-531.
[18]. Jacob Klen, Peder Wolkoff, Changes in eye blink
frequency as a measure of trigeminal stimulation
by exposure to limonene oxidation products,
isoprene oxidation products, isoprene oxidation
products and nitrate radicals. Int Arch Occup
Environ Health, (2004) 77: 235–243