Copyright © 2010 by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions Dept, ACM Inc., fax +1 (212) 869-0481 or e-mail permissions@acm.org. ETRA 2010, Austin, TX, March 22 – 24, 2010. © 2010 ACM 978-1-60558-994-7/10/0003 $10.00

Visual span and other parameters for the generation of heatmaps

Pieter BlignautDepartment of Computer Science and Informatics, University of the Free State, South Africa

Abstract

Although heat maps are commonly provided by eye-tracking andvisualization tools, they have some disadvantages and cautionmust be taken when using them to draw conclusions on eyetracking results. It is motivated here that visual span is anessential component of visualizations of eye-tracking data and analgorithm is proposed to allow the analyst to set the visual span asa parameter prior to generation of a heat map.

Although the ideas are not novel, the algorithm also indicates howtransparency of the heat map can be achieved and how the colorgradient can be generated to represent the probability for an objectto be observed within the defined visual span. The optionaladdition of contour lines provides a way to visualize separateintervals in the continuous color map.

Keywords: Eye-tracking, Visualization, Heatmaps

CR Categories: H.5.2 [Information Interfaces andPresentation]: User Interfaces; I6.9c [Simulation and Modeling]:and Visualization: Information visualization)

1. Introduction

A fixation may be thought of as the mean x and y positioncoordinates measured over a minimum period of time duringwhich the eye does not move more than a certain maximumamount [Eyenal 2001]. Therefore, the point of regard (POR), i.e.the gaze coordinates at a specific moment in time, mustcontinuously remain within a small area for some minimum timefor it to be regarded as a fixation.

Several techniques exist in which eye-tracking data can bevisualized. Bar graphs, for example, may be used to show thenumber of fixations or visitors or average time spent per area ofinterest (AOI). Techniques also exist to overlay the originalstimulus with visualizations in order to guide the analyst towardsconclusions. Scan paths, for example, may be used to indicate theposition of fixations with dots that overlie an image of the originalstimulus. The dots may be connected with lines to indicate thetemporal relationship or saccades between fixations while theradius of the dots can, optionally, represent fixation duration.

Heat maps are semi-transparent, multi-colored layers that coverareas of higher attention with warmer colors and areas of lessattention with cooler colors. Instead of highlighting the areas ofhigher attention with red, they can be left uncolored while theareas of lesser attention are dimmed to a degree that correspondsto the amount of attention [Tobii Technology 2008; Spakov and

Miniotas 2007]. Three-dimensional fixation maps can be used tomake the heat map graphically more attractive, but they tend to beless informative since the further parts of the image are shownwith less detail and are obscured with peaks at the near end [TobiiTechnology 2008; Wooding 2002].

Despite their informative nature, heatmaps have disadvantages aswell. Bojko [2009] lists several points of caution when usingheatmaps and provides a number of guidelines when usingheatmaps. Bojko [2009] and Blignaut [2009] highlight theimportance of the algorithm and parameters that are used toidentify fixations. Three other aspects that can lead to erroneousinterpretation of eye tracking data must also be considered.Firstly, if the difference in time spent between areas with littleattention and areas with much attention is large, the areas withlittle attention might not be colored clearly enough and can bemistaken as not being observed at all. Secondly, the visual span,or foveal field of view, of an individual determines the amount ofinformation that can be observed with peripheral vision. Thirdly,the transitions from one color to the next are not sharp and it isdifficult to interpret the colors in terms of a numeric value for thespecific metric of attention that is used.

This paper focuses on heatmaps as a visualization technique foreye-tracking data. An algorithm to generate heatmaps isdiscussed. User-defined parameters, such as visual span,transparency, color range, and the probability for an object to beobserved at a specific distance from the centre of a fixation areincluded in this algorithm. The use of contour lines to visualizeseparate intervals in the continuous color map is proposed.

2. Experimental set-up

The stimuli used as example in this paper were taken from amemory recall experiment during which chess players had to lookat a configuration of chess pieces for 15 seconds whereafter theyhad to reconstruct the configuration. The recall performance ofthe participants is beyond the scope of this paper and only the eye-tracking data that was captured during the fifteen secondsexposure time was used in the visualizations.

Data was captured with a Tobii 1750 eye-tracker. The stimuliwere displayed on a 17" screen with a resolution of 1024×768 atan eye-screen distance of 600 mm. The stimuli were sized sothat 1 of visual angle was equivalent to about 33 pixels or 10.5mm. The individual squares of the chess board spanned about 20mm (2) while each piece was displayed at about 7×8 mm (<1).

3. Generation of heatmaps

3.1 Visual span

Visual span refers to the extremes of the visual field of a viewer,i.e. the area that can be cognitively observed with a singlefixation. The visual span of a fixation is measured as the distance(in pixels) from the centre of a fixation to the furthest point wherean observer might be able to perceive objects. This is not the

125

same as the radius of a fixation, which is the distance from thecentre of a fixation to the POR that is the furthest away.

In Figure 2 circles are drawn around fixation centers to indicatethe visual field of highest acuity (diameter = 2). Fixations areshown as dots, with the size of the dots being representative of theduration of a fixation on a linear scale. The 2 visual fields ofFigure 2 might lead an analyst to conclude that the participant didnot see the pieces on a2, b8, g1 or h2. One could rightfully askwhy a participant would bother to look at g2.

Bearing in mind, however, that a person might be able to observeobjects at 2.5 from the centre of the foveal zone (5 visual span)with 50% acuity [Duchowski 2007], it might be possible that theviewer perceived the white king and white pawn on g1 and h2respectively, although he did not look at them directly. Using thealgorithm in Figure 1, a heat map was generated that illustratesthis possibility (Figure 3). The same data set was used as inFigure 2 but the visual span (Line 6) was set to 5.

3.2 Assigning weights to fixations and pixels

Analysts should be allowed to select the metric of attention theywish to plot in a heatmap. In other words, they should be able toselect whether they want to base a heat map on the number offixations, the duration of fixations or the number of participantswho observed a target area [Bojko 2009]. In the case of fixationduration, the fixation weight (W) is set to the total duration (in ms)of the fixation (Figure 1, Line 9). For the number of fixations orparticipant recordings, the fixation weight is set to a value that theuser may select to ensure smooth coloring, typically W=100.

Each fixation contributes to the total weight of all pixels within itsvisual field (Figure 1, Line 10). Since the visual fields of differentfixations may overlap, it is possible that various fixations cancontribute to the total weight of a specific pixel. For the durationand number of fixations all fixations within the visual field of apixel contribute to its weight. For the number of participantrecordings only the nearest fixation of a specific recording to apixel contributes to the total weight of that pixel, provided that thepixel falls in the visual field of the fixation.

3.3 Probability

The probability that an observer will perceive an object during afixation, p ε [0,1], decreases as the distance of the object from thecentre of a fixation increases. For each pixel within the visualspan of a fixation, the fixation weight is multiplied with p beforeadding it to the total weight of the pixel (Figure 1, Line 10).

For the algorithm proposed in this paper, a user may select fromthree different models for scaling the weight over the visual field,V, i.e. Linear, Gaussian and No scaling. For no scaling p=1 forall pixels within the visual field of a fixation, i.e. the completeweight of the fixation contributes to the total weight of all pixelswithin its visual field (example in Figure 3a). For linear scalingthe probability, p, at a distance D from the fixation center is

p = 1-D/V where D V.

8

7

6

5

4

3

2

1

a b c d e f g h

Figure 2. Circles around fixations to indicate the visual field ofhighest acuity (diameter = 2).

1. for each pixel of original stimulus2. Weight[pixel] := 0 //Init pixel weights3. end for

//User opted to let the system assign the//highest pixel weight to weight for red

4. WtRed := 0;5. for each fixation6. for each pixel within the visual span of

current fixation7. D := Distance pixel to fixation centre

//p and W determined as described above8. p := Probability9. W := FixationWeight10. Weight[pixel] := Weight[pixel] + (W*p)11. end for12. if Weight[pixel] > WtRed then13. WtRed := Weight[pixel]14. end if15. end for

16. for each pixel of original stimulus withWeight[pixel] > 0//Get respective colour components

17. r := GetRedValue(Weight[pixel], WtRed)18. g := GetGreenValue(Weight[pixel], WtRed)19. b := GetBlueValue(Weight[pixel], WtRed)

//Add transparency20. Pixel.Red := (T*Pixel.Red + (10-T)*r)/1021. Pixel.Grn := (T*Pixel.Grn + (10-T)*g)/1022. Pixel.Blu := (T*Pixel.Blu + (10-T)*b)/10

//Draw contours if selected23. if Draw contours then24. c := Contour interval25. if Weight[pixel] div c

<> Weight[neighbour pixel] div c then//Make the colour of the pixel brown

26. Pixel.Red := 20427. Pixel.Grn := 10228. Pixel.Blue := 029. end if30. end if

31. end for

Figure 1. Algorithm for generation of heat maps

126

For Gaussian scaling (example in Figure 3b), pixels near thecentre of a fixation are assigned more weight than would havebeen the case with linear scaling while those further off areassigned less weight (Figure 4). For Gaussian scaling,

p = a.e-(D-b)²/2c² , with a=1 and b=0.

The constant c can be expressed in terms of the full width of thedistribution at half maximum (FWHM), i.e.

FWHM = 2.3548 × c [Wikipedia].

If FWHM is defined to represent 0.4 of the maximum visual span,it follows that

c = 0.17 × (visual span).

3.4 Color model

The RGB color model is an additive model in which red, green,and blue light are added together to reproduce a broad spectrum ofcolors. When generating heat maps, each pixel of the stimulus isassigned an RGB triplet (R, G, B) where each one of thecomponents can be an integer in the range 0 through 255.

The algorithm of Figure 1 uses a set of functions, GetRedValue,GetGreenValue and GetBlueValue (Lines 17, 18 & 19) to returnthe intensities for red, green and blue respectively for a specificpixel based on its weight according to the composite linear modelof Figure 5. Other color models, such as CMYK and CIE can alsobe implemented.

3.5 Handling transparency

The analyst has to select a transparency index for the heat map, Tε [0,10], where 0 indicates no transparency (the stimulus is totallyobscured) and 10 indicates complete transparency (heat mapinvisible). Every pixel of the original stimulus that is covered bythe heat map, i.e. pixel weight > 0, is edited by decreasing the redcomponent by the transparency factor, T/10 (Figure 1, Line 20).Thereafter, 1-T/10 of the red component of the heat map at thatpixel is added to the red component of the pixel of the originalstimulus (Figure 1, Line 20). The green and blue components areedited likewise (Lines 21 & 22). For Figures 3 and 6 thetransparency index was set to 5 while for Figure 7 it was set to 8.

0.0 0.5 1.0 1.5 2.0 2.5

Distance from fixation centre (degrees)

0.00.10.20.30.40.50.60.70.80.91.0

Pro

ba

bili

tyto

be

ob

serv

ed

Figure 4: Graph of the probability to be observed against distancefrom fixation centre. The red curve is for linear scalingand the blue curve for Gaussian scaling (FWHM = 40%of 5 visual span).

8

7

6

5

4

3

2

1

a b c d e f g h

8

7

6

5

4

3

2

1

a b c d e f g h

Figure 3a (top): Heat map of the same data set of Figure 2. Noscaling. Duration for red=1264 ms.

Figure 3b (bottom): Heat map of the same data set of Figure 2.Gaussian scaling (FWHM = 40% of 5visual span). Duration for red=1264 ms.

Figure 5: A composite linear model for the relationshipsbetween RGB components and pixel weight.(Weight for red is set to 100.)

127

3.6 Color range

Besides the parameters for visual span, the model for scaling theweight and the transparency index, the analyst may decide to set aweight to be used for red or choose to let the algorithm assign thehighest weight of all pixels (as was done in Figure 1, Lines 4 &12-14). A fixed value is useful if the analyst wants to determinewhich areas received a certain minimum amount of attention[Bojko 2009]. Figure 6 shows an example of a heat map where theduration for red was set to 600 ms instead of the 1264 ms that wasdetermined by the algorithm and used for Figure 3.

3.7 Adding contours

A heat map provides a qualitative overview of viewers' attention.Although a specific color can be mapped quantitatively in termsof the selected metric of attention, it is not easy to communicatethe value. Contours can be added to separate intervals in thecontinuous color map.

Contour lines designate the borders between different intervals ofpixel weight. If two adjacent pixels belong to different contours,one of them should be colored differently to indicate a contourpoint (Figure 1, Lines 23 – 30). Figure 7 shows a heat map of thesame data as in Figure 3 with contour lines at intervals of 200 ms.

It is believed that the contour lines assist substantially towards theinterpretation of heatmaps. For example, it is now clear that thepawn on d4 received about twice as much attention (average 900ms) as the pawn on e5 (average 450 ms). The contour lines alsocompensate for the loss of color information if the transparency isincreased to improve visibility of the original stimulus.

4. Summary

Although heat maps are valuable to identify qualitative trends ineye-tracking data it is important to have control over varioussettings to enable sensible comparisons. A simple algorithm was

presented that allows analysts to indicate the amount of peripheralvision that should be accommodated. The algorithm also allowsthe analyst to select the metric of attention together with anappropriate weight. The drop-off in visual attention can be scaledlinearly, according to a Gaussian function, or not at all. Thethreshold value for red as well as the transparency can beadjusted. The addition of contour lines provides a means tovisualize areas of equal attention.

References

BLIGNAUT, P.J. 2009. Fixation identification: The optimumthreshold for a dispersion algorithm. Attention, Perception andPsychophysics, 71(4), 881-895.

BOJKO, A. 2009. Informative or Misleading? HeatmapsDeconstructed. In J.A. Jacko (ed.) Human-ComputerInteraction, Part 1, HCII 2009, LNCS 5610, 30-39, Springer-Verlag, Berlin.

DUCHOWSKI, A.T. 2007. Eye Tracking Methodology: Theory andPractice (2nd ed.). Springer, Londen.

EYENAL. 2001. Eyenal (Eye-Analysis) Software Manual. AppliedScience Group. Retrieved 12 June 2008 fromhttp://www.csbmb.princeton.edu/resources/DocsAndForms/site/forms/Eye_Tracker/Eyenal.pdf

SPAKOV, O. and MINIOTAS, D. 2007. Visualization of eye gazedata using heat maps. Electronics and Electrical Engineering,2(74), 55-58.

TOBII TECHNOLOGY AB. 2008. Tobii Studio 1.2 User Manualversion 1.0. Tobii Technology.

WIKIPEDIA. Gaussian function. Retrieved on 30 November 2009from http://en.wikipedia.org/wiki/Gaussian_function.

WOODING, D.S. 2002. Fixation maps: Quantifying eye-movementtraces. Proc. ETRA 2002, ACM, 31-36.

8

7

6

5

4

3

2

1

a b c d e f g h

Figure 6: Heat map of the same data set of Figure 3 but withthe duration for red set to 600 ms instead of allowing thealgorithm to allocate the highest aggregate duration to red.

8

7

6

5

4

3

2

1

a b c d e f g h

Figure 7: Heat map with contour lines at intervals of 200 ms.Duration for red = 1200 ms; Transparency = 8.

128