towards more accessible visualizations for color-vision-deficient individuals

80 Copublished by the IEEE CS and the AIP 1521-9615/13/$31.00 © 2013 IEEE Computing in SCienCe & engineering

V i s u a l i z a t i o n C o r n e r

Editors: Cláudio T. Silva, [email protected]

Daniel Weiskopf, [email protected]

Towards More accessible VisualizaTions for color- Vision-deficienT indiVidualsBy Manuel M. Oliveira

A ccording to recent estimates, approximately 200 million individuals have some form

of color vision deficiency (CVD).1,2 Such a condition affects their private and professional lives, impacting their ability to effectively interpret color-coded information. As such, these individuals can’t fully benefit from standard scientific, medical, and in-formation visualization techniques. Figure 1 illustrates the problem with a situation faced on a daily basis by indi-viduals with CVD. Figure 1b shows a simulation of how one such individual sees the pie chart shown in Figure 1a. Note the difficulty to identify sectors A and B. Because decisions based on ambiguous information might have undesirable and even catastrophic im-plications, addressing this ambiguity problem is relevant both socially and economically. Fortunately, digital me-dia has become the prevalent form of image and video consumption, mak-ing it practical for us to use enhance-ment techniques to compensate for the loss of color contrast experienced by these individuals.

This article describes a series of techniques that we developed over the past few years to enhance visualization experiences for indi-viduals with CVD. These include recoloring techniques, the use of

patterns to encode color, and a model for simulating the color per-ception of these individuals. While recoloring techniques and patterns can be used to disambiguate color-coded information, color-perception simulation provides valuable feed-back for visualization designers to create more effective color scales and transfer functions capable of reaching broader audiences. Be-fore discussing these techniques, let’s briefly review the classes and causes of CVD.

Color Vision Deficiency: A Brief ReviewHuman normal color vision requires three types of cone cells (the retina’s photoreceptors involved in color sensation). If one type of cone is missing, the individual is called a di-chromat. Dichromats can be further classified as protanopes, deuteranopes, or tritanopes, if the missing photo-pigment is more sensitive to the long, medium, or short wavelengths of the visible spectrum, respectively. Alternatively, changes in the com-position of the photopigments can shift their spectral sensitivity.1 In this case, the individuals are called anomalous trichromats, and likewise can be classified as protanomalous, deuteranomalous, or tritanomalous.

Individuals having a single or no type of photopigment are called cone monochromats and rod monochromats, respectively.

CVD is an inherited condition as-sociated with the X chromosome, and whose incidence varies with ethnic groups. For the male population, cur-rent estimates suggest that approxi-mately 7.40 percent of the Caucasian population, 4.17 percent of Asians, and 2.61 percent of Africans have some level of red-green CVD.1 The prevalence is considerably smaller in the female population: 0.50 percent, 0.58 percent, and 0.54 percent, re-spectively, for the same ethnic groups. Unfortunately, there’s no clinical or surgical treatment available for CVD.

Recoloring TechniquesIdeally, recoloring techniques in-tended to enhance color contrast for individuals with CVD should satisfy a long list of desirable features:

1. be automatic;2. be deterministic;3. preserve the “perceptual distanc-

es” among the original colors;4. preserve, as much as possible, the

original colors;5. satisfy a global consistency property;6. preserve the original image

luminance;

Various techniques can be used to improve visualization experiences for individuals with color vision deficiency, including recoloring, pattern superposition, and the use of a color-perception simulation model for assisting visualization designers.

CISE-15-5-VisCorn.indd 80 24/10/13 10:45 AM

September/oCtober 2013 81

7. exhibit temporal coherence; and8. achieve real-time performance.

Some of these requirements might conflict with others.

Global consistency means that all pix-els with a given color in the original image should be mapped to a single shade in the recolored one. Preserv-ing the luminance values of the origi-nal pixels is key to avoid luminance polarity reversal: a distracting artifact that happens if brighter pixels become darker or vice-versa after recoloring. Temporal coherence and real-time performance are necessary for inter-active visualization sessions.

Recently, several recoloring tech-niques have been proposed trying to recover the color contrast missed by dichromats.3–10 Some of these techniques require user assistance, and the quality of their results is highly dependent on the user-pro-vided parameters.3,7 Most of the automatic recoloring techniques are non-deterministic and insuf-ficiently fast for use in interactive applications.4–6,8

Two automatic recoloring tech-niques9,10 try to address the afore-mentioned list of requirements, and have been evaluated in visualization contexts by individuals with CVD. As with other automatic techniques, they’ve been designed for dichro-mats, but are also applicable to anomalous trichromats, although using a reduced color gamut (we dis-cuss these next).

Optimization-Based RecoloringA technique introduced by Giovane Kuhn, Leandro Fernandes, and myself9 models the recoloring pro-cess as a mass-spring optimization, which can be efficiently implement-ed on GPUs. The technique tries

to satisfy requirements 1 through 6. Similar to another approach,5 the optimization is first applied to a set of quantized colors, and the re-sults of this first step are then used to optimize the entire set of colors. Due to the use of quantization, it’s unclear how it can enforce tempo-ral coherence. Although this tech-nique achieves interactive rates, it’s still not sufficiently fast for real-time recoloring on the fly.

The algorithm consists of three steps:

• color quantization;• optimization of the set of quantized

colors; and• reconstruction of the recolored

image from the set of optimized (quantized) colors.

Quantization can be performed us-ing any technique, such as uniform quantization or k-means. In the L*a*b* color space, the quantized colors are orthographically project-ed onto a plane that approximates the dichromat’s color gamut (see Figure 2). Such a projection mimics the dichromat’s perception for the given color. Each projected color is then treated as a particle, and a spring is set between each pair of particles. The rest length of any giv-en spring is given by the perceptual difference between the pair of quan-tized colors associated with the par-ticles it connects. The mass of each particle is defined as the reciprocal

of the perceptual difference between its corresponding quantized color and the dichromat’s perception. The L* coordinate is kept constant to preserve luminance, and opti-mization is performed in 1D. After optimization, the particles’ positions on the dichromat’s plane define the optimized colors.

The use of perceptual differences between pairs of quantized colors as the spring rest lengths makes the technique try to enforce that the recolored image will preserve such differences. Also, the more similar-ly a color is perceived by a normal trichromat and by a dichromat, the bigger its mass—and therefore, the bigger its resistance to move. These are desirable features. On the other hand, the restrictions of luminance and original-color preservation im-pose constraints to the 1-D optimi-zation. This makes it harder for the optimization to preserve the per-ceptual color differences found in the original image and, as a result, it might be trapped in some local minimum.

Figures 3 and 4 show examples of images recolored with Kuhn’s tech-nique, and illustrate the points dis-cussed. Figure 3b shows a simulated view of a deuteranope for the graph shown on its left. Note how the per-ceived colors of the two lower bars are essentially indistinguishable. Figure 3c shows the recolored im-age, where the color of the middle bar has been preserved, as it can be

Figure 1. Ambiguity caused by color vision deficiency (CVD). (a) Pie chart. (b) Simulated view of an individual with CVD (deuteranope) for the chart in (a). The identification of sectors A and B is ambiguous.

(a) (b)

2% 2%

A B C D E A B C D E

28%

12%30%

28%

12%30%

28%28%



82 Computing in SCienCe & engineering

perceived by deuteranopes. Given a deuteranope’s reduced color gamut (Figure 2b), the original green bar has been recolored with a brighter shade of yellow (blue’s opponent color).

Figure 4 illustrates the case in which the set of constraints causes the optimization to be trapped in a local minimum. Figure 4b shows the simulated view of a protanope for the image shown in Figure 4a. Figure 4c shows the recolored ver-sion of Figure 4a using Kuhn’s technique. Note that although it contains some dark blue shades, it exhibits significantly less con-trast than the original image. To handle these situations, we can use a variant of the technique that ex-aggerates color contrast instead. It essentially modifies the afore-mentioned algorithm by scaling the springs’ rest length by a given factor α (typically a = 2). Figure 4d shows the result obtained with this

variant of the algorithm applied to Figure 4a.

Evaluation by Participants with CVDBoth variants of Kuhn’s technique were evaluated by a group of 14 male volunteers with CVD. For the experiment, we used three sets of images. The first set (Natural) consisted of photographs of natural scenes. The second set (InfoVis) con-tained information visualization ex-amples, while the third one (SciVis) consisted of scientific and medical visualization images. The original and recolored versions of these im-ages were evaluated by the volun-teers using the method of paired comparisons.11

The results of the evaluation in-dicated that for the InfoVis set, the volunteers preferred the recol-ored images obtained with Kuhn’s technique. For the SciVis set, the volunteers preferred the recolored

images with exaggerated contrast (as they can avoid the local minima situations depicted in Figure 4c). An interesting outcome of this experi-ment was the fact that for the Natu-ral set, the volunteers preferred the original images over the recolored ones, despite the acknowledged en-hancement in contrast in the latter (see Figure 5c). According to the participants, the colors in these im-ages didn’t match their previous ex-periences and, as such, didn’t look “natural.” The results of this experi-ment suggest that recoloring tech-niques for CVD can be effective for visualization applications, as the color scales used to encode informa-tion tend to be somehow arbitrary. A description of the experiment and its results are detailed elsewhere.9

Projection-Based RecoloringAs previously mentioned, a dichro-mat’s color perception can be estimat-ed by orthographically projecting the

Figure 3. Optimization-based recoloring. (a) Reference image. (b) Simulated view of a deuteranope for the reference image. (c) Recolored image produced by Kuhn’s mass-spring optimization technique.

(a) (b) (c)

Norway France

Pop

ulat

ion

0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0

Italy Norway France

Pop

ulat

ion

0.0 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0

Italy Norway FranceItaly

0.0

Pop

ulat

ion

2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.0

Figure 2. Planes that approximate the color gamut of each class of dichromacy in the L*a*b* color space (after my work with Giovane Kuhn9). (a) Protanopes, (b) deuteranopes, and (c) tritanopes.

θtθp θd

L* L*L*

a* a*

b* b*b*

a*

(a) (b) (c)



image colors onto the plane in Fig-ure 2 that approximates the dichro-mat’s color gamut. Thus, in previous work, my colleague and I10 introduced an efficient recoloring technique based on the observation that when-ever dichromats experience signifi-cant loss of color contrast, most of it can be recovered by orthographically projecting the original colors onto a plane π (in the L*a*b* color space and containing the L* axis) that maximiz-es (in a least-squares sense) the pro-jected colors’ separation. Using this approach, recoloring simply consists of using the projected coordinates on plane π as the new color coordinates in the dichromat’s gamut plane. Fig-ure 6 illustrates this process.

This technique is quite efficient, achieving real-time performance and avoiding the local minima is-sues associated with optimization-based approaches. The technique also supports temporal coherence (the details of which are explained elsewhere10). As such, it satisfies all the desirable features, except for preservation of original colors (although it preserves gray shades). Perceptual differences are partially preserved. Figure 7 shows an ex-ample of images recolored using this technique.

Because of its real-time perfor-mance, such a technique can be easily integrated with any exist-ing visualization application in a minimally invasive way. This is

achieved as follows: once the ap-plication has written an image to the frame buffer, it calls a function that implements the recoloring al-gorithm. Such a function then reads the frame buffer’s content, recolors it, and writes it back, returning the control to the application. The ap-plication then swaps the front and back buffers. Using this approach, it becomes practical to perform real-time recoloring for dichromats with temporal coherence during visual-ization sections. Figure 8 illustrates the concept.

Using Patterns to Encode Color InformationAn alternative to recoloring is to superimpose patterns on colored

image regions. Properly designed patterns allow individuals with CVD not only to disambiguate but also to identify colors, by memo-rizing the rules of pattern forma-tion.12 Its use, however, trades spatial resolution for the patterns. Thus, it becomes hard to use them in small areas. Moreover, apply-ing patterns over regions contain-ing textures or detailed content whose colors change dynamically, such as in the case of exploratory visualization, might be distract-ing and mask some important data features. Nevertheless, the use of patterns can be useful for disam-biguating static content, and for communicating color informa-tion among normal trichromats

Figure 4. Optimization-based recoloring can be trapped in a local minimum. (a) Reference image. (b) Simulated view of a protanope for the reference image. (c) Recolored image produced by Kuhn’s mass-spring optimization technique trapped in a local minimum. (d) Recolored result obtained with the exaggerated-contrast version of Kuhn’s technique.

(a) (b) (c) (d)

Figure 5. Example of a recolored natural image. (a) Reference image. (b) Simulated view of a deuteranope for the reference image. (c) Recolored image using Kuhn’s technique.

(a) (b) (c)




and individuals with CVD.12,13 Figure 9 illustrates this notion. Figures 9a and 9b show a colored pie chart and how it would be per-ceived by a protanope, respectively. Note that in Figure 9b it’s impos-sible to distinguish between sales in North America and Europe, as well as between sales in Australia and South America. By superimpos-ing (on the original chart) patterns whose orientations and contrast levels uniquely represent colors in a meaningful way (Figure 9c), we can solve the ambiguity problem (Figure 9d) and allow proper com-munication between normal color vision and CVD individuals. Note that the original colors remained unchanged for normal trichromats (Figure 9c). A detailed description

of how to create such patterns is provided elsewhere.12

Designing More Effective Visualizations for CVDIn previous sections, we discussed strategies aiming to minimize the loss of color contrast experienced by individuals with color vision defi-ciency. Here, we address the issue of assisting visualization designers in understanding how individuals with CVD perceive colors. This should help designers define more effective color scales and transfer functions to reach broader audiences.

Gustavo Machado, Leandro Fer-nandes, and I introduced a physio-logically based model for simulating color perception that can handle normal color vision, anomalous

trichromacy, and dichromacy in a unified and consistent way.14 For anomalous trichromacy, it can simu-late spectral sensitivity shifts of the affected photopigments in the 0- to 20-nm range. A shift of 20 nm re-sults in a perception similar to the one of the corresponding class of dichromacy.

Controlled experiments involving a group of 13 individuals with CVD (4 protanopes, 4 protanomalous, 2 deuteranopes, and 3 deuteranoma-lous) and 17 normal trichromats have shown that the results pro-duced by this model closely match the perception of people with CVD. Figure 10 shows some simulated results for protanomaly produced using this model. Note the gradual loss of color contrast as the amount of shift in the spectral sensitivity increases. Figure 11 shows similar results for deuteranomaly.

The simulation process can be implemented as a single matrix-vector multiplication per pixel. A detailed description of how to compute the matrix required for any CVD-severity degree is pre-sented elsewhere.14 Such matrices can also be obtained by linear in-terpolation from a set of precom-puted matrices available at www.inf.ufrgs.br/~oliveira/pubs_files/CVD_ Simulation/CVD_ Simulation.html.

Figure 7. Image recoloring using a projection-based approach.10 (a) Reference image. (b) Simulated view of a protanope for the reference image. (c) Recolored image.

(a) (b) (c)

Figure 6. Projection-based recoloring technique. (a) The perception of a protanope for colors C1 to C3 have little contrast (represented by the yellow dots C1’ to C3’). (b) Orthographic projection of the original colors onto a plane that maximizes color separation. (c) Recoloring consists of using the coordinates of these projections on the dichromat’s original plane. This is equivalent to rotating the plane shown in (b) to align it with the dichromat’s plane.

θp θp

(a) (b) (c)



Given its simple implementa-tion and real-time performance, this color-perception simulation model can be easily integrated with any visualization application. The visualization designer can then in-teractively select the class of CVD (s)he would like to simulate (such as protanopia, deuteranopia, prot-anomaly, or deuteranomaly), getting instant visual feedback. Figure 12

shows the visible male’s head, with a given transfer function, seen by a normal trichromat, a protanoma-lous (10 nm), and a protanope. The designer can switch between nor-mal color vision and the simulation of any degree of CVD instantly. This feedback allows her/him to interactively refine the trans-fer function to reach a broader audience.

Color vision deficiency affects approximately 200 million in-

dividuals worldwide, compromising their ability to effectively perform color-related tasks and, therefore, fully exploit the benefits of visualiza-tion. Here, we’ve considered some of the techniques used for improving visualization experiences for individ-uals with CVD, including recoloring, pattern superposition, and the use of

Figure 8. Real-time recoloring for dichromats during a visualization section. (a) Reference frame. (b) Recolored frame. (c) A simulated view of a deuteranope for the reference image, for comparison.

(a) (b) (c)

Figure 9. Using patterns to solve ambiguity. (a) Reference chart. (b) Simulated view of a protanope for the reference chart. (c) Reference chart with superimposed patterns. (d) Simulated view of a protanope for the chart in (c).

$30,000

$30,000

$21,000

$23,000

$12,000

Asia

S America

Europe

Sales region

Product Y

Australia

N America

Africa

$17,000

Asia

S America

Europe

Sales region

Australia

N America

Africa

$30,000

$30,000

$21,000

$23,000

$12,000Product Y

$17,000

Asia

S America

Europe

Sales region

Australia

N America

Africa

$30,000

$30,000

$21,000

$23,000

$12,000

Product Y

$17,000

Asia

S America

Europe

Sales region

Australia

N America

Africa

$30,000

$30,000

$21,000

$23,000

$12,000Product Y

$17,000

(a) (b) (c) (d)

Figure 10. Simulation of protanomaly. The numbers in parenthesis represent the amount of shift in the spectral response of the photopigment more sensitive to the long wavelengths. A shift of 20 nm results in a perception similar to the one of a protanope.

Reference 2 nm 8 nm 14 nm 20 nm




a color-perception simulation model for assisting visualization design-ers. Although all of these techniques have limitations, their judicious use can significantly improve visualiza-tion experiences for individuals with CVD.

AcknowledgmentsI would like to thank the many vol-unteers with CVD who tested the techniques described here, provid-ing invaluable insights and feedback. Several people contributed to the de-velopment and evaluation of these techniques, in particular Gustavo Machado, Giovane Kuhn, and Leandro Fernandes. This work was supported by Conselho Nacional de Desenvolvim-ento Científico e Tecnológico (CNPq) and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) grants (305613/2007-3, 476954/2008-8, 305613/2007-3, 142627/2007-0, 131327/ 2008-9, 200284/2009-6, and 308936/

2010-8). Figures 4a and 7a were ren-dered using software developed by Francisco Pinto. Figure 5a is from oth-er work,15 and Figures 3a, 10a, and 11a are from http://commons.wikimedia.org. Figure 9 is by Aditi Majumder.

References1. L.T. Sharpe et al., “Opsin Genes, Cone

Photopigments,” Color Vision: From

Genes to Perception, Cambridge Univ.

Press, 1999, pp. 3–51.

2. C. Rigden, “The Eye of the Be-

holder—Designing for Colour-Blind

Users,” British Telecomm. Eng. vol. 17,

1999; http://colinpurrington.com/

wp-content/uploads/2011/09/

Rigden19991.pdf.

3. R. Dougherty and A. Wade, Daltonize,

2002; www.vischeck.com/daltonize.

4. M. Ichikawa et al., “Preliminary Study

on Color Modification for Still Images

to Realize Barrier-Free Color Vision,”

Proc. IEEE Conf. Systems, Man, and Cy-

bernetics, vol. 1, 2004, pp. 36–41.

5. K. Rasche, R. Geist, and J. Westall,

“Recoloring Images for Gamuts of

Lower Dimension,” Computer Graphics

Forum, vol. 24, no. 3, 2005,

pp. 423–432.

6. K. Wakita and K. Shimamura, “Smart-

color: Disambiguation Framework for

the Colorblind,” Proc. Assets, ACM,

2005, pp. 158–165.

7. G. Iaccarino et al., “Efficient Edge-Ser-

vices for Colorblind Users,” Proc. WWW,

2006, pp. 919–920.

8. L. Jefferson and R. Harvey, “Accom-

modating Color Blind Computer

Users,” Proc. Assets, ACM, 2006,

pp. 40–47.

9. G.R. Kuhn, M.M. Oliveira, and L.A.F.

Fernandes, “An Efficient Naturalness-

Preserving Image-Recoloring Method

for Dichromats,” IEEE Trans. Visualization

and Computer Graphics, vol. 14, no. 6,

2008, pp. 1747–1754.

10. M. Machado and M.M. Oliveira, “Real-

Time Temporal-Coherent Color Con-

trast Enhancement for Dichromats,”

Figure 12. Integration of the color-perception simulation model with a visualization system. The visualization designer can instantly evaluate the perception of CVD individuals and refine the transfer function to effectively reach a broader audience.

Normal trichromat Protanomalous (10 nm) Protanope

Figure 11. Simulation of deuteranomaly. The numbers in parenthesis represent the amount of shift in the spectral response of the photopigment more sensitive to the medium wavelengths. A shift of 20 nm results in a perception similar to that of a deuteranope.

Reference 2 nm 8 nm 14 nm 20 nm



Computer Graphics Forum, vol. 29,

no. 3, 2010, pp. 933–942.

11. L.L. Thurstone, “A Law of Comparative

Judgment,” Psychological Rev., vol. 34,

no. 4, 1927, pp. 273–286.

12. B. Sajadi et al., “Using Patterns to

Encode Color Information for Dichro-

mats,” IEEE Trans. Visualization and

Computer Graphics, vol. 19, no. 1, 2013,

pp. 118–129.

13. P. Hung and N. Hiramatsu, “A Colour

Conversion Method Which Allows

Colourblind and Normal-Vision

People Share Documents with Colour

Content,” Konica Minolta Tech. Report,

vol.10, 2013.

14. G.M. Machado, M.M. Oliveira, and

L.A.F. Fernandes, “A Physiologically-

Based Model for Simulation of Color

Vision Deficiency,” IEEE Trans. Visualiza-

tion and Computer Graphics, vol. 15,

no. 6, 2009, pp. 1291–1298.

15. D. Martin et al., “A Database of

Human-Segmented Natural Images

and Its Application to Evaluating Seg-

mentation Algorithms and Measuring

Ecological Statistics,” Proc. 8th Int’l

Conf. Computer Vision, IEEE CS, vol. 2,

2001, pp. 416–423.

Manuel M. Oliveira is an associate pro-

fessor of computer science at the Federal

University of Rio Grande do Sul (UFRGS),

in Brazil. His research interests cover most

aspects of computer graphics, but espe-

cially the frontiers among graphics, im-

age processing, and vision (both human

and machine). Oliveira has a PhD from the

University of North Carolina at Chapel Hill.

Contact him at [email protected].

Selected articles and columns from IEEE Computer Society publica-

tions are also available for free at http://ComputingNow.computer.org.

Advertising Personnel

Marian Anderson: Sr. Advertising CoordinatorEmail: [email protected]: +1 714 816 2139 | Fax: +1 714 821 4010

Sandy Brown: Sr. Business Development Mgr.Email [email protected]: +1 714 816 2144 | Fax: +1 714 821 4010

Advertising Sales Representatives (display)

Central, Northwest, Far East: Eric KincaidEmail: [email protected]: +1 214 673 3742Fax: +1 888 886 8599

Northeast, Midwest, Europe, Middle East: Ann & David SchisslerEmail: [email protected], [email protected]: +1 508 394 4026Fax: +1 508 394 1707

Southwest, California: Mike HughesEmail: [email protected]: +1 805 529 6790

Southeast: Heather BuonadiesEmail: [email protected]: +1 973 585 7070Fax: +1 973 585 7071

Advertising Sales Representatives (Classified Line)

Heather BuonadiesEmail: [email protected]: +1 973 304 4123Fax: +1 973 585 7071

Advertising Sales Representatives (Jobs Board)

Heather BuonadiesEmail: [email protected]: +1 973 304 4123Fax: +1 973 585 7071

ADVERTISER INFORMATION • SEPTEMBER/OCTOBER 2013
