Jeffrey P. BighamRichard Ladner, Ryan Kaminsky, Gordon Hempton, Oscar DanielssonUniversity of WashingtonComputer Science & Engineering

Browsing while blindScreen readers

Images cannot be read

W3C accessibility standardsProvide a text equivalent for every non-text element

What if no alternative text?NothingFilename (060315_banner_253x100.gif)Link address (http://www.cs.washington.edu)

nav_svcs.gif

OutlineWeb Studies

Providing Labels

WebInSight System

Evaluation

Future Work

Web Studies: All Images !=Significant images need alternative text Informative alt, title, and longdesc HTML attributes

Insignificant images need empty alt text

Automatic Determination?

More than one color AND both dimensions > 10 pixels An associated action (clickable, etc.)

Web StudiesPrevious studies img tags with defined alt attribute:27.9%[1], 47.7%[2], and 49.4%[2] Significant images have a defined alt attribute?76.9%[3]Gaps Some Ignore Image Significance Some Ignore Image Importance

[1] T. C. Craven. Some features of alt text associated with images in web pages. (Information Research, Volume 11, 2006).[2] Luis von Ahn et al. Improving accessibility of the web with a computer game. (CHI 2006)[3] Helen Petrie et al. Describing images on the web: a survey of current practice and prospects for the future. (HCII 2005)

Web StudiesUniversity of Washington CSE Department Traffic

Significant images without alternative text.Significant images withalternative text.~1 week 11,989,898 images. 40.8% significant 63.2% alt text

Chart4

25.8

15

Insignificant

Labeled Significant Images

Sheet1

Labeled Significant25.8

Unlabeled Significant15

Insignificant

Sheet1

Labeled Significant Images

Sheet2

Sheet3

Study ResultsPercentage of significant images provided alternative text, pages with over 90%of significant images provided alternative text, number of web sites in group,and number of images examined.

OutlineWeb Studies

Providing Labels

WebInSight System

Evaluation

Future Work

Providing Labels: Context LabelingMany important images are links Linked page often describes image What happens if you click

People of UW

People

Providing Labels: OCR Labeling [4] Jain et al. Automatic text location in images and video frames. (ICPR 1998)

(Optical Character Recognition)Improves recognition 25% relative to base OCR!

Providing Labels: Human LabelingHumans are bestRecent games compel accurate labelingWebInSight database has over 10,000 imagesCould do this on demand[5] Ahn et al. Labeling images with a computer game. (CHI 2004)[6] Ahn et al. Improving the accessibility of the web with a computer game. (CHI 2006)

[5][6]

OutlineWeb Studies

Providing Labels

WebInSight System

Evaluation

Future Work

WebInSight SystemTasks Coordinate multiple labeling sources Insert alternative text into web pages Add code to insert alternative text laterFeatures Browsing speed preserved Alternative text available when formulated Immediate availability next time

The Internet Blind User

OutlineWeb Studies

Providing Labels

WebInSight System

Evaluation

Future Work

Evaluation Measuring System Performance WebInSight tested on web pages from web studies Used Context and OCR Labelers Labeled 43.2% of unlabeled, significant images Sampled 2500 for manual evaluation 94.1% were correct

Proper Precision/Recall Trade-off

Evaluation: Demo

ConclusionLack of alternative text is pervasive

WebInSight calculates alternative text

WebInSight inserts alternative text

High precision and moderate recall

Future WorkUser StudiesWhat do users want?How can we provide it?Maintain experience.UsersContent ProducersUser StudiesDesigner motivation.Tools for Web DesignPeople can always be betterAdapt user techniques

Common ThemesTechnologyImproved labelingBring closer to userMove beyond imagesMore challengesContent StructureDynamic ContentWeb applications

WebInSight

http://webinsight.cs.washington.eduThanks to: Luis von Ahn, Scott Rose, Steve Gribble and NSF.

Screen readers enable the web to be browsed by people who cant see a visual display. They convert content that that is often designed to be viewed visually to either sound or refreshable Braille.

Images can pose a problem because they are inherently visual but often convey important information. On the web, the solution standardized by the World Wide Web Consortium is to provide alternative text for images that can be read in place of the image. If there is no alternative text, then the screen reader can do several different things dependent on the particular screen reading product and user settings. Options include skipping the image, reading the image filename, or, if the image is a link to another page, reading the URL of the linked page.

Thank you. My name is Jeffrey P. Bigham and today Ill be presenting work that Ive done with the WebInSight project on making images accessible which is joint work with my advisor Richard Ladner and others at the University of Washington Computer Science & Engineering Department.

In this talk I first hope to convince you that images that lack alternate text are pervasive on the web. Then Ill describe the WebInSight system which calculates alternative text for web images and inserts it on-the-fly as users browse the web.

The WebInSight system is able to provide alternative text for many images that dont have it already.Now that weve seen some background material that shows us what the problem is, Ill discuss some web studies that we did to show that the lack of alternative text is pervasive on the web.Web designers arent generally malicious and they usually dont actively avoid assigning alternative text or purposely assign incorrect alternative text. Assigning alternative text is often a subjective process and many web designers simply dont know what alternative text to provide. Because the majority of web designers are not blind, they dont see the immediate reward for providing alternative text.

Complicating matters, some images need alternative text and some images dont. If an image conveys information to a sighted user then it should be assigned alternative text that conveys that information to a blind user. If an image does something when you click on it, then it needs alternative text.

Conversely, if its purpose is inherently visual if its used for decorative or layout purposes then it should be assigned a zero-length alternative text. Simply omitting the alt attribute for such images is not good because the screen reader does not know the significance of the image in this case and will announce the image. A common example of this are images that are used for spacing on web pages. Often these images are used 10s ot 100s of times on a single web page. Forgetting to assign zero-length alternative text may cause the user to hear spacer.gif repeated multiple times on that page.

Using our subjective idea of what significance is as a guide, we presented an automatic test that correctly determines significance in nearly all cases.First, some previous studies have shown that less than half of images are assigned alternative text.

Some look only at the percentage of images that have an alt attribute defined and ignore significance.

A study conducted by Petrie et al. manually labeled significance, but only evaluated 100 manually-selected web pages.

By calculating significance automatically, we were able to consider many more web pages and explicitely consider the popularity and importance of these web pages. What is important isnt just how many images on the web are provided alternative text, but how many of the images that a user views are provided alternative text.

In our first study, we examined all web traffic generated by members of the CSE department at the University of Washington. We counted images viewed multiple times as separate images, in effect weighting the contribution of each image by its popularity. From this we saw that 40.8% of the approximately 12,000,000 images viewed in a week were significant of these, 63.2% were assigned alternative text.By calculating significance automatically, we were able to consider many more web pages and explicitely consider the popularity and importance of these web pages. What is important isnt just how many images on the web are provided alternative text, but how many of the images that a user views are provided alternative text.

In our first study, we examined all web traffic generated by members of the CSE department at the University of Washington. We counted images viewed multiple times as separate images, in effect weighting the contribution of each image by its popularity. From this we saw that 40.8% of the approximately 12,000,000 images viewed in a week were significant of these, 63.2% were assigned alternative text.We also considered 5 important groups of web sites and measured the percentage of significant images labeled. These included the 500 high-traffic websites, the 136 computer science department homepages, and the top 100 international universities. We also considered the 137 federal agencies and 51 U.S. States plus D.C. which we saw as an important group because it shows what effect legislation can do because both of the latter groups are compelled to provide accessible information by Section 508.In order to help ameliorate this problem, we developed a system that automatically labels many images and inserts these labels into web pages on the fly. The obvious question is where do labels come from? We present 2 fully automatic methods and 1 method that leverages recent work that compels humans to label images.The first method is Context Labeling. The insight here is that many important images are links and the pages they link to often serve as good alternative text for the images. In this labeling method we simply follow image links and use important fields such as the title or heading tags to construct labels.The second method is OCR Labeling. Many images on the web contain text, but screen readers cannot directly use it in its graphic form. Optical Character Recognition works well when applied to black text on white backgrounds scanned in at a known resolution, but can have problems with web images. To label images using OCR, we first preprocess the image to convert it to a resolution and format that the OCR prefers. Next, following a process similar to that developed by Jain et al. we convert it to several black & white versions by highlighting major color groups. The OCR can more easily handle these black and white images.

The example here shows that for this image the OCR originally produces no text. Our method produces six new black & white images. The OCR produces no output for 4 of the 6 new images, garbage text for one, and correct text for the last. We verify the output of the OCR using a web-based spelling checker.The final method of labeling images is motivated by the fact that our automatic methods of labeling images dont apply to all images and providing alternative text for arbitrary images is currently beyond the state-of-the-art. However, using recent popular games originating from Luis von Ahn et al. at CMU, we can get humans to label such images. These games provide a convenient mechanism to get humans to provide labels quickly and to help ensure that they labels they provide are correct.Now that youve heard the motivation for why we want to add labels to web images without them and a little about how we formulate those labels, Im going to briefly discuss the WebInSight system.

The WebInSight system dynamically inserts alternative text into web pages and coordinates the labeling sources that I just described. The main concerns we have are that it avoid harming the user experience and maintain the security and privacy expectations of the user.The important part of any system like this is how well it works. We think it does a good job.To measure the performance of our system, we attempted to label the images contained on the pages in our web studies using the system. These tests were done completely automatically and, therefore, we used only the context and OCR labeling methods.

The system was able to label 43.2% of the unlabeled significant images and through manual evaluation we determined that it provided correct labels 94.1% of the time. Therefore, the recall of the system is approximately 40%, but the precision quite high at nearly 95%. Although, wed like our recall to be higher, were encouraged that we are able to provide alternative text to so many images that need it but dont and do so with very high precision.If we return to the homepage of the Yale Alumni Association, youll see that the system was able to correctly label most of the important images on the site. Here it has relied mainly on the labels produced by the context labeler and has provided valuable alternative text for 18 of the 21 images on this page that link to another page.In conclusion, weve presented a system that can dynamically provides alternative text for images on the web as users browse to increase web accessibility.While the current WebInSight System involves only a client interface, our goal is to attack inaccessibility with a dual approach: tools for users that help them alter their environment to increase web accessibility and tools for content producers that help them create more accessible content. Our belief is that many of the techniques for users can be adapted to help producers.

While weve been consulted from the beginning with users of screen readers, were currently designing user studies that will attempt to measure what users want out of a system like ours, understand how we can provide it, and how all of this can be consistent with the expected user experience.

At the same time, were always trying to improve our system with better labeling modules. Were currently working on a version of our system that is implemented as a Firefox extension to allow users increased privacy and security. Finally, were exploring ways to move beyond images to content restructuring and handling dynamic content.

We believe that at the producing side of content accessibility the primary concern remains understanding the user. What forces create inaccessibility and how can we use information about the motivation of web designers to create tools that will help them make more accessible content. For the foreseeable future people will be better at creating accessible content than our automatic methods but they need tools that better support this. This is an incredibly important time for developing information accessibility tools. Even as the Internet settles into its vital role in society, were still dealing with the old struggles of images without alternative text and content that is structured inconveniently for accessibility tools. The web is currently undergoing another revolution that will bring dynamic content and new web applications, both of which pose even harder problems for accessibility. I hope we will be there to meet the challenge.

I encourage you to visit our website at http://webinsight.cs.washington.edu/ for more information about this and other WebInSight projects.

Thank you.