codefocus acessibility web

Windows Accessibility

What's New in Windows 7 Automation API Testing Tools vs. Accessibility Guidelines Internet Explorer 8 New Accessibility Features

.NET

WWiinnddoowwss AAcccceessssiibbiilliittyy

Volume 5 / Issue 4

Features6 A Letter from the Director of Accessibility at Microsoft

“Our vision is to create innovative technology that is accessi-ble to everyone and that adapts to each person’s needs. Acces-sible technology eliminates barriers for people with disabilities and it enables individuals to take full advantage of their capabilities.” —Bill Gates, Chairman, Microsoft CorporationRob Sinclair

8 Open AccessibilityOpen accessibility is a term that describes both the need for and the realization of interoperability in assistive technolo-gies. This article describes open accessibility and the ways Microsoft and other software development organizations, like Novell and the AIA, are working together to help user access technology across platforms.Norm Hodne

10 Accessibility 101What is accessibility? Learn about the fundamentals of accessibility, the people that accessible technologies help, and the things you can do to help them interact smoothly and successfully with each other.Jennifer Linn, Annuska Perkins

16 Windows Automation API 3.0 OverviewThe ecosystem of Windows® automation technologies includes Microsoft® Active Accessibility and Windows implementations of the UI Automation specifi cation, which provides more fl exible programmatic access to UI elements.Masahiko Kaneko

22 What's new in Windows 7 Automation APIWindows® 7 offers end–to-end accessibility with better performance, seamless interoperability, and improved framework design.Nazia Zaman, Matthew Karr

28 Creating Accessibility-aware Silverlight 2 Content

This article provides you with guidelines for taking advan-tage of Silverlight(TM) 2 features and building an accessible Silverlight application.Mark Rideout

36 Making Custom Controls AccessibleWhile there are many ways to make custom controls accessi-ble, choosing the right solution can be tricky. With Microsoft® technologies, you can implement UI Automation, create or override properties with Dynamic Annotation, or close the gap between Microsoft Active Accessibility and UI Automation.LeAnne Fossmeyer, Michael Bernstein, David Tseng, Vidhya Sriram

42 Microsoft Accessibility Testing ToolsThe few informal tests described in this article can expose a plethora of usability and accessibility shortcomings, over-sights, and other issues in your application. But how do you test, assess, and rectify them?Karl Bridge

48 Internet Explorer 8 New Accessibility Features

Windows® Internet Explorer® 8 has a lot of cool new features that make Web page browsing more accessible, including Caret Browsing, Zoom version 2, high DPI, and support for ARIA and UI Automation.JP Gonzalez-Castellan

56 A Pragmatic Approach to WPF AccessibilityA Microsoft MVP shares his experience addressing ac-cessibility with Windows® Presentation Foundation and adaptive technology vendors.Alvin Bruney

62 Writing a UI Automation ProviderUI Automation providers retrieve data about UI elements for assistive technologies and automation tools. This article walks through code samples for a UI Automation provider for a Win32-based custom control.Brendan McKeon

70 Creating UI Automation Client ApplicationsUI Automation enables a client application to investigate and manipulate the UI of all running applications on the system. This article walks through codes sample that demonstrate the basic features of UI Automation for client applications.Matthew Karr

Departments 7 Code Compilers

59 Advertisers Index

US subscriptions are US $29.99 for one year. Subscriptions outside the US pay US $44.99. Payments should be made in US dollars drawn on a US bank. American Express, MasterCard, Visa, and Discover credit cards are accepted. Bill me option is available only for US subscriptions. Back issues are available. For subscription information, email [email protected] or contact customer service at 832-717-4445 ext 10.

Subscribe online at www.code-magazine.com

CoDe Component Developer Magazine (ISSN # 1547-5166) is published bimonthly by EPS Software Corporation, 6605 Cypresswood Drive., Suite 300, Spring, TX 77379. POSTMASTER: Send address changes to CoDe Component Developer Magazine, 6605 Cypresswood Drive., Suite 300, Spring, TX 77379.

TAB

LE O

F C

ON

TEN

TS

4 Table of Contents www.code-magazine.com

6 www.code-magazine.comA Letter from the Director of Accessibility at Microsoft

cessible applications. Eleven years later, Micro-soft Active Accessibility still powers millions of applications around the globe. UI Automation extends the legacy of Microsoft Active Accessi-bility and provides support for managed applica-tion development through the Windows Presen-tation Foundation (WPF). Windows 7 introduces additional accessibility features for end users, as well as UI Automation improvements in perfor-mance, extensibility, and expanded platform sup-port.

In November of 2007, Microsoft entered into an agreement with Novell to extend UI Automation support to the Linux Accessibility Toolkit. This agreement, entered into under the Microsoft Com-munity Promise Specifi cation initiative, extends UI Automation support to the Linux platform, in-creasing interoperability and further demonstrat-ing Microsoft’s commitment to the accessibility community.

Within these pages you’ll fi nd a number of techni-cal articles describing the state of accessible ap-plication development, as well as overviews of the latest platform support for accessibility in some of Microsoft’s most exciting products, including Win-dows Presentation Foundation (WPF), Silverlight, and Internet Explorer 8. For those interested in learning the basics, this issue provides a thorough overview of accessible application development, intended for those who are completely new to the space.

Our mission at Microsoft is to help people every-where meet their full potential. Our commitment to accessibility is core to who we are—it’s a fi rst-class service in every product that we ship, and a set of investments we continue to grow year after year, product after product. Quite simply, making the computer easier to see, hear, and use is what we’re all about.

At Microsoft®, our commitment to creating the most accessible development platform in the world runs deep. In 1997, Microsoft

Active Accessibility® was released, providing a solid foundation for the fi rst generation of ac-

ON

LIN

E Q

UIC

K I

D 0

8110

12 A Letter from the Director of Accessibility at Microsoft“Our vision is to create innovative technology that is accessible to everyone and that adapts to each person’s needs. Accessible technology eliminates barriers for people with disabilities and it enables individuals to take full advantage of their capabilities.” —Bill Gates, Chairman, Microsoft Corporation

Rob SinclairDirector of Accessibility

Microsoft Corporation

CO

DE

CO

MP

ILER

S

Volume 5 Issue 4

Group Publisher Markus Egger

Associate PublisherRick Strahl

Managing EditorEllen Whitney

Content EditorsKevin Fansler

Writers In This IssueMichael Bernstein Karl BridgeAlvin Bruney LeAnne FossmeyerJP Gonzalez-Castellan Norm HodneMasahiko Kaneko Matthew KarrJennifer Linn Brendan McKeonAnnuska Perkins Mark RideoutRob Sinclair Vidhya SriramDavid Tseng Nazia Zaman

Technical ReviewersMarkus Egger

Art & LayoutKing Laurin [email protected]

ProductionFranz WimmerKing Laurin GmbH39057 St. Michael/ Eppan, Italy

PrintingFry Communications, Inc.800 West Church Rd.Mechanicsburg, PA 17055

Advertising SalesLake Quincy Media [email protected]@lakequincy.com

Sales ManagersErna Egger+43 (664) 151 [email protected] Ferguson832-717-4445 ext [email protected]

Circulation & DistributionGeneral Circulation: EPS Software Corp.Newsstand: Ingram Periodicals, Inc. Media Solutions Worldwide Media

SubscriptionsCirculation ManagerCleo Gaither832-717-4445 ext [email protected]

US subscriptions are US $29.99 for one year. Subscriptions outside the US are US $44.99. Payments should be made in US dollars drawn on a US bank. American Express, MasterCard, Visa, and Discover credit cards accepted. Bill me option is available only for US subscriptions. Back issues are available. For subscription information, email [email protected] or contact customer service at 832-717-4445 ext 10.

Subscribe online atwww.code-magazine.com

CoDe Component Developer MagazineEPS Software Corporation / Publishing Division6605 Cypresswood Drive, Ste 300, Spring, Texas 77379 Phone: 832-717-4445 Fax: 832-717-4460

c o m p o n e n t d e v e l o p e r m a g a z i n e

8 www.code-magazine.comOpen Accessibility

Norm [email protected],+1 (425) 704-7324

Norm Hodne is the Lead Program Manager in the Windows Experience Group responsible for the accessibility APIs, accessibility applications, and the speech commanding and dictation features shipped in Windows.

Norm also is a representative to the INCITS V2 standards organization where he works on standards related to accessibility.

number of users access technology in their daily lives. By making UI Automation freely available Microsoft hopes to increase the accessibility of many informa-tion and communication technologies and to drive cross-platform interoperability.

Microsoft UI Automation: An Innovative Accessibility Model

UI Automation, a next-genera-tion accessibility programming model, simplifi es development and reduces costs for AT devel-opers as well as IT application developers who want to make their software compatible with AT products, such as screen readers for people who are blind. UI Automation, which is available for Windows® XP,

Windows Vista®, Windows 7, and is built into the Windows Presentation Foundation (WPF), Silver-light™ 2 and in additional products, offers developers a richer and more effi cient way to convey user inter-face commands and behaviors to a person through their AT product.

UI Automation represents what the interface can do rather than how it is presented visually, making it easier for both the application developer and as-sistive technology developers to create rich experi-ences for their users, including people who are blind. AT products built using UI Automation will require fewer updates and fewer customizations for specifi c applications, because they can intelligently interact with user interfaces and controls that have yet to be invented.

Linux Implementation

Microsoft developed UI Automation from the start to be compatible for implementation on platforms other than Windows. Starting in 2008, a Novell team is implementing UI Automation for Linux and will support SUSE, Ubuntu, and Red Hat and make the code available to other Linux distributions. Their implementation work has several goals:

ON

LIN

E Q

UIC

K I

D 0

8110

22 Open AccessibilityCreating a natural user interface requires designers, testers, and developers working in concert to develop the right support that makes multi-modal access to an operating system and applications possible. To assist in this work through the Accessibility Interoperability Alliance (AIA), Microsoft® released its UI Automation Specifications with a Community Promise and released testing tools as open source projects via CodePlex. Microsoft is committed to interoperable accessibility.

Making Technology Accessible and InteroperableThe purpose of the AIA, formed recently by a coali-tion of the world’s leading information and assistive technology (AT) companies, is to foster industry-wide collaboration aimed at reducing many of the barri-ers that people with disabilities frequently encounter when they try to access information and communi-cation technologies, including Web sites.

To create accessible technol-ogy products today, developers work across multiple platforms, application models, and types of hardware. To address interoper-ability issues arising from this current environment, the AIA is using a two-part strategy: First, the group is working to harmo-nize current technologies to in-teroperate more easily, reducing costs for developers and creating more usable products for customers. Second, it is working to create a unifi ed accessibility model that will serve the entire industry. In order to support backwards compatibility and bridging tech-

nology to a full UI Automation implementation, Microsoft re-leased the UI Automation Speci-fi cation with the UI Automation, Microsoft Active Accessibility, and IAccessibleEx specifi cations.

Microsoft also released the UI Au-tomation Specifi cation to the AIA, an engineering working group dedicated to making it easier for developers to create software, hardware, and Web-based prod-ucts that are accessible to people with disabilities. The UI Auto-

mation specifi cation describes the latest accessibility framework technology developed by Microsoft, and will help developers include advanced accessibility functionality into implementations designed for use on any operating system.

UI Automation allows developers to create assistive technologies across all platforms and help a larger

UI Automation allows developers to

create assistive technologies across all platforms and

help a larger number of users access technology in their daily lives.

Novell will implement UI Automation for Moonlight that will provide accessibility

capabilities for Silverlight applications on Linux.

Fast Facts

9www.code-magazine.com Open Accessibility

AIA MembershipThe Accessibility Interoperability Alliance offers membership to IT and AT companies, as well as to non-governmental organizations (NGOs).

The AIA is committed to developing and enhancing standards and promoting the adoption of the standards to support the estimated one in four computer users that can benefi t from AT applications.

AIA Steering CommitteeThe Steering Committee for the Accessibility Interoperability Alliance includes fi ve members. The Steering Committee includes Qualilife and GWMicro in the AT member category, IBM and Adobe in the IT member category, and Microsoft as the at-large member.

tion framework that features the User Interface Auto-mation Test Library (UIA Test Library) and Visual UI Automation Verify (Visual UIA Verify), the graphical user interface (GUI) tool. The framework facilitates manual and automated testing of the Microsoft UI Automation Provider implementation of a control or application. The majority of the UIA Verify func-tionality is provided through a dynamic link library (such as UIA TestLibrary.dll) that contains the code for testing spe-cifi c UI Automation functionality and supports logging of the test results.

About the Community PromiseAs a member of the AIA, Micro-soft has agreed to grant a royal-ty-free license for any Microsoft patents necessary to implement required portions of the UI Au-tomation Specifi cation, as the specifi cation may be modifi ed and eventually published by the AIA. Companies also can implement the latest version of the UI Automation Specifi cation, which is publicly available from Microsoft. The Community Promise that accompanies the UI Automation Specifi ca-tion permits royalty-free access to Microsoft patent claims necessary to implement required portions of both mandatory and optional parts of the UI Auto-mation Specifi cation.

Committed to Open Accessibility

Microsoft is continuing its commitment to accessi-bility by creating innovative accessibility APIs and testing tools and releasing them with a Community Promise or as open source projects. This open ap-proach to accessibility technology allowed Novell to start development of UI Automation for Linux; it should also enable the development of lower-cost, interoperable solutions that ultimately make technol-ogy accessible to greater numbers of people through-out the world.

Resources

Accessibility Interoperability Alliance http://www.accessinteropalliance.org/

UI Automation Community Promise http://msdn.microsoft.com/en-us/accessibility/bb892136.aspx

Novell UI Automation Accessibility Project http://www.mono-project.com/Accessibility

CodePlex project hosting Web site http://codeplex.com/

Make Windows Forms accessible• Make Moonlight accessible•

Implement UI Automation technologies•

Mono is an open source project sponsored by No-vell which implements software to develop and run .NET client and server applications on Windows, Li-nux, Solaris, Mac OS X, and UNIX. Based upon the Mono Web site, Novell believes that Mono is well positioned to become the leading application devel-opment platform for Linux.

Implementing UI Automation on Linux retains the existing Linux accessibility frameworks. UI Automa-tion will integrate with the AT-SPI and ATK acces-sibility frameworks by implementing the client inter-faces in a bridge that will act as a translation layer between the frameworks.

Open Source Tools

Keeping with its commitment to help the community build better experiences for people that use assistive technology, Microsoft released two new testing tools as open source on the CodePlex shared code site. The UI Accessibility Checker (AccChecker) a Microsoft Active Accessibility® testing tool and UI Automation Verify (UIA Verify), a UI Automation testing tool, were released in March 2008. These tools are targeted at both developers and testers to provide support for the development process to create and ship products that meet the needs of assistive technology users.

UI Accessibility Checker

AccChecker enables testers, without prior experience with accessibility testing, to easily discover accessi-bility problems with Microsoft Active Accessibility and other user interface (UI) implementations. Acc-Checker was born from the realization that existing Microsoft Active Accessibility tools, such as Inspect and AccExplorer, provided in-depth details on the implementation, but no information on the correct-ness of the accessibility implementation.

UI Automation Verify Test Automation Framework

UIA Verify is the second accessibility-testing tool available on CodePlex. UIA Verify is a test automa-

…Microsoft has agreed to grant a royalty-free license for any patents necessary to implement required portions

of the UI Automation Specifi cation…

Microsoft is continuing its commitment to accessibility

by creating innovative accessibility APIs and testing

tools and releasing them with a Community Promise or

as open source projects.

Norm Hodne

10 www.code-magazine.comAccessibility 101

ON

LIN

E Q

UIC

K I

D 0

8110

32

Jennifer [email protected]

Jenny is a writer in the Windows Experience group and has been working with Accessibility for 8 years. Currently her main focus is on the MSDN Accessibility Developers Center (http://msdn.microsoft.com/en-us/accessibility/default.aspx) and internal accessibility guidelines for product groups. In her spare time, she enjoys walking homeless dogs at the Seattle Humane Society (http://www.seattlehumane.org) and reading about history.

Accessibility 101We know what you’re thinking: Why should I read an article about the fundamentals of accessibility? Well, if you already know the percentage of computer users who have disabilities, can name at least ten different categories of assistive technologies, and can describe the key concepts involved in designing an accessible application, then you can probably skip to the next article. However, if you’re unsure what accessible technology is, then take a few minutes and keep reading. You’ll learn about the main concepts around accessible technology, the people they help, and things you can do to help them interact smoothly and successfully with each other.

of solutions that makes computer use more comfort-able for some people and possible for others.

As a developer, one of your main concerns is to en-sure that you provide programmatic access to your application’s user interface elements. This enables the assistive technologies to get information about the UI elements and expose that information to the assistive

technologies, which in turn relays the information to the user. For example, important information that an AT needs includes type, name, location, and current state of a UI element. They also need to know when changes in the UI occur. For information about how to provide programmatic access using Microsoft’s acces-sibility frameworks, Microsoft UI Automation and Microsoft Active Accessibility, see the Ac-cessibility Developer Center on MSDN (http://msdn.microsoft.com/en-us/accessibility/default.aspx).

Programmatic access is just one of the key concepts of creating accessible applications. The following section presents ad-ditional concepts about acces-sible design.

Who Benefi ts from Accessible Technology?Although most accessible technology was originally intended and designed for individuals with severe disabilities, accessible technology is widely used by computer users of all abilities today. Among adult computer users in the United States:

1 in 4 has a visual disability.• 1 in 4 has a dexterity disability.• 1 in 5 has a hearing disability.•

What We Mean When We Talk about AccessibilityAccessible technology is a piece of software or hard-ware that makes it easier for someone to see, hear, and use a technology product, such as a computer or a mobile device. It can be an assistive technology (AT) product, a specially designed piece of software or hardware that accommodates someone’s disability (or multiple disabilities) and enables them to interact with a computer. These products are developed to work with a computer's operating sys-tem and software. For example, a blind computer user may use as-sistive technology called a screen reader to navigate an applica-tion’s interface and a computer user with ALS may use a word prediction program to help fa-cilitate communicating thoughts and ideas.

Many companies offer hardware devices, accessories, aids, and software applications that fall under the umbrella of assistive technology. For an overview of assistive technology categories, see Descriptions of Assistive Technology Products at the end of this article.

Accessible technology can also be a feature built into a product that allows someone to adjust the settings to meet their needs. Examples of accessibility fea-tures include those that allow a user to increase font size, change font settings, or choose different colors for their computer screen display. Other examples are the option for users to receive announcements from their computer through sound notifi cations (a "ding" when new e-mail messages arrive), or visual notifi cations (a modal dialog fl ashes and beeps when the user tries to click away from it). So overall it is helpful to think of accessible technology as a range

57% of computer users are likely or very likely to benefit from the use of accessible technology.

44% of computer users use some form of

accessible technology.

Users seek solutions to make their computers

easier to use, not for solutions based on their

health or disability.

Making accessibility options easier to discover and

use will result in computers that are easier to use,

more convenient, and more comfortable for

computer users.

Fast Facts

11www.code-magazine.com Accessibility 101

Annuska PerkinsAnnuska Perkins has called Microsoft home since 1998. She began as a program manager for MSN Side-walk.com, an online city guide. Afterwards, she joined the Accessibility group, where her role evolved from program manager, to product planner, to User Experience expert. Check out her blog and projects on the MSDN Accessibility site: http://msdn.microsoft.com/accessibility.

Profi le: Low Vision

Olivia, a managing editor, has had low vision as a result of glaucoma. Low vision is a term commonly used to mean partial sight, or sight that isn't fully correctable with surgery, pharmaceuticals, contact lenses, or glasses. It includes moderate vision im-pairment, such as tunnel vision or blind spots, as well as legal blindness and almost total blindness. Sometimes it involves a lack of acuity, meaning that objects appear blurred. Other times, it involves a re-duced ability to distinguish colors, see contrasts, or determine spatial relationships among objects.

Olivia can see type and images on her computer screen when they are enlarged to about 1.5 inches in height. As her glaucoma advances, print will appear faded and words will be diffi cult to read. By using a screen enlargement program, Olivia can effectively view and interact with documents and tools that are an important part of her job. In addition, she uses high DPI for sharpness and turns on the high con-trast accessibility setting with the enlarger because having the magnifi ed screen displayed in black on white improves the readability for her.

While she is currently able to use a mouse, she knows she needs to learn more about keyboard ac-cess because she will have to switch to keyboard use as her vision continues to deteriorate. Even with her high level of magnifi cation, Olivia tends to sit ex-tremely close to the monitor. In addition, due to the increased level of magnifi cation, she needs to scroll horizontally and vertically many times in order to see what other customers will see all at once on the screen. She has to work to remember where items are in order to retain context, for example, where controls and text are meant to be seen together but aren’t due to her level of magnifi cation. These factors make her prone to headache, eyestrain, and fatigue.

Although many accessible technologies are de-signed to help people with disabilities optimize their abilities, research shows that the majority of all computer users can benefi t from adjusting their display, mouse, keyboard, and sound settings. Us-ers can fi nd accessible technologies helpful when, for example, recovering from shoulder surgery. Another example is using a mobile device when driving a car or riding a bus, in which case, voice recognition or output can be helpful. According to a two-part study commissioned by Microsoft and conducted by Forrester Research (The Market for Accessible Technology—The Wide Range of Abili-ties and Its Impact on Computer Use: http://www.microsoft.com/enable/research/phase1.aspx and Accessible Technology in Computing—Examining Awareness, Use, and Future Potential: http://www.microsoft.com/enable/research/phase2.aspx), the majority of computer users can benefi t from using accessible technology. Figure 1 shows that 57% (74.2 million) of computer users are likely or very likely to benefi t from the use of accessible tech-nology due to having mild or severe disabilities. Specifi cally:

40% (51.6 million) of computer users are likely • to benefi t from the use of accessible technol-ogy due to mild diffi culties/impairments.17% (22.6 million) of computer users are • very likely to benefi t from the use of accessible technology due to severe diffi culties/impair-ments.

What does that mean for you? That’s a lot of people who will depend on the accessibility of your appli-cation. Accessible technology has the potential to improve computer use for a wider audience because it makes computers easier to use. A key component of encouraging the use of accessible technology is to make it easier to fi nd and highlight the functionality and benefi ts rather than the impairments they seek to ameliorate.

To get a more personal view of how accessible tech-nology helps people, here is a look at some profi les of computer users today. The following profi les de-scribe how people with common types of disabili-ties might use accessible technology to help them in their job and everyday life. Each profi le is followed by design and development considerations for cre-ating accessible applications.

Computer Users with Vision Disabilities

Vision disabilities include low vision, color blindness, and blindness. Among adult computer users in the United States, 1 in 4 (27%) have a vision disability. There are many options for individuals with vision disabilities to modify the computer displays and ap-pearance so it is more legible, or receive information through sound or touch. Those who are blind can-not use a computer monitor, but have the option to receive information from their computers through hearing or touch offered through screen readers and Braille displays.

Figure 1: This chart also shows the percentages of computer users who are likely or very likely to benefi t from the use of accessible technology due to a range of mild to severe diffi culties and impairments.


Accessibility ResourcesWhether you’re developing accessibility technology or designing an accessible Web site, you’ll need a bounty of resources. The following are tools, specifi cations, guidelines, and resources you can use to make computer-based content and technology accessible to all.

Tools

accChecker is a Microsoft® Active Accessibility® API test tool with additional accessibility test automation features. http://www.codeplex.com/AccCheck

UIA Verify is a UI Automation API test tool with additional accessibility test automation features. http://www.codeplex.com/UIAutomationVerify

Fujitsu ColorDoctor simulates display content according to grayscale and color characteristics. http://design.fujitsu.com/en/universal/assistance/colordoctor/

Wat-C Color Contrast Analyser tests whether foreground and background color combinations provide good color visibility. It also simulates certain visual conditions such as color blindness. http://www.paciellogroup.com/resources/contrast-analyser.html

Ensure that your feature can be used without • a mouse. Provide powerful, intuitive keyboard shortcuts.Support prefi x navigation (typing the fi rst few • characters to reach a known destination).

Ensure tab orders are logical.•

Computer Users with Hearing Disabilities

Hearing disabilities encompass a wide range of con-ditions—from slight hearing loss to deafness. People who have hearing disabilities might be able to hear some sound, but might not be able to distinguish words. Among adult computer users in the United States, 1 in 5 (21%) have a hearing disability.

Profi le: Deafness

Tim is a network architect and has been deaf from birth. He works very effectively with his computer; in addition to using e-mail, he uses instant messaging to communicate with his colleagues and co-workers.

Since he is deaf, he does not need sound information from his computer so he has turned sound off com-pletely. To be sure that he gets information visually that would otherwise be conveyed by sound, he uses SoundSentry to provide visual warnings for system sounds, ShowSounds to display captions for speech and sounds, and Visual Notifi cation to provide visual warnings when features are turned on or off. He has turned on the captioning features where available in products.

Tim also uses a TTY (Telephone Typewriter) device (replacing a handset phone) and a text pager with the keyboard set to vibrate attached to his belt. This pager does text to voice for outgoing messages and voice to text for incoming messages.

Design and Development Considerations: Deaf Computer Users

To ensure that a deaf computer user’s experience is equal to that of users not requiring assistive technol-ogy, keep the following design guidelines in mind:

If your application plays video, provide a way • to turn on and off captioning. Use the Show-Sounds system setting as a cue to the user’s preference.Be sure that your feature works with Visual • Notifi cation (Sound Sentry) system setting so that users receive visual cues in lieu of sound notifi cations.

Providing visual customizations can greatly en-• hance the customer experience. For example, provide ways for users to change the size, color, and fl ashing characteristics of sound alternatives.

Computer Users with Dexterity Disabilities

Individuals with dexterity disabilities experience pain, discomfort, or complete loss of feeling in their

Design and Development Considerations: Low Vision

To ensure that a low vision computer user’s experience is equal to that of users not requiring assistive technol-ogy, keep the following design guidelines in mind:

Ensure that individual objects in the UI can • easily be distinguished and that each object has a clear, unique, meaningful label that makes sense when read out of context.Respect the Windows user system settings re-• lated to accessibility, such as color choices and icon sizes.Do not specify hard-coded fonts. Instead, en-• able users to change the font size and face. Use default font size of 10 points or larger.Ensure that users can set color settings on a sys-• tem-wide basis and have them respected within their applications.Check for appropriate use of color in fore-• ground and background combinations.Ensure programmatic access to UI elements.• Provide full keyboard access to your applica-• tions functionality via keyboard focus, access keys, and shortcut keys.

Ensure that the UI adapts well to high DPI set-• tings.

Profi le: Blindness

Jorge, a fi nancial analyst, has been blind for fi ve years, ever since a car accident. He can see a little light but no images and uses a guide dog to navigate within buildings and outside.

After his car accident, Jorge went through extensive re-habilitation to learn how to use assistive technology to continue working. He uses a screen reader to relay in-formation from his computer, along with an earphone so that he can work without disturbing co-workers. He also uses a refreshable Braille display to supplement the voice information he receives from the screen reader. He uses Braille copies of the fi rm’s reports and analyses when needing to work with hardcopy.

Jorge also relies on the keyboard to navigate and on memorization of spatial relationships and keyboard shortcuts. Consistency and predictability within and across applications are great aids to his productivity.

Design and Development Considerations: Blind Computer Users

To ensure that a blind computer user’s experience is equal to that of users not requiring assistive technol-ogy, keep the following design guidelines in mind:

Provide programmatic access to the UI for as-• sistive technologies such as screen readers and Braille displays.Use text that is easily understood and that • makes sense when read out loud with no visual association. Supply alternative text for images.Group information logically and place the main • idea of each paragraph in the fi rst sentence.


Microsoft® Internet Explorer® Developer Toolbar provides a variety of tools for quickly creating, understanding, and troubleshooting Web pages. http://www.microsoft.com/downloads/details.aspx?FamilyID=e59c3964-672d-4511-bb3e-2d5e1db91038&DisplayLang=en

ARIA Resources

Web Accessibility Initiative WAI provides guidelines, resources, and strategies to make the Web accessible to people with disabilities. http://www.w3.org/WAI/

WAI-ARIA Primer provides background on the accessibility problem space solved with ARIA. http://www.w3.org/TR/wai-aria-primer/

ARIA Specifi cation defi nes the ARIA syntax. http://www.w3.org/TR/wai-aria/

WAI-ARIA Best Practices describes how to develop accessible rich Web applications using ARIA. http://www.w3.org/TR/wai-aria-practices

talking dictionary, phonetic spell-checking, word-prediction, and homophone support to address key aspects of reading and writ-ing processes. In addition, she has turned on features in the operat-ing system to help her focus on tasks, and she sometimes uses a headset to block out background noise.

Design and Development Con-siderations: Learning Disabili-ties

To ensure that a computer user with a learning disability has an end-user experience equal to that of users not requiring assistive technology, keep the following design guidelines in mind:

Enable users to perceive information in mul-• tiple ways, such as visually and aurally.Avoid unnecessary animations and distracting • UI elements. Provide a way to turn off anima-tions.

Adhere to common UI visual guidelines to pro-• vide a consistent and intuitive experience. For more information, see the Windows Vista User Experience Guidelines on MSDN (http://msdn.microsoft.com/en-us/library/aa511258.aspx).

Aging Computer Users

Do you have trouble seeing things on your com-puter screen? By the time we reach our fi fties, two-thirds of us have vision, hearing, or dexterity im-pairments that will impact our use of the computer. Some functional losses are accelerated by the onset of age-related degenerative diseases and ailments, including hypertension, osteoporosis, diabetes, and macular degeneration. Disabling conditions, includ-ing arthritis and orthopedic impairments resulting from sports, vehicle, and occupational injuries ex-perienced earlier in life tend to manifest themselves as the body ages.

Profi le: Aging Population

Fran is a retired librarian and spends her mornings hosting a book club and taking an aerobics class. She and her husband, Roy, bought a new computer when their kids started sharing photos and videos of their families.

Both Fran and Roy have diffi culty reading text on the computer screen and prefer different color schemes and settings. Roy prefers larger icons and a slower mouse cursor, while Fran prefers photos of her grandchildren on the desktop, large blue text, and a stylized mouse pointer for easier visibility. They each set up a profi le with their preferred settings so they can share fi les while personalized settings make it easier for each to read text on the computer screen.

fi ngers, hands, wrists, or arms, making it diffi cult to use a standard keyboard or mouse. Among adult computer users in the United States, 1 in 4 (26%) have a dexterity diffi culty. Dexterity diffi culties and impairments can be caused by a wide range of com-mon illnesses and accidents such as carpal tunnel, arthritis, stroke, cerebral palsy, Parkinson's disease, multiple sclerosis, loss of limbs or digits, spinal cord injuries, and repetitive stress injury, among others.

Profi le: Repetitive Stress Injury

Jason is a medical researcher, studying molecular bi-ology. In high school, Jason played tennis and has been playing regularly for the last 15 years. Unfortu-nately, years of playing have been hard on his joints and he recently had surgery for a rotator cuff injury.

Jason is undergoing physical therapy to strengthen the muscles of his rotator cuff. He is able to con-duct his work by using a speech recognition program installed on his computer. He also answers phone calls right on his computer and has a mobile phone equipped with voice command.

Design and Development Considerations: Dexter-ity Disabilities

To ensure that a computer user’s experience is equal to that of users not requiring assistive technology, keep the following design guidelines in mind:

Provide full keyboard access to your applica-• tions functionality via keyboard focus, access keys, and shortcut keys.

Enable programmatic access.•

Computer Users with Learning Disabilities

Learning disabilities can range from conditions such as dyslexia and attention defi cit disorder to intellec-tual disabilities. Some individuals with learning dis-abilities are better able to process information when it is presented to them in a form and at a pace that is appropriate to them individually. During the learning process, many people with learning disabilities ben-efi t from having a multisensory experience of audio speech paired with a visual representation. Reduc-ing visual and auditory distractions can also aid the learning process for many people.

Profi le: Dyslexia

Amelia is a fi fth grade student who wants to be a photographer when she grows up. A bright and ca-pable student, she struggles with dyslexia and admits that she used to be scared of reading books aloud in class. Fortunately, Amelia's parents and teacher rec-ognized the dyslexia early, and the school’s assistive technology specialist helped fi nd a software program that helps her read text in class and at home.

Amelia's computer is now outfi tted with a reading and writing software program. The program high-lights text as it reads the text aloud and includes a

Microsoft’s Commitment to Accessibility:

“Our vision is to create innovative technology that is

accessible to everyone and that adapts to each person's needs.

Accessible technology eliminates barriers for people with disabilities

and it enables individuals to take full advantage of their

capabilities.”—Bill Gates, Microsoft Corporation


Microsoft Resources

Profi les of Accessibility in Action details inspiring profi les of accessibility in action among fi ve individuals using accessibility options and assistive technology products on their computers. http://www.microsoft.com/enable/profiles/default.aspx

Microsoft® Accessibility Developer Center offers guidance, essential information, and tools for developing accessible applications and writing accessible code. http://msdn.microsoft.com/accessibility

Microsoft Active Accessibility on MSDN Library describes Microsoft Active Accessibility in detail. http://msdn.microsoft.com/en-us/library/ms697707(VS.85).aspx

UI Automation Specifi cation and Community Promise http://msdn.microsoft.com/en-us/accessibility/bb892133.aspx

Microsoft Accessible Technologies http://www.microsoft.com/enable/

sticks, trackballs, and touch screens; and alternative output systems such as speech synthesizers, Braille embossers and displays, and screen readers.

Below are descriptions of the various types of assis-tive technology products that are currently available on the market today. For more information, you can search the catalog of assistive technology products for products compatible with the Windows operat-ing system on the Microsoft.com/enable site.

Screen magnifi ers (or screen enlargers) work like a magnifying glass for the computer by enlarging a portion of the screen, increasing legibility, and mak-ing it easier to see items on the computer. Some screen magnifi ers allow a person to zoom in and out on a particular area of the screen.

Screen readers are used to verbalize, or "speak," items on the screen including text, graphics, control buttons, and menus into a computerized voice that is spoken aloud. In essence, a screen reader transforms a graphic user interface (GUI) into an audio interface. Screen readers are essential for computer users who are blind.

Speech recognition, or voice recognition, pro-grams allow people to give commands and enter data using their voices rather than a mouse or key-board. Voice recognition systems use a microphone attached to the computer, which can be used to cre-ate text documents such as letters or e-mail mes-sages, browse the Internet, and navigate among ap-plications and menus by voice.

Refreshable Braille displays provide tactile output of information represented on the computer screen. A Braille "cell" is composed of a series of dots. The pattern of the dots and various combinations of the cells are used in place of letters. Refreshable Braille displays mechanically lift small rounded plastic or metal pins as needed to form Braille characters. The user reads the Braille letters with his or her fi ngers, and then, after a line is read, can refresh the display to read the next line.

Braille embossers transfer computer generated text into embossed Braille output. Braille transla-tion programs convert text scanned-in or generated via standard word processing programs into Braille, which can be printed on the embosser.

On-screen keyboards provide an image of a stan-dard or modifi ed keyboard on the computer screen that allows the user to select keys with a mouse, touch screen, trackball, joystick, switch, or elec-tronic pointing device. On-screen keyboards often have a scanning option that highlights individual keys that can be selected by the user. On-screen keyboards are helpful for individuals who are not able to use a standard keyboard due to dexterity or mobility diffi culties.

Alternative input devices allow individuals to con-trol their computers through means other than a standard keyboard or pointing device. Examples include:

They now use the computer for nearly every aspect of their lives including banking and fi lling prescrip-tions while traveling, managing digital photos, and communicating with family members using e-mail

and video phone conferencing.

Design and Development Con-siderations: The Aging Population

To ensure that customers like Fran and Roy have an end-user experience equal to that of users not requiring assistive technolo-gy, keep the design guidelines for low vision, dexterity, and cogni-tive changes in mind.

High-level Concepts for Creating Accessible UIIn addition to providing programmatic access and following the considerations described above, what are the high-level concepts to consider to ensure that your application’s UI is accessible?

Ensure that all functionality of the UI is usable • by people with vision, dexterity, hearing, and cognitive limitations.Make your application’s default UI accessible. • For example, chose a color palette with con-trasting colors and avoid color combinations that are not perceivable by people with color-blindness.Make your UI fl exible and customizable so that • users can optimize the UI. As the user specifi es their preferences through Windows settings en-sure your application responds appropriately. (The user’s settings are discoverable program-matically through the Windows SystemParam-etersInfo() API: http://msdn.microsoft.com/en-us/library/ms724947(VS.85).aspx.)

Make the UI easy to navigate regardless of the • input device. Ensure that it can be accessed whether a customer uses a mouse, keyboard, on-screen keyboard, or speech recognition.

Descriptions of Assistive Technology ProductsAssistive technology products are designed to pro-vide additional accessibility to individuals who have physical or cognitive diffi culties, impairments, and disabilities. When selecting assistive technol-ogy products, it is crucial to fi nd products that are compatible with the computer operating system and programs on the particular computer being used.

More than 100 companies offer hardware devices, accessories, aids, and software applications that fall under the umbrella of assistive technology. These al-ternative input products include speech recognition systems, alternative keyboards, electronic pointing devices, sip-and-puff systems, wands, sticks, joy-

Among adult computer users in the United States:

1 in 4 has a vision diffi culty• 1 in 4 has a dexterity diffi culty• 1 in 5 has a hearing diffi culty•


hear what they are typing and also provide a spo-ken voice for individuals who cannot communicate orally, but can communicate their thoughts through typing.

Talking and large-print word processors are soft-ware programs that use speech synthesizers to pro-vide auditory feedback of what is typed. Large-print word processors allow the user to view everything in large text without added screen enlargement.

Conclusion

Whether you know it or not, many people with disabilities probably use your applications. Under-standing their needs is the fi rst step to designing ac-cessible applications that can be used by everyone.

Alternative keyboards• feature larger- or smaller-than-standard keys or keyboards, al-ternative key confi gurations, and keyboards for use with one hand.Electronic pointing devices• are used to con-trol the cursor on the screen without use of hands. Devices used include ultrasound, infra-red beams, eye movements, nerve signals, or brain waves.Sip-and-puff systems• are activated by inhal-ing or exhaling.Wands and sticks• are worn on the head, held in the mouth or strapped to the chin and used to press keys on the keyboard.Joysticks• are manipulated by hand, feet, chin, etc., and used to control the cursor on screen.Trackballs• are movable balls on top of a base that can be used to move the cursor on screen.Touch screens• allow direct selection or activa-tion of the computer by touching the screen, making it easier to select an option directly rather than through a mouse movement or keyboard. Touch screens are either built into the computer monitor or can be added onto a computer monitor.TTY/TDD conversion modems• are connected between computers and telephones to allow an individual to type a message on a computer and send it to a TTY/TDD telephone or other Baudot-equipped device.Light signaler alerts• monitor computer sounds and alert the computer user with light signals. This is useful when a computer user cannot hear computer sounds or is not directly in front of the computer screen. As an example, a light can fl ash alerting the user when a new e-mail message has arrived or a computer com-mand has completed.

Keyboard fi lters are typing aids such as word pre-diction utilities and add-on spelling checkers that reduce the required number of keystrokes. Key-board fi lters enable users to quickly access the let-ters they need and to avoid inadvertently selecting keys they don't want.

Reading tools and learning disabilities programs include software and hardware designed to make text-based materials more accessible for people who have diffi culty with reading. Options can in-clude scanning, reformatting, navigating, or speak-ing text out loud. These programs are helpful for those who have diffi culty seeing or manipulating conventional print materials; people who are devel-oping new literacy skills or who are learning Eng-lish as a foreign language; and people who compre-hend better when they hear and see text highlighted simultaneously.

Text-to-Speech (TTS) or speech synthesizers re-ceive information going to the screen in the form of letters, numbers, and punctuation marks, and then "speak" it out loud in a computerized voice. Using speech synthesizers allows computer users who are blind or who have learning diffi culties to

Jennifer LinnAnnuska Perkins

16 www.code-magazine.com

ON

LIN

E Q

UIC

K I

D 0

8110

42

Windows Automation API 3.0 Overview

Masahiko [email protected]

Masahiko Kaneko is a Senior Program Manager on the User Interface Platform Team in the Windows Experience Division. A program manager in accessibility at Microsoft for more than 10 years, he has been involved with several releases of the Windows Operating System as well as many other Microsoft products.

Masahiko enjoys road biking, XC skiing, music, and movies in his free time but most of it is now dedicated to his fi rst son who is turning 18 months in September.

UI Automation is designed for a modern and evolving user experience. The latest Windows 7 implementation

helps applications be compatible with existing accessibility applications while fully supporting new user interface paradigms. UI Automation is actually faster and more reliable

than its predecessor, Microsoft Active Accessibility.

Fast Facts

cessibility refers to the application or control offering the UI for accessibility as the server, while UI Auto-mation refers to this as the provider.

Microsoft Active Accessibility offers a single COM interface with a fi xed, small set of properties. UI Au-tomation offers a richer set of properties, as well as a

set of extended interfaces called control patterns to manipulate automation elements in ways Microsoft Active Accessibility cannot.

While UI Automation previ-ously had both managed and unmanaged API for providers, the original release had no un-managed interfaces for clients. With Windows Automation API 3.0, you can fi nally write UI Au-tomation clients entirely in un-managed code.

The new API also provides sup-port for transitioning from Mi-crosoft Active Accessibility serv-

ers to UI Automation providers. The IAccessibleEx interface enables legacy Microsoft Active Accessibil-ity servers to add support for specifi c UI Automation patterns and properties without rewriting their whole implementation. The specifi cation also allows in-pro-cess Microsoft Active Accessibility clients to access UI Automation provider interfaces directly, rather than through UI Automation client interface.

The ecosystem of Windows automation technologies, now called Windows Automation API, includes classic Microsoft Active Accessibility and Windows implementations of the UI Automation specifi ca-tion. At Microsoft, the UI Automation specifi cation is implemented on Windows Vista, Windows Server 2008, Windows Presentation Foundation (WPF), XPS Viewers, and many other upcoming Microsoft products. Windows 7, Windows Internet Explorer 8, and Silverlight 2.0 are joining the pack soon.

Windows Automation API 3.0 OverviewWhile general accessibility requirements (such as font colors in UI rendering) are important, programmatic access to the graphical user interface (GUI) is a crucial element to improving accessibility. On the Windows® operating system, Microsoft® Active Accessibility® and User Interface (UI) Automation support this programmatic access. This article provides a quick overview of Windows Automation API 3.0 featured in Windows 7.

Windows Automation API 3.0: a Bit of HistoryToday, the Windows operating system offers two application programming interfaces (API) specifi -cations for user interface accessibility and software test automation. The legacy API, Microsoft Active Accessibility, was introduced to Windows 95 as a platform add-on in 1996. The new API is a Windows implementation of the User Interface Automation specifi cation called UI Automa-tion. UI Automation was intro-duced in Windows Vista® and .NET Framework 3.0.

Although the two technologies are different, the basic design principles are similar. Both ex-pose the UI object model as a tree hierarchy rooted at the desktop. Microsoft Active Ac-cessibility represents individual UI elements as accessible ob-jects, and UI Automation rep-resents them as automation elements. Both refer to the accessibility tool or software automation pro-gram as the client. However, Microsoft Active Ac-

Figure 1: Microsoft Active Accessibility uses OLEACC.dll to communicate between clients, like screen readers, and servers, such as Windows applications.

17www.code-magazine.com Windows Automation API 3.0 Overview

The Architecture: Microsoft Active Accessibility, UI Automation, and Interoperability

The goal of Microsoft Active Accessibility is to ex-pose basic information about custom controls such as control name, location on screen, and type of con-trol, as well as state information such as visibility and enabled/disabled status. The UI is represented as a hierarchy of accessible objects; changes and actions are represented as WinEvents. The following com-ponents comprise the Microsoft Active Accessibility architecture:

Accessible Object—A logical UI element (such • as a button) that is represented by an IAcces-sible COM interface and an integer ChildID.WinEvents—An event system that enables serv-• ers to notify clients when an accessible object changes.

OLEACC.dll—run-time, dynamic-link library • that provides the Microsoft Active Accessibility API and the accessibility system framework.

For Microsoft Active Accessibility, the system com-ponent of the accessibility framework (OLEACC.dll) helps the communication between accessibility tools and applications (Figure 1). The applications (Microsoft Active Accessibility servers) provide UI accessibility information to tools (Microsoft Active Accessibility clients), which interact with the UI on behalf of users. The code boundary can be a pro-grammatic or process boundary.

The goal of UI Automation is similar but broader, as described later in this article. From an archi-tecture point of view, UI Automation loads the UI Automation Core component into both the acces-sibility tools’ and applications’ processes (Figure 2). This component manages cross-process com-munication and provides higher level services. This core component enables bulk fetching or caching of properties, which improves the cross-process performance over Microsoft Active Accessibility implementation.

Interoperability between Microsoft Active Accessibility-based and UI Automation-based Applications

The UIA-to-MSAA Bridge enables Microsoft Active Accessibility clients to access UI Automation provid-ers by converting the UI Automation object model to a Microsoft Active Accessibility object model (Fig-ure 3). Similarly, the MSAA-to-UIA Proxy (Figure 4) translates Microsoft Active Accessibility-based server object models for UI Automation clients.

Now with IAccessibleEx, you can also improve ex-isting Microsoft Active Accessibility server imple-mentations by adding only required UI Automation object model information. The MSAA-to-UIA Proxy takes care of incorporating the added UI Automa-tion object model.

Figure 2: User Interface Automation (UI Automation) uses the UI Automation Core to communicate between clients and providers and uses proxies to communicate with legacy implementations.

Figure 3: UIA-to-MSAA Bridge enables Microsoft Active Accessibility clients to access UI Automation providers.

Figure 4: The MSAA-to-UIA Proxy enables UI Automation clients to access Microsoft Active Accessibility servers.

Limitations of Microsoft Active AccessibilityMicrosoft designed the Microsoft Active Acces-sibility object model about the same time as Win-dows 95 released. The model is based on “roles” defi ned a decade ago, and you cannot support new

18 www.code-magazine.comWindows Automation API 3.0 Overview

is broader in scope than just an interface defi nition. UI Automation provides:

An object model and functions that make it easy • for client applications to receive events, retrieve property values, and manipulate UI elements.A core infrastructure for fi nding and fetching • across process boundaries.A set of interfaces for providers to express tree • structure and some general properties.A set of interfaces for providers to express other • properties and functionality specifi c to the con-trol type.

To improve on Microsoft Active Accessibility, UI Au-tomation aims to address the following goals:

Enable effi cient out-of-process clients, while • continuing to allow in-process access.Expose more information about the UI in a way • that allows clients to be out-of-process.Coexist with and leverage Microsoft Active Ac-• cessibility without inheriting its limitations.Provide an alternative to IAccessible that is • simple to implement.

The implementation of the UI Automation specifi ca-tion in Windows features COM-based interfaces and managed interfaces.

UI Automation Elements

UI Automation exposes every piece of the UI to cli-ent applications as an Automation Element. Provid-ers supply property values for each element; invoking a method on an element invokes the corresponding method on the provider. Elements are exposed as a tree structure, with the desktop as the root element.

Automation elements expose common properties of the UI elements they represent. One of these proper-ties is the control type, which describes its basic ap-pearance and functionality (for example, a “button” or a “check box”).

UI Automation Tree

The Automation Tree represents the entire UI: the root element is the current desktop and child elements are application windows. Each of these child elements

UI behaviors or merge two or more roles togeth-er. There is no text object model, for example, to help assistive technologies deal with complex Web content.

Another limitation involves navigating the object model. Microsoft Active Accessibility represents the UI as a hierarchy of accessible objects. Clients navi-gate from one accessible object to another using

interfaces and methods available from the accessible object. Serv-ers can expose the children of an accessible object with prop-erties of the accessible object or with the IEnumVARIANT COM interface. Clients, how-ever, must be able to deal with both approaches for any server. This ambiguity means extra work for client implementers, and the complexity can contribute un-foreseen problems of server im-plementations.

Just as important is the inability to extend Micro-soft Active Accessibility properties or functions without breaking or changing the IAccessible COM interface specifi cation. The result is that you cannot expose new control behavior or properties through the object model. The object model tends to be both static and stagnant.

UI Automation Specifi cation

The UI Automation specifi cation provides more fl exible program-matic access to UI elements on the desktop, enabling assistive technology products such as screen readers to provide infor-mation about the UI to end us-ers and to manipulate the UI by means other than standard input. The specifi cation can be support-ed across platforms other than Microsoft Windows.

The implementation of that specifi cation in Win-dows is also called UI Automation. UI Automation

The ecosystem of Windows automation technologies, now

called Windows Automation API, includes classic Microsoft Active Accessibility and

Windows implementations of the UI Automation specifi cation.

Figure 5: This UI Automation tree represents all elements on the Run dialog.

UI Automation offers much better performance for

out-of-process client practices (400% or faster for some scenarios than Microsoft Active Accessibility

running out-of-process), while adding richness and fl exibility to support the

latest user interface designs.


UI Automation Control Patterns

A control pattern describes the attributes and func-tionality of an automation element. For example, a simple “clickable” control like a button or hyperlink should support the Invoke control pattern to repre-sent the “click” action.

Each control pattern is a canoni-cal representation of possible UI features and functions, as shown in Table 3. There are 22 control patterns defi ned to date, some of which are new in Windows 7, and the Windows Automation API can support custom control patterns. Unlike with Microsoft Active Accessibility role or state properties, one automation ele-ment can support multiple UI Automation control patterns.

UI Automation Control Types

A control type is another automation element prop-erty that specifi es a well-known control that the ele-ment represents. Currently, UI Automation defi nes 38 control types, including; Button, Check Box, Combo Box, Data Grid, Document, Hyperlink, Im-age, ToolTip, Tree, and Window.

Before you can assign a control type to an element, the element needs to meet certain conditions, includ-

can contain elements representing menus, buttons, toolbars, and so on. These elements in turn can con-tain elements like list items, as shown in Figure 5.

UI Automation providers for a particular control support navigation among the child elements of that control. However, providers are not concerned with navigation between these control subtrees. This is managed by the UI Automation core, using information from the default window providers.

To help clients process UI information more ef-fectively, the framework supports alternative views of the automation tree: (1) Raw View, (2) Control View, and (3) Content View. As Table 1 shows, the type of fi ltering determines the views, and the client defi nes the scope of a view.

UI Automation Properties

The UI Automation specifi cation defi nes two kinds of properties: (1) automation element properties and (2) control pattern properties. Automation element properties apply to most controls, provid-ing fundamental information about the element such as its name. Examples are listed in Table 2. Control pattern properties apply to control pat-terns, described next.

Unlike with Microsoft Active Accessibility, every UI Automation property is identifi ed by a GUID and a programmatic name, which makes new properties easier to introduce.

Automation Tree Description

Raw View The full tree of automation element objects for which the desktop is the root.

Control View A subset of the raw view that closely maps to the UI structure as the user perceives it.

Content View A subset of the control view that contains content most relevant to the user, like the values in a drop-down combo box.

Table 1: Types of Automation Tree views.

Property Description

AutomationId A string containing the UI Automation identifi er (ID) for the automation element. The AutomationId property of an element is expected to be the same in any instance of the application, regardless of the local language.

BoundingRectangle The coordinates of the rectangle that completely encloses the element. The returned rectangle is expressed in physical screen coordinates.

Name A string for the text representation of the element. This string should always be consistent with the label text on screen. For example, the Name property must be “Browse…” for the button labeled “Browse…”.

ControlType A ControlType of the automation element, which defi nes characteristics of the UI element by well known UI control primitives such as button or check box.

FrameworkId A string for the name of the underlying UI framework. FrameworkId enables client applications to apply special cases to a particular UI framework. Examples of property values are "Win32", "WinForm", and "DirectUI".

Table 2: Examples of UI Automation Element properties.

UI Automation Community Promise Specifi cationsThe entire accessibility framework specifi cation is available under Microsoft Open Specifi cation Promise (http://www.microsoft.com/interop/osp/default.mspx). Windows Automation API 3.0 is a Windows implementation of the specifi cation.

Find more information at the MSDN Accessibility Developer Center http://msdn.microsoft.com/accessibility/

From a UI Automation client’s perspective,

there is no difference between UI Automation providers and

Microsoft Active Accessibility servers that implement IAccessibleEx correctly.

20 www.code-magazine.comWindows Automation API 3.0 Overview

Control DescriptionDock Used for controls that can be docked in a container, like toolbars or tool

palettes.ExpandCollapse Used for controls that can be expanded or collapsed, like the File menu.Grid Used for controls that support grid functionality such as sizing and moving

to a specifi ed cell, without header information. The “large icon view” in Windows® Explorer is an example of a control that follows the Grid control pattern.

GridItem Used for controls within grids. For example, each cell in Explorer’s “details view” could follow the GridItem pattern.

Invoke Used for controls that can be invoked, such as a button.ItemContainer Used for controls that hosts a number of children that may be virtualized.MultipleView Used for controls that can switch between multiple representations of the

same set of information, data, or children. For example, a list view control where data is available in thumbnail, tile, icon, list, or detail views.

RangeValue Used for controls that have a range of values. For example, a spinner control containing years might have a range of 1900 to 2010, while another spinner control presenting months would have a range of 1 to 12.

Scroll Used for controls that can scroll.Selection Used for selection container controls. For example, list boxes and combo

boxes.Table Used for controls those have a grid as well as header information. For

example, Microsoft Excel worksheets.Text Used for edit controls and documents that expose textual information.Toggle Used for controls where the state can be toggled. For example, check boxes

and checkable menu items.Transform Used for controls that can be resized, moved, and rotated. Typical uses for

the Transform control pattern are in designers, forms, graphical editors, and drawing applications.

VirtualizedItem Used for a virtualized object in a container that supports the ItemContainer pattern.

Window Used for controls that provide fundamental window-based functionality within a traditional graphical user interface.

Table 3: Examples of UI Automation Control Patterns.

IAccessibleEx Interface

The IAccessibleEx interface helps existing applica-tions or UI libraries extend their Microsoft Active Accessibility object model to support UI Automa-tion without rewriting everything from scratch. With IAccessibleEx, you can implement only the differ-ences between Microsoft Active Accessibility and UI Automation object models.

Because the MSAA-to-UIA Proxy translates the object models of IAccessibleEx-enabled Microsoft Active Accessibility servers as UI Automation object mod-els, UI Automation clients don’t have to do any extra work. The IAccessibleEx interface can enable classic in-process Microsoft Active Accessibility clients to in-teract directly with UI Automation providers, too.

Which to Support: Microsoft Active Accessibility, UI Automation, or IAccessibleEx?

While IAccessibleEx can be a cost effective way of supporting UI Automation, a couple of technical con-siderations should be made prior to the decision.

ing a particular automation tree structure, property values, control patterns, and events. However, you are not limited to these. You can extend a control with custom patterns and properties as well as with the pre-defi ned ones.

The total number of pre-defi ned control types is sig-nifi cantly lower than Microsoft Active Accessibil-ity accRole defi nitions, because you can combine UI Automation control patterns to express a larger set of features while Microsoft Active Accessibility roles cannot. You can also customize the description of control type by LocalizedControlType property while keeping the baseline type as defi ned.

UI Automation Events

UI Automation events notify applications of changes to and actions taken with automation elements. The four different types of UI Automation events, as list-ed in Table 4, do not necessarily mean that the visual state of the UI has changed. The UI Automation event model is independent of the WinEvent framework in Windows, although the Windows Automation API can make UI Automation events interoperable with the Microsoft Active Accessibility framework.

Supported PlatformUI Automation is supported on the following platforms to date:

Windows XP•

Windows Vista•

Windows Server 2003•

Windows Server 2008•

Windows 7•


Event DescriptionProperty change Raised when a property on an UI Automation element or control pattern

changes. For example, if a client needs to monitor an application's check box control, it can register to listen for a property change event on the ToggleState property. When the check box control is checked or unchecked, the provider raises the event and the client can act as necessary.

Element action Raised when a change in the UI results from end user or programmatic activity. For example, clients can listen for the Invoked event when a user clicks a button when the InvokePattern is invoked programmatically.

Structure change Raised when the structure of the UI Automation tree changes. The structure changes when new UI items become visible, hidden, or removed on the desktop.

General event Raised when actions of global interest to the client occur, such as when the focus shifts from one element to another or when a window closes.

Table 4: Types of UI Automation Events.

Clients’ Golden Question: Microsoft Active Accessibility, UI Automation, or Something Else?

Microsoft Active Accessibility is a “chatty” architecture and is slow for clients that run out of process. To mitigate this, many acces-sibility tool programs choose to hook into and run in the target application process. While assis-tive technologies widely employ this in-process code practice, the risk and complexity are extremely high. Thus, you need an out-of-process solution with better per-formance and reliability.

UI Automation offers much better performance for out-of-process client practices (400% or faster for some scenarios than Microsoft Active Accessibility running out-of-process), while adding richness and fl exibility to support the latest user interface designs.

Conclusions and Resources

Windows Automation API 3.0 in Windows 7 fea-tures the best of Microsoft Active Accessibility and the UI Automation specifi cation, while providing a cleaner migration path from one to the other. The object model is easier to use and more fl exible, the automation elements refl ect the evolution of modern user interfaces, and developers can defi ne custom UI Automation control patterns, properties, and events.

UI Automation Specifi cations are offered as cross-platform accessibility API under the Microsoft Open Specifi cation Promise. For more information about the specifi cation and about how Microsoft imple-ments it, visit the MSDN Accessibility Developer Center (http://msdn.microsoft.com/accessibility/).

If you are developing a new application or control, I recommend UI Automation as it provides the best fl exibility. Microsoft Active Accessibility may look simpler in the short term, but there are a lot of seri-ous limitations to overcome, like its Windows 95-in-spired object model and the inability to support new UI behaviors or merge roles.

These shortcomings surface quickly when you try to introduce new controls.

The UI Automation object model is based on canoni-cal functionalities of UI features. Control developers choose from abstracted features (control patterns) that best suit their controls’ behaviors. This is supple-mented with control types that offer role information to users.

When to Use IAccessibleEx

Consider the following requirements before you use the IAccessibleEx interface for your application.

Rule #1: The baseline Microsoft Active Accessi-bility Server’s accessible object hierarchy must be clean.

IAccessibleEx cannot fi x problems with existing ac-cessible object hierarchies.

Rule #2: Your IAccessibleEx implementation must be compliant with both Microsoft Active Accessibil-ity and UI Automation specifi cations.

Tools are available to validate compliance with both specifi cations.

If either of these baseline requirements is not met, you should consider implementing UI Automation natively. You can keep legacy Microsoft Active Ac-cessibility server implementations for backward com-patibility if it is necessary. From a UI Automation client’s perspective, there is no difference between UI Automation providers and Microsoft Active Ac-cessibility servers that implement IAccessibleEx cor-rectly.

The UI Automation object model is based on canonical

functionalities of UI features. Control developers choose from abstracted features

(control patterns) that best suit their controls’ behaviors.

Masahiko Kaneko


The Windows 7 Automation API is now available unmanaged

from the client to provider sides, boasting a 400%+ improvement in

performance.

Fast Facts

What’s New in Windows 7 Automation API

ON

LIN

E Q

UIC

K I

D 0

8110

52

Nazia Zaman

[email protected],Program Manager, User Interface Platform

Nazia is a PM on the User Interface Platform Team in the Windows Experience Division, and is very passionate about Accessibility.

She graduated from Stanford University with a Masters in Computer Science. She is interested in Human Computer Interaction, usability, and kernel programming. Check out her blog at: http://blogs.msdn.com/dreamingreality/

What’s New in Windows 7 Automation APIWindows® 7 offers end–to-end accessibility with better performance, seamless interoperability, and improved framework design. The Windows Automation API 3.0 combines the advantages of existing accessibility implementations in Microsoft® Active Accessibility® and a modern API design in UI Automation to provide easier migration from and integration with precursor technologies and industry standards.

Once you have the Automation object, you can discover the entire user interface (UI). The UI is

modeled as a tree of automation elements (IUIAutomationEle-ment objects), each element representing single pieces of UI: a button, a window, the desktop, and so on. The IUI-AutomationElement interface has methods relevant to all con-trols, such as checking proper-ties or setting focus. Here, you get the element with focus and its name:

IUIAutomationElement *pElement;pAutomation->GetFocusedElement(&pElement);BSTR name;pElement->get_CurrentName(&name);std::wcout << L"Focused Name:" << name << L"\n";

Furthermore, elements can expose functionality specifi c to a particular control through control pat-terns. Control patterns are collections of associated properties, events, and methods, and more than one can apply to an element. Here, I use the Invoke pat-tern to press a button:

IUIAutomationInvokePattern * pInvoke;pElement->GetCurrentPatternAs(UIA_InvokePatternId, __uuidof(IUIAutomationInvokePattern), (void **)&pInvoke);pInvoke->Invoke();

To navigate the tree of automation elements, you can use tree walkers. Here, I create an IUIAutoma-tionTreeWalker object to navigate to the fi rst child of an element in the Control View of the tree:

IUIAutomationTreeWalker * pWalk; pAutomation->get_ControlViewWalker(&pWalk);IUIAutomationElement * pFirst;pWalk->GetFirstChildElement(pElement, &pFirst);

To address performance issues when communi-cating across processes, clients can fetch multiple properties and patterns at a time. Here, I create a cache request (IUIAutomationCacheRequest ob-

The major improvements in this accessibility frame-work are:

End-to-end unmanaged • solution, including a native client API, faster and more robust UI Automation im-plementations, extensible provider implementations, and Win32 proxy for performance and avail-ability.Harmonization with in-• dustry standards such as W3C ARIA and Section 508 specifi cations.New properties and control patterns.•

Custom properties, events, and patterns.•

An Unmanaged Client API

For Windows 7, UI Automation has a new COM client API for use from unmanaged and managed clients alike. Unmanaged applications can now use UI Automation without changing languages or loading the CLR, while simultaneously benefi ting

from all the latest features. This new API is similar to the previ-ous managed API, but is friend-lier to C++ developers.

The heart of the new COM API is the IUIAutomation interface, which enables clients to get au-tomation elements, register event handlers, create objects, and ac-cess other helper methods. To begin using UI Automation, simply include the Automation header (UIAutomation.h), and

then CoCreate the Automation object:

IUIAutomation * pAutomation;CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (void **)&pAutomation);

Unmanaged applications can now use UI Automation without changing languages

or loading the CLR,while simultaneously benefi ting

from all the latest features.

23www.code-magazine.com What’s New in Windows 7 Automation API

Matthew KarrMatthew is a Software Development Engineer on the User Interface Platform Team, Accessibility, Application, and Automation group in the Windows Experience Division.

Matthew Karr has been a developer at Microsoft for 5 years, working on Automation and Accessibility. He’s helped ship UI Automation V1 and the Magnifi cation API with Vista, and is currently working on the COM UI Automation API for Windows 7.

In his free time he enjoys juggling, snowboarding, science fi ction, and video games.

Proxy factory registration consists of creating proxy factory entries and inserting them into a proxy fac-tory mapping. The mappings are processed in order as clients register proxies. In this example, I create the proxy for all HWNDs with a class name con-taining the word “MYCUSTOMBUTTON”:

// Instantiate the proxy factory.IUIAutomationProxyFactory * pCF = new CustomFactory();

// Create an entry with the factory.IUIAutomationProxyFactoryEntry * pCEnt;pAutomation->CreateProxyFactoryEntry(pCF, &pCEnt);pCEnt ->put_ClassName(L"MYCUSTOMBUTTON");pCEnt ->put_AllowSubstringMatch(TRUE);

// Get the mapping.IUIAutomationProxyFactoryMapping * pMapping;pAutomation->get_ProxyFactoryMapping(&pMapping);

// Insert the entry at the start of the mapping.pMapping->InsertEntry(0, pCEnt);

Optimizations in the Event Model

UI Automation clients, like screen readers, track events raised in the UI by UI Automation providers and notify users of the events. To improve effi ciency in Windows 7, providers can raise an event selec-tively, notifying only those clients that subscribe to that event.

Common UI Automation events that providers should support include the following:

Changes in a property or control pattern for a • UI Automation element.End user or programmatic activity that affects • the UI.Changes to the structure of the UI Automa-• tion tree.

Shift in focus from one element to another.•

To listen to an event, create a COM object imple-menting the event handler interface, and then call the corresponding method on the IUIAutomation object to subcribe the handler to the event. Here, I subscribe a handler to the Focus Changed event:

class FocusChangedHandler: public IUIAutomationFocusChangedEventHandler;

FocusChangedHandler * focusHandler = new FocusChangedHandler();

// Register the event with default cache request.pAutomation->AddFocusChangedEventHandler( NULL, focusHandler);

Interoperability with Microsoft Active Accessibility-based ControlsTo improve interoperability, UI Automation trans-lates between Microsoft Active Accessibility and

ject) from the Automation object, identify the prop-erty (Automation ID property) to prefetch, cache it for an element, and then get the cached ID:

IUIAutomationCacheRequest * pCR;pAutomation->CreateCacheRequest(&pCR));pCR->AddProperty(UIA_AutomationIdPropertyId);

IUIAutomationElement * pCachedElement;pElement->BuildUpdatedCache(&pCachedElement);

BSTR autoID;pCachedElement->get_CachedAutomationId(&autoID);std::wcout << L"Cached ID:" << autoID << L"\n";

With UI Automation property conditions, looking for specifi c values for a property is easy; you can combine conditions with AND, OR, and NOT op-erators to search for UI scenarios. Here, I search for a check box:

IUIAutomationCondition * pCheckBoxProp;VARIANT varCheckBox;varCheckBox.vt = VT_I4;varCheckBox.lVal = UIA_CheckBoxControlTypeId;pAutomation->CreatePropertyCondition( UIA_ControlTypePropertyId, varCheckBox, &pCheckBoxProp);

IUIAutomationElement * pFound;pElement->FindFirst(TreeScope_Descendants, pCheckBoxProp, &pFound);

In addition to these features, UI Automation pro-vides custom proxy registration, events, and the Text Pattern for manipulating documents.

Proxy Factory

Clients can customize how UI Automation sees HWND-based controls by registering customized providers with a proxy factory implementation. Custom proxies affect only the client’s own view. The provider answers CreateProvider calls with the HWND to be proxied, as well as the idOb-ject and idChild associated with the WinEvent, if needed.

Here, I verify the expected values of OBJID_CLIENT and CHILDID_SELF and create a custom proxy:

IFACEMETHODIMP CustomFactory::CreateProvider( UIA_HWND hwnd, LONG idObject, LONG idChild, IRawElementProviderSimple **ppRetVal){ *ppRetVal = NULL; if(idObject == OBJID_CLIENT && idChild == CHILDID_SELF) { // Create the custom proxy. *ppRetVal = new CustomProxy((HWND)hwnd); } return S_OK;}

24 www.code-magazine.comWhat’s New in Windows 7 Automation API

Some frameworks such as HTML, Windows Pre-sentation Foundation (WPF), and Silverlight™ have a metadata system that can associate accessibility-related properties with an element. Consequently, developers can easily fi x common bugs such as an incorrect accessibility name. However, Win32 does not have a similar feature, and the very basic property system for HWNDs does not apply to sub-items.

However, with Direct Annotation, developers can mark up properties on Win32 controls with accessi-bility property/value information, enabling develop-ers to fi x bugs without needing a full UI Automation or Microsoft Active Accessibility implementation. Here, I set the AutomationId on a simple control with the hwnd:

IAccPropServices *pAccPropServices;CoCreateInstance(CLSID_AccPropServices, NULL, CLSCTX_SERVER, IID_IAccPropServices, (void**)&pAccPropServices);pAccPropServices->SetHwndPropStr(hwnd, OBJID_CLIENT, CHILDID_SELF, AutomationId_Property_GUID, L"Foo ID");

Custom Control Patterns, Properties, and EventsWith Windows 7, you can extend the platform with custom control patterns, properties, and events. Be-cause of this support, developers of UI Automation clients and providers can introduce new accessibil-ity specifi cations independent of future operating system releases.

Developers register and use custom patterns, properties, and events on both the client side and the provider side. If a provider registers for a prop-erty that the client hasn’t, the client can’t retrieve it, and if a client registers for a property that the provider hasn’t, the provider can’t respond to a request for it. Neither of these cases causes errors; one side merely remains unaware of the other’s capabilities.

Once the new property, pattern, or event is regis-tered, using it is just like using a built-in pattern, property, or event.

Custom Properties

Registration of properties is identical for both cli-ent and provider. In the following sample, I have a native UI Automation control and I want to add a custom string property, LongName:

// The PropertyId for the LongName property.PROPERTYID longNamePropertyId;

// This is the predefi ned property GUID,// the name of the property, and the type.UIAutomationPropertyInfo longNamePropertyInfo = { GUID_LongNameProp,

UI Automation implementations. UI Automation clients can use the new UI Automation services to interact with earlier Microsoft Active Accessibil-ity implementations, and Microsoft Active Acces-sibility clients can interact with UI Automation providers.

In Windows 7, Microsoft Active Accessibility imple-mentations can add UI Automation properties and

control patterns by supporting the IRawElementProviderSimple interface to expose patterns and properties and the IAccessibleEx interface to handle ChildIds. Figure 1 shows the relationship between the IAccessible, IAcces-sibleEx, IRawElementProvider-Simple, and IRangeValuePro-vider interfaces.

With the IAccessibleEx interface, developers can extend Microsoft Active Accessibility implementa-tions by adding required UI Auto-mation object model information. The new MSAA-to-UIA proxy featured in the Windows Auto-

mation API provides a variation of a UI Automation “provider” to Microsoft Active Accessibility imple-mentations. UI Automation clients can interact with all variations of UI Automation implementations: native UI Automation, IAccessible (Microsoft Active Accessibility), and IAccessible + IAccessibleEx.

New Win32 Support via OLEACC Plus IAccessibleEx

In Windows 7, OLEACC proxies expose information about common controls that Microsoft Active Ac-cessibility cannot express. The MSAA-to-UIA proxy recognizes IAccessibleEx and forwards this addi-tional information to UI Automation. For example, the OLEACC slider proxy adds the IRangeValue pattern, exposing minimum and maximum values that Microsoft Active Accessibility cannot expose. Extending the OLEACC proxies with IAccessibleEx has the dual benefi t of leveraging existing code and keeping the OLEACC proxies up to date.

Figure 1: COM diagram shows the role of IAccessibleEx in extending legacy implementations.

Clients using the UI Automation client API can use its services to

search existing IAccessible implementations.

Code that writes to the UI automation provider interfaces will still be visible to existing

IAccessible clients.


UI Automation as an Automated Test ToolUI Automation is specifi cally designed to support automated testing, as well as accessibility. UI Automation separates two models of UI interaction: one exposes information about the UI and the other collects information needed by test tools.

UI Automation providers are applications, such as Microsoft Word and other third-party applications, or controls based on the Microsoft Windows operating system. UI Automation clients include automated test scripts and assistive technology applications. Both a provider and client are required for UI Automation to be useful as an automated test tool.

Custom Control Patterns

Creating a custom control pattern requires the fol-lowing:

Arrays of events, proper-• ties, and methods associated with the pattern.IIDs of the pattern’s corre-• sponding provider interface and client interface.Code to create a client inter-• face object.

Code to perform marshal-• ling for the pattern’s proper-ties and methods.

On the client side, the code that registers a pattern must supply a factory for creating instances of a Cli-ent Wrapper that forwards property requests and method calls to an IUIAutomationPatternInstance provided by UI Automation. The UI Automation framework then takes care of remoting and marshal-ling the call.

On the provider side, the code that registers a pattern must supply a “pattern handler” object that performs the reverse function of the Client Wrapper. The UI Automation framework forwards the property and method requests to the pattern handler object, which in turn calls the appropriate method on the target object’s provider interface.

The UI Automation framework takes care of all communication between the client and provider, both of which register corresponding control pat-tern interfaces. For more details please refer to the Windows 7 SDK.

New Properties and Control Patterns

Some other useful properties for automation ele-ments have been added to UI Automation for Win-dows 7:

ControllerFor is an array of elements manipulated by the automation element. Without this property it is hard to determine the impact of an element’s op-eration.

DescribedBy is an array of ele-ments that provide more infor-mation about the automation element. Instead of using the object model to discover infor-mation about the element, cli-ents can quickly access that in-formation in the DescribedBy property.

FlowsTo is an array of elements that suggest the reading order af-ter the current automation element. FlowsTo is used when automation elements are not exposed or struc-tured in the reading order users perceive.

L"LongName", UIAutomationType_String };

// This yields the property ID for new property.pAutomationRegistrar->RegisterProperty( &longNamePropertyInfo, &longNamePropertyId);

Retrieving the property is similar to retrieving a normal property; you use the property ID that the RegisterProperty method initialized:

VARIANT longNameValue;pElement->GetCurrentPropertyValue( longNamePropertyId, &longNameValue);std::wcout << longNameValue.bstrVal << L"\n";

Again, on the provider side, you call the same regis-tration method except you add an entry in the con-trol’s implementation of IRawElementProviderSimple::GetPropertyValue.

In this code sample, the UI Automation implemen-tation keeps a reference to the control in _pOur-Control, and the control supports a GetLongName method:

GetPropertyValue(PROPERTYID propertyId, VARIANT * pRet){ pRet->vt = VT_EMPTY;

if(propertyId == longNamePropertyId) { // Get the long name from the control. pRet->bstrVal = _pControl.GetLongName(); pRet->vt = VT_BSTR; }

// Deal with other properties... return S_OK}

Custom Events

Events follow a nearly identical model to properties: you register the event ID, and then you can use the custom EventID exactly as you use a normal Even-tID. However, custom events cannot have argu-ments; they are merely notifi cations that the event occurred.

Here, I register the event. Both listening and raising would then be identical to other events:

// The EventId for the ThingHappened eventEVENTID thingHappenedEventId;

// Event Information// This is the event GUID, the name of the event.UIAutomationEventInfo thingEventInfo = { GUID_ThingHappenedEvent, L"ThingHappened" };

// This gives you the Event ID for the new event.pAutomationRegistrar->RegisterEvent( &thingEventInfo, &thingHappenedEventId);

However, with Direct Annotation, developers can mark up

properties on Win32 controls with accessibility

property/value information...

To improve interoperability, UI Automation translates

between Microsoft Active Accessibility and

UI Automation implementations.

26 www.code-magazine.comWhat’s New in Windows 7 Automation API

UIA_ItemContainerPatternId, __uuidof(IUIAutomationItemContainerPattern), (void**)&pContainer));

// Search the container for the property.VARIANT varNameStr;varNameStr.vt = VT_BSTR;varNameStr.bstrVal = SysAllocString(name);

IUIAutomationElement * pFound;pContainer->FindItemByProperty(NULL, UIA_NamePropertyId, varNameStr, &pFound);

// Realize the virtual element.IUIAutomationVirtualizedItemPattern * pVirt;pFoundElement->GetCurrentPatternAs( UIA_VirtualizedItemPatternId, __uuidof(IUIAutomationVirtualizedItemPattern), (void**)&pVirt));

pVirtualizedItem->Realize();

New UI Automation Properties for Accessible Rich Internet Applications (ARIA)

Every day, Web sites are increasing their utility with dynamic content and advanced UI controls by us-ing technologies like Asynchronous JavaScript and XML (AJAX), HTML, and JavaScript. However, as-sistive technologies are frequently unable to inter-act with these complex controls or expose dynamic content to users. Accessible Rich Internet Applica-tions (ARIA) is a W3C technical specifi cation for developing Web content and applications so that they are accessible to people with disabilities.

IsDataValidForForm identifi es whether data is valid in a form.

ProviderDescription identifi es source information for the automation element’s UI Automation pro-vider, including proxy information.

Control Patterns for Virtualized Child Objects

When a control has too many children to load at once, a common solution is to treat the excess chil-dren as virtualized controls. This creates problems

for the UI Automation tree map because there are only a handful of real controls and the virtual-ized controls simply don’t exist in the UI Automation tree.

To manage this, UI Automation offers two control patterns. The ItemContainer pattern lets a user search a container of virtualized controls for specifi c properties. This gives the client a reference to a virtualized control, but the

user can’t do anything with it. The VirtualizedItem pattern enables the client to force the item to exist, either by realizing it internally, or by having it scroll on screen.

In this client-side code example, I search for a spe-cifi cally named item in a virtualized list:

// Get the ItemContainer pattern.IUIAutomationItemContainerPattern * pContainer;pElement->GetCurrentPatternAs(

W3C ARIA Role MSAA Role UIA Control Type UIA AriaRole Property

button ROLE_SYSTEM_PUSHBUTTON button button

checkbox ROLE_SYSTEM_CHECKBUTTON Checkbox checkbox

combobox ROLE_SYSTEM_COMBOBOX Combobox combobox

grid ROLE_SYSTEM_TABLE DataGrid grid

gridcell ROLE_SYSTEM_CELL DataItem gridcell

group ROLE_SYSTEM_GROUPING Grouping group

img ROLE_SYSTEM_GRAPHIC Image img

link ROLE_SYSTEM_LINK HyperLink link

list ROLE_SYSTEM_LIST List list

menu ROLE_SYSTEM_MENUPOPUP Menu menu

presentation ROLE_SYSTEM_PANE Pane presentation

progressbar ROLE_SYSTEM_PROGRESSBAR ProgressBar progressbar

radio ROLE_SYSTEM_RADIOBUTTON RadioButton radio

slider ROLE_SYSTEM_SLIDER Slider slider

tooltip ROLE_SYSTEM_TOOLTIP Tooltip tooltip

tree ROLE_SYSTEM_OUTLINE Tree tree

treegrid ROLE_SYSTEM_TABLE DataGrid treegrid

Table 1: W3C ARIA Roles can be mapped to Microsoft Active Accessibility roles and UI Automation control types and AriaRole properties.

Windows 7 offers further extensibility through

the registration of custom control patterns, properties, and events.


to various properties and functions of Microsoft Active Accessibility and UI Automation.

UI Automation also offers a simple text object model with the TextPattern pattern, which supports embedding objects in a document object. This en-ables user agents and client applications to treat Web content either as an HTML document or as a traditional desktop UI depending on the end-user scenarios.

With these features, UI Automation enables both the support of and extension of the W3C ARIA specifi cation without a dependency on a specifi c application or browser.

Conclusion

With application user interfaces growing more and more complex, getting accessibility right is a challenge for developers. Programmatic access to the UI is criti-cal in the development of assistive technologies like screen readers and magnifi ers. To address this, the Windows 7 Automation API aims to provide a com-plete end-to-end, fl exible, extensible, and consistent framework with improved design and performance.

For further details, please refer to the Windows 7 SDK or the MSDN Accessibility Developer Center (http://msdn.microsoft.com/accessibility/).

To support the ARIA specifi cation, the UI Automa-tion specifi cation enables developers to associate UI Automation AriaRole and AriaProperties at-tributes with W3C ARIA Roles, States, or Proper-ties. This helps user applications such as Internet Explorer support the ARIA object model in the context of UI Automation while keeping a baseline accessibility object model.

Some parts of the ARIA specifi cation can be mapped to the desktop-oriented Microsoft Active Accessibility object model; however, much of the specifi cation can only be applied to rich Internet applications. Table 1 lists some examples of map-pings from W3C ARIA Roles to Microsoft Active Accessibility Roles and UI Automation Control Types.

For example, the ARIA Role checkbox is sup-ported in Microsoft Active Accessibility by the role ROLE_SYSTEM_CHECKBUTTON and in UI Automation by the combination of con-trol type Checkbox and AriaRole checkbox. The ARIA state checked is supported in Microsoft Ac-tive Accessibility by the state STATE_SYSTEM_CHECKED and in UI Automation by the control pattern Toggle Pattern and the AriaProperties property checked.

ARIA States and Properties are supported by the UI Automation AriaProperties property with the following exceptions: ARIA properties that take object references (like the describedby property), and ARIA properties already supported by the ac-cessibility object model. Table 2 lists examples of mappings from W3C ARIA States and Properties

W3C ARIA States and Properties

Microsoft Active Accessibility UI Automation Control Patterns and Properties

UI Automation Aria-Properties Property

checked STATE_SYSTEM_CHECKED Toggle Pattern, checked checked

controls n/a ControllerFor n/a

describedby n/a DescribedBy n/a

disabled STATE_SYSTEM_UNAVAILABLE IsEnabled False disabled

fl owto n/a FlowsTo n/a

invalid n/a IsDataInvalidForForm invalid

labelledby n/a LabeledBy n/a

live n/a n/a live

multiselectable STATE_SYSTEM_EXTSELECTABLE CanSelectMultiple multiselectable

readonly STATE_SYSTEM_READONLY IsReadOnly readonly

required STATE_REQUIRED IsRequiredForForm required

secret STATE_SYSTEM_PROTECTED IsPassword secret

valuemax n/a Maximum Property in RangeValue Pattern

valuemax

valuemin n/a Minimum Property in RangeValue Pattern

valuemin

valuenow IAccessible::get_accValue Value Property in RangeValue Pattern

valuenow

Table 2: W3C ARIA States and Properties can be mapped to Microsoft Active Accessibilty properties and UI Automation control patterns and AriaProperties properties.

Nazia ZamanMatthew Karr


ON

LIN

E Q

UIC

K I

D 0

8110

62

Creating Accessibility-aware Silverlight 2 Content

Accessibility is an ever-increasing requirement as you develop applications.

Silverlight 2 provides the tools you need to make your application accessible but you need to understand

how best to do it.

Fast Facts

Creating Accessibility-aware Silverlight 2 ContentIf you haven't heard, accessibility is one of the most important aspects of a Web site experience. By using the accessibility features in Silverlight™ 2, you can provide the best experience for all users. Building a rich Internet experience can be a daunting task when you have to balance a cool visual design with usability.

Mark [email protected](425)-703-1688

Mark Rideout is a Program Manager on the Silverlight Team at Microsoft. Mark is responsible for features around text and text input in addition to input and accessibility features. Mark has worked on multiple Microsoft UI frameworks such as Windows Forms and has been working with the .NET Framework since 1.0. Mark started at Microsoft in 1997, and prior to that, Mark worked as a consultant where he designed and developed business applications.

Mark enjoys everything Lego, pinball, and Disneyland. He enjoys teaching his 2-year-old daughter about everything he likes.

elements into two types: simple visuals that usually derive directly from the FrameworkElement class, or functional controls that derive from the Control class. A TextBlock, Image, or MediaElement are ex-amples of simple visuals while a Button, TextBox, or ListBox are examples of functional controls.

Along with the visual tree of elements, your Silver-light 2 application has a corresponding accessibil-ity tree. UI Automation presents three accessibility views of the visual tree: a control view, a content view, and a raw view. Generally, all elements that

derive from Control show up in the control view while ele-ments that derive directly from FrameworkElement show up in the content view. TextBlock can appear in either the con-tent view when used inside a control or data template or the control otherwise. If you cre-ate custom Silverlight controls, you can identify whether it’s a control, content, or both (or none).

There are different ways to think of elements be-ing visible or invisible. Silverlight 2 reports elements whose Visibility property is equal to Visibility.Col-lapsed as hidden, but reports elements that are in the tree and "hidden" from view as being visible el-ements. This distinction is important because you might style your app to have different UI "modes" by just z- ordering the elements. If you have rea-sons for not setting the Visibility property, then you should at least disable controls that are "hidden" from view to ensure that screen readers don't at-tempt to interact with them.

XAML Tag Your Elements

As you develop your Silverlight application, you sometimes need to provide additional accessibility information to ensure your content is readable. For example, screen readers won't know anything about your Image element except that it is an image. As with HTML, you need to provide "alternative" text-based information. With Silverlight, you provide al-

Usability goes beyond the initial determination that a user interface (UI) "makes sense" as the UI has to be accessible to everyone. Silverlight

2 enables building a cool and accessible experience, whether you are building an entire Silverlight appli-cation or just a few Silverlight controls. This article provides you with guidelines for taking advantage of these features and building an accessible Silver-light application.

If you attempted to add accessibility to your Sil-verlight 1 content, you probably found that there wasn't much support except for something analogous to "alt" text for screen readers. Silver-light 2 on the other hand pro-vides necessary features that you can build on.

In general, when you build your Web application with accessibil-ity in mind, there are a general set of areas to think about:

Screen reader support.• Keyboard navigation.•

High contrast presentation.•

Ensuring Your Silverlight 2 Content Supports Screen ReadersSilverlight 2 provides information to screen readers about your application via the new UI Automation framework. At the application level, you work with UI Automation using the AutomationProperties class and its corresponding properties. When creat-ing custom controls or extending controls, you work with AutomationPeer and the UI Automation pro-vider interfaces. This article looks at some impor-tant UI Automation concepts you should be aware of as you build your Silverlight 2 application.

Silverlight Accessibility Tree

Your Silverlight 2 application is comprised of many visual elements from Borders to Control-based ele-ments. For the most part, you can split the visual

29www.code-magazine.com Creating Accessibility-aware Silverlight 2 Content

tems, Microsoft® Active Accessibility®-based screen readers can read Silverlight 2 content via the UI Automation UIA-to-MSAA Bridge. Because the ac-cessibility technologies are somewhat different, not all data transfers into Microsoft Active Accessibility properties and some concepts are simplifi ed.

For more information on the UIA-to-MSAA bridge, see the Windows Automation API 3.0 Overview article by Masahiko Kaneko in this issue. In addition, you can search MSDN for Active Accessibility Bridge to UI Auto-mation for specifi c details on the limitations.

Ensure Keyboard Interactivity

Creating an accessible Silverlight application means creating a keyboard-accessible application. Try us-ing your application with only a keyboard to iden-tify where functionality is unavailable to keyboard-only users. Sometimes you'll fi nd that you require mouse actions to use certain features. Other times you'll fi nd that you cannot tab to an element.

By thinking about keyboard accessibility during ap-plication design and development, you will avoid these types of issues. For more information on key-board best practices, see Guidelines for Keyboard User Interface Design in MSDN (http://msdn.mi-crosoft.com/en-us/library/ms971323.aspx).

IsTabStop and Tab Order

With the new keyboard and focus model in Silverlight 2, you can create a keyboard-friendly experience. You specify what controls can receive focus via the IsTab-Stop property. Most Silverlight controls, such as But-ton, already set the IsTabStop property to true. After you design the UI, take the time to set the TabIndex property to control the tab order. One thing to note is that only Control-based elements can receive focus.

Image and TextBlock (Non-Control-based Elements)

If you don't think about keyboard accessibility, you might add interactivity to an Image, Path, or Media-Element with mouse input, but visual elements that don’t derive from the Control class cannot receive focus. When you enable mouse actions on these types of elements, you may also want to enable the functionality with a keyboard command or with a control in the UI.

Alternatively, if you want to enable focus on an Im-age, you can create a custom UserControl element that contains the Image and add your keyboard logic there. Do note that you will also need to provide a fo-cus visual indication, as described later in this article.

ternative accessibility information by using the Auto-mationProperties static class.

There are cases when you won't have to add addi-tional markup via AutomationProperties. For ex-ample, controls that have text in them (like Buttons) already provide default information. In other cases, you do need to provide additional data.

Using AutomationProperties is very easy both in your XAML markup and in code. Using the Image element as an example, you set the AutomationProp-erties.Name attribute with XAML markup like this:

<Image x:Name="image" Source="/dogandcats.png"AutomationProperties.Name="Image of a dog andcat on a couch"/>

In code, you use the appropriate attached property set/get method:

AutomationProperties.SetLabeledBy(this.nameInput, this.nameLabel);

For best practices on providing good alternative text, see the MSDN technical article "Creating Text Equivalents for Images" (http://msdn.microsoft.com/en-us/library/ms971334.aspx).

Sometimes you might have a TextBox, Image, or other control labeled with text. For example, a TextBox for the user's name might have text to the left specifying "Name:". You can use the TextBlock element to pro-vide that static text label, and the AutomationProper-ties.LabeledBy property to tell that the TextBlock is labeling the TextBox. Because Silverlight 2 does not support element Binding syntax (using the Element-Name syntax) to set the LabeledBy property in XAML, you have to resort to some code, like this XAML:

<StackPanel Orientation="Horizontal"> <TextBlock Text="Name:" x:Name="nameLabel"/> <TextBox Width="100" x:Name="nameInput"/></StackPanel>

Then you set the LabeledBy property like so:

AutomationProperties.SetLabeledBy(this.nameInput, this.nameLabel);

To properly support screen readers, you need to en-sure that all your non-text controls that show up in the accessibility tree provide at least the Automa-tionProperties.Name value. I recommend that you spend the time to provide other values such as Help-Text. There are some great accessibility verifi cation tools recently released by Microsoft that work great with UI Automation. Check it out by searching for Microsoft Accessibility Labs on MSDN.

UI Automation and Microsoft Active Accessibility Screen Readers—Bridging the Gap

As I've mentioned earlier, Silverlight 2 relies on the operating system's UI Automation support. This does not imply that only UI Automation-based screen readers are supported. On supported operating sys-

Along with the visual tree of elements,

your Silverlight 2 application has a corresponding accessibility tree.

AutomationPeer is Based upon the WPF DesignThe design for the AutomationPeer class and derived classes comes from the Windows Presentation Foundation set of classes. This provides you with an opportunity to write your AutomationPeer accessibility code once and use it both in WPF and Silverlight.

Sharing designs also helps you as you learn about adding accessibility since there are already materials out about WPF and accessibility.

30 www.code-magazine.comCreating Accessibility-aware Silverlight 2 Content

To help you understand some of the ways people might use your application, think about what your application would look like if you printed it on a grayscale printer. Think about what it would be like to use it with the keyboard only. Think about whether your application is usable with the sound turned off. Finally, think about whether you could easily explain how to use your app over the phone. All of these will help your design be more usable for more people in more situations.

Support Changing Font Sizes

Silverlight 2 doesn't automatically change size based upon the computer's DPI settings nor does the content automatically change size based upon the browser's zoom or text size settings. There are three ways that you can scale your content to en-sure your application can provide a large font ex-perience: based upon the Web browser's zoom set-ting, based upon a user-selectable UI, and based upon a user-selectable font size UI. The last option only focuses on scaling the text size vs. scaling the whole UI.

Modern Web browsers have support for whole page zooming. When users zoom, the Silverlight browser plug-in receives a resize event. Silverlight provides a public Resized event that, with a little code in your application’s Loaded event, you can use to dynami-cally resize your Silverlight content:

App.Current.Host.Content.Resized += new EventHandler(Content_Resized);

In the XAML, just put a ScaleTransform that you will use when the Silverlight plug-in resizes:

<UserControl.RenderTransform> <TransformGroup> <ScaleTransform x:Name="zoomTransform" ScaleX="1" ScaleY="1"/> </TransformGroup> </UserControl.RenderTransform>

Lastly, apply the scale as a factor of your applica-tion’s size:

void Content_Resized(object sender, EventArgs e){

System.Windows.Interop.Content content = App.Current.Host.Content;

double scalePlayerX = (content.ActualWidth / this.Width); double scalePlayerY = (content.ActualHeight / this.Height);

this.zoomTransform.ScaleX = scalePlayerX; this.zoomTransform.ScaleY = scalePlayerY;}

A second approach is based upon the same code that you use for scaling based upon the Web browser's zoom except you provide your own UI for scaling.

Custom Keyboard Actions

The trick in providing custom keyboard actions with a hotkey combination is to make sure your

combination doesn't confl ict with the browser keyboard ac-tions because the browser will always win.

A good keyboard combina-tion that I've used is combining a number key with Shift and Control keys. For example, to provide keyboard hotkeys for a particular view in my UI, I listen (hook up event handlers) for the Shift, Control, and number keys

associated with the view. I add a KeyDown event handler to the root visual of my application since keyboard events bubble when not handled:

if (e.Key == Key.D1 && ((Keyboard.Modifi ers & (Modifi erKeys.Control | Modifi erKeys.Shift)) == (Modifi erKeys.Control | Modifi erKeys.Shift))){ // Perform hotkey action here}

Regardless of what custom keyboard actions you add you need to let users know that they exist. You can do this by setting the AutomationProperties.Accel-eratorKey property. In addition, you should add a tooltip or UI that describes the hotkey combination.

With Great Power Comes Great ResponsibilityAs you develop a great UI with complex gradients and cool graphics, you need to ensure you have a good experience with high contrast settings. I won't go so far as to say you shouldn’t make your UI cool; just put in the extra effort to make an accessible ex-perience. Here are a few things to think about:

Design with accessibility in mind.• Support different text sizes or browser zoom.•

Honor high contrast settings.•

Design with Accessibility in Mind

Using XAML and styling your application is great, but you can create some ugly UI that doesn't pres-ent well to users who need higher contrast for read-ability.

One important thing you can do during your de-sign is to convey information with visual cues other than color. For example, if you highlight particular list items with a red font, you could also bold them, so the user has a non-color cue to distinguish high-lighted items. You can optionally add some sound as the user moves over list items to note that the list item is "active".

With Silverlight, you provide alternative

accessibility information by using the AutomationProperties

static class and attached properties.

Migration Headache?

For immediate relief, visit

www.VFPConversion.com today!

VFP Conversion is a migration services brand of EPS Software Corp., providing expert upgrading of VFP applications to the latest technologies available from Microsoft. To learn more about how VFP Conversion can assist your enterprise, call toll free: 1 (866) 529-3682. [email protected] www.VFPConversion.com


Custom Controls/UI

So far, I've addressed application-level accessibility, but what’s good for the application developer is good for the control developer. There are a few more con-cepts that you need to understand fi rst though.

Derive from Existing Controls

The Silverlight team has spent a lot of time making their controls accessible; you should try to derive from the closest Silverlight control. In addition, you need to extend the control’s AutomationPeer, which encapsulates accessibility functionality. If you can't fi nd an AutomationPeer that matches your needs, then you should at least derive from FrameworkElementAutomationPeer.

Implement AutomationPeer and Associated Provider Interface

The AutomationPeer class exposes UI Automation by rolling up most of the non-pattern related UI Au-tomation properties exposed by the Automation-ElementIdentifi ers class.

Silverlight primarily chooses to focus on Auto-mationPeer concepts where it can. The best way to describe this is to say that the AutomationPeer class provides the necessary default UI Automation property values, and you identify a control’s acces-sibility support via functions rather than a role.

UI Automation identifi es control functionality by a set of interfaces called control patterns. For exam-ple, to identify that you can "invoke" a control, you implement the IInvokeProvider interface in your control's AutomationPeer. To add a control pattern, you create a new AutomationPeer class and over-ride its GetPattern method to return the interface corresponding to the pattern requested. Now that you have a custom AutomationPeer class, you need to make your custom control return an instance of it. Override the OnCreateAutomationPeer meth-od on your control and return a new instance of your custom AutomationPeer. Listing 2 is an ex-ample of adding the ITog gle Provider to a custom AutomationPeer created for a light switch control. The important parts of the custom control are in Listing 1.

To simplify technology and focus on the purpose of Silverlight, Silverlight supports only the APIs for creating provider-side UI Automation information. Silverlight does not provide the client-side UI Au-tomation APIs.

Change is Good, But Notify Others

Because accessibility properties change often, screen readers rely on change notifi cation to be effi cient. Change notifi cation covers such things as knowing when focus changes, knowing when a control be-

The last approach is to provide an option to increase the font size. Silverlight 2 does not recognize the browser's text size settings, so you need to provide the user with a custom UI (as simple as a set of buttons for selecting text size). Adjusting the font size can be a great approach if you have a lot of text and the overall UI doesn't need scaling. Since adjusting the font size causes the text to take up more space, you need to use layout elements such as Grid and StackPanel instead of hard coding widths/heights. Silverlight elements inherit the FontSize property throughout, so setting it at the root of your application causes the value to propagate.

Bring Contrast to Your UI

Supporting high contrast is an important part to making your application accessible. Silverlight 2 provides the SystemParameters.HighContrast value to indicate when you should render in a high con-trast mode. A high contrast mode can be as simple as omitting non-important images, gradients, or pat-

terns behind text to more com-plex styling that makes your ap-plication design a bit simpler. In addition, a high contrast mode should remove or reduce transi-tion animations, including the use of fl ashing.

You can take advantage of Silver-light's styling system to provide additional styles for use in high contrast scenarios. Silverlight 2

does not ship with high contrast styles for built-in controls, so you will need to create this as you style your application.

Expression Blend provides support for creating cus-tom styles for your controls. I recommend creating two styles that you can switch between based upon the SystemParameters.HighContrast value. There is one catch: you can only apply a Silverlight con-trol style once, and it has to be before or during the control's constructor. The more holistic approach is defi ning custom control styles during application startup, but you can apply a custom style in your control's constructor as well:

public class MyButton : Button{ public MyButton() { if (SystemParameters.HighContrast) { this.Style = (Style)Application. Current.Resources["btnContrast"]; } } }}

For an example of a Silverlight control "theme" framework, see Nikhil Kothari's blog for details (http://www.nikhilk.net/Silverlight-Themes.aspx).

Creating an accessible Silverlight application means

creating a keyboard-accessible application.

Best Accessible Silverlight™ 2 Experience Comes from a UI Automation-based Screen ReaderIt should make sense that you will get the best screen reader experience with Silverlight when the screen reader uses UI Automation. The technology offers much more information than predecessors. UI Automation-based screen readers can only read the rich UI Automation concepts and additional information provided.

33www.code-magazine.com Creating Accessibility-aware Silverlight 2 Content

for identifying and notifying others about changes. See Listing 1 and Listing 2 for an example of rais-ing property changes.

Provide Focus Indication

Visually, your control can do most anything. When people talk about accessibility support, one im-portant visual is focus. In Silverlight 2, the control template and styling model is very rich, but what doesn't exist is a common defi nition for visually identifying focus.

If your control derives from a common Silverlight control, you should use the FocusVisualElement identifi cation defi ned for the control and just re-style it as necessary. Using Silverlight's Visual State Manager (VSM), you can provide a custom control template that defi nes a new focus element.

If your control is totally unique, you should create your custom focus visual and add code in the Got-Focus and LostFocus event handlers to show and hide the focus indicator. You should also create a control template and provide a named focus ele-ment, similar to what Silverlight does.

High Contrast

Lastly, instead of putting the onus on the applica-tion developers using your custom control, you

comes disabled, and knowing when control pattern properties change.

Silverlight provides two methods to raise a change notifi cation: You use the AutomationPeer.RaisePropertyChangedEvent method to signal property changes, and use the AutomationPeer.RaiseAutomationEvent method to signal other types of automation changes.

Luckily, Silverlight already tells screen readers about most changes for you. Silverlight provides notifi ca-tions to UI Automation clients for the Automation-Peer IsEnabled, IsOffscreen, Name, and ItemStatus values. In addition, Silverlight provides structure change notifi cations that tell you when items are added and removed from the visual tree. By using or deriving from a Silverlight control, you also take advantage of control-specifi c notifi cations, such as when a CheckBox is checked.

If you create your own control with its own Auto-mationPeer functionality, then you are responsible

… when you design, do it with accessibility in mind.

public class LightSwitch : Control{ // Not a complete Silverlight custom control but enough // to demonstrate providing accessibility support via // AutomationPeer and UI Automation private bool _stateOn; public LightSwitch(){}

public bool State { get { return this._stateOn; }

set { changeState(value); } }

protected override void OnMouseLeftButtonDown(MouseButtonEventArgs e) { e.Handled = true; base.OnMouseLeftButtonDown(e); this.Toggle(); }

public void Toggle() { changeState(!this._stateOn); }

protected override AutomationPeer OnCreateAutomationPeer() { return new LightSwitchAutomationPeer(this); }

public void changeState(bool newState) { // notify accessibility that the state changed LightSwitchAutomationPeer ap = FrameworkElementAutomationPeer.FromElement(this) as LightSwitchAutomationPeer;

if (ap != null) { ap.RaiseStateChangedEvent(this._stateOn, newState); }

// fl ip light switch state this._stateOn = !this._stateOn;

// update visuals this.updateVisuals(this._stateOn); }}

Listing 1: Example control raising UI Automation change notifi cations.


erating system does not support UI Automation, and the Mozilla plug-in technology does not sup-port interaction with the OS X accessibility frame-work.

Conclusion: Silverlight’s Accessible FutureWith Silverlight 2, you can control high contrast styles, keyboard navigation, and screen reader sup-port. You can even take advantage of UI Automa-tion wherever it’s implemented. Silverlight’s new features can help you build Web applications and controls that support accessibility, so all users can gain enjoyment and value from your work. And re-member, when you design, do it with accessibility in mind.

should add the logic necessary to customize the controls visual style based upon the high contrast option mentioned earlier in this article.

OS Limitations

For Silverlight 2, the operating system and, to some extent, the browser dictate the level of accessibility that Silverlight can provide.

Silverlight is a Web browser plug-in based upon ei-ther Mozilla plug-in or Microsoft ActiveX technol-ogy, depending upon the browser. The Web browser is responsible for providing operating system inter-action and notifi cations with the plug-ins contained on a Web page. In some cases, a plug-in can work directly with the operating system and bypass the browser.

Silverlight can provide accessibility information when used inside of browsers running on operat-ing systems that support the UI Automation speci-fi cation. Currently, Microsoft Windows XP and later support UI Automation; Novell has made a commitment to supporting the UI Automation interfaces on Linux. Currently, Apple’s OS X op-

public class LightSwitchAutomationPeer : FrameworkElementAutomationPeer, IToggleProvider{ public LightSwitchAutomationPeer(LightSwitch light) : base(light) { }

// Helper function for raising notifi cation internal void RaiseStateChangedEvent(bool oldValue, bool newValue) { if (oldValue != newValue && AutomationPeer.ListenerExists( AutomationEvents.PropertyChanged)) { base.RaisePropertyChangedEvent( TogglePatternIdentifi ers.ToggleStateProperty, oldValue == true ? ToggleState.On : ToggleState.Off, newValue == true ? ToggleState.On : ToggleState.Off); } }

// Let UI Automation know that the pattern is supported public override object GetPattern( PatternInterface patternInterface) { if (patternInterface == PatternInterface.Toggle) { return this; } return base.GetPattern(patternInterface); }

protected override AutomationControlType GetAutomationControlTypeCore() { // Button is the closest match return AutomationControlType.Button; }

protected override string GetClassNameCore() { return "LightSwitch"; }

#region IToggleProvider Members

public void Toggle() { if (!base.IsEnabled()) { // Don't toggle if the control is not enabled throw new ElementNotEnabledException(); } ((LightSwitch)this.Owner).Toggle(); }

public ToggleState ToggleState { get { ((LightSwitch)this.Owner).State == true ? ToggleState.On : ToggleState.Off; } }

#endregion}

Listing 2: Custom AutomationPeer with Togglefor example control.

Mark Rideout

36 www.code-magazine.comMaking Custom Controls Accessible

To meet the accessibility requirements, a small change in standard

controls can cause more code changes than

the change itself. Never underestimate the

implication of a small modification.

Fast Facts

LeAnne FossmeyerLeAnne is a Content Publishing Lead in the Windows Experience organization at Microsoft. Quick Tips: “Do I Really Need that

Control?”Before investing time in designing a new control or customizing a standard control, you should ask yourself: “Do I really need that control?”

If you’re lucky, you may fi nd a standard control that does the job. Many modern UI libraries support accessibility and cus-tomization.

If you customize a standard control, try not to overload it with features. Unnecessary complexity can result in ex-pensive accessibility support cost, not to mention reduced usability. Never underesti-mate the implication of a small modifi cation. If you have to in-

troduce major customizations, consider creating a control from scratch

If you have to introduce a new control, try to follow standard practices to support UI accessibility. For example, never assume that all users have access to pointing devices, and try to use system or desktop theme colors to avoid accessibility problems with custom colors. While the focus of this article is on

The Windows® operating system offers several technologies to help make controls accessible: Microsoft Active Accessibility, UI frameworks,

and UI Automation. As Table 1 shows, the most effective technology depends on how developers in-troduce their new controls.

For controls and frameworks written from scratch, UI Auto-mation is recommended today. While Microsoft Active Acces-sibility is handy for relatively simple controls, the technology doesn’t support the complexity of modern user interfaces (UI).

For controls based on common controls (such as COMCTL32.DLL or USER32.DLL), Dynam-ic Annotation enables modest customization. However, it may not be suffi cient for signifi cant customization or for sub-classed controls. For con-trols based on solid Microsoft Active Accessibility implementations, developers can use the IAcces-sibleEx Interface specifi cation to enhance accessi-bility with new UI Automation interfaces.

This article introduces the basics of using UI Au-tomation, Dynamic Annotation, and the IAcces-sibleEx interface for developing controls.

Making Custom Controls AccessibleWhile custom controls are introduced every day, not all of them are easily accessible. This article provides a quick summary of Microsoft® technologies that help make Win32-based custom controls programmatically accessible. Techniques range from implementing UI Automation, to creating or overriding properties with Dynamic Annotation, to using the new IAccessibleEx interface to close the gap between UI Automation and Microsoft Active Accessibility®.

ON

LIN

E Q

UIC

K I

D 0

8110

72

Type of Custom Control

Type of Customization Suggested Approach

New control Write new UI code from scratch Implement UI Automation or Microsoft Active Accessibility for relatively simple controls such as a button.

COMCTRL/USER controls with minor modifi cation

Make small changes that have no functional impact (background color)

Use Dynamic Annotation to modify (or add) Microsoft Active Accessibility/UI Automation properties.

Microsoft Active Accessibility-based controls

Add new functionality not compatible with Microsoft Active Accessibility design

Implement the IAccessibleEx interface and address only the gap from Microsoft Active Accessibility to UI Automation.

Table 1: Suggested solutions by type of custom controls.

37www.code-magazine.com Making Custom Controls Accessible

// IRawElementProviderSimpleSTDMETHODIMP GetPatternProvider( PATTERNID patternId, IUnknown** pRetVal);

STDMETHODIMP GetPropertyValue( PROPERTYID propertyId, VARIANT* pRetVal);

STDMETHODIMP get_HostRawElementProvider( IRawElementProviderSimple** pRetVal);

STDMETHODIMP get_ProviderOptions( ProviderOptions* pRetVal);

For the UI Automation Framework to recognize your UI Automation provider implementation, you must respond to the WM_GETOBJECT message. The UiaReturnRawElementProvider function helps produce the key to establish a connection to the UI Automation Core:

case WM_GETOBJECT: { if (_pProxy == NULL) { // Create an automation element object _pProxy = CCheckbox3UIAProxy::Create(this); } return UiaReturnRawElementProvider( _hwnd, wParam, lParam, _pProxy); } break;

Now you are ready to implement the support for UI Automation properties and pattern interfaces.

Step 2: Implementing the Properties and Control Patterns

UI Automation providers add support for prop-erties with the GetPropertyValue method. For control patterns, providers support the corre-sponding patternId for the GetPatternProvider method.

You can register the properties and control patterns with GUIDs (such as the Toggle_Pattern_GUID) that you map to the corresponding IDs with the Uia LookupId function. For existing control pat-terns and properties, you can fi nd corresponding IDs in UIAutomationClient.h (such as UIA_Toggle-PatternId).

programmatic access to the user interface, these fun-damental accessibility criteria are very important.

Many accessibility practices are available in the form of industry standards (such as Section 508 of the Federal Rehabilitation Act). Some useful re-sources are introduced at the Microsoft Accessibil-ity Developer Center (http://msdn.microsoft.com/en-us/accessibility/default.aspx).

Part I: Implementing UI Automation

This article talks about implementing UI Automa-tion for new controls in a few simplifi ed steps. De-signing a UI Automation object model for a new user interface is straightforward if you follow a few basic principles:

Defi ne user scenarios clearly. Actual user sce-1. narios are helpful. During the design process, you shouldn’t rely on a single operation device such as mice, custom color, or screen size.Defi ne the UI functionality as simply as pos-2. sible. Is it clickable (such as a button or hy-perlink)? Does it allow for selection (such as a list box)?Isolate fundamental UI elements by function 3. or feature. Structure a complex UI in a way consistent with user scenarios. A keyboard navigation model is a useful guide.

Match a UI feature from the previous step with 4. existing controls. The UI Automation Control Type specifi cation is a useful guideline. If you have trouble fi nding the correct control type, see the Control Pattern specifi cations and lo-cate a control type that has the control pat-terns you want.

Step 1: The Foundation

Note: This sample project is available for download at MSDN Online (http://msdn.microsoft.com/en-us/library/ms771315(VS.85).aspx). The Windows SDK also features two additional UI Automation provider samples.

The foundation of UI Automation provider objects is the IRawElementProviderSimple interface im-plemented by the applications or by the UI Frame-work that supports UI Automation. Objects that contain a more complex tree structure should also implement IRawElementProviderFragment, and the root for this structure implements IRawEle-mentProviderFragmentRoot. Objects whose per-formance is very important can use IRawElement-ProviderAdviseEvents to know which events they should raise. Finally, you should use the IRawEle-mentProviderHwndOverride interface when the provider needs to reorder the automation tree. Most providers do not implement this interface.

In this project, you implement only IRawElement-ProviderSimple, which contains the following methods:

Before investing time designing a new control or

customizing a standard control, you should ask yourself:

“Do I really need that control?”

MEET MEMBERS OF MICROSOFT'S

ACCESSIBILITY TEAM

Michael Bernstein(Lead Software Design Engineer, User Interface Platform Team)

My team builds great Accessibility and Speech user experiences in Windows, as well as the developer libraries and tools that enable those experiences. This is a very rewarding area for me: I can open a new world of computer experiences to those who have never been able to access them before

David Tseng(Software Design Engineer in Test, User Interface Platform Team)

I'm helping to ensure that the user has a great experience no matter how he or she interacts with a computer—whether it be through speech recognition, screen reading, or screen magnifi cation. This happens by making sure the developer is empowered with high quality mechanisms to build the best experience possible.


Vidhya Sriram(Software Development Engineer in Test, User Interface Platform Team)

My team not only works on making programmatic access to GUI applications easier but also enables application developers to expose richer information about their application to assistive technology products. I love working with this team for the commitment and passion shown towards making computers easier to use for physically challenged users and thereby bringing a positive change in their lives.

IUnknown **pRetVal){ // Clear out param *pRetVal = NULL;

if (patternId == UIA_TogglePatternId) { *pRetVal = static_cast<IRawElementProviderSimple*>(this); AddRef(); } return hr;}

Part II: Using Dynamic Annotation

Now, I will talk about using Dynamic Annotation for Win32® controls with relatively minor modifi ca-tions. The technique was originally introduced with Microsoft Active Accessibility version 2.0. Its pur-pose is to enable a custom control to expose acces-sibility information without having to fully imple-ment the IAccessible interface.

With Windows 7 Automation API, you can now use Dynamic Annotation to override either Micro-soft Active Accessibility or UI Automation proper-ties of the proxy objects that OLEACC creates for standard Windows controls (many of those sup-ported by COMCTL32.dll or USER32.dll). If your custom control is a slight modifi cation of an exist-ing control or requires support for only a few ac-cessibility properties, Dynamic Annotation is a good candidate for implementing your accessibility solution.

Dynamic Annotation provides three different mech-anisms for handling annotations: Direct Annota-tion, Value Map Annotation, and Server Annota-tion. In this article I discuss only Direct Annotation and Server Annotation.

In this example, you get the property value of a check box:

HRESULT CCheckbox3UIAProxy::GetPropertyValue( PROPERTYID propertyId, IUnknown **pRetVal){ // Clear out param pRetVal->vt = VT_EMPTY; // Specify ControlType.CheckBox for the // ControlType property if (propertyId == UIA_ControlTypePropertyId) { pRetVal->vt = VT_I4; pRetVal->lVal = UIA_CheckBoxControlTypeId; } return S_OK;}

To support controls with a binary status such as the check box (checked or cleared), the Toggle Pattern is best, and the implementation is straightforward as shown in Listing 1.

Here, you get the pattern provider for the toggle pattern:

HRESULT CCheckbox3UIAProxy::GetPatternProvider( PATTERNID patternId,

// IToggleProvider to support the Toggle control pattern.HRESULT CCheckbox3UIAProxy::Toggle(){ HRESULT hr = CheckAlive(); if (SUCCEEDED(hr)) { _pControl->MoveToNextState(); } return hr;}

HRESULT CCheckbox3UIAProxy::get_ToggleState(ToggleState* pRetVal){ HRESULT hr = CheckAlive(); if (SUCCEEDED(hr)) { switch(_pControl->CheckboxCheckedState()) { case CHECKED: {

*pRetVal = ToggleState_On; } break;

case MIXED: { *pRetVal = ToggleState_Indeterminate; } break;

case UNCHEcKED: { *pRetVal = ToggleState_Off; } break; } } return hr;}

Listing 1: The Toggle Pattern supports the binary status of a check box control.

Many accessibility practices are available in the form of

industry standards (such as Section 508 of

the Federal Rehabilitation Act).

39www.code-magazine.com Making Custom Controls Accessible

Other Considerations: WinEvents

Applications and the UI Frame-work use WinEvents to no-tify accessibility applications of changes in the user interface. When one of the following hap-pens, call NotifyWinEvent:

An object is created, de-• stroyed, shown, hidden, re-ordered, or invoked.A selection is changed.•

A Name, Value, State, De-• scription, Location, Parent, Help, Default-Action, or Accelerator accessibility property changes in an object.

These calls are needed only for objects for which you implement accessibility. The system provides accessibility for common objects and will call the appropriate events when needed. Not all annota-tions require an additional WinEvent.

Part III: Implementing IAccessibleExIf you already have a solid Mi-crosoft Active Accessibility im-plementation of a control, you can use the IAccessibleEx in-terface to add more accessibil-ity functionality. The advantage of this interface is that you can reuse existing accessible object implementations, properties, object tree structures, and WinEvents. As long as the baseline Micro-soft Active Accessibility implementation is clean, IAccessibleEx is an attractive solution to enhance the custom control for UI Automation.

To extend your Microsoft Active Accessibility im-plementations with the IAccessibleEx interface, do the following:

A sample Dynamic Annotation project is available for download at MSDN Online.

How It Works

Dynamic Annotation allows you to modify some of the Microsoft Active Accessibility properties, but not every Microsoft Active Accessibility property is relevant. The supported properties are:

Name (PROPID_ACC_NAME)• Description (PROPID_ACC_DESCRIPTION)• Role (PROPID_ACC_ROLE)• State (PROPID_ACC_STATE)• Help (PROPID_ACC_HELP)• KeyboardShortcut (PROPID_ACC_KEY-• BOARDSHORTCUT)

DefaultAction (PROPID_ACC_DEFAULTAC-• TION)

While annotating many of these Microsoft Active Accessibility properties is straightforward, be care-ful when you annotate the Microsoft Active Ac-cessibility Role property. Annotation of Role won’t affect the behavior of the control or proxy object. For example, do not annotate a button as a slider because the baseline Win32 button control may not offer all of the functionality of a slider con-trol.

To annotate a control, create an AccPropServices COM object, call the appropriate SetHwndProp methods to annotate the control, and then release the AccPropServices object. Do not forget to call CoInitializeEx to support all COM practices in your WinMain routine (Listing 2).

All annotations made in a Microsoft Active Acces-sibility property are refl ected in the UI Automa-tion translation as well as in the Microsoft Active Accessibility-to-UI Automation Proxy. If you want to override or add a UI Automation property to the control, you can specify a UI Automation Identi-fi er GUID instead of the Microsoft Active Acces-sibility PropID.

If you customize a standard control, try not to overload it with features. Unnecessary complexity can result in expensive

accessibility support cost, not to mention reduced usability.

HRESULT CFancyTextControl::SetAccessibleProperties(){ // COM has been assumed to be initialized... IAccPropServices* pAccPropServices = NULL;

HRESULT hr = CoCreateInstance(CLSID_AccPropServices, NULL, CLSCTX_SERVER, IID_IAccPropServices, (void**)&pAccPropServices);

if (SUCCEEDED(hr)) { // Annotating the Role of this object be STATICTEXT VARIANT var; var.vt = VT_I4; var.lVal = ROLE_SYSTEM_STATICTEXT;

hr = pAccPropServices->SetHwndProp(_hwnd, OBJID_CLIENT, CHILDID_SELF, PROPID_ACC_ROLE, var);

pAccPropServices->Release(); } return hr;}

Listing 2: Annotation supports extending UI Automation properties for existing controls.

… you can now use Dynamic Annotation to override

Microsoft Active Accessibility or UI Automation properties.


Implement IAccessibleEx. 1. Listing 3 shows the implementation of the GetObjectForChild and GetIAccessiblePair methods so UI Automation can map an IAccessible and ChildId pair to a corresponding IAccessibleEx instance.Expose IAccessibleEx with IServiceProvider::5. QueryService.Implement IRawElementProviderSimple to 6. provide property values and pattern providers for your control.Implement control patterns and properties 7. that describe your control.

Implement WinEvents for your control.8.

A sample IAccessibleEx project is available for download at MSDN Online. The sample project has baseline accessibility support for a custom List Box control and the UI Automation RangeValue control pattern.

HRESULT CListboxAccessibleObject::GetObjectForChild( long idChild, IAccessibleEx **pRetVal){ VARIANT vChild; vChild.vt = VT_I4; vChild.lVal = idChild; HRESULT hr = ValidateChildId(vChild); if (FAILED(hr)) { return E_INVALIDARG; } // List item accessible objects are stored as an array of // pointers; for the purpose of this example, I assumed // the list contents will not change. Accessible objects // are created only when needed. if (itemProviders[idChild - 1] == NULL) { // Create an object that supports UI Automation and // IAccessibleEx for the item. itemProviders[idChild - 1] = new CListItemAccessibleObject(idChild, g_pListboxControl); if (itemProviders[idChild - 1] == NULL) { return E_OUTOFMEMORY; } } IAccessibleEx* pAccEx = static_cast<IAccessibleEx*> (itemProviders[idChild - 1]);

if (pAccEx != NULL) { pAccEx->AddRef(); } *pRetVal = pAccEx; return S_OK; }

HRESULT CListItemAccessibleObject::GetIAccessiblePair( IAccessible **ppAcc, long *pidChild){ if (!ppAcc || !pidChild) { return E_INVALIDARG; } CListboxAccessibleObject* pParent = m_control->GetAccessibleObject();

HRESULT hr = QueryInterface( __uuidof(IAccessible), (void**)ppAcc); if (FAILED(hr)) { *pidChild = 0; return E_NOINTERFACE; } *pidChild = m_childID; return S_OK; }

Listing 3: These methods help UI Automation map to a corresponding IAccessibleEx instance.

Designing a UI Automation object model for a new user

interface is straightforward if you follow a few basic principles.

Conclusions and Resources

While there are many ways to make custom con-trols accessible, choosing the right solution can be tricky. It is easy to underestimate the cost of cus-tomizing controls. Be aware of the hidden costs of a complex UI design, and make sure you have a plan for making your wonderful new controls accessible to everyone.

The examples introduced in this article are avail-able for download from the MSDN article, “Mak-ing Custom Controls Accessible.” (http://msdn.mi-crosoft.com/en-us/accessibility/cc307845.aspx) A more thorough discussion of this topic is available on the MSDN Accessibility Development Center (http://msdn.microsoft.com/en-us/accessibility/).

LeAnne FossmeyerMichael Bernstein

David TsengVidhya Sriram

42 www.code-magazine.comMicrosoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines ComplianceClose your eyes, ignore your mouse, navigate with your keyboard, and rely on your ears alone. Now try to use an application you’ve built or tested. Can you? The few informal tests described in this article can expose a plethora of usability and accessibility shortcomings, oversights, and other issues in your application. But how do you test, assess, and rectify them?

Karl BridgeKarl Bridge is a Programming Writer, specializing in developer content, for the Windows Experience (WEX) organization at Microsoft.

As the name implies, WEX is focused on the Windows user experience for consumers, IT professionals, and developers. Karl has written extensively for MSDN and the Windows SDK with a focus on making the Windows developer community a happy and integral part of the Windows experience. Prior contributions include the developer documentation for managed UI Automation and the Windows Vista Sidebar platform.

Causing confusion, consternation, and occasional despair amongst Karl's family, friends, and co-workers are his mid-Atlantic sensibilities—a confl uence of English eccentricity tempered with Canadian reserve and further jumbled by numerous strolls through Europe, the Middle East, North Africa, and Asia. Fortunately, Karl has a spell-checker and an editor to buffer readers from the inevitable mélange of mid-Atlantic spelling and grammar.

According to a study that Microsoft commissioned and Forrester Research

conducted:

40% (51.6 million) of • computer users with mild difficulties or impairments are likely to benefit from

the use of accessible technology.

17% (22.6 million) of • computer users with severe difficulties or

impairments are likely to benefit from the use of accessible technology.

Fast Facts

by exposing control properties, methods, and events such that you can use a tool to manipulate and interact with the UI manually or through au-tomation.

By incorporating the relevant components of the Windows Automation API and follow-ing common accessible de-sign practices, developers and testers can make applications running on Windows signifi -cantly more useful for people with vision, hearing, or motion disabilities.

In the following sections I briefl y describe the frameworks that constitute the Windows Automation API.

Microsoft Active Accessibility

Microsoft Active Accessibility is a set of COM interfaces and APIs that provide the means to

expose and collect information about UI elements and controls. This allows assistive technologies, test tools, and automation frameworks program-matic access to and manipulation of the UI.

UI Automation

UI Automation is similar to Microsoft Active Ac-cessibility in that it provides a means for exposing and collecting information about UI elements and controls to support user interface accessibility and software test automation. However, UI Automation

To ensure the most consistent user experience across the Microsoft® Windows® product line, Microsoft has developed a number of tools to

assist developers and testers to verify the accessi-bility implementations in their applications.

In this article, I describe each of these tools and explain what they are designed to test and the scenarios in which each tool, alone or in combination, may be useful. I also describe how you can use these tools throughout the development cycle and how to incorporate them into automated and non-automated test frameworks.

An Overview of the Windows Automation Frameworks

The tools I discuss in this ar-ticle are designed to test and report on a set of standardized accessibility issues typical of an application user interface (UI). The tests rely on programmatic ac-cess of the UI implementing the relevant API com-ponents defi ned in the framework known as the Windows Automation API.

Despite the name, the Windows Automation API encompasses both accessibility and automa-tion frameworks, including Microsoft Active Ac-cessibility®, UI Automation, the IAccessibleEx interface, as well as other related and evolv-ing technologies. Each of these frameworks en-ables programmatic access to the UI at run time

ON

LIN

E Q

UIC

K I

D 0

8110

82

43www.code-magazine.com Microsoft Accessibility Testing Tools vs. the Ten-ton Gorilla of Accessibility Guidelines Compliance

execution based on a set of predefi ned test cases. Depending on the sophistication of the tool, I will supply generic recommendations for specifi c issues uncovered during testing.

In addition to compiling reports for basic unit tests, some of the tools discussed also offer support for regression and integration testing. This abil-ity to fi nd and track an accessibility bug through to resolution in a process you can automate makes these tools indispensible additions to a tester’s toolkit.

And in this Corner, We Have… The Tools

The following Microsoft tools enable developers to test accessibility implementations and functionality in applications using Microsoft Active Accessibility or UI Automation.

UI Automation Verify

UI Automation Verify (UIA Verify) is a test auto-mation framework that facilitates automated testing for specifi c Microsoft UI Automation implementa-tions. This framework provides the basis for the UI Automation Test Library command-line tool and Visual UI Automation Verify (Visual UIA Verify), a Windows-based graphical user interface (GUI) for the test framework. Most of the UIA Verify frame-work functionality is provided through a DLL (UI-ATestLibrary.dll) that enables the testing of specifi c UI Automation functionality and that logs the test results.

Note: The Visual UIA Verify GUI does not sup-port event monitoring or testing and validating text-based content using the UI Automation Text-Pattern and TextPatternRange control pattern in-terfaces. The UI Automation Test Library does not share these limitations.

To use the UI Automation Test Library, a driver (or client) application obtains an Automation Element object from a control that requires verifi cation. The driver, in turn, supplies the AutomationElement object to the UI Automation Test Library, which ex-ecutes the tests required to validate the UI Automa-tion implementation. Figure 1 illustrates the typi-cal workfl ow when using the UI Automation Verify framework to test an application.

The Visual UIA Verify framework is a Windows driver for the UI Automation Test Library. This tool

is a newer technology that provides a much richer object model than Microsoft Active Accessibility, and it is compatible with both Win32® and .NET.

Note: UI Automation exposes each object in the UI as an AutomationElement object to client applica-tions. An AutomationElement object exposes com-mon properties of the UI element it represents, such as the control type and relevant control patterns that provide properties specifi c to the control type.

The IAccessibleEX Interface

The IAccessibleEX interface is a new interface that provides the ability for developers to add par-tial support for UI Automation to controls with existing IAccessible COM interface implementa-tions. This is transparent to testers and the tools discussed in this article.

Evaluating the Accessibility of Your ApplicationThere are two primary approaches to evaluating the accessibility of an application: a manual audit based on a set of commonly accepted heuristics for an accessible UI (also known as static testing) and a partial audit to a fully automated test platform that evaluates the implementation of an accessibil-ity framework within your application’s code (dy-namic testing). Both methods can provide results based only on a set of guidelines, either from the heuristics of the audit or those built into the design of the accessibility framework.

Accessibility Auditing

Typically, auditing your application for accessi-bility issues is a manual process that involves at-tempting to access and use the full functionality of the application exclusively through keyboard input and other assistive technologies (such as screen readers). As an auditor, you assess each is-sue discovered during the audit and log a bug for each issue as necessary. You then assign a priority and severity to each bug and, where possible and appropriate, provide recommenda-tions for fi xing or working around the issues that the audit uncovers. These recommendations are based on a combination of experience and acces-sibility guidelines from industry or advisory groups.

Since there is an inherent risk of testing the func-tionality of the tool rather than the application, auditing should be conducted by seasoned testers familiar with accessibility requirements and the broad range of assistive technologies.

Accessibility Testing

For the purposes of this article, accessibility test-ing is associated with automated, repeatable code

Despite the name, the Windows Automation API

encompasses both accessibility and automation frameworks.

Where to Get the ToolsThe tools discussed in this article are available from various locations, which I’ve listed below.

UI Automation Verify

Available from CodePlex, the Microsoft open-source project-hosting Web site: http://www.codeplex.com/UIAutomationVerify

UI Spy

UI Spy typically ships with the Windows SDK. However, since it did not ship in the Windows SDK for Windows Server 2008 and .NET Framework 3.5, released in February of 2008, you can fi nd it in the earlier Windows SDK for Vista Update: http://www.microsoft.com/downloads/details.aspx?familyid=4377F86D-C913-4B5C-B87E-EF72E-5B4E065&displaylang=en


Figure 2: The fi ve functional areas of the Visual UI Automation Verify framework.

Figure1: Here you can see the UI Automation Verify framework workfl ow between driver, application, and UIATestLibrary.

is designed for manual testing. It provides an inter-face to UI Automation Test Library functionality and eliminates the code overhead of a command-line tool. Figure 2 illustrates the fi ve functional ar-eas of the Visual UIA Verify framework.

UI Spy

UI Spy, the precursor to the UIA Verify framework, is a Windows application designed for UI Automa-tion testing of specifi c accessibility issues.

While the lack of built-in automation support limits the effectiveness of UI Spy, it can still be an invalu-able tool for viewing an application's UI hierarchi-cal structure, property values of individual controls, and event tracking. You can also use UI Spy to in-teract with controls through UI Automation control patterns. This extensive functionality provides the


content using the TextPattern and TextPattern-Range control pattern interfaces of UI Automation.


UI Accessibility Checker (AccChecker) verifi es the design and implementation of Microsoft Active

means for developers and testers to verify that an application UI is programmatically accessible to as-sistive technology devices. Figure 3 shows UI Spy querying and displaying the UI Automation proper-ties of an element.

UI Spy is similar to Visual UIA Verify in that it does not support the testing and validation of text-based

Figure 3: The control view of UI Spy querying displays the UI Automation properties of an element, running with limited user access.

If you don't need the entire SDK, you can get the .NET Framework Tools with UI Spy by downloading the Windows SDK for Vista Update. This will install side by side with the new Windows SDK for Windows Server 2008. For example: C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin (with UISpy) or C:\Program Files\Microsoft SDKs\Windows\v6.1\Bin


AccChecker is available from CodePlex, the Microsoft open-source project hosting Web site: UI Accessibility Checker: http://www.codeplex.com/AccCheck

Inspect Objects, Accessible Explorer, and Accessible Event Watcher

These tools are available as individual downloads from the Microsoft Active Accessibility 2.0 Software Development Kit (SDK) Tools Web site: http://www.microsoft.com/downloads/details.aspx?familyid=3755582A-A707-460A-BF21-1373316E13F0&displaylang=en#filelist


Accessibility in a control or application UI, regard-less of the underlying UI framework.

The AccChecker architecture incorporates three levels, or stages, of functionality:

A Windows GUI application designed to sup-1. port manual testing, message logging, and suppression generation.An API designed for use in automated testing 2. frameworks.

A console application that supports unman-3. aged test automations typically unable to use the AccChecker-managed API.

All three levels of AccChecker provide similar functionality through verifi cation routines that in-clude Microsoft Active Accessibility programmatic access, programmatic event generation, control layout and keyboard navigation validation, and a basic screen-reader transcription service. Figure 4 illustrates the initial view of the AccChecker UI with default verifi cations.

Inspect Objects

Inspect Objects (Inspect) is a Microsoft Active Ac-cessibility-based tool designed for inspecting and testing IAccessible COM interface properties as well as navigating UI elements and controls.

The Inspect tool tracks elements by using key-board focus, mouse cursor movement, or its navi-gation tools. This built-in navigation is based on a snapshot view of the Microsoft Active Accessibility element tree hierarchy.

The Inspect tool is very useful for quickly discov-ering UI elements that lack unique and meaning-ful Microsoft Active Accessibility Name property values. The Name property is required for assistive technology devices, such as screen readers, to iden-tify controls such as buttons, text boxes, list boxes, images, and links. Figure 5 shows the Inspect tool querying the Microsoft Active Accessibility prop-erties of an element in the Notepad menu.

Accessible Explorer

Accessible Explorer (AccExplorer) is a Microsoft Active Accessibility-based tool that offers func-

Figure 4: The AccChecker UI runs with these default verifi cations.

Figure 5: The Inspect tool UI queries Notepad.

This ability to fi nd and track an accessibility bug

through to resolution in a process you can automate makes these

tools indispensible additions to a tester’s toolkit.


tionality that is very similar to the Inspect tool. However, it differs in two important features:

A somewhat limited ability to track and dis-• play a live view of the Microsoft Active Ac-cessibility element tree.

Microsoft Active Accessibility verifi cation of • specifi c UI elements. The AccExplorer tool provides a passed, failed, and warning system on selectable Microsoft Active Accessibility properties based on the HRESULT returned by the automated query.

Figure 6 illustrates how to use the AccExplorer tool to verify specifi c Microsoft Active Accessibil-ity properties for the Notepad content pane.

Accessible Event Watcher

Accessible Event Watcher (AccEvent) is a com-mand-line tool that provides developers and tes-ters with a means of validating Microsoft Active Accessibility-based WinEvents raised by UI ele-ments. The AccEvent tool reports events when an element is invoked or selected, when it changes state, and when there is a change in focus.

You can track in-process and out-of-process events based on the options available with the Microsoft Active Accessibility SetWinEventHook function.

Even though the AccEvent tool may report some UI Automation-specifi c WinEvents, you should use a dedicated UI Automation client for testing UI Automation events due to the differing de-signs of the event object models in the two frameworks.

Figure 7 illustrates how the AccEvent tool catches the toolbar clock events.

Conclusion

No tool alone can defi nitively test and guaran-tee compliance with all accessibility guidelines. However, the tools described in this article make achieving that goal signifi cantly easier through their comprehensive coverage of the many accessi-bility compliance issues that software vendors face today.

Figure 7: AccEvent reports time changes for the toolbar clock.

Figure 6: The AccExplorer tool can test specifi c Microsoft Active Accessibility properties in the Notepad content pane.

Karl Bridge

48 www.code-magazine.comInternet Explorer 8 New Accessibility Feature

ON

LIN

E Q

UIC

K I

D 0

8110

92

JP Gonzalez-CastellanJP is the Program Manager responsible for improving the Accessibility experience in Internet Explorer. He was born in Argentina and got his Computer Science degree at Rose-Hulman Institute of Technology in Indiana. He has been involved at Microsoft with Accessibility since 2004.

Internet Explorer 8 New Accessibility FeaturesWindows® Internet Explorer® 8 has a lot of cool new features that make Web page browsing more accessible. I’m going to cover Caret Browsing, Zoom Version 2, High DPI, Accessibility Rich Internet Applications (ARIA) support and User Interface Automation (UI Automation) support. Caret Browsing is particularly helpful for low-mobility users. Zoom 2 and High DPI support targets low-vision users, and the new ARIA and UI Automation support targets screen-reader users. Low-mobility users prefer to use the keyboard or devices that interact with a virtual keyboard. Some low-vision users require specialized assistive software to interact with computers while others can do well with features and tools shipped with the operating system.

You can hold the SHIFT key down and press the arrow keys to select text.

As I mentioned before, many users prefer the key-board to the mouse because they fi nd it faster for certain tasks. Users can select a word or phrase, as you see in Figure 2, bring up the Accelerators

through the context-menu key, select Translate with Windows Live, and see the selection’s meaning in Spanish, without ever taking their hands off the keyboard.

Zoom 2

Page zoom lets you enlarge or reduce the view of a Web page to improve readability. The feature is particularly useful on very large and very small displays, allowing

for scaling of content while maintaining the intended page layout. The second iteration of the zoom feature set (the fi rst shipped in Internet Explorer 7) focuses on providing a higher-quality, more predictable, and persistent zooming experience. Primary features in this release include the elimination of horizontal scroll bars for the majority of mainstream scenar-ios and the introduction of persistent zoom states. I personally fi nd this feature useful when using my TV to browse the Web. I sit far away from the TV; using a wireless keyboard, I use the Zoom to make the pages more readable. The extensive keyboard shortcut’s list allows me to use the mouse as little as possible.

Zoom 2 works by adjusting the internal state of the dots-per-inch (DPI) calculations within Internet Explorer 8. To understand this better, consider how different display settings affect a page's layout.

When you make something accessible you are not only impacting the accessibility com-munity but the entire user base. My favorite

example is the wheelchair access ramps. After the American Disability Act was passed, public gather-ing places, like airports, added wheelchair ramps. Airports soon noticed that mothers with baby stroll-ers and passengers with rolling suitcases were using the ramps too, since it was easier than picking up the strollers and suitcases over the ledge. Im-prove the accessibility of your software and everybody wins. When you provide robust key-board navigation to benefi t us-ers who can’t use the mouse, you are also enabling users who want to perform tasks faster, since there are now fewer tasks they need to use the mouse for.

Caret Browsing

Caret Browsing is a new Acces-sibility feature that allows users to navigate a Web page using a moveable cursor on the screen with the keyboard. Users can select and copy text as well as tables and images using only the keyboard.

Moving the cursor within the text of a Web page is similar to moving the cursor within the text of a Microsoft Word docu-

ment, as shown in Figure 1. Pressing F7 toggles Caret Browsing on or off. You can enable Caret Browsing for one tab or for all tabs and windows.

…users can select a word or phrase, bring up the

Accelerators through the context menu key, select

Translate with Windows Live, and see the selection’s meaning in Spanish, without ever taking their hands off the keyboard.

According to a Harris Poll survey,

Americans with disabilities spend twice as much time

on the Internet as those without disabilities.

Roughly 750,000 million people the world over have some disability.

Fast Facts

49www.code-magazine.com Internet Explorer 8 New Accessibility Feature

as big on the 800 × 600-pixel screen as it was on the 1200 × 1600-pixel screen.

System DPI: Changing from 96 DPI to 192 DPI, but keeping all other factors the same, implies more

Screen Resolution: Changing from a 1200 × 1600-pixel screen resolution to an 800 × 600-pixel resolution means changing the spacing between pixels. In other words, pixels are effectively bigger. A screen element of 100 × 100 pixels is now twice

Figure 1: Caret is visible on the screen for the fi rst time.

Figure 2: Pressing the context menu key brings up the context menu; so you can access Web Accelerators using only the keyboard.

Resources and ReferencesYou can learn much more about ARIA and how to use it in your Web content from the following W3C resources:

Introduction to ARIA http://www.w3.org/TR/wai-aria-primer/

ARIA Roles, States and Properties http://www.w3.org/TR/wai-aria/

ARIA Best Practices http://www.w3.org/TR/wai-aria-practices/

To learn more about Windows Accessibility APIs go to http://msdn.microsoft.com and search for Microsoft Active Accessibility and UI Automation.


html#percentage-units). The Internet Explorer page zoom feature cannot change any of the above set-tings except for the zoom factor. However, it can lay out or render a page at a DPI setting that is different from the confi gured system DPI, thus achieving zoom results.

The sample code in Listing 1 divides the screen into quadrants of color, using absolute positioning, to show the difference between the Internet Explorer 8 adaptive page zoom and the Internet Explorer 7 optical page zoom

The boxes in Figure 4 are absolutely positioned, and they are anchored to the corners of the view-port. Each is sized using relative dimensions (%). The box in the middle is anchored to the center of the body. In Figure 5, you can see how Inter-net Explorer 7 displays a 175% zoom while Figure 6 shows how Internet Explorer 8 handles a 175% zoom.

You can see the importance of carefully selecting the anchor for positioned screen elements and using relative dimensions instead of absolute or device-dependent dimensions (such as pixels). If you had positioned the center DIV using pixels, the DIV would have moved from the center when zoomed.

High DPI

You can use the Windows DPI Scaling (https://window-shelp.microsoft.com/Windows/en-US/help/454f5078-2b51-4cda-b4c0-6391e870c41d1033.mspx) feature to scale up Windows fonts and user interface (UI) elements (such as buttons, icons, input fi elds, and so on) by a given percentage. This is different from the scaling that occurs when you lower the display resolution be-cause in the DPI Scaling case, Windows is providing fonts and UI elements that are drawn with more pix-els, resulting in a larger, higher fi delity, and sharper Windows experience.

Like Windows Vista, the Internet Explorer 8 user interface is entirely DPI Scaling-aware, and you will notice that all UI elements and fonts are scaled accordingly. Additionally, larger and higher fi delity icons are used. Notice the difference in the Internet Explorer chrome when Windows DPI Scaling is set to 96 DPI (Figure 7) and 120 DPI (Figure 8).

pixels in an inch. The spacing between pixels has not changed. However, more pixels are required to form a logical inch.

Zoom Factor: Changing the zoom factor from 1 to 2 means that everything should be twice as big. Figure 3 shows how to calculate the Zoom Factor.

HTML Layout Measurements: This value is al-ways 96 DPI, as explained in the CSS 2.1 speci-fi cation (http://www.w3.org/TR/CSS21/syndata.

Figure 3: Zoom factor = System DPI/Layout DPI.

You can see the importance of carefully selecting

the anchor for positioned elements and using relative dimensions

instead of absolute or device-dependent dimensions

(such as pixels).

<html> <head> <title> Zoom 2 Sample </title> </head>

<style>

div { width: 50%; height: 50%; position: absolute; }

#center { width: 20%; height: 20%; left: 40%; top: 40%; border: 5px black solid; }

</style>

<body style="border:0; padding:0;">

<div id="red" style="left:0; top:0; width:50%; height:50%; background-color:red;"> Red </div>

<div id="blue" style="left:50%; top:0; width:50%; height:50%; background-color:blue;"> Blue </div>

<div id="yellow" style="left:0; top:50%; width:50%; height:50%; background-color:yellow;"> Yellow </div>

<div id="green" style="left:50%; top:50%; width:50%; height:50%; background-color:green;"> Green </div>

</body>

</html>

Listing 1: Sample Web page to understand better the new Zoom.

51www.code-magazine.com Internet Explorer 8 New Accessibility Feature

Figure 4: Notice that at 100% the square is centered in the viewport.

Figure 5: 175% zoom in IE 7 magnifi es the text and divs in the Web page and forces the horizontal and vertical scrollbars to show up.

By default, Internet Explorer 8 will zoom the con-tent of the Web to match your Windows DPI Scal-ing settings. (Please note that this is a change from Internet Explorer 7, which did not zoom to match Windows DPI Scaling settings.) For example, if you set your DPI Scaling to 120 DPI, Internet Ex-plorer 8 will zoom the content of Web pages by 125%.

For more information, especially for developers who want to take advantage of High DPI in their Web pages and WebOCs, please see: Making the Web Bigger (http://go.microsoft.com/fwlink/?LinkID=125391&clcid=0x409)

ARIA syntax is a great mechanism to unlock your dynamic, rich Web applications

for everyone.


fi ed Web pages for accessibility, you can use ARIA to mark up your rich Web applications with roles, states, and properties. For example, to match the behavior you create through script, you can defi ne a DIV element as a button, check box, or another ARIA role.

ARIA syntax is a great mechanism to unlock your dy-namic, rich Web applications for everyone. Today Web pages with dynamic content and custom UI controls (such as TreeView controls) have to reuse existing HTML controls. For example, you can make custom TreeView controls accessible by defi ning each item as an HTML list element. This approach can add com-plexity to the code, make it more diffi cult to imple-ment, and prevent all users from getting the same rich behavior.

Code Sample: Create an Interactive TreeView Control Using ARIA

The code sample in Listing 2 uses ARIA to create an interactive TreeView control using HTML and

ARIA and UI Automation Support

In the latest release of Internet Explorer 8, Micro-soft has made a big investment in Accessible Rich Internet Applications (ARIA). The W3C defi nes ARIA as a syntax for making dynamic Web con-tent and custom UI accessible. Windows Internet Explorer 8 recognizes the ARIA role, state, and property information from controls and exposes it to assistive technologies. Assistive technologies can then use Microsoft Active Accessibility® and/or UI Automation Accessibility APIs, which are support-ed for the fi rst time in Internet Explorer, to retrieve the information. Instead of building separate simpli-

Figure 6: 175% zoom in IE 8 magnifi es the Web page’s text but not the divs. So the layout of the page is maintained and hence no horizontal and vertical scrollbars are needed.

Figure 7: Internet Explorer Chrome at 96 DPI. Figure 8: Internet Explorer Chrome at 120 DPI.

53www.code-magazine.com

Conclusion

With Internet Explorer 8 and its support for ARIA and UI Automation, developers have a number of useful tools for developing accessible, rich Web pages. With Caret Browsing, the new Zoom func-tionality, and High DPI support, Internet Explor-er 8 helps make content more accessible to your users.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<html>  <head> <title>TreeView ARIA Example</title> <style type="text/css"> li { list-style-type: none; } </style> <script type="text/javascript"> // Function to expand and collapse tree items by referencing the ariaExpanded attribute. function Toggle(root, itemId) { var el = document.getElementById(itemId) var treeitem = document.getElementById(root) if (el) { if (treeitem.ariaExpanded == "true") { treeitem.ariaExpanded = "false"; el.style.display = "none"; treeitem.childNodes[0].innerText="+ "; } else { treeitem.ariaExpanded = "true"; el.style.display = "block"; treeitem.childNodes[0].innerText="- ";

} } } </script> </head> <body style="margin: 0px">  <ul id="Tree" role="tree">  <li role="treeitem" id="A1" aria-expanded="false" title="Transportation"> <a href="javascript:Toggle('A1','sub1')">+</a> <a href="http://en.wikipedia.org/wiki/Transportation">Transportation</a>  <ul style="display:none" id="sub1" role="group"> <a href="http://en.wikipedia.org/wiki/Car" role="treeitem" title="Car"><li> Car </li> </a> <a href="http://en.wikipedia.org/wiki/Boat" role="treeitem" title="Boat"><li> Boat </li> </a> <a href="http://en.wikipedia.org/wiki/Bike" role="treeitem" title="Bike"><li> Bike </li> </a> </ul> </li> </ul> </body> </html>

Listing 2: Creating an interactive TreeView control.

Internet Explorer 8 New Accessibility Feature

JavaScript. Users of Internet Explorer 8 using an as-sistive technology (such as a screen reader) can in-teract with this control and experience the same rich behaviors users without vision impairments do. If you don’t have access to a screen reader you can use the Inspect32 testing tool available at http://www.micro-soft.com/downloads/. You can experience the benefi ts of ARIA fi rst hand by hovering with the mouse or tabbing with the keyboard through the TreeView. The role displayed in Inspect 32 is “treeview item” instead of “outline item”. Similarly a screen reader will speak out “treeview item” instead of “outline item”.

Users of Internet Explorer 8 using an assistive technology

(such as a screen reader) can interact with a custom the

TreeView control and experience the same rich behaviors users without vision impairments do.

JP Gonzalez-Castellan

Are You Ready To Start?

www.eps-software.com

EPS Can Help!EPS Software provides:

Application Development• .NET 3.0 (WPF, WCF)• Windows & Smart Client• ASP.NET & AJAX Web• Tablet PC & Mobile PC• Windows Mobile

Prototyping ServicesSituation AssessmentProject ManagementMentoringTraining

Contact us at:[email protected]

56 www.code-magazine.comA Pragmatic Approach to WPF Accessibility

ON

LIN

E Q

UIC

K I

D 0

8111

02

Alvin BruneyAlvin Bruney is a Technology Specialist working for Royal Bank of Canada in the .NET Centre of Excellence program. He is a Microsoft Press author and a long-time ASP.NET MVP.

in WPF is due to the fact that the new UI Automa-tion handles accessibility support behind the scenes. You can take advantage of this framework with new projects and use the framework’s bridge and proxy technology to extend existing projects’ feature sets.

UI Automation is more robust and exposes more programmatic information to assistive technolo-gies, allowing for an improved experience. You

can read more about UI Auto-mation elsewhere in this mag-azine as well as at the MSDN® Accessibility Developer Cen-ter. For now, I want to focus on testing accessibility.

Testing WPF Applications

When you build WPF applica-tions, you can test accessibil-ity support using tools such as Jaws, UI Spy, Windows-Eyes, and ZoomText to name a few. There’s even an accessibil-ity checker, shown in Figure 1,

on the Visual Studio tools menu for applications based on Web site project templates. This tool ana-lyzes Web pages against Web Content Accessibility Guidelines (WCAG) and Section 508 of the U.S. Rehabilitation Act.

A strategy that I recommend is to use a combi-nation of tools so that you can achieve great-er accessibility testing coverage. For instance, the Accessibility Checker in Visual Studio tar-gets Web pages; UI Spy targets UI Automa-tion compliance; AccChecker targets Microsoft® Active Accessibility® compliance and general acces-sibility testing. You can decide which tool is a better fi t based on your needs and the underlying acces-sibility framework you’re using. You might use UI Spy and UIA Verify for testing WPF applications based on UI Automation and Accessibility Checker for those based on Microsoft Active Accessibility.

Bear in mind that none of these tools help with designing accessibility into your applications. They

A Pragmatic Approach to WPF AccessibilityAs is often the case, applications are not typically designed with accessibility in mind. Usually, applications are designed to satisfy business requirements. If those business requirements do not include accessibility, more likely than not, the application as a whole will be inaccessible to important segments of users. There are, however, steps you can take to mitigate this common lack of foresight in requirements analysis.

To start off a project right, address accessibility at the design level, even when overlooked in the business requirements. Treat accessibility

as you would usability: neither is explicitly a busi-ness requirement, but both are required parts of a successful user interface. Think about how users can interact with your user interfaces and plan for alternate accessibility with screen readers, magni-fi ers, visual cues, keyboard usage, and so on.

Another way to ensure acces-sibility is to use the right devel-opment tools. The good news is that all versions of Visual Studio® address accessibility; the bad news is that this sup-port doesn’t guarantee perfect coverage, and you should in-clude test and validation tools in your development arsenal. Once you’ve got your design in place and you’ve built what you believe to be an accessible application, control, Web page, or service, you can use these tools to test your deliverables for compliance with accessibility requirements.

In this article, I’ll talk about these issues with respect to Win-dows® Presentation Foundation (WPF). I’ll also touch on some areas where current offerings from adaptive technology ven-dors present challenges to both users and developers.

WPF Has Accessibility Built In

When you build a WPF application, you still need to design for accessibility; however, it’s an easier road to travel today because of the new accessible technology framework that you can build on.

One reason for the increased accessibility support

Treat accessibility as you would usability: neither is explicitly a business requirement,

but both are required parts of a

successful user interface.

Fast FactsAccording to the U.S.

President's Committee on Employment of People with Disabilities, the majority of accommodations cost less

than US$500, and many have little or no cost at

all. However, the average recruiting cost equals

more than 13 percent of an employee's annual salary.

57www.code-magazine.com A Pragmatic Approach to WPF Accessibility

a price for supporting this backward compatibility: bitmaps and images that use the legacy raster technol-ogy begin to blur and distort under increased DPI. Users with certain types of visual impairments may have diffi culty working with such an application.

Even users without visual impair-ments who work with these ap-plications at non-native DPIs can suffer eye strain after prolonged exposure. To put this in perspec-tive, consider a desk clerk work-ing eight hours a day for several months with a software application where some of the images are blurred. This situation can easily lead to a health issue, which can quickly turn into a li-ability issue.

To make working with your applications easier on ev-eryone’s eyes, you should create vector-based images and bitmaps using tools such as Microsoft Blend, or convert existing raster-based images to vector-based images. Vector-based images can respond automati-cally to changes in DPI settings while retaining vi-sual clarity. There are already a number of products that offer this conversion for free or for a fee.

ZoomText

Users with visibility impairments use programs like ZoomText to read content on screen. ZoomText, created by Ai Squared, is a screen reader-magnifi er for Microsoft

can only validate that what you did design and build complies with known, testable requirements.

Challenges with the New Technology

For the most part, supporting accessibility with the UI Automation framework works as intended. However, I want to walk through some cases where potentiality may not align with reality. UI Automa-tion is new, and some assistive technology vendors (ATVs) do not yet support it, which presents sev-eral challenges. Here are some specifi cs you may run into, as well as a few workarounds.

JAWS Screen Reader

The screen reader JAWS has a “JAWS Cursor” mode that allows users to read text and UI information line by line on the screen. When in this mode, the reader does not read the screen information for applications built with WPF native controls. The root cause is the current lack of support for UI Automation in JAWS. Fortunately, Jaws Cursor mode is not the primary inter-face of the screen reader, and the workaround is simply to avoid this mode where possible. As of this writing, JAWS plans to fully support UI Automation in 2009.

Win32 Controls

WPF supports legacy Win32 controls with the Hwnd-Host control, providing an easier migration path from Win32 to XAML-based applications. However, there’s

Figure 1: The accessibility checker in Visual Studio 2008 analyzes Web pages and displays results.

However, there’s a price for supporting this backward

compatibility: bitmaps and images that use the legacy

raster technology begin to blur and distort under increased DPI.

www.code-magazine.com

Expanding LawOn October 2, 2007, National Federation of the Blind (NFB) v. Target Corporation was certifi ed as a class action lawsuit. The plaintiff, NFB, alleges that the Target.com Web site violates the rights of the disabled because some parts of the site are inaccessible to the visually impaired.

The case raises eyebrows as the fi rst accessibility-based class action lawsuit to target a public facing Web site owned by a non-federal entity. These lawsuits have tended to target government institutions.

If NFB prevails, the mandate for and scope of accessibility requirements may apply to all Web pages. But, as I have shown you in this article, there are proactive steps that IT organizations can take to comply with these requirements.

fi er that supports UI Automation. Today, the choices available are pretty close to nil. The situation is expect-ed to improve as ATVs start supporting WPF, UI Auto-mation, and other standard accessibility technologies.

A second option is to disable hardware acceleration for Microsoft Windows in the dialog shown in Figure 3. Hardware acceleration uses DirectX® to improve video and animation quality. When you disable hard-ware acceleration, DirectX is not used.

Hardware Refreshes

Another accessibility issue may arise particularly in the workplace with hardware refreshes. Periodically, the computers or workstations that employees use are upgraded in a hardware refresh cycle, includ-ing new monitors. While applications built on WPF technology will automatically scale correctly to fi t the increased screen resolution, legacy-based con-trols may not scale as you expect.

Some common controls such as the WPF dialog box are not WPF native. Instead, they are simply wrappers around legacy Win32 technology. Usually, issues with these controls manifest themselves in any number of ways, from controls with chopped or unreadable text to controls that do not maintain their layout integrity under different screen conditions, such as the DPI changes and high contrast mode in Figure 4.

Migration from Legacy Technologies

Even legacy Visual Basic® 6 (VB6) migration can inadvertently introduce accessibility issues in WPF. Let’s examine one common scenario. Most VB6

Windows. In some instances, the part of the magnifi ed screen on a WPF native application may completely dis-appear or become unreadable as in Figure 2.

This happens because the hardware-accelerated WPF rendering bypasses the software rendering layer. While you can avoid this in general either by turning off hard-ware acceleration or by using the Magnifi cation API, the latter option is not available with the ZoomText application. It’s important to note that while this is not strictly a WPF issue, the end result is that WPF-based applications may be inaccessible to ZoomText.

Unfortunately, the workarounds are not particularly pleasant. One option is to use another screen magni-

Figure 2: Parts of the user interface are cut off on this magnifi ed screen.

Figure 3: The hardware acceleration slider lets you disable hardware acceleration in Windows XP.

58 A Pragmatic Approach to WPF Accessibility

59www.code-magazine.com A Pragmatic Approach to WPF Accessibility

ble! You need to plan for this progression to avoid in-troducing accessibility issues into your applications.

Conclusion

In my experience, WPF has made the most progress on the accessibility front by far. However, you cannot rely on a platform and tools alone to cover accessibility mandates. The onus is on you to design for accessibility so that you can deliver a better product to your clients.

As the WPF technology gains traction, assistive technol-ogy vendors will come on board in increasing numbers with support for the new UI Automation framework. At that point, you can expect WPF-based applications to have full accessibility support. Until that time, you should plan for accessibility and use the available tools described earlier to help test for accessibility coverage.

common controls have an autosize property that al-lows the control to grow or shrink as the form or parent container is resized. By default, the autosize property is set to false.

When these applications are migrated to .NET, the default setting for the autosize property remains set to false, and the result is that the contents of the con-trol cannot grow as screen conditions change. Con-trast this to a new application developed using native WPF controls where the autosize property isn’t even present. Here, the control automatically grows as re-quired to meet users’ needs.

Another accessibility issue arises when these legacy VB6 controls are migrated fi rst to .NET, where they retain their “inaccessible” settings, and then to WPF inside a WindowsFormsHost control. It’s important to note that this not a WPF issue at all. In fact, its roots are buried deep in VB6 legacy settings. However, WPF-based applications can inherit the accessibility burden.

The visually impaired user couldn’t care less which technology is to blame; the application is inaccessi-

Figure 4: The effects of legacy technology can manifest as unreadable text in high contrast mode.The visually impaired user

couldn’t care less which technology is to blame;

the application is inaccessible! You need to plan for this

progression to avoid introducing accessibility issues into your

applications.

Why Bother Designing and Building Accessible Applications?For one, an accessible application caters to the largest possible audience, and that’s always good for business. The services you offer won’t exclude the estimated 20 million Americans with some sort of disability (source: American Census Bureau, 2006).

Alvin Bruney

AD

VER

TIS

ING

IN

DEX

Active Software Professionalswww.aspalliance.com 6

Applied Innovationswww.appliedi.net 76

Asposewww.aspose.com 75

Code Magazinewww.code-magazine.com 7

Developer Expresswww.devexpress.com 2

DevMavenswww.devmavens.com 15

EPS Software Corp.www.eps-software.com 54, 60

Microsoftwww.microsoft.com/enable/ 5

Tech Conferenceswww.devconnections.com 41

VFP Conversionwww.VFPConversion.com 31

WestWindwww.west-wind.com 35

Xiinewww.xiine.com 69

Advertising Sales:Lake Quincy Media [email protected]@lakequincy.com

Sales ManagersErna Egger+43 (664) 151 [email protected]

Tammy Ferguson832-717-4445 ext [email protected]

Advertisers Index

This listing is provided as a courtesy to our readers and advertisers.

The publisher assumes no responsibility for errors or omissions.


ON

LIN

E Q

UIC

K I

D 0

8111

12

Writing a UI Automation Provider for a Win32-based Custom Control

Brendan McKeonBrendan is a senior software development engineer in the Windows division. He’s worked at Microsoft for 10 years and was a key senior contributor to UI Automation.

providers expose information about the UI as a tree of IRawElementProviderSimple objects. The IRawEle-

mentProviderSimple interface is the base interface representing and exposing information about a single UI element.

The MyCustomControlProvider class only exposes information for the overall HWND. Here, I complete the implementations of these IRawElementProvid-erSimple methods. In this ex-ample, I expose a custom name and specify a control type:

IFACEMETHODIMP MyCustomControlProvider :: get_ProviderOptions(ProviderOptions * pRetVal){ *pRetVal = ProviderOptions_ServerSideProvider | ProviderOptions_UseComThreading; return S_OK;}

This method tells UI Automation that the class is a server-side provider rather than a client-side provider. In general, UI Automation uses client-side providers to support built-in controls such as Win32 buttons, while server-side providers expose information about custom controls to UI Automa-tion.

The second provider options fl ag tells UI Automa-tion to use COM’s threading rules when it makes calls using the interface: incoming calls must come in on the same thread on which you handed out this object, which saves you the trouble of having to do synchronization ourselves. This assumes that CoInitialize was called on the main UI thread dur-ing application startup so that the main UI thread is Apartment-threaded.

Writing a UI Automation Provider for a Win32-based Custom ControlDo you have a complex custom control that you want to make programmatically accessible, but you aren’t sure how? Custom controls, by their nature, tend to be very diverse: each is typically written for a specific purpose, making it difficult to generalize implementation details. How do you know what to implement? You should consider supporting accessibility for any custom control that performs its own rendering and input management—routing mouse and keyboard input—within the HWND that it owns.

In this article, I walk through the steps you need to follow to implement a server-side provider for a Win32-based custom control

using Microsoft® UI Automa-tion. The example I use assumes the custom control has its own Handle to a Window (HWND) and Window Procedure. While the emphasis is on unmanaged C++ implementations, the tech-niques are equally applicable to managed code.

What is a Provider?

UI Automation presents the UI on the desktop to a client application in the form of a tree of IUIAuto-mationElement objects, each of which can expose properties describing the UI. The UI itself must sup-ply these properties using APIs and messages. The work of calling the UI-specifi c APIs and returning the information back to UI Automation is done by a component called a provider.

UI Automation comes with providers for the vari-ous standard OS controls, such as Win32 button and list controls, already built in. To make a cus-tom control accessible through UI Automation, you need to supply a UI Automation provider for that control.

Exposing Built-in Properties of a Single ElementThe fi rst step to making your custom control ac-cessible is to create a class that implements the UI Automation provider interface, IRawElementPro-viderSimple, as shown in Listing 1.

While UI Automation clients see the entire user inter-face (UI) as a tree of IUIAutomationElement objects,

In general, UI Automation uses client-side providers to support built-in controls such

as Win32 buttons, while server-side providers expose information about

custom controls to UI Automation.

Fast Facts

63www.code-magazine.com Writing a UI Automation Provider for a Win32-based Custom Control

IFACEMETHODIMP MyCustomControlProvider:: GetPropertyValue(PROPERTYID idProp, VARIANT * pRetVal ){ pRetVal->vt = VT_EMPTY; if(idProp == UIA_NamePropertyId) { pRetVal->bstrVal = SysAllocString(L“MyControlName“); pRetVal->vt = VT_BSTR; } else if(idProp == UIA_ControlTypePropertyId) { pRetVal->lVal = UIA_ButtonControlTypeId; pRetVal->vt = VT_I4; } return S_OK;}

Finally, get_HostRawElementProvider tells UI Au-tomation which HWND this control is associated with:

IFACEMETHODIMP get_HostRawElementProvider( IRawElementProviderSimple ** pRetVal ){ return UiaHostProviderFromHwnd(_hwnd, pRetVal);}

Once you’ve implemented the class, you have to ensure that UI Automation can actually access an

Patterns are interfaces that UI Automation uses to expose control-specifi c functionality such as selection or command invocation. I’ll revisit them later, so I’ll set the return value to NULL here:

IFACEMETHODIMP MyCustomControlProvider:: GetPatternProvider(PATTERNID idPattern, IUnknown ** pRetVal ){ *pRetVal = NULL; return S_OK;}

The GetPropertyValue method handles element properties. UI Automation divides properties into two categories: those related to a specifi c area of functionality—for example, those dealing with se-lection, which only apply to controls that manage selection—and those that broadly apply to any el-ement, including Name, Enabled, and Control Type. This latter category of property is handled in the GetPropertyValue method.

For now, I’m supplying suitable values for the element’s Name and Control Type properties. These are two of the more important properties, as screen readers typically read the name and type of a control as focus changes, but I can support any number of properties later:

#include <UIAutomation.h>

// New class that exposes a custom control to UI Automation. // This implements the provider-side interfaces.class MyCustomControlProvider: public IRawElementProviderSimple{ HWND _hwnd;

// Reference to control data or object model goes here

public:

MyCustomControlProvider(HWND hwnd): _hwnd(hwnd) { }

// Standard implementations of AddRef, Release, // QueryInterface go here

// IRawElementProviderSimple methods IFACEMETHODIMP get_ProviderOptions(ProviderOptions * pRetVal); IFACEMETHODIMP GetPatternProvider(PATTERNID idPattern, IUnknown ** pRetVal ); IFACEMETHODIMP GetPropertyValue(PROPERTYID idProp, VARIANT * pRetVal ); IFACEMETHODIMP get_HostRawElementProvider( IRawElementProviderSimple ** pRetVal );};

Listing 1: This example is an initial implementation of a provider class, MyCustomControlProvider.

LRESULT MyCustomControlWndProc(HWND hwnd, UINT uMsg, WPARAM wParam, LPARAM lParam){ switch(uMsg) { case WM_GETOBJECT: { MyCustomControlProvider * pProvider = new MyCustomControlProvider(); LRESULT lres = UiaReturnRawElement(hwnd, wParam,

lParam, pProvider); pProvider->Release(); return lres; } } return DefWindowProc(hwnd, uMsg, wParam, lParam);}

Listing 2: The WM_GETOBJECT message ensures UI Automation can access an instance of your provider.

64 www.code-magazine.comWriting a UI Automation Provider for a Win32-based Custom Control

JECT message to see if the HWND supports UI Au-tomation, and uses the returned provider to obtain more properties.

All this happens only on demand: it’s only when a client requests an object for this specifi c HWND that the WM_GETOBJECT message is sent. Get-PropertyValue is called only when a client requests a property.

Connecting a Provider to Real Control DataTo make this example more realistic, I’m going to modify the provider code to contain a reference to the custom control’s own data so that it can return real data from GetPropertyValue. Depending on how your control works, this might involve a pub-lic or internal COM object, window messages, or direct access to the control’s data.

An important issue to address here is that of pro-vider lifetime. UI Automation holds onto a refer-ence to the control on behalf of clients, and clients are not required to release their references at any particular point in time. This means that a provider can outlive its corresponding UI.

One way to deal with this situation is to add an “alive” fl ag to the provider and set it to a “dead” state when the UI is no longer present. Further incoming calls to the IRawElementProvider can then check this fl ag and return the defi ned value UIA_E_ELEMENTNOTAVAILABLE.

instance of it. This is done by handling the WM_GETOBJECT message in the window’s Window Procedure, as shown in Listing 2.

In response to WM_GETOBJECT, I create and re-turn an instance of the provider to UI Automation via the UiaReturnRawElement API from UIAuto-mationCore.dll. Standard COM reference counting rules apply: UiaReturnRawElement calls AddRef() internally to claim its own reference; so I call Re-lease() when I’m done with the WndProc’s refer-ence.

Also, the code here creates a new instance of the provider each time: this is allowable, as is creating a single instance the fi rst time and caching that value for later reuse. Just remember to use the Release() method on that reference in the WM_DESTROY handler.

Note: Throughout this article, I refer to the Inspect Objects tool, which is available in the Active Ac-cessibility 2.0 SDK Tools download on the Micro-soft Download Center (http://www.microsoft.com/downloads/details.aspx?familyid=3755582a-a707-460a-bf21-1373316e13f0&displaylang=en). This tool helps you see what UI Automation sees. You can use it to test and verify your provider implementations.

If I point the Inspect Objects tool at the custom con-trol, it should report the Name and Control Type that I’ve specifi ed above. What’s happening here is that Inspect Objects asks UI Automation for the ob-ject at a specifi ed point on the screen. UI Automa-tion determines the HWND, sends a WM_GETOB-

class MyCustomControlProvider: public IRawElementProviderSimple{ HWND _hwnd;

// Create a pointer to internal control state, set to // non-NULL in the constructor. // OnUIDisposed sets it to NULL to indicate that UI is no // longer present. Using NULL to indicate this avoids having // a separate fl ag. // Incoming UI Automation interface methods must check this // before use. // IMyInternalControlInterface _pControl;

...

// Called when corresponding UI is deleted void OnUIDisposed();};

void MyCustomControlProvider::OnUIDisposed(){ _pControl = NULL;}

IFACEMETHODIMP MyCustomControlProvider::GetPropertyValue( PROPERTYID idProp, VARIANT * pRetVal ){ // Always clear out parameter before anything else pRetVal->vt = VT_EMPTY;

// Now I check to see if the UI is still present if(_pControl == NULL) { return UIA_E_ELEMENTNOTAVAILABLE; }

// UI is still present, so code continues as before if(idProp == UIA_NamePropertyId) { ... // get name from _pControl and return via pRetVal. }...

Listing 3: One technique for handling a provider’s lifetime is to create and update a state fl ag.


Note on Microsoft® Active Accessibility CompatibilityUI Automation automatically makes some of the structure, properties and events of your custom control available to Microsoft Active Accessibility clients. The information UI Automation translates is the subset of data that Microsoft Active Accessibility supports.

WM_GETOBJECT requests both Microsoft Active Accessibility support and UI Automation support from a HWND, with the lParam indicating which type the sender is looking for. When you pass the provider to UiaReturnRawElement, it checks the lParam and wraps the provider in a UIA-to-MSAA adaptor, which it then hands back to the client.

This UIA-to-MSAA translation wrapper also implements the IAccessibleEx interface, which allows Microsoft Active Accessibility clients to query for the UI Automation properties and patterns that are outside the scope of the Microsoft Active Accessibility interfaces.

The returned IRawElementProviderFragment in-stance can be a new instance of a provider or a cached instance, so long as it obeys COM’s refer-ence counting rules. Note that the object that this is called on continues to point to the same UI element that it was previously pointing to.

The tree structure that is returned by this meth-od should be consistent: if you can navigate to a child, then when you navigate back to the par-ent, you should end up at a provider instance that represents the same UI element. If you cache providers, this may be the same provider object instance.

The root provider of your structure, the one that oc-cupies the entire HWND, must return NULL when asked for its parent or siblings. You should provide just the subtree structure that corresponds to the UI that your control is providing; UI Automation will do the work of integrating this into the larger tree-of-peer-HWNDs.

Next, for the GetRuntimeId method, UI Automa-tion requires that you return an integer array that uniquely identifi es this specifi c element within the scope of the owning HWND:

IFACEMETHODIMP MyCustomControlProvider:: GetRuntimeId (SAFEARRAY ** pRetVal);

If you already have a unique integer assigned to each UI element, using that would work perfectly here as a single element array. Be cautious about using a pointer value because you have to take into account that pointers on 64-bit systems require two integers.

Once you have your array, prepend to it the value UiaAppendRuntimeId and use the API’s SafeArray-CreateVector and SafeArrayPutElement functions to convert the C-style array to a SAFEARRAY.

The root element of your control can simply return NULL here: UI Automation automatically con-structs Runtime IDs for the host HWND, and from UI Automation’s point of view, the root element of the control is essentially the same element as that HWND.

The get_BoundingRectangle method returns the bounding rectangle coordinates of the control. For this, return the location of the UI element in screen coordinates. If the element is not vis-ible, return a rectangle consisting of all 0s. You may need to use MapWindowPoints to convert from HWND client-relative coordinates to screen coordinates:

IFACEMETHODIMP MyCustomControlProvider:: get_BoundingRectangle(UiaRect * pRetVal);

As with GetRuntimeId, the root element doesn’t have to do anything here, since its location is the same as the host HWND, and UI Automation can already get that information from the HWND.

Listing 3 shows the new code that takes provider lifetime into account.

Now if you point the Inspect Objects tool at the control, you should see the control’s actual name instead of the string.

Adding Structure by Adding IRawElementProviderFragmentSo far, I’ve simply exposed a single element. How-ever, most custom controls have some internal structure, whether items in a ListBox or a tree of el-ements in a HTML-like control. The next step is to expose that structure to UI Automation by adding the IRawElementProviderFragment interface.

You need to consider what sort of class structure to use to expose your control’s internal structure. If you have a two-level homogenous container—for example, a list that contains only one type of list item—then it may make sense to have one class for the overall parent list and another class for the items. On the other hand, if the control has a more generalized object tree—as is the case with HTML—then it may make more sense to have a hierarchy of provider objects that mirrors the UI’s own class structure. It could have a base class that implements common functionality and derived classes that han-dle element-specifi c behavior.

Whatever structure you choose, you need to imple-ment both IRawElementProviderSimple and IRawElementProviderFragment interfaces for each class that represents an element in the structure. As you’ll see later, the root element also needs to imple-ment IRawElementProviderFragmentRoot.

You only need to access the tree structure when a cli-ent requests it. Furthermore, you don’t expose a tree structure all at once; instead, you access or traverse it element by element as needed.

Because internal structure is very control-specifi c, I won’t provide concrete examples in this walk-through. I’ll just outline what each method does. Note that, as with IRawElementProviderSimple, all the methods should check that the corresponding UI element is “alive” fi rst before they go on to do the real work.

The Navigate method is perhaps the most obvious method in this interface. It’s called with a specifi ed direction (Parent, Next or Previous sibling, or First or Last child), and returns an instance of IRawEle-mentProviderFragment that represents the element in the specifi ed direction. This object must also im-plement IRawElementProviderSimple or else UI Automation would be able to navigate to it but not get any information about it:

IFACEMETHODIMP MyCustomControlProvider:: Navigate(NavigateDirection direction, IRawElementProviderFragment ** pRetVal );

66 www.code-magazine.comWriting a UI Automation Provider for a Win32-based Custom Control

Note on Client-side ProvidersUI Automation also supports client-side providers, which are loaded by the client application and often provide the default UI Automation support for many of the standard Win32 controls.

Server-side and client-side providers are very similar in terms of general class structure: they implement the same set of IRawElement* interfaces and expose properties and patterns in the same way. But there are differences.

While server-side providers live in the same process as the control itself; client-side providers live in the client application’s address space. Server-side controls can obtain information from the underlying control by accessing the control’s internal data if necessary. Client-side controls, however, must rely on APIs or window messages.

IFACEMETHODIMP MyCustomControlProvider:: ElementProviderFromPoint(double x, double y, IRawElementProviderFragment ** pRetVal);

IFACEMETHODIMP MyCustomControlProvider:: GetFocus( IRawElementProviderFragment ** pRetVal);

These methods should return a provider instance representing the focused element or the element at the specifi ed point (in screen coordinates), or NULL if the point or focus is on the root element itself or if the point or focus is not on the ele-ment at all. For both these methods, the provider should return the deepest element possible.

Finally, I’ll return to the implementation of IRawElementProviderSimple::get_HostRawEle-mentProvider. You should modify this such that it only returns the HWND for the root. For all child elements within the control, it should return NULL.

You can take a piecemeal strategy for implement-ing all of these methods. First, start with imple-menting the Navigate and BoundingRectangle methods. This enables you to get to the root el-ement by hovering the cursor over any part of the control, and then walking through the tree using the Navigation functionality of the Inspect Objects tool. Use the highlight rectangle feature to quickly verify that the locations are being ex-posed correctly as you navigate. Then you can implement ElementProviderFromPoint. When you hover the mouse over an element within the control, Inspect Objects should go straight to that element instead of stopping at the HWND. Final-ly, implement the GetRuntimeId and GetFocus methods.

Deciding What Part of the Tree Structure to Expose

When deciding what parts of the tree structure to expose, remember that UI Automation is only inter-ested in elements that are important to an end user. This includes any element that an end user would perceive as being a separate control in its own right. For example, a push-button and its text are typically not considered distinct elements from an end-user’s point of view, even if they are represented as such in a control’s internal structures. In such a case, you should expose just a single element representing the entire button.

Exposing Control-specifi c Functionality Using PatternsThe next step in implementing a provider is to cus-tomize the provider with information specifi c to the corresponding UI. I’ve already covered prop-erties (such as Name and Control Type), so it’s time to turn to control patterns. A pattern is just an interface that contains both properties and meth-

You only use the GetEmbeddedFragmentRoots method when your control hosts other HWNDs within it. Since I’m not covering this issue yet, this method returns NULL here.

Again, the root element doesn’t have to do any-thing because its location is the same as the host HWND:

IFACEMETHODIMP MyCustomControlProvider:: GetEmbeddedFragmentRoots(SAFEARRAY** pRetVal);

UI Automation calls the SetFocus method when it wants you to set your control’s internal focus state to a particular element. UI Automation automati-cally focuses the parent HWND fi rst, so you only need to update your control’s internal state:

IFACEMETHODIMP MyCustomControlProvider:: SetFocus();

The get_FragmentRoot method provides UI Au-tomation with a convenient way to get to the root element of your control without having to call Navigate(Parent) repeatedly:

IFACEMETHODIMP MyCustomControlProvider:: get_FragmentRoot( IRawElementProviderFragmentRoot ** pRetVal);

Next, you need to implement two structure-related methods only on the root node. These are on the IRawElementProviderFragmentRoot and allow UI Automation to request the focused element or the element at a specifi c screen location:

IFACEMETHODIMP MyCustomControlProvider::GetPatternProvider( PATTERNID idPattern, IUnknown ** pRetVal ){ // Always start by setting output parameter to NULL *pRetVal = NULL;

// Check that control is still alive if(_pControl == NULL) { return UIA_E_ELEMENTNOTAVAILABLE; }

// Now check if it‘s a pattern that this element supports if(idPattern == UIA_ExpandCollapsePattern && _pControl->CanExpand()) { // Assuming here that this class also implements // IExpandCollapseProvider *pRetVal = this; AddRef(); }

return S_OK;}

Listing 4: This method returns a pattern provider based on the patterns the control implements.


There are three main categories of events in UI Automation:

Events associated with a property, such as • changing the Name or IsEnabled values.Events associated with an action, such as the • pressing of a button.

Events associated with the creation of deletion • of elements, called structure change events.

The general technique for notifying UI Automa-tion of an event is the same in all three cases: the control creates an instance of a provider for the UI element that corresponds to the event, and then calls one of the UI Automation “Raise” APIs, pass-ing in the ID of the event or property as appropri-ate.

For example, to signal focus change, a control would use code similar to the following, which as-sumes that pFocusedItem is a reference to an item that has just received focus:

// Use UiaClientsAreListening to avoid creating a// provider if no clients are listening

if(UiaClientsAreListening()){ IRawElementProviderSimple pProvider = new MyCustomControlProvider(hwnd, pFocusedItem); UiaRaiseAutomationEvent(pProvider, UIA_FocusChangedEventId);

pProvider->Release();}

If clients are listening, the UiaRaiseAutomation-Event call will create a reference to the provider that remains in place until the client is done with the ele-ment.

Before the Inspect Objects tool can track focus in the control, I need to implement the HasKeyboardFo-cus property to return a VT_BOOL variant with value VARIANT_TRUE when the element has focus. When that’s done, Inspect Objects should be able to track focus as you move keyboard focus around with-in your control using tab or arrow keys.

NOTE: The UiaClientAreListening method returns TRUE if a client is listening to any event from this process. You can get more fi ne-grained information by implementing the optional IRawElementPro-viderAdviseEvents interface on the root element of your control. UI Automation will then call the Advi-seEventAdded and AdviseEventRemoved methods on this interface as clients register and unregister for specifi c events.

Deciding which Properties, Patterns, and Events to Implement

So far, I’ve discussed the mechanics of exposing the structure of a control and of exposing the prop-

ods related to a common theme. For example, the SelectionItem pattern interface has properties like get_IsSelected that return the current selected state, and methods like Select that change the se-lection.

Implementing a pattern is a matter of fi guring out which interfaces to implement, and then implement-ing the interface in terms of the underlying control data. The mechanics of exposing the interface are similar to using QueryInterface, but instead using the method IRawElementProviderSimple::GetPatternProvider. This method takes a parameter indi-cating the pattern that is requested and returns an IUnknown object that implements the correspond-ing interface.

While it is often most convenient to implement the pattern interface on the same provider object, this is not required. You can implement the interface on a separate object if desired, though this separate object will need to adhere to the same warnings re-garding lifetime that the original provider does.

Listing 4 shows a simple implementation of Get-PatternProvider which implements an interface on the same object as the provider.

It isn’t necessary to cast the returned object to the interface corresponding to the requested pattern: UI Automation calls QueryInterface on the re-turned object as a separate step after this to get the correct interface.

Ensure that manipulating the control using these interfaces is consistent with manipulating the con-trol using the mouse or keyboard. For example, if selecting some element with a mouse or keyboard causes some other element to change state, that same change should occur when the element is selected using the ISelectionItemProvider::Select method.

Notifying UI Automation of UI Events

The last piece of UI Automation functionality that you need to address is events. When control state changes, you need to tell UI Automation so that it can inform any interested clients. For example, if the focus changes from one element to another, UI Automation can inform a screen reader, which can in turn query for the name of the element with fo-cus.

[Name and Control Type] are two of the more important properties, as screen readers

typically read the name and type of a control as focus changes…


erties, patterns, and events for elements of the con-trol. But how do you decide what to expose? Much of that depends on the purpose and functionality of the control. However, there are a few properties that almost all elements should expose, as listed in Table 1.

UI Automation provides some other properties automatically, such as ProcessId and NativeWin-dowHandle. Even though these are defi ned in the header fi le, you don’t need to implement support for them.

In determining which other properties and patterns to support, a useful guide is the control type: many control types have a set of associated properties and patterns that are expected for controls of that type. For example, buttons are expected to imple-ment the Invoke pattern, and combos and lists are expected to implement the Selection pattern.

You can fi nd further information on these proper-ties and patterns, as well as the entire Windows Automation 3.0 API, in the upcoming Windows® Software Development Kit for Windows 7.

Writing a UI Automation Provider for a Win32-based Custom Control

Property Description

Name Identifi es the control. This is often the text on the control or the label. The name shouldn’t contain the type of the control. For example, “OK” rather than “OK button.”

ControlType Specifi es one of the defi ned UIA_XxxxControlTypeId values from UIAutomationClient.h, like UIA_ButtonControlTypeId.While ControlType is effectively an integer enumeration value, LocalizedControlType is a human-readable string. UI Automation automatically supplies the LocalizedControlType based on the ControlType that you supply, so you don’t need to implement the LocalizedControlType property.

IsEnabled Identifi es whether the element is enabled or disabled.

HasKeyboardFocus Identifi es whether the element has keyboard focus.

IsKeyboardFocusable Identifi es whether the element can have keyboard focus.

AcceleratorKey and AccessKey

Identifi es the accelerator key or access key for the control.

Table 1: Your custom control should support these properties for each of its elements.

Brendan McKeon

When deciding what parts of the tree structure to expose,

remember that UI Automation is only interested in elements that are

important to an end user.

Have You Xiine It?

Experience the next generation in digital reading FREE. Go to www.xiine.com to read CoDe Magazine and other great publications.

All You Can Read. Digitally.


ON

LIN

E Q

UIC

K I

D 0

8111

22

Creating UI Automation Client Applications

Matthew KarrSoftware Development Engineer

Matthew Karr has been a developer at Microsoft for 5 years, working on Automation and Accessibility. He’s helped ship UI Automation V1 and the Magnifi cation API with Vista, and is currently working on the COM UI Automation API for Windows 7.

In his free time he enjoys juggling, snowboarding, science fi ction, and video games.

Automation enables programmatic access through fi ve components:

The UI Automation tree facilitates navigation • through the structure of the UI. The tree is built from the collection of window handles (HWNDs).Automation elements are individual com-• ponents in the UI. These can often be more granular than a window handle.Automation properties provide specifi c infor-• mation about UI elements.

• Control patterns defi ne a particular aspect of a control’s functionality. They can consist of in-formation about properties, methods, events, and structures.

• Automation events provide event notifi cations and information.

Basics of a UI Automation Client Application

For the Windows® 7 operating system, UI Automation has a new COM client API for both unmanaged and managed cli-ents. Unmanaged applications can now use UI Automation without requiring a change in languages or loading the common language runtime (CLR).

Perhaps the best way to un-derstand UI Automation in general, and the COM client specifi cally, is to see them in ac-tion. Listing 1 demonstrates a complete program that gets the UI element at the current cur-sor position, and then prints its name and control type.

Next I’ll walk through the various pieces that make up this example.

Microsoft® UI Automation is a tool for doing just that. It provides an abstracted model of the UI, and allows a client application

to both investigate and manipulate the UI of all running applications on the system. In this article, I explain how you can use UI Automation for au-tomated testing, and I walk through code samples that demonstrate the basic features of UI Automa-tion.

Using UI Automation for Automated Testing

The goal of many automated test tools and scenar-ios is the consistent and repeatable manipulation of the user interface. This manipulation can involve simple unit testing of specifi c controls, or complex recording and playback of test scripts that iterate through a series of generic actions on a group of controls.

One complication that arises from automated applications is the diffi culty of synchroniz-ing a test with a dynamic tar-get. As an example, consider a list box control—such as one contained in the Windows Task Manager—that displays a list of currently running ap-plications. Since the items in the list box are dynamically updated outside the control of the test application, consis-tently repeating the selection of a specifi c item in the list box is impossible. Similar problems arise when attempting to re-peat simple focus changes in a UI that is outside the control of the test application.

Programmatic access provides the ability to imitate, through code, any interaction and experience ex-posed by traditional mouse and keyboard input. UI

Creating UI Automation Client ApplicationsSometimes an application needs to interact with the user interface (UI) of a second application. The first application might be a test application that drives the UI of the target to run through some automated tests. It might describe the UI out loud, as an aid to users that are blind. It might be a speech application that allows users to give vocal commands. In each of these cases, the application needs a way to inspect and interact with the UI of the system and other running applications.

Microsoft UI Automation was available for

both managed and unmanaged applications

from the beginning; however, the framework

interface was limited for managed API

for its client applications. Now with new

introduction of COM based unmanaged client API,

the UI Automation clients can also be managed

or unmanaged.

Fast Facts

71www.code-magazine.com Creating UI Automation Client Applications

UI Automation and ThreadingBecause of the way UI Automation uses Windows messages, confl icts can occur when a client application attempts to interact with its own UI on the UI thread. These confl icts can lead to very slow performance or even cause the application to stop responding.

If you intend your client application to interact with all elements on the desktop—including its own UI—you should make all UI Automation calls on a separate thread. This recommendation also applies when your application locates elements and uses control patterns.

It is safe to make UI Automation calls within a UI Automation event handler because the event handler is never called on a UI thread. However, when subscribing to events that may originate from your client application’s UI, you must make the call on a non-UI thread. Remove event handlers on the same thread.

Directly Obtaining UI Automation Elements

Once you have the UI Automation object, you can discover the entire UI. The UI is modeled as a tree of automation elements (IUIAutomationElement interface objects), where each element represents a single piece of UI. The IUIAutomationElement interface has methods relevant to all controls, such as checking properties or setting focus.

The root element of the UI Automation tree is the desktop. You can obtain this element by calling either the IUIAutomation::GetRootElement or IUIAutomation::GetRootElementBuildCache method. Each method retrieves an IUIAutomationElement inter-face pointer, from which you can search or navigate the rest of tree, as described later in this article.

If you have screen coordinates—such as the cur-sor position in this example—you can retrieve an IUIAutomationElement interface by calling the IUIAutomation::ElementFromPoint method.

To retrieve an IUIAutomationElement interface from a window handle (HWND), call the IUIAutomation::ElementFromHandle method.

You can retrieve an IUIAutomationElement inter-face that represents the focused control by calling the IUIAutomation::GetFocusedElement method.

UI Automation Properties for Clients

Properties of IUIAutomationElement objects con-tain information about UI elements, usually controls.

Creating the CUIAutomation Object

The heart of the new COM API is the IUIAuto-mation interface, which lets clients get automa-tion elements, register event handlers, create various helper objects, and access other helper methods.

To get started using UI Automation in your applica-tion, do the following:

Include Uiautomation.h in your project head-1. ers. This header will bring in the other headers that defi ne the API.Declare a global pointer to an 2. IUIAutomation interface.Initialize COM.3.

Create an instance of the 4. CUIAutomation class and retrieve the IUIAutomation interface in your global pointer.

The following example function creates the object instance and stores the retrieved interface address in the global pointer g_pAutomation:

HRESULT InitializeUIAutomation(){ CoInitialize(NULL); HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (void**)&g_pAutomation); return (hr);}

int _tmain(int argc, _TCHAR* argv[]){ // Initialize COM and create the main Automation object IUIAutomation *g_pAutomation; CoInitialize(NULL); HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL, CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation), (void**)&g_pAutomation); if(FAILED(hr)) return (hr);

// Get the element under the cursor // Use GetPhysicalCursorPos to interact properly with // High DPI POINT pt; GetPhysicalCursorPos(&pt);

IUIAutomationElement *pAtMouse; hr = g_pAutomation->ElementFromPoint(pt, &pAtMouse); if(FAILED(hr)) return hr; // Get the element‘s name and print it BSTR name; hr = pAtMouse->get_CurrentName(&name);

if(SUCCEEDED(hr)) { wprintf(L“Element‘s Name: %s \n“, name); SysFreeString(name); } // Get the element‘s Control Type (in the current languange) // and print it BSTR controlType; hr = pAtMouse->get_CurrentLocalizedControlType(&controlType); if(SUCCEEDED(hr)) { wprintf(L“Element‘s Control Type: %s \n“, controlType); SysFreeString(controlType); } // Clean up our COM pointers pAtMouse->Release(); g_pAutomation->Release(); CoUninitialize(); return 0;}

Listing 1: The program gets the UI element at the current cursor position and prints its name and control type.

72 www.code-magazine.comCreating UI Automation Client Applications

simple like the Invoke pattern, which lets clients invoke a control. In contrast, the more compli-cated Value pattern supports getting and setting a control’s value, and then checking whether the value is read-only.

You can obtain a control pattern by calling the IUIAutomationElement::GetCurrentPattern or IUIAutomationElement::GetCurrentPatternAs method, or their cached versions. Once you get a control pattern interface, use it as you would the element itself, either by directly calling control pattern methods or by accessing control pattern properties.

The TurnItUp method in Listing 2 uses the Ran-geValue pattern to set a control (such as a volume slider) to its maximum value.

In addition to checking for errors when getting the current pattern, this example also checks for NULL because NULL is a valid return value when a control does not support the pattern requested.

Control Types

While a control type is mechanically a simple enu-meration property, on a conceptual level it is much more important. An element’s control type is a broad classifi cation of the control, such as button, window, or list box. Each control type has certain expected control patterns. For example, a button control should support either the Invoke pattern or the Toggle pattern, and a hyperlink should support the Invoke Pattern and possibly the Value pattern.

Some controls have conditional support for several control patterns. For example, the menu item con-trol has conditional support for the Invoke, Expand Collapse, Toggle, and SelectionItem control pat-terns, depending on the menu item control’s func-tion within the menu control.

Searching and Navigation

There are two major ways of getting to elements. The fi rst is to use a Find method, which lets the UI

The properties of an element are generic to all ele-ments and not specifi c to a control type. Control pat-terns, discussed later, expose control-specifi c proper-ties.

UI Automation properties are read-only. To set prop-erties of a control, you must use the methods of the appropriate control pattern; for example, use the IScrollProvider::Scroll method to change the posi-tion values of a scrolling window.

To improve performance, you can cache property values of controls and control patterns when you retrieve elements.

Some generic properties, and all control pattern properties, are available as properties on the IUIAutomationElement interface or control pattern interface, and you can retrieve them with accessor methods.

When using the generic accessors, you must spec-ify properties with the property identifi ers defi ned in UIAutomationClient.h. You use them to specify properties when retrieving property values, con-structing property conditions, and subscribing to property-changed events.

If a UI Automation provider does not implement a property, UI Automation is able to supply a default value for that property. For example, if the pro-vider does not support the property identifi ed by UIA_IsDockPatternAvailablePropertyId, UI Auto-mation returns FALSE.

Control Patterns

Control patterns complement properties. Control patterns are collections of associated properties, events, and methods that describe an element. More than one pattern can apply to a single ele-ment.

A control pattern represents a collected group of capabilities. The pattern might be something

HRESULT TurnItUp(IUIAutomationElement *pElement){ IUIAutomationRangeValuePattern * pRangeVal;

HRESULT hr = pElement->GetCurrentPatternAs(UIA_RangeValuePatternId, __uuidof(IUIAutomationRangeValuePattern), (void **)&pRangeVal);

if(FAILED(hr) || pRangeVal == NULL) { return hr; }

double max; hr = pRangeVal->get_CurrentMaximum(&max);

if(SUCCEEDED(hr)) { hr = pRangeVal->SetValue(max); }

pRangeVal->Release(); return hr;}

Listing 2: This method uses the RangeValue pattern to set a control to its maximum value.

Programmatic access provides the ability to imitate,

through code, any interaction and experience exposed by traditional

mouse and keyboard input.

73www.code-magazine.com Creating UI Automation Client Applications

ticular name) is quite simple. However, conditions can be considerably more complex.

Using Tree Walkers

Another method for navigating through the tree is the use of tree walkers. Tree walkers allow use of direct-navigation methods such as moving between parent, child, and sibling to walk through a fi ltered view of the tree. There are several built-in fi ltered views:

Raw or unfi ltered, which shows all elements.• Control view (the default), which fi lters out • elements that either are redundant or are just used for layout.

Content view, which fi lters controls even more • selectively than Control view does.

Users can also create their own custom views with Conditions.

The following simple example shows how to walk to the fi rst child of the control referenced by pEle-ment:

// Get the control view walkerIUIAutomationTreeWalker * pWalk; g_pAutomation->get_ControlViewWalker(&pWalk);

// Go to the element‘s fi rst childIUIAutomationElement * pFirst;pWalk->GetFirstChildElement(pElement, &pFirst);

UI Automation Events for Clients

UI Automation lets clients subscribe to events of interest. This capability improves performance by eliminating the need to continually poll all UI ele-ments in the system to see if any information, struc-ture, or state has changed.

Effi ciency is also improved by the ability to listen for events only within a defi ned scope. For example,

Automation core minimize cross-process calls and improve searching time. The second is to use a tree walker to walk a UI tree.

Using Find Methods

The Find methods require three things: a parent ele-ment, a scope to search on, and (most importantly) a condition to search for.

You create conditions by calling various APIs on the CUIAutomation object. You can create simple Property conditions, or combine them with And, Or, and Not operators to produce more complex conditions.

Once you create a condition, you need to specify the element to start searching from and the search scope. The element you want to start from is the element on which you call the FindFirst or Fin-dAll method. The search scope is typically one of three choices: the element itself, its children, or its descendants.

You usually want to limit search scope as much as possible to improve performance and to reduce the chance of fi nding an element you weren’t looking for.

Listing 3 is a simple example of searching for a named application window using FindFirst meth-ods. In the example, the search condition (a par-

IUIAutomationElement* GetTopLevelWindowByName(LPWSTR windowName){ if (!windowName) { return NULL; }

IUIAutomationElement* pRoot; IUIAutomationElement* pFound; VARIANT varProp; varProp.vt = VT_BSTR; varProp.bstrVal = SysAllocString(windowName);

// Get the desktop element HRESULT hr = g_pAutomation->GetRootElement(&pRoot);

// Get a top-level element by name, such as „Program Manager“ if (pRoot) { IUIAutomationCondition* pCondition; g_pAutomation->CreatePropertyCondition(UIA_NamePropertyId, varProp, &pCondition); pRoot->FindFirst(TreeScope_Children, pCondition, &pFound); pRoot->Release(); pCondition->Release(); }

VariantClear(&varProp); return pFound;}

Listing 3: The FindFirst method here fi nds a particular application window.

If a UI Automation provider does not implement

a property, UI Automation is able to supply a default value

for that property.

74 www.code-magazine.comCreating UI Automation Client Applications

an administrative application the operating system prompts the user for consent to run the application with elevated user rights. Similarly, when a user at-tempts to perform a task that requires administra-tive user rights, Windows presents a dialog box that asks the user for consent to continue. This dialog box is protected from cross-process communica-tion, so that malicious software cannot simulate user input.

Only signed client applications are trusted to com-municate with applications running at a higher us-er-rights level. To gain access to the protected sys-tem UI, applications must be built with a manifest fi le that includes a special attribute. This UIAccess attribute is included in the <requestedExecution-Level> tag where the value of the level attribute is an example only, as follows:

<trustInfo xmlns= „urn:0073chemas-microsoft-com:asm.v3“> <security> <requestedPrivileges> <requestedExecutionLevel level=“highestAvailable“ UIAccess=“true“ /> </requestedPrivileges> </security> </trustInfo>

The UIAccess attribute value is “false” by default; that is, if the attribute is omitted, or if there is no manifest for the assembly, the application will not be able to gain access to protected UI.

With access to the system and target application’s UI, your client applications can navigate the UI, retrieve properties and control patterns for UI ele-ments, and listen to relevant UI events.

a client can listen for focus change events on all UI Automation elements in the tree, or on just one ele-ment and its descendants.

Client applications subscribe to events of a particu-lar kind by using methods such as AddAutomation-EventHandler or AddFocusChangedEventHandler to register an event handler These methods take a callback interface, which should be implemented in an object created by the client. A method such as IUIAutomationEventHandler::HandleEvent will then be called on the appropriate thread whenever the event occurs.

On shutdown, or when UI Automation events are no longer of interest to the application, UI Auto-mation clients should remove the events. They do so either by calling a removal method for specifi c events such as RemoveAutomationEventHandler or RemoveFocusChangedEventHandler, or by calling the RemoveAllEventHandlers method to remove all events.

UI Automation and Screen Scaling

Starting with Windows Vista®, users can change the screen resolution (in dots per inch or dpi) so that most UI elements appear larger. Previously, the scaling had to be implemented by applications. Now, the Desktop Window Manager performs de-fault scaling for all applications that do not handle their own scaling. UI Automation client applica-tions must take this feature into account.

The UI Automation API does not use logical coor-dinates. Methods and properties either return phys-ical coordinates or take them as parameters.

By default, UI Automation applications that run at a screen resolution other than 96 dpi will get incor-rect results from these methods and properties. For example, because the cursor position is in logical co-ordinates, the client cannot simply pass these coor-dinates to the IUIAutomation::ElementFromPoint method to get the element that is under the cursor. In addition, the application will not be able to cor-rectly place windows outside its client area.

The solution to this situation has two parts:

Make your client application aware of screen • resolution by calling the Win32 function Set-ProcessDPIAware at startup. This function makes the entire process aware of screen reso-lution, so that no windows belonging to the process are scaled.

Call • GetPhysicalCursorPos to get the current cursor coordinates.

User Account Control and User Rights

Finally, you need to consider User Account Con-trol and user rights to ensure your client can access the information it needs. Before a user launches

Matthew Karr

The properties of an element are generic to all elements and not

specifi c to a control type. Control patterns,

discussed later, expose control-specifi c properties.

codefocus acessibility web

Documents