Audio Programming Assistant

A system that supports blind and vision impaired people to learn C# programming

Team APAPhilip Haines, Ngoc Khuu, Van Tieu-Vinh, Ping Li and

Dr Dat Tran (mentor)University of Canberra, Australia

Outline

• Problem Definition• Our Solution• System Design• System Architecture• System Demonstration• Future Development• Questions

from Imagine Cup 2006

Teaching

Software Design… technology enables a better education for all

Software Design… technology enables a better education for all

Learning

Looking for an idea

ClassroomStudent ???

Material

It may be.. I haven’t seen blind students at

my university. Why?

At Our University

… So what are the Disability Standards for Education anyway? The Human Rights Commissioner, Graeme Innes will give a fascinating address today in 2B9, from 12 - 1.00This is a unique opportunity to hear this eminent speaker, who has an international reputation.

If you know that…

Go to http://www.hreoc.gov.au/about_the_commission/president_commissioners/innes.html for a bio of Graeme, who will be accompanied by his guide dog Jeordie

Research – People with Sight Problems

1.4 million children under 15 are blind

(WHO, 2003)

Every minute: a child goes blind

Blind Vision Impaired

Global (WHO, 2003) 45 million 135 million

Australia (Blind Citizens, 2004) 50,000 480,000

Blind and Vision Impaired People

four and a half times four and a half times more likely to be unemployed than the national average

(Vision Australia, 2007)

Education Level Unemployed

Postgraduates 34%

All Levels 69%

How can technology…

…help them achieve equal opportunity in education

and employment?

Current Technology and Hardware

Vision Australia

Canberra Blind Society

… and Software

Screen Reader

Key Findings (Vision Australia, 2007)

There is a higher need for technical skills amongst people who are blind or vision impaired

The more formats of material they can access, the higher their employment opportunities

What Technical Skill?

I still think it's worthwhile for a blind person to try a career as a programmer

Michael Freeman blind programmer in

Vancouver, Washington

Technical Skill would be…

ProgrammingSkill

Help blind and vision impaired people become programmers

… Our Idea

How to Learn Programming?

WhatSoftware?

Current: Audio Scanner, JAWS, Window-Eyes

WhatSoftware?

No audio programming tool available

How to enter?

Our Solution

C# .NET programming

Edit, Compile , Debug, and Run C# programs

Audio Programming ToolAudio Audio

Studio.NETStudio.NET Provide better access to Web pages,Make websites voice enabled,and more …

Audio Web Browser

Audio Audio Web BrowserWeb Browser

Provide audio learning materialsDownload audio toolsProvide a forum

Audio Web SiteAudio Audio Web SiteWeb Site

Assumptions and Requirements

Blind users can use computer keyboardsBlind users can use computer keyboards

System design: not only for blind and vision impaired users but also for normal vision usersSystem design: not only for blind and vision impaired users but also for normal vision users

Interface design: comply Microsoft and W3C standards for vision impaired usersInterface design: comply Microsoft and W3C standards for vision impaired users

No commercial screen reader software is includedNo commercial screen reader software is included

The Challenge

Current Visual C# programming tool is not an open source Current Visual C# programming tool is not an open source

We cannot ask Website providers to add voice to their Web pagesWe cannot ask Website providers to add voice to their Web pages

1. Audio Studio.NET (Programming tool)

Load or create a project, then edit, compile, debug, run and save the project

Blind users: Voice available on every control

Vision impaired users: Large font

Use short cut keys or just Tab and Enter keys

Values entered

OKButton

OK

200, 400

200

50

Arial

16px

1

Add a Button

Name

Text

Location

Width

Height

Font

Font Size

Tab Index

CCoConConsole

1. Audio Studio.NET

Load or create a project, then edit, compile, debug, run and save the project

Outputs text Outputs text or graphicsor graphics

Outputs text Outputs text or graphics or graphics and voiceand voice

2. Audio Web Browser

Use arrow keys to move up and down

Press Enter to select that link

Use arrow keys to move up and down

Press Enter to select that link

Audio Web Browser

Link is spoken out and highlighted

Link is spoken out and highlighted

Audio Web Site

System Architecture

System Demonstration

Audio Studio.NET

Get C# Lessons

to learn

Download &

install APA System

Audio Web Browser

Audio Web Browser

Have question

Search answers

on the Internet

Practice programs

Upload question

on Audio Website

System Evaluation

Rishi, vision impaired, music student Robert, blind, lawyerSharon, blind, graduate

Costs

Costs

C# SDK and Speech SDK: free

Programming and Web Browser Tools: free

Web Administration: minor cost

Benefits

Benefits

Learning tools for everyone

Higher chance to be employed

Increase number of students

Increase productivities of employees

Accessibility to government & organization websites

Intended Impacts

change policies

Develop voice-enabled

software

change education programs

Equal oppoturnity in education and employment

Future Development

Audio Studio.NET

Audio Website

Audio Web Browser

Text-to-Speech

Speech Recognizer

Braille

Thank you!

Many thanks to University of Canberra,Vision Australia, and Canberra Blind Society.

Many thanks to University of Canberra,Vision Australia, and Canberra Blind Society.

Special thanks to Graham, Paul Brooks, Rishi, Robert and Sharon.