AISN

http://www.cs.bgu.ac.il/~royif/AISN

Auditory Imaging for Sightless Navigation

Final PresentationMay, 2011

Academic Advisor:Prof. Yuval Elovici

Technical Advisor:Dr. Rami Puzis

Team Members:Yakir Dahan

Vitali SepetnitskyRoyi Freifeld

Project Team

3

General Overview Introduction

› Motivation & Problem Domain› Existing Solutions› Our Solution› Project Goals

System Architecture› General Diagram› Description of modules – A closer look

Technologies The system in action …

Introduction

4

5

Introduction Motivation

Most of our navigation in the everyday life heavily depends on visual feedback that we get from our environment

When the ability to see the surroundings is missing due to visual impairments, the ability to navigate is also damaged

Physical sense: White Cane Guide DogDrawbacks:

• Limit the freedom of their owner• Guide dogs are expensive to train and intended to

only a small part of the blind community

Sensory substitution: Warning of obstacles (e.g. Optical Radar) Sonar-like images scanning (e.g. The vOICe)Drawbacks:

• Requires an intensive training and is hard to get used to, due to a non-intuitive conversion between image and sound

6

Introduction Existing Solutions

A combination of visual information processing and 3D sound creation and positioning:

Taking a stream of frames Processing the frames and

retrieving visual information Creating appropriate sounds Performing an auditory

spatialization of the sound7

Introduction Our Solution

Sightless navigation by sensory substitution

A development of an application that allows a person to navigate, relying primarily on the sense of hearing

Integration with a spatial auditory environment

Providing a flexible environment for future research

8

Introduction Project Goals

System Architecture

9

10

3RD PARTY

PIXEL_T

SOUND_T

System Architecture

11

System Architecture A Closer Look

UI Unit:› Enables the user to

configure the system through a user-friendly GUI

Core Unit:› The system’s engine› Responsible for:

Initializing the system Running all the components Communicating with hardware and

database

12

System Architecture A Closer Look (cont.)

Image-Processing Unit:› Receives images

obtained from a camera and retrieves visual information

Sound-Creation Unit:› Responsible for creating appropriate

sounds according to the visual information

Sound-Positioning Unit:› Performs auditory spatialization of the

sound in the 3D space

System Architecture A Closer Look (cont.)

General Database:› Contains the data

relevant to an interaction of the system with users e.g. Profiles Data about installed technologies

Technologies Data Storage:› Contains the installed

technologies used by the system

14

Technologies

15

The application was designed for Linux OS

Code written in C/C++

Eclipse IDE & Qt SDK for GUI

Handling HRTF data by calling MATLAB routines from the code

Database by MySQL RDBMS

Used Technologies & Platforms General

Integration with Primesense 3D camera using OpenNI API

Image processing by OpenCV library

Sound positioning by Integration with Sound Scape Renderer (SSR) and OpenAL API

Audio connection handling using JACK audio server daemon

Used Technologies & Platforms Within the system

16

The System In Action…

17

ANY QUESTIONS?THANK YOU

!

18