vcs technology release 1 - vis caeca soni · vcs technology methology and basis ... v s is the...

VCS Technology

Release 1.0

VCS Technology Methology and basis

Release .1.0

2 / 47


Release .1.0

3 / 47

INDEX

Coordinate System ............................................................................................................................................................................................. 5

Presentation........................................................................................................................................................................................................ 6

Calibrating the hearing threshold of the listener~user. .................................................................................................................................. 7

How to calibrate the hearing of the listener. .............................................................................................................................................. 9

VCS OBJECT STRUCTURE ..................................................................................................................................................................................10

TURNING SOUND .........................................................................................................................................................................................15

STEPPING SOUND ........................................................................................................................................................................................15

COMPASS .....................................................................................................................................................................................................16

ANNOUNCEMENT OF AREA / ROOM / CHAMBER.....................................................................................................................................16

INVENTORY ..................................................................................................................................................................................................16

RECORDING & LISTENING .........................................................................................................................................................................17

LINEAR SONAR .............................................................................................................................................................................................17

CIRCULAR SONAR.........................................................................................................................................................................................18

TRIDENT.............................................................................................................................................................................................................19

AIR FLOW......................................................................................................................................................................................................20

SUDDEN DROPS, EDGES, ETC. .....................................................................................................................................................................21

GUIDING THREAD ........................................................................................................................................................................................22

HOW DO THEY WORK?...........................................................................................................................................................................23

What is a safe perimeter? ......................................................................................................................................................................29

GUIDING THREAD WITH AUTOMOTION (v.2.)...........................................................................................................................................32

OTHER TOOLS...............................................................................................................................................................................................33

USER PROFILE...............................................................................................................................................................................................33

VCS APPLIED ......................................................................................................................................................................................................34

FIRST-PERSON ..............................................................................................................................................................................................34

CPU ...............................................................................................................................................................................................................34

MENU ...........................................................................................................................................................................................................34

GPU ...............................................................................................................................................................................................................35

SCENE ...........................................................................................................................................................................................................35


Release .1.0

4 / 47

DISPLAY INDICATORS...................................................................................................................................................................................35

GLOBAL SOUND ...........................................................................................................................................................................................35

CREDITS / ABOUT … ....................................................................................................................................................................................36

FORCE BACK DEVICES ..................................................................................................................................................................................36

BRAILLE .........................................................................................................................................................................................................36

VIDEOS / PRESENTATIONS ..........................................................................................................................................................................36

CHARACTERS (videogames) ........................................................................................................................................................................37

HELP ..............................................................................................................................................................................................................37

SOUND CATALOGUE ....................................................................................................................................................................................37

VOICE-OVER .................................................................................................................................................................................................37

INPUT (KEYBOARD / MOUSE / OTHERS) ....................................................................................................................................................38

STEREO SOUND / 5.1 / VCS 3D STEREO .....................................................................................................................................................38

SOUND POOL ...............................................................................................................................................................................................38

BACKGROUND DESCRIPTION (Videogames) ..............................................................................................................................................38

VCS ALGORITHMS .............................................................................................................................................................................................39

ALGORITHM OF (definite/pronounced/ broad/ clear Fields/Bell??) .......................................................................................................39

VCS 3D STEREO / SOUND 5.1 ......................................................................................................................................................................42

VCS SOUND POOL / SOUND FLAGS ............................................................................................................................................................45

WORKING WITH VISUAL IMPAIRMENT and other disabilities..................................................................................................................46

VCS TOOLKIT for programmers...................................................................................................................................................................47


Release .1.0

5 / 47

Coordinate System


Release .1.0

6 / 47

PRESENTATION

VIS CAECA SONI (The hidden power of sound)

VCS refers to a collection of rules, procedures, protocols and routines which permits a visually impaired person and/or blind persons

orientate themselves in a virtual three dimensional space on a computer.

VCS is not an external shell which encloses a pre-existing application to convert it into compatible software. It is a methodology

which must be fully understood to be applied and taken into account at the init ial steps of development, in the process of previous

analysis, as its application implies conceptual alternations which affect the design and final functionality.

VCS is the Research and Development basis of the Project “Entornos virtuales para la integración”, (Virtual environments towa rds

integration), awarded in the conference REMPRENDE 2005 as the most innovative technological project. (Madrid, June 2005).

Initial concepts to be taken into consideration

The quantity of audio information in each scene must be measured, as is normally done with visual information. Excess data can

daze and confuse the listener. On the contrary, poor quality and incomplete information can ruin and impoverish the development.

The audio channel must include music, background sound effects, sounds of communication to the listener and any o ther audible

aspects depending on the nature of our product. It is logical to suppose that a videogame would need a soundtrack and backgro und

sound effects, while a business presentation would only require music, and a 3D positioning map would require neit her of these

elements. What each of these former examples would need, would be the VCS communication channel.

For correct use of the audio channel, it is essential that sound priority be defined. For example, in videogames, music and

background sound effects must be lowered as other audio information is given to the player so as to continue the game. Sounds

defined as background effects could continue following the standard 3D resources, such as 3D SOUND or 5.1. This will avoid

saturation of the communication channel and our only concern will be controlling the volume. This will also provide the desired

background effect. However, for the 3D sounds which we want to serve as orientation, we must make use of VCS technology.


Release .1.0

7 / 47

CALIBRATING THE HEARING THRESHOLD OF THE LISTENER~USER.

Neither STEREO sound nor SURROUND sound or 5.1 sound exists.

All sounds are MONO in Nature, with a single focus or origin. Sound is produced and it propagates through waves in a spherical

expansion which are alternated by obstacles it encounters.

These expanding waves create spheres which continually increase in size and become weaker until they eventually disappear

diffused by the surroundings.

Our brain is only capable of processing the information that comes from our ears in a sequential manner, one by one, (first one then

the other), but it is processed so quickly that we have the sensation of it being a parallel process.

The same process occurs regarding the frequencies heard by each ear. We only actually process one frequency of the sound we hear

at any given moment, then another, and so forth until all frequencies have been processed. This audio sweep is produced with such

speed that it gives the sensation of polyphony. Hence, algorithms of digital compression exist such as the mp3 or wav.

The spatial location of the origin of a sound would not be possible without the existence of ECHO. Every source of sound produces

an infinite swell of waves which follow the original sound, originated by ricochets, reverberations, etc. These waves reach our ears

sequentially and permit our brain to calculate by approximation, the location of the source of the sound.

In fact, without the ECHO and the swell of waves produced it would be impossible to determine the location. If a person were to

walk blindfolded in the desert, (terrestrial location with extremely limited ECHO), and heard something fall in the sand close by, one

could only determine if the fall were to the left or to the right, but would not know if it were in front or behind.


Release .1.0

8 / 47

This is due to the fact that the brain can only determine this data only according to the intensity of the sound signal in eac h ear. It

needs the data provided by the posterior swell of waves to be able to establish, (with small head tilt to modify the orientation), the

origin of the sound. In fact, many animals move their ears to provide their brain with this information when they hear and so they

evaluate the intensity and origin of the sound signal.

Therefore, if the measurement of the intensity of the sound is the basis of calculation to locate the source, it is logical to assume

that everything which affects this process is decisive

Each person hears differently, not even each ear hears the exact same information and our hearing ability changes on a daily basis.

Furthermore, each person’s acoustic threshold is distinct. Factors, such as a common cold, an ear infection, etc, influence the sound

measurement process in a decisive way.

3D sound systems, background or surround sound, already exist with various names in today’s market. These systems, such as the

mp3 player, are created so that optimal use is within the common thresholds of hearing. For this reason they are a good acous tic

complement for audio-visual productions such as videos, videogames or presentations. They permit the creation of generic sound

effects in a space. However, as they are adjusted to the common hearing threshold, (as with any other ergonomic products), th ey

cause diffuse low-quality sound and are not appropriate as an efficient method of spatial orientation through sound.

Based on what has been previously explained, we can see the necessity of adjusting sound rules to the real and present audio

capacities of each listener, and even these capacities may not be valid for that same person some days after the initial measurement

The following graph shows how a standard product, such as 3D SOUND or Surround (provided by the operative system by default), is

adjusted. A listener is positioned in the centre of a scene and a sound is emitted from a determined spot, in front of the listener and

to the right. The origin of the sound is seen as a point in the upper-right side of the graph and the shaded area represents where the

majority of listeners will position the sound using default sound rules.


Release .1.0

9 / 47

This range of results is valid when the sound is used as a complement to the atmosphere of a scene, a diffuse effect which do es not

require precise positioning of a sound.

The algorithms of calibration for VCS technology level out and equalize the differences which exist among listeners.

Once the listener’s hearing has been calibrated, the rules will permit our orientation and our ability to pinpoint the origin of the

sound signal in comparison to a previous vague positioning. Consequently, VCS technology is an effective and adequate tool for

accurate positioning through sound.

HOW TO CALIBRATE THE HEARING OF THE L ISTENER.

The importance of this process has been illustrated in the previous chapter.

Ideally the listener/user should calibrate their hearing each time they use the program, however, users will probably find this

routine monotonous, but we recommend it be done a minimum of two times per month. In reality, what we are doing is a hearing

test and then inserting this data into a program which adjusts to your hearing capacity at any given time.

The calibration process involves a series of steps:

The first step consists of having a sound swing from side to side and asking the listener to centre the sound. After various test runs

(5 test runs recommended) of this exercise we know where the centre of the listener’s hearing balance is. The most typical method

of evaluation makes use of a program which does the various tests on the listener and takes the average of the three most similar

measurements.

It is convenient to measure the listener’s sensitivity to an arc of frequency to identify the upper and lower auditory thresholds. No

frequency outside of these thresholds should be used. All frequencies used in the rules and tools should be rev ised and there should

be a warning as to any incompatibilities.

In the next step we proceed to measure the audible volume of each ear and its sensitivity. This will determine the limits of pitch and

acute sounds which must be taken into consideration in the tools set.

Check that the listener responds positively to standard routines of sound effects such as 3D sound or 5.1 sound.

Using examples, we must then measure the listener’s sensitivity in differentiating dull sounds, therefore obtaining a parameter of

dullness/opacity which will be used in the algorithm of 3D VCS stereo sound. (The concept of acoustic opacity/dullness will be

further explained at a later stage.)

If an EAX system is installed, ensure that the listener is using it correctly.


Release .1.0

10 / 47

VCS OBJECT STRUCTURE

VCS technology links objects of a scene to a collection of attributes. These attributes are adapted to be used by the different

routines, processes, protocols, etc which the technology consists of.

These attributes are not limited to individual objects or a collection of groups; they can also be applied to defined physical areas,

specific coordinates or events. The number of attributes varies according to the nature of the object, or of the group, or area as well

as the type of project or program which requires the use of VCS technology.

The attributes provide the opportunity for interaction; they allow for the possibility of having an oral description of the object/area,

etc., of having a step counter, a scoreboard of energy/achievements (i.e. in videogames) or any other function we wish to apply or

associate with an object/area, etc.

The following chart describes the majority of these attributes; however they can be amplified/changed/separated as seen fit.

Name Description

Obj_id Object Identifier.

This is the object identifier for the program. It is the link between VCS and the collection of 3D objects of the

application. It is usually a code which identifies the 3D object.

Obj_Aux Object Aux

It is usually a short text which identifies the object and often contains the name of the 3D file.

Obj_name Name

Internal name of the object for all VCS modules. It is not unique by definition but should be treated as such.

Snd_des_id

(1..n)

Sound Descriptor Identifier

This is a recording of the description which can be heard on request using a tool and/or routine. There can be as

many (and as much information) as we wish to include in the application. For example, a voice that says “Door”,

or a specific Beep sound that we associate to a door. This field contains the sound identifier for the system. It is

usually a code.


Release .1.0

11 / 47

Snd_des_Aux

(1..n)

Sound Descriptor Aux

As with the previous structure, this attribute usually includes information about the language of the message of

the identifier, or the name of the audio file which has been loaded in this position.

On occasions, instead of loading various Snd_des_id , only one audio file is loaded with the sound descriptors

within. In these cases, this variable collects the information regarding its starting position and duration. For

example:

“Eng”

“Alarm.Wav”

“Alarm.Wav/0/20/21/3/24/40”

Txt_des (1..n) Text Descriptor

Similar to previous attribute but contains a series of texts which are literally read by the system’s Speech Engine,

if available.

Txt_des_Aux

(1..n)

Text Descriptor Aux.

Identical structure/array to the previous attribute, with texts associated to the first Text Descriptor. Normally

contains the language code of the text to be read.

Snd_Active_Id

(1..n)

Sound Active Id

This is the code of a sound loaded in memory which can be independently reproduced simply by using the

algorithms of distance. This may be perhaps the sound of a fountain which can only be heard at a certain

distance. It is basically an array as there are objects which can have various sounds, for example a fighter plane

which would have one particular sound when it is flying well, a distinct sound when it has been hit and another

when it is falling. These are often sounds not voices.

Snd_Active_Aux

(1..n)

Sound Active Aux

An array parallel to the previous one, with extra information such as the sound file or the description of the

sound, or the circumstances which cause the sound to appear, etc.

Snd_Passive_Id

(1..n)

Sound Passive Id

This is the identifier of a loaded sound which is only heard upon interaction (i.e. interaction with the object). For

example: the sound made when calling at a door.

Snd_Pasive_aux

(1..n)

Sound Passive Aux.

Similar array to previous ones described.


Release .1.0

12 / 47

Text_Title (1..n) Text Title

This is a text associated with the object and seen as subtitles, or specialized texts of descriptions, as might be

needed for people with partial visual impairment where a magnified text can be used.

Braille_Txt

(1..n)

Braille Text

As its name indicates it is a formatted series which is sent to the Braille device on request of the application.

CAT/FAM (1..n) Category / Family

This is an internal code which permits the objects to be grouped together in Families or Categories, depending on

the use we wish to give an object.

ForceBack_Str

(1..n)

ForceBack String

This is a control string which is sent to the connected devices which FORCE BACK technology incorporated.

Obj_Status Object Status

This states the status of the object.

Obj_Color Object Colour

This contains the colour code of the object.

Obj_Image

(1..n)

Object Image

Code of the image in the system or of the image file, which can be used in display units, etc. based on its use in

the application.

Obj_Url_link Object Url Link

For applications which associate the object with an URL.

Obj_Counter

(1..n)

Object Counter

This can contain the number of times of interaction with object, or times the object has been passed, or times the

object has emitted a sound, etc.

Obj_Timer

(1..n)

Object Timer

For timing devices, such as a bomb (count down), or time until something is activated (for example a robot), etc.

Obj_Mem_Size Object Memory Size

The amount of memory that an object needs.


Release .1.0

13 / 47

Obj_Eax_param

(1..n)

Object Eax Params

To link EAX parameters to an object.

Obj_Vol Object Volume

For objects which require a special volume.

Obj_Pan Object Pan

For objects with special panning.

Obj_St_Typ Object Stereo Type

Object stereo mode ( if needed ).

Obj_Shape Object Shape

Can be used for objects without a visible shape such as a whirlwind or a beam of light. Using this, we can indicate

to the program an object which can be represented on programs such RADHARC3D.

Obj_record_Id Object Record ID

VCS offers the possibility of voice recording. The recording can then be associated with an object, a coordinate or

an area so that the recording will be reproduced when the area/object, etc. is passed (or nearby). This recording

is associated with a sound program.

Others Others

Depending on the use or needs, any amount of structures can be created.

… …

All the above attributes should be stored in variables within the structures of the programming.


Release .1.0

14 / 47

VCS ROUTINES / ALGOR ITHMS / VCS TOOLS

The following information describes the tools & main routines of the VCS nucleus which provides the listener with all the data

describing the 3D environment they are in.

IMPORTANT NOTE ABOUT THE ROUTINES USED:

The majority of the routines describe processes in which sounds, voices or varied files are emitted. All of them can be substituted

and/or be used to complement the acoustic outlet with signals sent to devices such as BRAILLE DEVICE or FORCEBACK input

devices. This is NOT specifically stated in each case but must ALWAYS be taken into account.

The sounds used are not all freely emitted to the system’s buffer. Depending on their type, the sounds can be sent to the VCS sound

pool server or directly to the system buffer.

All the routines should be configured so that they may be interrupted at any time on request, thus clearing the buffer at that

moment. Voices are interrupted by pressing the CONTROL key. If the sound has been sent to the sound pool, the pool will control

the interruption.


Release .1.0

15 / 47

TURNING SOUND

Turning on the vertical axis (Y), the listener receives a sound signal every T degrees; sounds which can be made distinct depending

whether you are turning left or right.

As an option, it is possible to define special sounds every time certain angles are reached, i.e. 0º,90º,180º, or 270º.

In addition, it is possible to consider angles which are relative to where you are facing, or to use the cardinal points or the hands of

the clock as a reference as to where we are moving.

STEPPING SOUND

This is applied in VCS maps, videogames using VCS or in other types of 3D maps/plans using VCS.

As a starting parameter, the length of the user’s stride is inserted. A sound is emitted every time that distance (one full stride) is

reached, with a different sound to distinguish between the right and the left foot.

Similarly, the sound varies if the user/listener is walking, running or walking stealthily. These parameters permit other routines/tools

calculate possible collisions, depending on the number of strides left to reach an object.

It is appropriate to create an interface in the setup menu so that the user/listener can insert their own parameters, particularly in

the case of 3D VCS maps.


Release .1.0

16 / 47

COMPASS

By use of an associated key, or any other trigger, a sound is emitted, or a voice, which indicates the position in relation t o a NORTH

which has been previously defined. There are various modes and the voices depend on the language chosen:

.a) Traditional voice: “You are facing South…” (N-S-E-W)

.a+) Similar but with 8 points (N-NW-W-SW-S-SE-E-NE)

.a++) Similar but using the hands of the clock (12-3-6-9/1-2-4-5-7-8-10-11)

.b) Any of the above mentioned, but substituting the voice for a sound previously decided and associated to this

event.

ANNOUNCEMENT OF AREA / ROOM / CHAMBER

This can be requested by pressing a key and/or using a routine. It indicates where the listener is by using the VCS attributes of the

area/room we are located in. “ You are on a green bridge…”

The second time this information is requested, a more detailed description is given : “You are on a green bridge beside the castle of

Volgor”.

And so on.

Normally the SOUND DESCRIPTOR is used, but this can be programmed and configured for every project.

This usually comes accompanied by other extra information such as “You are on a green bridge. You have already been here”. It is

common to use this every time the listener/user passes from one area to another, regardless of ambient sound effects.

INVENTORY

This is typically used in videogames but is also very useful in VCS 3D maps.For example, so that the user can follow a “To Do” list.

This is usually done by voice sounds which are obtained from the information found in the VCS descriptors.


Release .1.0

17 / 47

RECORDING & LISTENING

VCS permits the listener/user record their voice in any 3D coordinate of the scene and link this sound to an object or area , where

possible. Using a microphone, the user can make notes or leave reminders such as “Careful with step”, “There is a dragon behind

the door on the right”, etc.

Once the voice has been recorded, anytime the user approaches the point where the recording was registered, the system

reproduces the sound and/or displays a menu where they can delete it, re-record it, silence the message, etc.

LINEAR SONAR

With a small angle fluctuation defined by the system and using a maximum distance which are also defined by the general

parameters, this tool, activated on request or pressing a key, simulates the sonar of a ship in a straight line.

From the position of listener/user, we can draw a virtual straight line whose length corresponds to the maximum distance and with

the previously mentioned fluctuating angle.

If an object crosses the path of this line, the sound descriptor will reproduce a sound. This sound will vary according to th e distance

between the object and the user/listener. For example “You are 3 strides away from a sofa”.

The most common case would be to describe only the first and nearest object to cross this virtual line, however it is possible, when

necessary, to describe all the objects which cross our path, stating the objects from nearest to furthest.

As voice recordings are slow, it is usual to change the voice descriptor for a shorter sound which permits objects to be identified

with greater speed. For example, the typical “Knock, Knock,” sound of wood when knocked on, could substitute the voice message

of “Door”.

If we loop the sonar, the number of times that an object emits a sound will give us the reference to its distance from us. An increase

in the number of times it emits a sound, the closer we are and the less frequent the sound emission are, the further we are from the

object, (as is used in car to assist parking).

Another variant would be to use the sonar but at a different angle

to frontal position ( 0º).

A further variation would be that the actual sound emitted varies

according to the distance from the object - the sound using a

higher or lower pitch the nearer or further we are.

If we choose the option where all the objects in our range are

described, it is advisable to provide the option of

interrupting/stopping the sounds on request.


Release .1.0

18 / 47

CIRCULAR SONAR

This is identical to linear sonar, except that it makes a circular sweep. We turn with a rotation set by parameters which can be 360 º,

or 0~90º/270º~360º.

We must take into account the same premises as before (with linear sonar) but also considering the following:

Before giving the information we must indicate the angle, for example “To the south, at three strides, there is a barrel”. In this case,

we should also take into consideration the option of changing voice messages for identifying sounds, not just for object sounds, but

also sounds associated with the angles.

It is recommended not to use the option of listing all the objects in the given radius as this process will be slow and tiring as it is

repeated with every set rotation we make. (The option of interrupting/stopping description should be included).

When, or if, substituting the voice messages for quicker sounds, these sounds will have to be identified according to the rot ation we

make (or angle we are facing) and as we move in a certain direction. In this case, a double Beep could help us identify the angles at

our rear (non-frontal angles).

Likewise, we could use the guides N-S-E-W, N-NW-W-SW-S-SE-E-NE,12-3-6-9/1-2-4-5-7-8-10-11 or any other guide we see fit.


Release .1.0

19 / 47

TRIDENT

This is a variation of circular sonar but it multiplies its effect and is applied to three fixed set angles, one at 0º head-on and another

two lateral angles possibly being 270º and 90º. These angles can vary and use other angles instead of right angles such as

0º/315º/45º.

Only the first object is detected by the sensor and alternative sounds are normally used instead of voice messages.

A specific parameter of the system registers the range.

On this occasion, the angle indicator is not used. Instead we use the balancing system or panning to locate the sound focus.

Using this method, sounds emitted at left angles will be 100% audible by the left ear and sounds originated from right angles will be

heard 100% by the right. Meanwhile head-on frontal sounds will be heard by both ears at the same time.


Release .1.0

20 / 47

AIR FLOW

The use of a property of a VCS object as a tool. If an object is used as an exit or a door, we deliberately produce the sound of a

draught or air FLOW.

The 3D sound is positioned as a 3D ambient sound effect thus facilitating the listener’s ability to orientate themselves and find the

path to be followed.

No more than two air currents should be heard at any one time to avoid confusion. The sound volume will indicate the distance

from the exit.

We are simply making use of a sound which naturally provokes the sensation of exiting/move through from one area to the next .


Release .1.0

21 / 47

SUDDEN DROPS, EDGES, ETC.

By calculating the next step, the program identifies when the listener/user is nearing an area which implies danger, such a s a steep

drop, or an unexpected step, and can deliberately make the next, potentially dangerous step more difficult to take, which serves as

a warning sign for the user when advancing towards this area of danger

This is achieved either by using sound signals (voice or other) or by devices such as FORCEBACK devices or Braille line.


Release .1.0

22 / 47

GUIDING THREAD

This consists of creating a net of linked points which establishes ‘paths’ that the listener can follow safely. These paths, to be

considered ‘safe routes,’ are adjusted to the surface of the scene and avoid conflictive zones.


Release .1.0

23 / 47

HOW DO THEY WORK?

Every time the listener reaches a point on the guiding thread, a sound is emitted. If the listener turns around on the vertic al axis (y),

instead of going straight ahead, when they are facing a connected point another sound will be emitted and will continue as long as

the listener is facing this point, and also while they are advancing towards it. It is important to follow the steps previously

mentioned in the section referring to calibrating the listener’s hearing in order to be able to face the new point on the guiding

thread.

It is important that no more than two or three points be directly connected and that the associated points be at wider rather than

narrower angles; the wider the degree of separation the better:

The sound of the points can provide extra information. For example, the point that we are coming from should sound different to

the other points around us, and it is possible to make a point sound different if we have already been there, or passed by it. One

quite useful variant, particularly in videogames, is that the sound varies between points that lead us to the adventure and points

that lead us away from the action.

With reference to the sounds, we should not limit ourselves to a single ‘Beep’, and remember that a sound can be a sequence of

many Beeps.


Release .1.0

24 / 47

To construct a Guiding Thread it is necessary to have an array of N elements, a second array with N elements, and M connections,

where M stands for the maximum number of connected points. This is merely the basis. It will be necessary to use other parallel

arrays to give us other groups of information such as counting the number of paces/strides, active and dormant points, etc.

For example, let’s look at the following scene

We can see two bedrooms, a winding path which leads to a fountain. We have drawn the guiding line with 28 occurrences which

have been connected.

We have an array with 28 occurrences which will contain the coordinates of the points, and another of (28,3) for linked points:


Release .1.0

25 / 47

TABLE A: Positions

Point X Y Z

1 X1 Y1 Z1

2 X2 Y2 Z2

3 X3 Y3 Z3

4 X4 Y4 Z4

…

28 X28 Y28 Z28

TABLE B: Links

Point R1 R2 R3

1 2 4

2 1 3

3 2 8

4 1 5

5 4 6

6 5 7

7 6 8

8 3 7 9

9 8 10 13

10 9 11

11 10 12 14

12 11 13

13 9 12

14 11 15

15 14 16

16 15 17

17 16 18

18 17 19

19 18 20

20 19 21

21 20 22 28

22 21 23

23 22 24

24 23 25

25 24 26

26 25 27

27 26 28

28 21 27

TABLE C: Visited?

Point VISITED?

1 0

2 0

3 0

4 0

…

28 0


Release .1.0

26 / 47

Furthermore, we will define various sounds:

BONG When the listener reaches a point

TUN When the listener faces a point not previously visited

TIN When the listener faces a point already visited

In this example we do not differentiate if we are facing the point of origin or the way towards the ‘adventure’, we only differentiate

if the point has been previously visited or not. For this, will need another third table with 28 occurrences and every time a point is

reached it will be registered in the table.

We begin by placing the listener at point 1, hence we will hear a BONG sound and we will put the number 1 in the third table beside

occurence1.

The listener rotates without advancing forward.


Release .1.0

27 / 47

If the angle in relation with any of the connected points is less than the angle previously set in the parameters, we will hear a TUN

Likewise, when on the way from point 1 to point 2, if we remain facing the latter point, we will hear the sound TUN. We could also

include the distance as a factor of repetition, giving more information in this case.


Release .1.0

28 / 47

Yet if we turn face point 1, we will hear TIN

As a consequence, the listener will always have knowledge of their orientation and even if the listener moves and tempo rarily loses

their orientation, they can stop and turn around until the recover their position facing in the correct direction.

As we reach point 2, we will hear a BONG sound and we will insert number 1 in the flag belonging to element 2 of Table 3. This

process will be repeated with all the connected points we visit.

The information given can be enhanced. In addition to the BONG we hear as we reach a point, we could also produces beeps

corresponding to the amount of points connected to this new location we have reached.


Release .1.0

29 / 47

WHAT IS A SAFE PERIMETER?

This is a concept associated with the Guiding Thread, just one of its multiple options.

As the listener/user can move freely around the scene, it is possible that they move so far away from the Guiding Th read that they

are incapable of re-finding it to continue on their path.

One solution to overcome this problem would be to resituate the listener, upon their request, at the last visited point, by providing

the pertinent sound messages to find their way.

However, in a videogame this is not very realistic and so a safe perimeter is used. Let us suppose the following situation where the

listener has passed beyond the Guiding thread lines and cannot reach them again even by facing the next point:

If the listener faces the next point and advances following the TUN TUN sound, in this case, he will fall into the hole.

For this reason an auxiliary sound to TUN is created, for example TUN* which indicates that the listener is facing a point however

the path/route there is not safe. Therefore the listener can then attempt to locate their point of origin in order to return there

safely. However, it could be the case that this point is also in similar circumstances to the point of destination (i.e. the path there is

surrounded by danger). The listener is lost and it will be necessary to use other tools such as linear sonar to return to the Guiding

Thread lines.


Release .1.0

30 / 47

To avoid these circumstances, the concept of a safe perimeter is used. It is an area parallel to the Guiding thread which guarantees

the safety of the listener while within this area.

When the player abandons the Safety Perimeter (SP), they are warned by means of a voice message or sound signal advising them

to return as soon as possible.

To calculate the SP, a simple trigonometry formula is used. Let us suppose that the width of the SP has already been set by a VCS

parameter that we shall call SPS.

If the distance of the listener to the nearest right angle of the Guiding Thread, is greater than SPS, then a message is generated.


Release .1.0

31 / 47

This distance is obtained by calculating the height of the triangle formed by point 1, point 2 and the position of the listener.

It is also possible to calculate the new position and impede the listener from leaving the SP, but this restricts freedom of movement.

Generally both the Guiding thread and the SP are used discarding one of the 3D dimensions, in other words, using only two of the

three axes of coordinates. However, it is possible to be used in complete 3D. In this case, it is recommendable to use a maximum of

one connecting point for every side of the virtual cube which surrounds the listener.

In 3D structures, we can use a fixed XYZ or we could use XYZ connecting point coordinates, however this greatly complicat es control

and management structures.


Release .1.0

32 / 47

GUIDING THREAD WITH AUTOMOTION (V.2.)

This is a variant of the Guiding Thread. The listener is not permitted to leave the path of the Guiding thread. Therefore, th is is

adequate for videogames but not for VCS 3D maps.

Taking into account that walking or moving around the scene should not be an advantage to players without visual disabilities, the

process of moving from area to area for listeners/players is controlled directly by the system.

This way, the listener decides when to advance. Then the character moves but only as far as the next point on the Guiding thread.

We will hear the associated beep sound indicating we have reach a point, and if there is no crossing paths or another alterna tive

point, the player continues until reaching the next point, where this process will be repeated. The listener can always stop

advancing.

In the case where the point we have reached offers various alternatives, the listener will stop at this point and by rotating on the

vertical axis (Y) they will have knowledge of the distinct routes within their reach. These rotations are guided by sound, as described

in previous sections.

In cases where there are only two alternatives, left and right, we can make the process easier, where by pressing a selected key we

follow the path it corresponds to. Likewise, the same can be done with the down key (to go backward) which would permit us to

turn and return to the previous point.

As with the previous algorithm, we the information given can be enhanced, informing the listener about the options with their

reach, describe points previously visited, etc.


Release .1.0

33 / 47

OTHER TOOLS

A videogame usually consists of an infinite number of mini-games which require individualized options. For example, option to

select objects from an inventory, an ammunition counter, energy counter, etc.

Common sense is the only rule to be followed here; the communication channel must not be saturated and i t is necessary to do a

thorough check-up of the variants and alternatives of the tool created.

Each option or the combination of all of them, should have the possibility of being configured by/for the list ener according to their

audio capacity at the time of playing the game.

USER PROFILE

As the calibration of the user must be personalized, VCS programs must offer a profile management system which permits us load

and record all adjustments associated with each player.

It must also be possible to export these profiles to other programs which use VCS technology, independently of who manufactures

the program. These layouts are subject to the version of the VCS technology they use.

The profiles should be kept in a file which is easily transferred and which the user can carry with them or download from an internet

website.

It is important to note that each application should allow for the voice recording of the user and the parameters which have been

previously configured.


Release .1.0

34 / 47

VCS APPLIED

From the information previously provided, we can deduce that VCS, apart from being a collection of algorithms, protocols, routines,

etc., is based on using logic and common sense to achieve a determined outcome, which is usually the description of a 3D

environment through the audio channel.

(¿Esto? ha de ser llevado a extremos). When taken out of context, much of the information, such as the Guiding thread or Circular

sonar, do not seem logical, but within our area of technology, the rules and routines become efficient and creative solutions ,

necessary to achieve our purpose.

FIRST-PERSON

As is patently obvious VCS technology implies that the sound is ALWAYS reproduced in first person in relation to the listener,

independently of position of the camera, if a camera exists.

This is a simulation exercise. In VCS tools it is habitual that the camera follows the character in third person, or moves freely around

the scene, but the sound is emitted in first person.

This asynchronicity between image and sound aids persons without visually disabilities, have a better understanding of the

technology.

CPU

One factor which must be taken into account is the limitations that many computers have, especially since we are working with 3D

technology which demands a high level of resources. Many visually impaired/blind persons do not have computers capable of

managing graphic 3D environments, due to lack of hardware, (poor graphic cards, etc.). When programming, it is fundamental to

take this into consideration and allow for the technology to be configured for its use on computers with a low CPU.

MENU

Menu management/administration is important and any special mentions require user interface. The program must adjust to the

visually impaired user, following laws and current rules in force, set by the National Council for the Blind or other. The pr ogram must

permit dynamic navigation of the options; how to replay and/or stop a recording, return to menu, skip an option, etc. Instructions

should appear in the language indicated in the profile of the user.


Release .1.0

35 / 47

GPU

In regards to the graphics (and taking into account previously mentioned CPU information), it is not necessary to renounce or

forsake the maximum level of graphics possible. VCS is a tool with the intention of integration. Its finished products should be

attractive to all players/users, independent to their visual capacities. Until now, products designed for the visually impaired/blind

have little allure or interest for others.

Taking into account the limitations apparent in the majority of products using technology for the visually impaired/blind, th e VCS

tools should use an ample range of graphic configuration, which offers zero visualization (without use of a monitor) to highly

advanced graphics (with use of shaders and all other visual options of cutting-edge technology). Option to be configured is

necessary.

SCENE

In each scene there should be a mix of passive and active elements, a layer of background sound effects and most likely backg round

music.

This background music will be the first to be sacrificed in the case of acoustic saturation if VCS is controlling the sound, as there

should always be the possibility of lowering the volume or cancelling the sound if the circumstances require so.

The background sound effects will be next in the priority order (¿¿que puede resultar alterado?? – alterado el orden o el sonido?.

On certain occasions, like entering a new area, this sound is heard but for a limited and short time so as to keep the audio channel

free.

DISPLAY INDICATORS

Display indicators must be complemented by sound indicators, activated on request, and which will provide the same information as

that seen on screen. Likewise we can define alarms/warning signals, for example when your energy or amm unition is running low.

GLOBAL SOUND

There should not be too many active sources of sound at any given time. As previously explained, tool messages will sound

independently to background music and sound effects but NO more than 3 passive sounds should be heard at once

It is fundamental that a GLOBAL parameter regulates the volume of ALL the sounds of the application.


Release .1.0

36 / 47

CREDITS / ABOUT …

All other common information/routines as found in other software, must be adapted so as to be also heard through the audio

channel (or is it that alternative information can also be offered???).

FORCE BACK DEVICES

All VCS objects/products are designed to be able to interact with these types of external devices, such as joysticks or game pads.

They can respond with active or passive commands.

VCS products must offer additional information for the listener/player, for example the proximity of an unstable floor, a sudden

drop, etc. It is even possible to improve this by linking the devices to the guiding thread lines.

BRAILLE

Similar to the previous section, all VCS objects and events can send a sequence of information to a Braille line if connected to the

computer

This property should be adjusted along with the drivers of the hardware manufacturer.

VIDEOS / PRESENTATIONS

Any video sequences must be very well prepared, ensuring that all information is also transmitted by the audio channel, and if

necessary, by subtitles too.

All videos should be VCS objects, managed by the technology and all its attributes.

This implies that traditional videos seen in videogames or other multimedia products, do NOT comply with this premis e and hence

are not compatible, except when VCS rules have been applied from the beginning of their creation.

VCS technology proposes to substitute these traditional videos for animated flash type sequences, with animated images and a

complete narration of the sequences. This narration should not be literal, but more like a novel as told on radio. This permits the use

of comic type animation used to show the information and which can subsequently be used in the program due to its graphic

content.

When creating this sequence of animated images it is important to note that it will not be possible to make abrupt camera and/or

scene changes as some visually impaired persons are very sensitive to such sudden changes. High contrast images with clearly

defined outlines will be used and will appear on screen in logical order.

The soundtrack of these videos should be of a similar style to old radio serials, capable of providing all the necessary info rmation

without a visual display and with superior use sound and acoustic special effects.

We must take into account the possibility of a multi-language program, should it be required in the finished product. This will entail

special care when synchronizing image and sound for each language option.

Likewise, we should take into account the use of subtitles, adjusted to each language option and braill lines, if required.


Release .1.0

37 / 47

CHARACTERS (VIDEOGAME S)

Characters are also treated/regarded as objects, with their own identifying sound. We must make use of our imagination and

carefully orchestrate their first appearance perhaps with the use of an introductory video, a voice recording, etc.

In the Help option, there should be a description of the character and its sound descriptor so that the users/listeners learn to

associate them with this each character.

HELP

The Help option should consist of a voice recording, and be much more comprehensive and detailed than traditional Help option s.

i.e. must include an explanation of each and every VCS tool incorporated in the product, and the workings of each tool. All

explanations must be accompanied be the associated sounds.

Evidently, multi-language use and Braille to be taken into account.

SOUND CATALOGUE

The Help option should include some type of sound catalogue which describes and identifies all of the product’s sounds.

It must be a combination of voice and sound, with an outlet for subtitles and Braille.

The catalogue should be organised and categorized so that the listener/user has rapid and easy access to the sounds.

The content may be variable, as it depends on when and from what area the user/player needs access to the catalogues (i.e. if

access is from level 2, catalogue will not reveal information/sounds from level 3) variable Puede incluir contenido variable en

función del área desde donde se llame o del nivel del juego, p.e.

VOICE-OVER

Eventhough the objects have sound descriptors which are activated at certain times, the use of a voice-over or a subtitle offering

information may be required underother circumstances.

For example, a warning giving information about a particular situation/event, a count-down in a game, etc. (Also necessary in

Braille).


Release .1.0

38 / 47

INPUT (KEYBOARD / MOUSE / OTHERS)

It must be possible to access all the controls of the program from a keyboard. Even when other devices such as a joystick or

gamepad are used, there should be keys which can perform the same function.

It is not necessary to get rid of the mouse while using the program, but as with other devices, the functions of a mouse shou ld have

an equivalent on the keyboard.

STEREO SOUND / 5.1 / VCS 3D STEREO

None of the above mentioned methods are used exclusively by VCS technology, rather it uses a combination of all to achieve its

objectives.

The majority of VCS tools/routines, use the appropriate/normal channel for/of communication through the VCS pools, but in the

case of 3D positioning, we must use the routine of VCS 3D STEREO.

The background sounds and 5.1 will continue to use the system’s resources.

NOTE: For correct use refer to sound routines for VCS 3DSTEREO.

SOUND POOL

Through mangement or sound and message pools, VCS controls and ensures that the audio channel of communication does not

become saturated.

Please consult Sound Pool routine for correct application.

BACKGROUND DESCRIPTI ON (VIDEOGAMES)

Any videogame using VSC technology provides visually impaired/blind players with a description of all the information which

appears on the screen in the various scenes of the game. This information should be compatible with the background sounds whi ch

accompany each scene, while also leaving space in the audio channel for background music, (if possible). For example, a sequence

on the sea front would include background sound of the sea breeze and the waves breaking on the shore, but without affecting the

communication channel with the player.

Additionally, and in agreement with the VCS standard, each object must have sound descriptors which permit players to fully b enfit

from this technology-

No more than 3 objects with passive sound descriptors, (those objects which emit an identifying sound without need of interaction

from the player) should be within the players reach, as more than this will cause confusion. This does not limit the sound ca pacity of

a scene, but means that the player should active and deactivate sound descriptors of objects according to their proximity.


Release .1.0

39 / 47

VCS ALGORITHMS

ALGORITHM OF (DEFINITE/PRONOUNCED/ BROAD/ CLEAR FIELDS/BELL??)

One attribute of all sounds is what is called ‘balance’ or ‘pan’. It refers to defining if a sound if heard more to the left or to the right

of the listener. A balanced sound is one which goes from one side to the other, creating a zig zag or spiral sensation. In reference to

sound orientation, this attribute marks the position of the listener in respect to the X axis.

Listening to a sound head-on or facing the source of the sound, has the effect of provoking the most neutral balance possible,

considering that we are more or less aligned with the sound source. This balance will be achieved whether the object emitting the

sound is directly in front of us or directly behind us. The volume diferencial when moving forward and/or backward will indicate

whether the sound is coming from in front or behind.


Release .1.0

40 / 47

To face the source of a sound, we rotate on theY (vertical) axis until the sound volume we perceive is as loud on the left as on the

right. We must bear in mind information regarding inital calibration of user. Some listeners will be more senitive than others at

detecting variations in the balance of sound, some needing more saltos (reach?) to detect diferences whereas others will have such

sharp hearing as to notice minimal differences in balance.

The algorithm of campana pronunciada exagerates/amplifies the variations of balance according to the parameter we have defined

as our auditory threshold when we did our caliberation. In other words, depending on the sensitivity of the listener, all sou nds

which move a determined number of degrees away from centred position (sound source head-on) will be exaggerated, creating a

controlled lateral effect. Therefore, for those not particularily sensitive to balancing sounds, all objects (sound sources), which are

not centred will sound well-off centre and more lateral than they are in reality.

This artifical behaviour of the sound permits all listeners face and identify sound sources with much more precision and thus

optimizing listener’s orientation

NOTE:

It is important to stress that independently of the balance value, the volume of the sound signal, will always ind icate the distance to

the emitting sound source.

The angle to be taken into account in this algorithm must be defined by the process of calibration done by each listener. Nor mally, it

is necessary to adjust depending on the distance of the object so that the angle coincides with the largest salto (Reach?leap?

degree? Grade?) of sensitivity to the balance variation obtained from the process of calibration. In other words, if the list ener

capable of hearing a range of P units in the left ear, and T units in the RIGHT?, the angle corresponds to that calculate so that the

distance of the object means a variation of the largest value of P or T (signifique una variación del valor mayor de P ó T).


Release .1.0

41 / 47

Of course there will be a minimum and a maximum value for this angle, regardless of the sensitive of the listener, to avoid the

abbreviation generated being excessive for short distances.

These determining facts show the importance of the calibration process before starting any other process.


Release .1.0

42 / 47

VCS 3D STEREO / SOUND 5.1

Stereo sound is not valid for 3D positioning as it only gives information regarding the balance or panning of the sound. In o ther

words, it only informs us if a sound is produced to our left or our right and gives us a sense of the distance to the soun d, however

we will never know whether the sound is in front of us or behind. With sustained sounds we can play around with our positioning,

and moving around we could determine this information, but not all sounds are continual or last long enough for us t o be able to

use this possibilty.

5:1 technology provides a second axis of balance, our axis referred to here as Z, which informs us if the sound is being prod uced in

front of us or from behind.

Technically this is possible when we offer 4 sound sources instead of the usual 2 used in traditional stereo, and in this way we

simulate 3D positioning


Release .1.0

43 / 47

There are two inconveniences of using the 5.1 system compared to stereo, hence VCS technology offers its own algorithm to create

the 3D effect:

The 5:1 system is not available on all computers, and even if it is, it requires precise location of speakers in order to be effective.

5.1 estándar routines do not avail of processes of optimization and are not user-adaptable, hence are not a valid option for precise

positioning.

The 3D Sound routine of VCS considers the adjustments necessary for each user so that the sound generated is optimized.

At a level of resources, the algorithm is costly therefore its use is recommended exclusively for 3D positioning. For the 3D

environments, we should use the routines which by default are offered by the operating.


Release .1.0

44 / 47

Stereo sound provides the position on the X axis with respect to the sound.To simulate the fade effect, or the postion of the sound

on the Z axis with respect to listener, it is necessary to use a second sound. Using the original sound as the frontal sound, and basing

ourselves on this sound, we recreate a second sound which is the result of applying a filter of sound opacity. Hence, we now have

two sounds. The first which we will refer to as FRONT (from the original frontal sound) and another which we will call REAR (created

from the original sound using an opacity filter ) .

Now it is just a question of creating a virtual balance depending on the position of the focus of both sounds on the Z axis with

respect to the listener. Therefore, if the sound is very much to the front (100%) only the first FRONT sound will be heard, a nd if it is

very far behind us, we will use just REAR sound.

This will NOT create an exact balance of both sounds, but rather both sounds will begin in the position 0 (Z) and one or the other will

be used depending on where they are, allowing the volume indicate the distance.

It is obvious that FRONT y REAR sounds must be clearly distinguishable and must give the desired sensation of FRONTAL/REAR

sounds. This routine should have the following parameters:

Px,Py,Pz: Position of listener.

Sx,Sy,Sz: Position of sound.

Snd_Front: ID of frontal sound.

Snd_Rear: ID of rear sound.

Usr_Pan_Step: Panning value of user.

Snd_vol: To control the total volume.

VCS 3D STEREO sound requires a process of a familiarization on the listener’s behalf. It is therefore convenient to add an option in

the menu which permits this process. This process whereby the user becomes familiar with the sound system could be independent

or within the routines of user calibration.

Why does REAR = FRONT + Opacity?

This is actually a simulation of the natural human process of hearing sounds. Eventhough a sound may seem to us t o be same

whether it is produced in front or behind this is not quite the case as what actually happens is that there is an opacity eff ect, caused

by our ear mass, which clouds the sound when ir comes from behind. Visually impaired/blind persons are famili ar with this effect

and therefore naturally and automatically associate these sound differences to the Z axis.


Release .1.0

45 / 47

VCS SOUND POOL / SOUND FLAGS

Due to the quantity of sounds that VCS technology generates, it is necessary to organize and control these sound pools to avoid

sound overlaps, saturations and other errors in the system.

To do this we must first catalogue what we do and don’t want to manage in the pools.

The music has its own channel. In principal, the channel available in the software by default will be used or the channel available for

the software will be used by default.

For the background music, and also for the minor sound positioning, we will also use the routines of the system itself, for example,

the sound of the doors in VCS or the background sounds of scenes.

VCS tools, whether they use voice or not, should be adjusted to a virtual pool to avoid errors. To do this, it would also be

appropriate to establish an order of priority according to the importance of the message.

For example, imagine that we are following the activated Guiding thread, and we can hear the sound of our point of destination. If

we cross a room (to a new area), when we reach the new point there is a sound, and if the VCS area warning system is activated

there will be another sound. This sound is not incompatible with the Guiding thread sounds and there won’t be an overlap but if for

example there is another voice recorded message from the user and at the same time a sudden drop, the sound order should be a s

follows:

Waring of sudden drop.

Warning of new area

Recorded message

Guiding thread.

It is the programmer who must prepare these sequences, creating a virtual stack/list/pile/ mountain for the correct use/order of

messages, adding a collection of flags which regulate the sound traffic.

For example, we are moving using the Guiding thread and we press a key for circular sonar, the Guiding thread should remain silent

until we stop using the circular sonar. The same thing would happen if we requested help; all other warnings should then be

silenced.

THE USE OF SOUND POOLS for system messages is fundamental and is actually one keys of success of the VCS application.


Release .1.0

46 / 47

WORKING WITH VISUAL IMPAIRMENT AND OTHER DISABILITIES

VCS is a technology created to orientate users in a 3D space through use of the audio channel, which makes it especially useful for

the visually impaired/blind

As a collective, visually impaired persons have a wealth of experience and knowledge accumlated as to how to adapt to different

environments using their hearing canal to guide them. Thus, their contribution is fundamental for the development of VCS

technology.

It is vital to include visually impaired/blind persons in the team, to aid in aspects of definition, design, development, etc of all VCS

tools.

Feeling comfortable in a team implies trust. Visually impaired/blind persons must treated in the same manner as other members of

the team, without need of overprotection or special treatment which is actually counterproductive and makes them feel distin ct

rather than comfortable in the team. Frankness is fundamental.

To start with, we should ask them exactly in which terms they wish to be referred to, as this varies from country to country, group to

group.

It is not always possible to put ourselves in their shoes. There are circumstances which can be understood by common sense, but

others which we will never be able to comprehend. For example, a person with normal vision will NEVER be able to conceive a

totally blind person’s reality. Hence, it is fundamental to open channels of communication and trust beyond our normal sensory

perceptions.

It is very important not to over-use test teams, whether thay are visually impaired/blind persons or not, as results may deteriorate.

It is recommendable to renew the testers every so often to ensure quality testing.

VCS is not just a tool for visually impaired/blind persons. It can also be used for many other parallel purposes, including u se for

those with auditory or other disabilites.

The possiblity of linking external devices by means of VCS attributes, means that the technology can be used for other purposes,

such as therapeutic software to treat injuries or for rehabilitation therapy.

Its multi-language dimension permits its use in language learning programs and software translation programs.


Release .1.0

47 / 47

VCS TOOLKIT FOR PROGRAMMER S

This is a collection of layouts and a definition of rules at a generic level used for the application of the technology in an y software. It

is adjusted/ it adjusts to the processes, protocols, axioms and theorems explained in the VCS documentation/information.

It is quite straight-forward giving examples and fonts/sources in DBpro and C++. We have not dismissed the possibility of amplifying

these options at a future stage.

vcs technology release 1 - vis caeca soni · vcs technology methology and basis ... v s is the...

Documents