pigeon project

29
Team Members : Team Members : Thomos Charilaos Thomos Charilaos Kasselakis Georgios Kasselakis Georgios Tsompanidis Ilias Tsompanidis Ilias Simaioforidis Zisis Simaioforidis Zisis Project Tutor : Athena Vakali Project Tutor : Athena Vakali

Upload: georgios-kasselakis

Post on 14-Jan-2015

1.628 views

Category:

Technology


3 download

DESCRIPTION

 

TRANSCRIPT

Page 1: Pigeon Project

Team Members : Team Members : Thomos CharilaosThomos Charilaos

Kasselakis GeorgiosKasselakis GeorgiosTsompanidis IliasTsompanidis Ilias

Simaioforidis Zisis Simaioforidis Zisis

Project Tutor : Athena VakaliProject Tutor : Athena Vakali

Page 2: Pigeon Project

Part IPart IIntroductionIntroduction

Page 3: Pigeon Project

Introduction VideoIntroduction Video

Page 4: Pigeon Project

In a world with ever increasing communication needs , In a world with ever increasing communication needs , there is great demand for tools that dissolve the barriers there is great demand for tools that dissolve the barriers includingincluding

different culturesdifferent cultures languageslanguages disabilitiesdisabilities

.Pigeon is a software solution which .Pigeon is a software solution which exploits the usability of web translationexploits the usability of web translation inherits real-time messaging of MSN Networksinherits real-time messaging of MSN Networks

IntroductionIntroduction

Page 5: Pigeon Project

Translation is an old and debated subjectTranslation is an old and debated subject Human interpretersHuman interpreters

Great costGreat cost Can be very slowCan be very slow

Machine translationMachine translation Poor quality of translationPoor quality of translation Media limitations [ only text ]Media limitations [ only text ] Limited variety of languages supportedLimited variety of languages supported

.Pigeon addresses these issues.Pigeon addresses these issues The best* translation availableThe best* translation available Range of media typesRange of media types Enhanced selection of languagesEnhanced selection of languages

Why .Pigeon ?Why .Pigeon ?

Page 6: Pigeon Project

.Pigeon at a glance.Pigeon at a glanceInterfaces and functionalitiesInterfaces and functionalities

Translated speech messagingTranslated speech messaging

Zero deploymentZero deployment

Translated instant text messagingTranslated instant text messaging

Voice command interfaceVoice command interface

Support for impaired usersSupport for impaired users

MSN MSN messengermessenger

.Pigeon bot.Pigeon bot .Pigeon .Pigeon

messengermessenger

Page 7: Pigeon Project

Translated speech messagingTranslated speech messaging

Zero deploymentZero deployment

Translated instant text messagingTranslated instant text messaging

Voice command interfaceVoice command interface

Support for impaired usersSupport for impaired users

MSN MSN messengermessenger

.Pigeon bot.Pigeon bot .Pigeon .Pigeon

messengermessenger

.Pigeon at a glance.Pigeon at a glanceInterfaces and functionalitiesInterfaces and functionalities

Messages to be sent are typed here

Messages that are sent by users in the conversation appear here

Page 8: Pigeon Project

Translated speech messagingTranslated speech messaging

Zero deploymentZero deployment

Translated instant text messagingTranslated instant text messaging

Voice command interfaceVoice command interface

Support for impaired usersSupport for impaired users

MSN MSN messengermessenger

.Pigeon bot.Pigeon bot .Pigeon .Pigeon

messengermessenger

.Pigeon at a glance.Pigeon at a glanceInterfaces and functionalitiesInterfaces and functionalities

Select source an target languages

Tap the input text and translate it (text input)

Or record it through the device microphone and translate it (voice input)

The translated text will appear here (text output)

If the option is selected the translated text will be spoken from the device loudspeakers (voice output)

Page 9: Pigeon Project

Translated speech messagingTranslated speech messaging

Zero deploymentZero deployment

Translated instant text messagingTranslated instant text messaging

Voice command interfaceVoice command interface

Support for impaired usersSupport for impaired users

MSN MSN messengermessenger

.Pigeon bot.Pigeon bot .Pigeon .Pigeon

messengermessenger

.Pigeon at a glance.Pigeon at a glanceInterfaces and functionalitiesInterfaces and functionalities

Other users’ messages appear here in user’s native language

Input message box

Other users are provided as contacts

The voice interface can manipulate all aspects of the application using spoken commands

Push and hold the record button to record a message

Voice output can be used by blind users

Page 10: Pigeon Project

.Pigeon in action….Pigeon in action… Let us introduce ….

•Danai – Greek. also speaks German Journalist who often travels abroad

•Anna Li – Chinese. Is a senior Executive Manager of a Conference & Exhibition Organization Company.

•Ivan – Argentinean. Is a mute. Student of Asian Literature, participating in an Exchange Program with Tokyo University

People on the move… Tourists

Journalists

Targeted Audiences :

Business Oriented Users

Impaired PeopleVisually Handicapped

Hard of Hearing or Mutes

Page 11: Pigeon Project

Typically: She would use a human Portuguese to Typically: She would use a human Portuguese to GreekGreek interpreterinterpreter

She speaks only Greek & German and has to communicate with Portuguese Representative

with .Pigeon…with .Pigeon…Uses her Pocket PCUses her Pocket PCSets up a phone conversation (input=speech, output=speech)Sets up a phone conversation (input=speech, output=speech)

Danai visiting the European Danai visiting the European ParliamentParliament

TranslationDanai’s

Voice VoiceRecognition

VoiceRecognition

Translation

Representative’sVoice

Voice Synthesis

Page 12: Pigeon Project

Anna Li organizing an International Anna Li organizing an International ConferenceConference

Physical presence

Human Interpreters or booths

GREAT COST !

The Multinational Company has vital partners located in different countries. Every single partner must take part at the meeting

Typically : She would:

with .Pigeon:Everyone Communicating in her native languageAt any desired formatReal-Time

organize a meeting

Company’sCDN

Partners’Devices

Page 13: Pigeon Project

Ivan visiting University’s LibraryIvan visiting University’s Library

Ηe can’t write Japanese. The Librarian does not know how tocommunicate in sign language

Typically: Ηe would rather find a Japanese who also knows how to communicate with Ivan

with .Pigeon… Ηe uses his PocketPC, writes his instructions to the Librarian and he selects Speech as Translation Output.

TranslationΙvan’s Text

VoiceRecognition

Translation

Librarian’sVoice

Voice Synthesis

Page 14: Pigeon Project

Part IIPart IIFunctionality OverviewFunctionality Overview

Page 15: Pigeon Project

Message in a bottleMessage in a bottle

Clients that have access to the .Pigeon framework include

• MSN messenger enabled devices

• Devices that support the .net framework

Both MSN and .Pigeon messengers communicate with the bot except when doing direct translation

Voice is recognized or synthesized as needed

The translation is performed on the input text using external servers

Page 16: Pigeon Project

The translation processThe translation process

Common issues in machine translation toolsCommon issues in machine translation tools Semantic errors in output textSemantic errors in output text Lack of two way translationLack of two way translation Diminished speedDiminished speed

Proposed solution of the .Pigeon frameworkProposed solution of the .Pigeon frameworkThe translation graph, combines :The translation graph, combines : Server measured speed ratingsServer measured speed ratings User feedback for quality of translationUser feedback for quality of translation

Page 17: Pigeon Project

The translation processThe translation process

The translation The translation availability is visualized availability is visualized as a vector between as a vector between languages with ratings of languages with ratings of qquality and uality and sspeed peed <q,s><q,s>The users provide the The users provide the graph with a quality to graph with a quality to speed ratio which is used speed ratio which is used to combine the ratings to combine the ratings under the formulaunder the formula

wwA,BA,B = (10-q) = (10-q)c+sc+s(1-c)(1-c)

De

El

En

Fr<10,<10,7>7>

<8,9<8,9>>

<8,8<8,8>>

<5,9<5,9>>

<6,7<6,7>>

Page 18: Pigeon Project

The translation processThe translation process

De

El

En

Fr<10,<10,7>7>

<8,9<8,9>>

<8,8<8,8>>

<5,9<5,9>>

<6,7<6,7>>

wwA,BA,B = (10-q) = (10-q)0.5+s0.5+s0.50.5wwA,BA,B = (10-q) = (10-q)c+sc+s(1-c)(1-c)

De

El

En

Fr

8.58.5

8.58.5

88

77

6.56.5

Page 19: Pigeon Project

The translation processThe translation process

De

El

En

Fr5.65.6

7.67.6

6.86.8

8.28.2

6.46.4

20%

De

El

En

Fr1.41.4

3.43.4

3.23.2

5.85.8

4.64.6

80%

Page 20: Pigeon Project

Part IIIPart IIITechnical InsightTechnical Insight

Page 21: Pigeon Project

Existing External Existing External ResourcesResources ServerServer

Windows 2003 Server Web EditionWindows 2003 Server Web Edition Development toolsDevelopment tools

Visual Studio .NET 2003Visual Studio .NET 2003 Smart Device programmability toolkit 2003Smart Device programmability toolkit 2003 SAPI 5.1 SDKSAPI 5.1 SDK

Technological frameworksTechnological frameworks .NET Compact Framework 1.0 SP1.NET Compact Framework 1.0 SP1 .NET Framework 1.1.NET Framework 1.1 Visual Basic .NETVisual Basic .NET XML Web services, ASP.NETXML Web services, ASP.NET

Technological ResourcesTechnological Resources

Page 22: Pigeon Project

Technological ResourcesTechnological Resources

Page 23: Pigeon Project

MSN Messenger BotMSN Messenger Bot Implementing the ‘Push’ design patternImplementing the ‘Push’ design pattern Connection with MSN servers using socketsConnection with MSN servers using sockets Message construction compatible with MSN6 protocolMessage construction compatible with MSN6 protocol Deals with inconsistencies with different protocols – Deals with inconsistencies with different protocols –

Alternative to contact listsAlternative to contact lists

Developed ResourcesDeveloped Resources

Technological ResourcesTechnological Resources

Page 24: Pigeon Project

Developed ResourcesDeveloped Resources

Technological ResourcesTechnological Resources

Exposed .Pigeon ServicesExposed .Pigeon Services Encapsulate the direct translation interfaceEncapsulate the direct translation interface Supports media transformationsSupports media transformations

Text to speechText to speech Speech to textSpeech to text

Speech to TextMS SAPI 5.1

Recorded Speech

Recognized text

Babel TechnologiesText to Speech

Translated text

Synthetic Voice

Page 25: Pigeon Project

Part IVPart IVExitExit

Page 26: Pigeon Project

.Pigeon … Flying ahead.Pigeon … Flying ahead

Support for optical Support for optical recognitionrecognition

Support for regular cell Support for regular cell phonesphones

Planned extensions for Planned extensions for making .Pigeon more making .Pigeon more appealing to CDN marketsappealing to CDN markets

Page 27: Pigeon Project

Exit videoExit video

Page 28: Pigeon Project

Part VPart VQuestionsQuestions

Page 29: Pigeon Project

Is there somethingIs there somethingyou wanted to ask ?you wanted to ask ?