next generation apps with google chrome-by dhruv gohil

GoogleIO @ Java Meetup Ahmedabad

Build next generation apps with eyes and ears using Google Chrome

Treat your users as human!

lUX - 101

lTools should fit human not the other way around!

Now we have books on UX!

lWith computers, we lost 'human touch' in interaction

lTouch screens brought it back, but frankly its not the natural way to do 'things'

Let's get back to basics..

lI have to tell you something.. I will speak and you will listen.

lSo why it has to be different for computers?!

Enter the new era!

lNow browsers can speak to you.

lAnd browsers can hear you too!

http://updates.html5rocks.com/2013/01/Voice-Driven-Web-Apps-Introduction-to-the-Web-Speech-API

Demo time!

https://www.google.com/intl/en/chrome/demos/speech.html

Do it yourself!

You talk too much...

lIt's time to listen..

lYes! That's all we need!

https://translate.google.co.in/#auto/hi/Welcome%20to%20Ishi%20Systems

Off-line mode too!

http://syl22-00.github.io/pocketsphinx.js/

This is how translation should work!

http://www.moreawesomeweb.com/demos/speech_translate.html

Command mode!

lI am married.... ammm.. Can I command my computer to do things?

https://www.talater.com/annyang/ https://github.com/TalAter/annyang

But can you see me?

http://wolframhempel.github.io/photobooth-js/

Ok but how its useful?

http://revealjs.herokuapp.com/

Time to demo!, again.! :-)

https://github.com/willy-vvu/reveal.js

What was that?

lMagic of 'Motion detection algorithms'

lOver simplified How To:

Capture video [getUserMedia()]

Draw frames [ html5 <canvas> ]

Run Algorithm to detect change between frames [ .js]

lIf you are deeply interested

Read : http://labs.littlemissrobot.com/behind-the-mirror/

http://www.adobe.com/devnet/html5/articles/javascript-motion-detection.html

More!

lhttps://github.com/foo123/HAAR.js#live-examples

Interactive Face Detection

Many Faces Detection

Interactive Mouth Detection

l

lhttp://www.soundstep.com/blog/experiments/jstracking/

Motion tracking

lhttp://www.soundstep.com/blog/experiments/jsdetection/

The Magical Xylophone

But Why chrome?

http://caniuse.com/#feat=web-speech

Its time to...

WebSpeech API fundametals

lAudio capture

lRequest authorization

lAudio processing pip line

Nodes (FFT , Gain, Filter...)

http://chimera.labs.oreilly.com/books/1234000001552/

lJoin to destination (Speakers)

l+ Visualizations

GetUserMedia API fundamentals

lAudio + Video capture

lRequest authorization

lCapture

lDraw (Hidden)

lAnalyze + Process

lRe-Draw

And finally...

लगता हैं बर्फ गगर रही हैं!

https://github.com/IonicaBizau/WebcamSnowhttp://ionicabizau.net/WebcamSnow/

Keep hacking!

http://topheman.github.io/parallax/

Thanks!