finally, a voice for the enterprise!
TRANSCRIPT
VOICE OF THE ENTERPRISEAdding Another Input Stream to Your Applications
From Google Car,
Brome, QC, CA
August 2015
INTRO
➤ William “Bill” Klos ➤ Senior Architect, Centric Consulting ➤ Columbus, OH ➤ Specialties
➤ Cloud ➤ Mobility ➤ Alternative Technologies
➤ @williamklos ➤ [email protected]
From Google Car,
Brome, QC, CA
August 2015
THE PROMISE
➤ Star Trek
➤ 2001: A Space Odyssey
➤ Interstellar
➤ Mother (Alien)
➤ The Beam (Canvas)
But what I’m talking about is not Artificial Intelligence or Machine Learning.
I’M TALKING ABOUT
➤ Quick Status Updates
➤ What happened with the overnight jobs?
➤ What’s the story with the dog?
➤ Did Mom take her medicine today?
➤ Executing Actions
➤ Route me home but don’t bug me unless there’s traffic.
➤ Put me down for 8 hours today on the Acme project.
➤ Blow up the ship.
WHAT’S SO GOOD ABOUT A VOICE UI?
➤ When It’s Good
➤ Less Friction, More Natural
➤ Pervasive/Ubiquitous
➤ Can be Conversational
➤ Truly Keeps Hands Free
➤ Requires Less Focus/It’s Freeing
➤ Only the Needed Information
➤ When It’s Bad
➤ Requires Focused Verification of Results
➤ Adds to the Environmental Chaos
➤ Global Thermonuclear War
THE CANDIDATES
➤ Mobile
➤ Siri
➤ Google Now
➤ Cortana
➤ Non-Mobile
➤ Alexa (Amazon Echo)
THE NOMINEE
➤ Alexa (Amazon Echo)
THE PLUSES
➤ Cheap
➤ There’s an API & SDK for it.
➤ Easy to develop to.
➤ Can Host Anywhere where HTTPS is Available
➤ Voice recognition is good.
➤ Can Own a Room
➤ Being Extended all the Time
➤ Good Support & Community
THE MINUSES
➤ It’s Voice Recognition, but not Necessarily YOUR Voice
➤ Requires an Internet Connection
➤ Testing Can be Wonky
➤ Will drive your family mad & leave you hoarse
➤ Can’t Take it With You
➤ Not as Feature-rich as Your Phone’s Capabilities Yet
➤ Have to buy a complete device everywhere you want to use it
➤ Cannot Initiate an Interaction
➤ Using your services a little less natural than native services
GENERAL USAGE
GENERAL USAGE - WHAT IS A VOICE UI (MANAGEMENT)
DueForward API
Mobile UI
Web UI
Voice UI
GENERAL USAGE - WHAT IS A VOICE UI (YOU GUYS)
Spicoli[devops-slack-hook-push]
Telemetri[telemetri-api]
DueForward[dueforward-api]
tbd-email[aws-ses-manager]
SLACK
tbd-push[aws-sns-manager]
RSSFEEDS[112]
WEB PAGES[~3500/mo]
Voice UI[alexa-voice-api]
Web UI[angular]
Dashboards[bi-bigdata]
GENERAL TAXONOMIES
COMPANIES[154]
CONCEPTS[85]
CITIES[35]
CLOUDS[14]
DATABASES[29]
HARDWARE[21]
SOFTWARE[26]
INDUSTRIES[26]
MATERIALS[3]
PLATFORMS[41]
LANGUAGES[36]
SYNONYMS[279]
Subscriber AccessCentric Access
CLIENT/INDUSTRY TAXONOMIES
HEALTHCAREINSURANCE FINANCIAL
MICROSOFT ALLIANCE DATA
PROCTOR & GAMBLE
CLIENT CLIENT POTENTIAL CLIENT
INDUSTRY SERVICE OFFERING
SERVICE OFFERING
Public Access
DEMO DASHBOARD
Blog Posts Opinions Reputation
OPERATIONAL
AWS
OTHER CENTRIC
APPS
BI SO
ATOMFEEDS
[20]
PHASE 2
PHASE 3
for CodeMash 2016 presentation
notifications
notifications
notifications
logs
AWS S3
“ALEXA”
“ALEXA”
“DO SOMETHING”
“DO SOMETHING”
“ASK MY APP TO…”
“HERE ARE YOUR
AWESOME RESULTS”
“HERE ARE YOUR
AWESOME RESULTS”
CUSTOM SKILL INTERACTION
NATIVE SKILL INTERACTION
GENERAL USAGE - INTERACTING WITH ALEXA
Alexa, turn on the lights downstairs.
Alexa, tell DevOps to spin up another order processor.
The lights are now on.
There are now 6 order processors running.
GENERAL USAGE - ADDRESSING ALEXA & YOUR SKILL
➤ Address the device with: “Alexa” or “Amazon”. Your pick.
➤ Address your Skill with:
➤ Ask, Tell (preferred, most natural)
➤ Talk to, Open, Launch, Start, Use, Resume, Run, Load, Begin
➤ There is no functional difference between the phrases it just comes down to what is easiest to convey the necessary meaning by the user to your application.
AskDueForward what is the current STATUS of the DATABASE AskDueForward what VERSION it is RUNNING
AskDueForward how many DOCUMENTS need to be DETERMINED TellDueForward to KICK OFF a DETERMINATION job AskDueForward how many DOCUMENTS need to be DETERMINED
HOW TO DEVELOP FOR IT
HOW TO DEVELOP FOR IT - WHAT YOU’LL NEED
➤ An AWS Account w/Alexa Development Option
➤ http://developer.amazon.com
➤ An Amazon Echo (though you can do some stuff w/o it)
➤ An Intents File
➤ A kind of template file for filtering your Utterances through
➤ An Utterances File
➤ example phrases
➤ Your Language of Choice (Google go for me)
➤ A Server with HTTPS Capabilities
HOW TO DEVELOP FOR IT - APPLICATION SPECIFICS
➤ If you work with Java, Node.js, or Python - you can use Amazon Lambda to host and execute your source in response to Alexa events.
➤ More languages coming soon.
➤ Or you are completely free to use whatever you want as long as you have an HTTPS endpoint to point Alexa to.
ALEXA SKILL KIT
ALEXA SKILL KIT - INTENTS
ALEXA SKILL KIT - UTTERANCES
Alexa, ask DueForward what were the top CONCEPTS for LAST MONTH Alexa, ask DueForward what were the top DATABASES for LAST MONTH Alexa, ask DueForward what are the top LANGUAGES for THIS MONTH Alexa, ask DueForward what was the top SOFTWARE for LAST MONTH
CONSIDERATIONS
➤ Designing Your Input Options
➤ How casual? How formal? How many ways to say it?
➤ Can you ask it easily or is it a complex request?
➤ Designing Your Response Options
➤ How casual? How formal? How many ways to say it?
➤ How much data can you retain when hearing vs. seeing?
➤ Do you want read-only (safe) or read-write (powerful)?
➤ Transaction size (short, to the point on both sides)
➤ You have about 10 seconds to put it all together.
➤ Testing
SETTING UP YOUR APP
SETTING UP YOUR APP
➤ For Amazon Certification & Publishing
➤ Verify that the Request was Sent by Alexa
➤ Check the Signature of the Request
➤ Check the Timestamp of the Request
➤ Don’t Need These for Testing
➤ Verify the Application Id Matches the One Assigned
CODING THE APP
STIMULUS/RESPONSE
CODING THE APP - STIMULUS/RESPONSE
Echo Captures Request
FormulateRequest Packet
Route RequestTo Supplied
Endpoint
Formulate & Return Voice
Response
Process Request
Echo Speaks Response
Formulate & Return Card
Info
Alexa App Shows
Companion Card
CODING THE APP - STIMULUS/RESPONSE / COMPANION APP RESPONSE CARDS
GENERAL ARCHITECTURE
CODING THE APP - GENERAL ARCHITECTURE
Shell APIPassthru[optional]
DueForward Application
DB
DB
DB DB
DB
DB
HTTPS
Future Application
Future Application
Future Application
Future Application
ECHO APP DATA
CODING THE APP - PASSTHRU CALL
CODING THE APP - PASSTHRU RESPONSE
CODE - INTENTS & UTTERANCES
CODE - INTENTS & UTTERANCES
CONVERSATIONS
DISAMBIGUATION
DISAMBIGUATION
CONVERSATION - DISAMBIGUATION
➤ Alexa, tell CampIO to check in Bill. ➤ Did you mean Bill Klos or Bill Chamberlain? ➤ Bill Chamberlain | The second one. ➤ OK. Bill Chamberlain is now checked in.
PROMPTING THE USER
CONVERSATION - PROMPTING THE USER
➤ Alexa, ask DueForward… | Alexa, ask DueForward for help. ➤ You can check system status, get metrics, or run a job. Which
would you like to do? ➤ Run a job, please. ➤ OK. I can re-determine documents or destroy the ship. Which
would you like me to do?
CONVERSATION - TELL ME MORE
➤ Alexa, ask DueForward what is the current status of the database?
➤ All database servers are currently operating normally. ➤ What about memory usage? ➤ Memory usage is at 11%. ➤ And how many documents need to be re-determined? ➤ Currently, there are 177 documents that need to be re-determined.
Would you like me to go ahead and clear them out? ➤ Please. ➤ Done.
NAVIGATING A PROCESS
“Computer, initiate self-destruct sequence 1, code 1-1 A, set for five minutes and I want it to be silent except for a ticking clock sound to mysteriously play ship-wide over the speakers. Thanks.
-The Captain
CONVERSATIONS - NAVIGATING A PROCESS
Initiate self-destruct sequence.
CONVERSATIONS - NAVIGATING A PROCESS
Destruct sequence activated. Please provide authorization code.
CONVERSATIONS - NAVIGATING A PROCESS
alpha, alpha, alpha
CONVERSATIONS - NAVIGATING A PROCESS
Authorization accepted. Please confirm your decision.
CONVERSATIONS - NAVIGATING A PROCESS
Confirmed | Yep | Do it | Let’s light this candle | Affirmative…
CONVERSATIONS - NAVIGATING A PROCESS
Self-destruct sequence in process. Sorry you’re having a bad day.
WHAT ABOUT SECURITY?
CONVERSATIONS - WHAT ABOUT SECURITY?
Tell Me Your Authorization
Code
Confirm Your Request One
Last Time
CAPT?
Geez, Use “The Force”
Go Find The Captain
JEDI?
Request Self-Destruct
Sequence
UNDERSTANDING
WHAT DID I SAY?
WHAT DID I SAY? - METAPHONES, SOUNDEX, & NYSIIS
➤ Metaphone
➤ William Klos = WLM KLS
➤ Incidentally, “Galluzzo” also = KLS
➤ SoundEx
➤ William Klos = W450 K420
➤ NYSIIS
➤ New York State Identification & Intelligence System
➤ 8-25 Step Process
➤ William Klos = WALAN CL
CONSIDER “WILLIAM KLOS”
WHAT DID I SAY? - THE MANY PHASES OF KLOS
➤ KLOS / K420 / KLS / CL
➤ KLAS / K420 / KLS / CL
➤ CLAUS / C420 / KLS / CL
➤ CLOS / C420 / KLS / CL
➤ CLOSE / C420 / KLS / CL
➤ KIDS / K420 / KTS / CAD
➤ CLASS / C420 / KLS / CL
➤ KOLS / K420 / KLS / CAL
➤ KOLB / K410 / KLB / CALB
WHAT DID I SAY? - VARIABLE INPUT
BUT, WHAT IF I GO BY BILL?
EXACTNESS
WHAT DID I SAY? - EXACTNESS
➤ Be mindful that Alexa tries to be forgiving.
➤ If what you speak vs. the matched Utterance has some instances of the wrong tense, transposed words, or missing/substituted words, it will try to give you the benefit of the doubt.
➤ If you need exactness, you’ll have to use Slots and bounce the associated data against a database when matching instead of simply matching on Intents.
WHAT DID YOU SAY?
WHAT DID YOU SAY? - PLAIN TEXT
WHAT DID YOU SAY? - SSML
➤ audio (recorded voice files)
➤ break (adding pauses)
➤ p (paragraph)
➤ phoneme (pronunciation based on defined alphabets)
➤ s (ending a sentence with a period)
➤ say-as (spell-out, digits, fraction)
➤ speak (root element)
➤ w (verb, noun, past-participle, alternate pronunciations
➤ (e.g. bass vs. bass)
WHAT DID YOU SAY? - SSML
<speak> five<break time="1s"/>four<break time="1s"/>three. Abort sequence canceled. </speak>
MANUAL PHONETICS
VARYING THE RESPONSE
WHAT DID YOU SAY? - VARYING THE RESPONSE
THINGS TO REMEMBER
➤ Enunciation helps, but it is not critical to success.
➤ But ambient noise can wreak havoc.
➤ You should have to have thousands of samples, but more than a few.
➤ Don’t strive for perfection, but don’t blow up your ship on a mis-understanding.
➤ Treat phonetics/pronunciation codes like you would multi-lingual set ups.
➤ 2 Words: DevOps
➤ Be polite. Give bonus points for niceties.
➤ Can be a cheap way to get cool in to the enterprise and sneak in some alternate technologies.
HTTPS://GITHUB.COM/CENTRICCONSULTING/ALEXA-FRAMEWORK-GO
Q&A