cognitive services: building smart apps with speech, nlp & vision
TRANSCRIPT
Nick LandrySenior Technical Evangelist – Microsoft
Blog: AgeofMobility.com
@ActiveNick | github.com/ActiveNick
Microsoft Cognitive Services:
Building Smart Applications
with Speech, NLP & Vision
Microsoft Cognitive ServicesGive your apps a human side
Microsoft Cognitive ServicesGive your apps a human side
Cognitive Services API Collection
Bringing it all together The Seeing AI App
Computer Vision, Image, Speech Recognition, NLP,
and ML from Microsoft Cognitive Services
Watch Video HereRead Blog Here
EmotionSpeaker
Recognition
Speech
Custom
RecognitionComputer Vision
Face
Video
microsoft.com/cognitive
Linguistic Analysis
Language
Understanding
Bing Spell Check
Entity Linking
Knowledge
Exploration
Academic
Knowledge
Bing
Image Search
Bing
Video Search
Bing
Web Search
WebLM
Text Analytics Recommendations
Bing
Autosuggest
Bing
News SearchTranslator
EmotionSpeaker
Recognition
Speech
Custom
RecognitionComputer Vision
Face
Video
microsoft.com/cognitive
Linguistic Analysis
Language
Understanding
Bing Spell Check
Entity Linking
Knowledge
Exploration
Academic
Knowledge
Bing
Image Search
Bing
Video Search
Bing
Web Search
WebLM
Text Analytics Recommendations
Bing
Autosuggest
Bing
News SearchTranslator
Computer Vision API
Distill actionable information from
images
Video API
Analyze, edit, and process videos within
your app
Face API
Detect, identify, analyze, organize, and
tag faces in photos
Emotion API
Personalize experiences with
emotion recognition
Vision
Updated Computer Vision API
Content of Image:
Categories v0: [{ “name”: “animal”, “score”: 0.9765625 }]
V1: [{ "name": "grass", "confidence": 0.9999992847442627 },
{ "name": "outdoor", "confidence": 0.9999072551727295 },
{ "name": "cow", "confidence": 0.99954754114151 },
{ "name": "field", "confidence": 0.9976195693016052 },
{ "name": "brown", "confidence": 0.988935649394989 },
{ "name": "animal", "confidence": 0.97904372215271 },
{ "name": "standing", "confidence": 0.9632768630981445 },
{ "name": "mammal", "confidence": 0.9366017580032349, "hint": "animal" },
{ "name": "wire", "confidence": 0.8946959376335144 },
{ "name": "green", "confidence": 0.8844101428985596 },
{ "name": "pasture", "confidence": 0.8332059383392334 },
{ "name": "bovine", "confidence": 0.5618471503257751, "hint": "animal" },
{ "name": "grassy", "confidence": 0.48627158999443054 },
{ "name": "lush", "confidence": 0.1874018907546997 },
{ "name": "staring", "confidence": 0.165890634059906 }]
Describe0.975 "a brown cow standing on top of a lush green field“
0.974 “a cow standing on top of a lush green field”
0.965 “a large brown cow standing on top of a lush green field”
Start coding
microsoft.com/cognitive
Speech
Bing Spell Check API
Detect and correct spelling mistakes within your app
Language Understanding Intelligent Service
Teach your apps to understand
commands from your users
Web Language Model API
Leverage the power of language models trained on web-scale
data
Linguistic Analysis API
Easily parse complex text with language
analysis
Text Analytics API
Detect sentiment, key phrases, topics, and language from
your text
Language
Reduce labeling effort with interactive featuring
Seamless integration to Speech API
Deploy using just a few examples with active learning
Supports 5 languages (English, Chinese, Italian, French, Spanish)
Language Understanding Models
{“entities”: [
{“entity”: “flight_delays”,“type”: “Topic”
}],“intents”: [
{“intent”: “FindNews”,“score”: 0.99853384
},{
“intent”: “None”,“score”: 0.07289317
},{
“intent”: “ReadNews”,“score”: 0.0167122427
},{
“intent”: “ShareNews”,“score”: 1.0919299E-06
}]
}
“News about flight delays”
Language Understanding Models
Your conversation logic
Logic
Web Service
Your Bot
LUIS
Start coding
microsoft.com/cognitive
https://www.microsoft.com/cognitive-services/en-us/text-analytics-api
Start coding
microsoft.com/cognitive
https://www.microsoft.com/en-us/translator/products.aspx
Start coding
microsoft.com/cognitive
Easy
F lex ib le
Qual i ty
Why Microsoft Cognitive Services?
Developer Resources
Preview Pricinghttps://www.microsoft.com/cognitive-services/en-us/pricing
Documentationhttps://www.microsoft.com/cognitive-services/en-us/computer-vision-api/documentation
Client SDKshttps://github.com/Microsoft/ProjectOxford-ClientSDK
https://github.com/felixrieseberg/project-oxford (nodejs)
https://github.com/southwood/project-oxford-python
Example Codehttps://github.com/jsturtevant/happy-image-tester-django
https://github.com/jsturtevant/happy-image-tester-nodejs
Join Our Communityhttps://stackoverflow.com/questions/tagged/microsoft-cognitive
https://social.msdn.microsoft.com/forums/azure/en-US/home?forum=mlapi
https://cognitive.uservoice.com/
https://blog.xamarin.com/performing-ocr-for-ios-android-and-windows-with-microsoft-cognitive-services/
https://github.com/pierceboggan/smarter-apps/tree/master/ExpenseIt/
https://blog.xamarin.com/adding-facial-recognition-to-your-mobile-apps/
https://github.com/pierceboggan/smarter-apps/tree/master/talk/employee-directory/
https://github.com/xamarin/mini-hacks/tree/master/microsoft-cognitive-services/solutions
https://github.com/DotNetToscana/See4Me
https://github.com/ActiveNick/BingTranslateDemo
https://github.com/ActiveNick/CognitiveServices-Demo-Face
https://github.com/ActiveNick/CognitiveServices-Demo-EntityLinking
https://github.com/ActiveNick/CognitiveServices-Demo-TextAnalytics
https://channel9.msdn.com/Events/Build/2016/B878
https://channel9.msdn.com/Events/Build/2016/B855
https://channel9.msdn.com/Events/Build/2016/T620
https://channel9.msdn.com/Events/Build/2016/B834
https://channel9.msdn.com/Events/Build/2016/B833
http://developer.microsoft.com/cortana
http://AgeofMobility.com/Cortana
Online Microsoft training delivered by experts to help technologists continually learn
Hundreds of courses for developers, IT Pros, students, entrepreneurs and enthusiasts
11 different languages
3M+ students registered
Build your own Learning Plan
All free!
http://mva.microsoft.com
• Universal Windows App Developmentwith Cortana and the Speech SDK
• Available for on-demand viewing now:http://aka.ms/CortanaMVA
• Channel 9 Show
• Visual Studio Toolboxwith Robert Green• New Voice Commands
• Integration with Cortana’s canvas
• Background Voice Commands
• Continuous dictation
• Poutine in Montreal!
https://channel9.msdn.com/Shows/Visual-Studio-Toolbox/App-Development-with-Cortana
More Cortana Dev on Windows 10
Thank You!Slides are in SlideShare. Demos are on GitHub.Contact me and let me know what you build, I will be happy to help promote your apps.
Blog: AgeofMobility.com
Twitter: @ActiveNick
Email: [email protected]
Apps: www.bigbaldapps.com
LinkedIn: linkedin.com/in/activenick
GitHub: github.com/ActiveNick
Slideshare: slideshare.net/ActiveNick
