learn about ios10 siri kit
TRANSCRIPT
Say Hello to Siri !
Snehal Patil – iOS Developer @ Yahoo
Only Supported Domains ! • VoIP calling • Messaging • Payments • Photo • Workouts • Ride booking • CarPlay (automotive vendors only) • Restaurant reservations (requires
additional support from Apple)
So before you go and start thinking about all the different potential use cases for your app, keep in mind that Apple has only opened support for these six domains.
!
2 new Frameworks Siri Kit was introduced in WWDC 2016, giving users the ability to control your app’s behavior using their voice.
Intents framework - communication between your app and the system.
Intents UI framework - presenting a custom interface when one of your tasks is performed.
Siri Workflow 1. First, Siri takes the audio of the user's Speech and converts it into text. 2. Next, the text is converted into an Intent, a structured representation of the
user's request. 3. Based on the Intent, Siri will take Action to perform the user's request. 4. Finally, Siri will present Responses (both visual and verbal) to the user based
on the Action taken.
Speech Intent Action Response
Siri + App Workflow 1. Vocabulary - This is how the app tells Siri the words it needs to know to interact with it. 2. App Logic - These are the actions and responses that the app will take based on given Intents. 3. User Interface - This is the optional, custom user interface that the app can give its responses in.
Speech Intent Action Response
Vocabulary App Logic User Interface
Intent Object - Messaging Domain The structured Intent will contain the following information:
Domain: Messages Intent: sendMessage Recipient: John Content: lets go!
INSendMessageIntentResponseCode - unspecified, ready, Inprogress, success, failure, failureRequiringAppLaunch, failureMessageServiceNotAvailable NSUserActivity
INSendMessageIntent INSendMessageIntentResponse
The Intent Lifecycle
1. The app must Resolve every parameter on an event. As a result, the app will call Resolve multiple times (once
per each parameter), and sometimes multiple times on the same parameter until the app and the user agree on what is being requested.
2. The app must Confirm that it can handle the requested Intent and tell Siri about the expected outcome. 3. Finally, the app must Handle the Intent and perform the steps to achieve the requested outcome.
The Intent Extension
● The Intents Extension is responsible for handling the main interactions between the app and Siri ● Implements resolve, confirms and handle results ● Support one or more intents - choose as per your apps requirements ● Runs in the background while Siri is in foreground ● Security can be handled by - restricting access while device is locked locked and by asking
permission - for ex, dont allow user to send payments when app is locked.
Intent Methods - Messaging Domain
Resolve func resolveRecipients(forSendMessage intent: INSendMessageIntent, with completion: ([INPersonResolutionResult]) -> Swift.Void) { } Resolve response
Success(with:) Your app matched the user's request
ConfirmationRequired(with:) Siri makes sure about users request
Disambiguation(with:) Your app needs the user to select from a list of people provide options to choose from
Confirm func confirm(sendMessage intent: INSendMessageIntent, completion: @escaping (INSendMessageIntentResponse) -> Void) { let userActivity = NSUserActivity(activityType: NSStringFromClass(INSendMessageIntent.self)) let response = INSendMessageIntentResponse(code: .ready, userActivity: userActivity) completion(response) }
• Tell Siri the expected result of an intent using intent response • Siri prompts for confirmation using NSUserActivity
Handle func handle(sendMessage intent: INSendMessageIntent, completion: @escaping (INSendMessageIntentResponse) -> Void) { let userActivity = NSUserActivity(activityType: NSStringFromClass(INSendMessageIntent.self)) let response = INSendMessageIntentResponse(code: .success, userActivity: userActivity) completion(response) }
• Perform the requested action • Provide as much info about the result as possible • Provide response in few seconds, otherwise use inprogress intent (for ex, network call)
Intent UI Extension • Bring your apps interface into siri ( for ex. Brand, visual
identity)
• Provide a UIViewcontroller - which can be customized
• Completely Optional
• Displayed alongside siri content
• Not interactive - you can’t put "controls" in your UI intent interface.
Creating Intent Extension
Extensions
Siri Entitlement
Configuration plist
NSExtension
NSExtensionAttributes
IntentsSupported
IntentsRestrictedWhile- Locked (optional)
Requesting Authorization in plist Include the NSSiriUsageDescription key in your iOS app’s Info.plist file.
Requesting Siri Authorization in Your iOS App
INPreferences.requestSiriAuthorization() { (status) in print("New status: \(status)") }
Structuring Your App’s Services
Implement your core services in a private shared
framework : A private shared framework lets you define your core services in one code module and link the code dynamically into your iOS app and your Intents extension. Using a framework minimizes the size of both executables and ensures that both use the same code.
Use a shared container to store common
resources. Put relevant images and data files into a shared container so that they can also be accessed by your Intents extension. You can enable shared container support in the Capabilities tab of your iOS app and Intents extension targets.
Specifying Custom Vocabulary - INVocabulary object
• Defining custom vocabulary helps Siri understand commands used in conjunction with your app, which improves the user experience.
• Choose terms that could be misunderstood by someone not familiar with your app.
• Do not register terms that are easily understood, ex. “My Photo Album” or “My Workout”.
• Focus on terms whose literal meaning differs from your app’s usage of those terms. • The most important terms should always be first in the NSOrderedSet object that you
create. • To register terms that are specific to a single user, use the INVocabulary object.
let workoutNames = self.sortedWorkoutNames() let vocabulary = INVocabular.shared() vocabulary.setVocabularyStrings(workoutNames, of: .workoutActivityName)
Global Vocabulary file - AppIntentVocabulary.plist
Demo
Limitation • Only support few domains. Out of the 2 million apps in the App Store, how many fall under the
support of the six categories? This is the primary downside of the system Apple devised.
• If you are very excited to add speech recognition into your app which might not fall into supported Siri kit domain - Try Speech Recognition Framework -- https://realm.io/news/tryswift-marc-brown-say-it-aint-so-implementing-speech-recognition/?utm_campaign=This%2BWeek%2Bin%2BSwift&utm_medium=email&utm_source=This_Week_in_Swift_103
Example Code • Unicorn Chat - https://developer.apple.com/library/prerelease/content/samplecode/UnicornChat/
Listings/SiriExtension_UCSendMessageIntentHandler_swift.html
Thank you !