CMU Usable Privacy and Security Laboratoryhttp://cups.cs.cmu.edu/
Sensor-Based Interactions
Kami Vaniea
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
OutlineOutline
Coping with Uncertainty
Making Sense of Sensing Systems: Five Questions for Designers and Researchers
Distributed Mediation of Ambiguous Context in Aware Environments
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
UncertaintyUncertainty
State of knowledge in which one or more alternatives result in a set of possible
specific outcomes, but where the probabilities of the outcomes are neither
known nor meaningful.
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Uncertain CommunicationUncertain Communication
Sensors produce concrete data with uncertain or ambiguous interpretation•Location Data (GPS)
•Temperature
Computers can make bad decisions based on uncertain data• Interrupting user
•Making poor decisions on user’s behalf
•Communicating wrong information
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
OutlineOutline
Coping with Uncertainty
Making Sense of Sensing Systems: Five Questions for Designers and Researchers
Distributed Mediation of Ambiguous Context in Aware Environments
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
HHI and HCIHHI and HCI
Human to Human Interaction (HHI) has been well studied by researchers
How can we use some of the lessons about Human to Human communication to facilitate Human to Computer communication?
Recognize that communication is a shared responsibility and requires management and repair
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Norman’s Seven Stages of Action
Norman’s Seven Stages of Action
Forming the goal
Forming the intention
Specifying an action
Executing the action
Perceiving the state of the world
Interpreting the state of the world
Evaluating the outcome
Goal
Execution
Evaluation
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Five Issues in CommunicationFive Issues in Communication
Address: Directing communication to a system
Attention: Establishing that the system is attending
Action: Defining what is to be done with the system
Alignment: Monitoring system response
Accident: Avoiding or recovering from errors or misunderstandings
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Stages of Action vs. Conversation IssuesStages of Action vs. Conversation Issues
SystemHuman
Address
Attention
Action
Alignment
Accident
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Address: How do I address or of many possible devices?
Address: How do I address or of many possible devices?
GUI Answers• Keyboard
• Mouse
Exposed Challenges• Disambiguate signal-to-noise
• Disambiguate intended target system
• How to not address the system
Possible Problems• No response
• Unwanted response
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Attention: How do I know the system is ready and attending to my actions?Attention: How do I know the system is ready and attending to my actions?
GUI Answers• Graphical feedback such as blinking cursors
Exposed Challenges• Giving feedback so the user knows it has the system’s
attention
• Periphery feedback
Possible Problems• Wasted input if system not responding
• Unintended action
• Privacy and Security problems(?)
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Action: How do I effect a meaningful action, control its extent and
possibly specify a target or targets of my action?
Action: How do I effect a meaningful action, control its extent and
possibly specify a target or targets of my action? GUI Answers
• Use standard GUI widgets, such as clicking and selecting text, to interact with the system
Exposed Challenges• Identifying and selecting possible actions• Avoid unwanted selection• Handling complex operations
Possible Problems• Limited operations available• Failure to execute action• Unintended action
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Alignment: How do I know the system is doing the right thing?
Alignment: How do I know the system is doing the right thing?
GUI Answers• Graphical feedback such as text appearing• Auditory feedback• Detectible new state
Exposed Challenges• Making system state perceivable• Providing distinctive feedback on results and state
Possible Problems• Differentiation problems• Inability to detect mistakes• Unrecoverable problems
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Accident: How do I avoid mistakes?Accident: How do I avoid mistakes?
GUI Answers• Control/guide in direct manipulation
• Stop, cancel, undo or delete
Exposed Challenges• Controlling or canceling system actions in progress
• Disambiguating what to undo in time
• Intervening when user makes obvious error
Possible Problems• Unintended action
• Inability to recover state
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
DiscussionDiscussion
UbiComp is still a new growing field
Now is the time to create new methods of interaction between Humans and Machines
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
OutlineOutline
Coping with Uncertainty
Making Sense of Sensing Systems: Five Questions for Designers and Researchers
Distributed Mediation of Ambiguous Context in Aware Environments
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Ambiguous Context DataAmbiguous Context Data
Many systems are based on the assumption that context data is unambiguous
However, accurately determining context is a hard problem since the system isn’t in possession of all the facts
Techniques such as AI algorithms can be applied but are not always accurate and require some training
Sometimes it is necessary to ask the human in order to get unambiguous context data
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Architecture of the Extended Context Toolkit
Architecture of the Extended Context Toolkit
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Disambiguating Sensor DataDisambiguating Sensor Data
In an ubiquitous system where all the devices behave in a distributed manor disambiguating data is a shared problem
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Issues in Building Realistic Context-Aware Applications
Issues in Building Realistic Context-Aware Applications
Distribution: Devices are in a distributed environment
Storage: When should ambiguous data be stored?
Multiple Subscription Types: Allow subscribers to opt-in/out and choose their type of subscription
Pre-emption of Mediation: Prevent multiple simultaneous mediations
Forced Mediation: Request mediation by other devices
Feedback: Provide feedback to users about what the system thinks is happening
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
StorageStorage
Should ambiguous data be stored?
Should mediated data be stored?
Both?
How do we store ambiguous data since it is actually a graph of options
Storing only unambiguous data is easier
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
SubscribersSubscribers
Opt-in and Opt-out subscriptions allow subscribers to get only what they want and no more
Those who get ambiguous data will have to mediate it, wait for another to mediate it or force mediation
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Pre-Emption of MediationPre-Emption of Mediation
Mediation may occur simultaneously in several components
If several components are mediating at once the first one to succeed pre-empts the others
Does this mean the users will get multiple requests for mediation?
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
Forced MediationForced Mediation
If a subscriber is unable or doesn’t want to mediate it can request another component to mediate
It passes all ambiguous data to the other component which mediates if possible
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
FeedbackFeedback
Provide feedback about what the system is doing and what its current state is
Feedback can come in many forms from messages to speech
• CMU Usable Privacy and Security Laboratory • http://cups.cs.cmu.edu/
ExamplesExamples
In/Out Board•Monitored when residents of a single building
where present or not
CybreMinder•A situation aware reminder system which can
use situational data to trigger reminders
Word Predictor•Makes typing words easier since it guesses
context
CMU Usable Privacy and Security Laboratoryhttp://cups.cs.cmu.edu/
Questions?