ux scotland 2016 - designing voice and natural language experiences

38
Copyright 2016 © Sabio All Rights Reserved Designing voice and natural language experiences Dan Whaley UX Scotland 2016

Upload: daniel-whaley

Post on 12-Jan-2017

212 views

Category:

Technology


2 download

TRANSCRIPT

Page 1: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All Rights Reserved

Designing voice and natural language experiences

Dan Whaley

UX Scotland 2016

Page 2: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Why now?

User Acceptance

Core Speech Recognition/NLU

Data – fast mobile connections

Exposure – Siri, Google Now, Cortana

Conversational

Interfaces

Page 3: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

User Expectation

Expectation for low effort, intelligent experiences…

Conversation is our most natural communication medium

Well design conversational and speech UIs are the next generation of low effort interface

Intuitive

•User doesn’t need to learn how to interact

Unobtrusive

•Technology or interface doesn’t get in the way

Personalised

•Tailor interaction to the specific user

Anticipatory

•Predicts users next action or intent

Cross Channel

•Switch between channels and time

Page 4: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Platform Explosion

Nina Web/Mobile

Page 5: UX Scotland 2016 - Designing voice and natural language experiences

Design

Page 6: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Design Principles

Start with the user

Design with data

Get out of the way

Do the hard work to make it simple

Design for all

Understand context

Design for the interface

Deliver solutions, not technology

Be consistent, not uniform

Reuse and refine

Test

Design

Analyse

Page 7: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Start with the user… design with data

Page 8: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Conversation as interface

Natural Language

Temporal

Linear

Repetetive

RedundantAmbiguous

Non-spatial

Transient

Humans are wired for

conversation

Part of evolutionary

development

Instinctively interpret both

words and dialogue acts

“The conscious knowledge that speech can have a non human origin

is not enough to overcome the historically appropriate activation of

social relationships by voice.” Nass and Brave 2005

Page 9: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Understand context

When is voice/nlu the right interface?

• Hands free environment

• Low background noise

• Number of questions to qualify task

• Very large number of options

• Controlled domain

When is voice/nlu the wrong interface?

• Lots of graphical information

• Selecting from a large number of defined options

• Noisy environment

• Difficult to say items (email address)

Page 10: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Understand context

Page 11: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Get out of the way

Conversational UI’s can be the least intrusive interface… truly invisible

Keep it that way

Page 12: UX Scotland 2016 - Designing voice and natural language experiences

Conversation

Page 13: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Cooperative Principle

Paul Grice (1913-1988)

Philosopher of language

Conversation is built on cooperation towards mutual understanding by conversing parties

Participants must make every effort to listen and respond appropriately and at the right moment

"Make your contribution such as it is required, at the stage at which

it occurs, by the accepted purpose or direction of the talk exchange

in which you are engaged." (Grice, Studies in the Way of

Words1989: 26).

Page 14: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Maxims Of Conversation

Quantity

• Provide the appropriate amount of information

Quality

• Be truthful, do not say what you know to be false

Relevancy

• Make your contribution directly relevant, advance the conversation

Manner

• Choose words and concepts that can be understood by all participants to ensure clarity

Page 15: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Dialogue Entities

Agent

• People or systems engaged in conversation

Utterance

• Whatever is spoken or written including inherent semantics

Discourse makers

• Verbal or non verbal markers used to signal or mark a change in the conversation

Turn

• Period of time that a particular agent ‘has the floor’ or controls the conversation

Page 16: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Components of Conversation

State

State

Agent AgentAction

Action

Context

Signal

Signal

Me: “hello” (Action: initiate conversation. State: Waiting)

Friend: “hey Dan” (Action: reply, acknowledge. State: Conversing)

Me: “how are you finding the workshop?” (Action: Question, offer turn.

State: waiting response)

Page 17: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Prompts

Prompts are means to communicate or request information from the user or signal change of state or turn

Providing Information

• Order status, Cab ETA, Balance, etc…

Requesting Information

• Destination, date of birth, credit card number

Confirming Information

• Check understanding. Implicit or explicit.

Offering Choices

• Transfer funds, check account balance, etc…

Signalling an Error Condition

• Re-establish dialogue, get the user back on track

Page 18: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Tips for good prompting…

Speak like a human

Avoid jargon or slang

Start with the most

important information

Don’t explain the

interface. Avoid “speak”

Confirm explicitly only

where necessary

Be consistent, but not

uniform

Avoid mixing recorded

and synthesised audio

Don’t focus on errors

Avoid repeating “sorry”

and “please”

Use contractions, “I’m”

Keep choices simple and

distinct

Page 19: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Lost in time…

Mark where you are

• Implicit confirmation, e.g., “Card Payment…”

Use verbal/non-verbal discourse markers

• Silence to signal turn taking

Indicate any likely delay or latency

• Avoid unexpected silence, “one moment while I…”

Establish islands of safety

• Ok, I’ll take you back to the menu

Allow user to escape

• “start over”

Voice is a temporal interface: its easy to get lost

Page 20: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

When it all goes wrong…

No Match

• When the system doesn’t understand the user

No Input

• When the user doesn’t respond to a question

No to confirm

• When the user rejects a confirmed response

Think about what drives each type of error in a

particular context. User is confused, weary,

distracted? Background noise?

Page 21: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Handling error

Take the blame

Give a couple of attempts

Tier prompting

Give examples of how to respond

Islands of safety

Don’t be repetitive

Switch modality

Page 22: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Persona

Function

Congruence

Persona

Coherence

Persona defines the overall

communication style

Diction (familiar vs formal)

Verbosity (concise vs wordy

Tone of voice (warm vs austere)

Characteristics (friendly, aloof,

warm, etc…)

Coherent matching of application function with

coherent persona creates congruent experience

Page 23: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Dom

Page 24: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Jess

Page 25: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Page 26: UX Scotland 2016 - Designing voice and natural language experiences

Implementation

Page 27: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

How Speech Works

• Phonetics

• /k/a//n//s/el/

• “cancel”

• “can sell”

Acoustic Model

• “cancel my account” {.8}

• “can sell my account” {.2}

Linguistic Model • NLU: Extract

meaning

• Action: ‘Cancel’

• Entity: ‘Account’

Semantic Model

I want to

cancel

my

account

Text only starts here

Page 28: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Architecture

Application

Data

Model

Controller

View

Prompts

Recog/

NLU

Action:

Entity

- Users response mapped

to action/entity model

- Dialogue crafted around

these

“Book a ticket to see

Batman tomorrow about

eight”

Book: CinemaTicket

(Film: Batman,

Date: Tomorrow,

Time: 20:00)

Next

view

Page 29: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Training

Setup: Map example ‘utterances’ to semantic model

“Book a cinema ticket” -> Book:CinemaTicket

“make a reservation to see a film” -> Book:CinemaTicket

“get me a ticket to batman” -> Book:CinemaTicket

Craft dialogue based onOptimisation: Review and update regularly

“Need a pass to see batman” -> ??

“I wanna see a movie” -> ??

Page 30: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Data Sources

Contact Centre

Listen to calls –speech analytics

Web Chat

Web transcripts –

text analytics

Mobile or Web

Forum pages –site analytics

Page 31: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Artificial Intelligence

iStock

Page 32: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

AI - Turing Test

Page 33: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

AI Today

Se

lf-d

rivin

gC

ars

Na

tura

l L

an

gu

ag

e P

roce

ssin

g

Pla

yin

g ‘G

o’

Specific Intelligence General Intelligence

Page 34: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

AI Journey

DesignedIntelligence

Human Assisted

Intelligence

Full MachineIntelligence

Page 35: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Future - AI Driven Customer Service

Artific

ial In

tellig

en

ce

Page 36: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Future drivers

Connected devices: no screen

• Wearables

• Smart homes

More data: personalisation/anticipation

• Biometrics

• Location

• Consumer preference

Health: cost of care

• Elder care

• Non-physician care

Page 37: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

Page 38: UX Scotland 2016 - Designing voice and natural language experiences

Copyright 2016 © Sabio All rights reserved

[email protected]

@whaley_d

https://uk.linkedin.com/in/dwhaley-speech-ux

Contact