advanced speech and language technology for complex...

Advanced Speech and Language Technology for Complex

Customer Care Automation and Self-Service

Roberto PieracciniChief Technology Officer

SpeechCycle, Inc.26 Broadway, 11th Floor

New York, NY [email protected]

What is SpeechCycle?

SpeechCycle is the leading provider of 3rd Generation speech applications for digital service providers.

Started in 2001; located in NYC.Rapidly growing software company (about 70 people)Telephone based automatic spoken dialog systems for complex customer care.Deployment models: on-demand and on-premise managed service.Processing millions of complex support calls every month for the largest cable and telecommunication operators in the US and Australia.Full automation rates up to 40%Experts in speech recognition, speech science, software engineering, advanced voice interaction design, and the strategic value of speech systems in the contact center.

A brief history of commercial spoken dialog systems

GENERATIONFIRST SECOND THIRD

Time Period 1994-2001 2000-2005 2004-todayType of

Application Informational Transactional Problem Solving

ExamplesPackage Tracking, Flight Status

Banking, Stock Trading, Train Reservation

Customer Care, Technical Support, Help Desk.

Architecture Proprietary Static VoiceXML Dynamic VoiceXMLComplexity

(Number of DMs) 10 100 1000

Interaction Turns A couple 5-10 More than 10

Dialog Modality directed

directed + natural language (SLU) + mixed initiative

directed + natural language (SLU) + intelligent mixed initiative

LangTech 2008 4February 29, 2008

The Problem

• Cost of customer care is on a steep increasing curve; this is especially true in the Digital Service Provider arena– Devices and services are becoming more and more complex—more

things can go wrong– Number of customer grows year by year– …and so the number of agents needed to provide quality support– Outsourcing and offshoring reached a point of diminishing returns

• Quality of customer care is on a decreasing curve– Long waiting queues– New products and services offered every so often– Agents are not always up to date– Turnover makes agent training difficult and costly– Difficult to maintain consistent quality of service– Infrastructures are not always up to date


…been there

The switchboards were something to behold, with many, many operators sitting in long rows plugging countless plugs into countless jacks. The cost of adding new subscribers had risen to the point unforeseen in the earlier days, and that cost was continuing to rise, not in a direct, but in a geometric ratio. One large city general manager wrote that he could see the day coming soon when he would go broke merely by adding a few more subscribers.

AT&T, Early 1900s


Customer Care AutomationValue Proposition

• Value for the customer (the provider)– Reduced costs

• Value for the final user (the subscriber)– No waiting in a queue– Consistent quality of customer care

• Value for the technology vendor (i.e. us)– Revenue - Profit


Technical Support Automation is Difficult

• Acquisition of knowledge• Keeping up to date with new products and services• Emotional state of callers• Problem identification• Caller mental model• Instructing non technical savvy callers• Uncontrolled events• Challenging acoustic environment• Cultural barriers against speaking to machines• Callers do not trust automated systems can help them• Callers are not cooperative


SpeechCycle’s Approach to Automated Technical Support

• Speech recognition over the telephone with sophisticated Voice User Interface

• Detect call reason using advanced natural language technology• Ask simple questions when needed• Don’t ask questions...if possible. Integrate with provider’s

customer information systems• Instruct caller to follow simple diagnostic and troubleshooting

steps if needed.• Perform diagnostic and troubleshooting steps automatically if

possible. Integrate with diagnostic systems. • Automated reporting and call performance classification• Use log data to monitor and continuously improve automation and

caller experience– Speech– Language– Logic


The Doctor Analogy

• Know your patients even before they talk– Medical records

• Let your patients talk– What’s your problem?

• TAKE THE INITIATIVE• Then ask simple clarification questions

– Does it hurt when you laugh?• Take measurements, run tests• Prescribe cure• If it does not work:

– Try something else• If it does not work:

– Send to a specialist with an updated medical record.


The Doctor Analogy applied toAutomated Technical Support

• Know your callers even before they talk– Integration with Customer Account DB

• Let callers talk– Problem identification with natural language

• TAKE THE INITIATIVE• Then ask simple clarification questions

– Directed dialog• Take measurements, run tests

– Integration with diagnostic tools• Prescribe cure

– Step by step resolution• Problem solved?

– How did we do?• If it did not work:

– Try something else• If it did not work:

– Escalate to human agent


The long term vision for automated technical support

Please describe your problemMy internet connection is slowSit back and relax. I will fix it for you and call you back when I am done!

SPECH ONLY

FULLY INTEGRATED

Customer account information

Home device informationDiagnostic tools

Network state

IMMERSIVECALLER

EXPERIENCE

LangTech 2008 12February 29, 2008 Confidential

The Continuous Improvement Cycle

Call-Flow Design

Call-Flow TestingPrompts and Grammars Management

Real-time VUI optimization and learning

Reporting

Call Data Analysis

Natural Language grammar development, tuning and testing

Speech Recognition Performance analysis

Dynamic Content Refresh

DATA


Handling complexity

CanYouSeeModem

yes

no

WaitGetBrand continue ExpertPrompt

~UserOption.help~InteractionError

MistookRouter

true

ExpertPrompt

nobrand

NeedMakeModel

~exit

~exit

CallerConfirmedCabledataModem=true

TopEnd_CompareModemVars

TopEnd_MakeModel

success

CustomerAccount_Success

SupportedModem

SupportedOhMyModem

client

cox

!!C/C/C!!

Unsupported

Rpt_OutOfScopeExit=UnsupportedModem

HandleTransferOkay

~exit

TopEnd_VerifyModem

CustomerHasMultipleModems

true

ModemHasBatteryBackup

true

ModemHasBatteryBackup

true

TopEnd_NoBrandModem

WaitUnplugModem

ModemProximity

continue ~UserOption.help

yes

no WaitIdentify

~UserOption.help

~InteractionError


WaitUnplugModem_deux

continue

StillNeedHelpIdentifyingno

yes

ProximityBad

~exit

yes

no

noModemProximity_deux

CantReachOrFindPowerCord_deux

otherModemUnplugProblem=true

other

~UserOption~InteractionError

Thanks

ProximityBad Escalate ~exit

reach

find

CanYouReachModemPowerCord

yes

WaitUnplugModem_deux

continueno

yes

Rpt_WantAgent=true

~InteractionError

NeedHelpIdentifying_2

HelpUnplugModemNMUnplugModem

MistookRouter

true

WaitUnplugModem_2

~InteractionError

~UserOption.help continue

OtherProblemCapture_2

TransferQueue=normal

HandleTransferOkay

ThxAndMoreInfoBlurb

ThxAndMoreInfoBlurb

ThxAndMoreInfoBlurb

HelpUnplugModem

CycleModemRecently_1

yes

no

~InteractionError

WaitUnplugModemAndCoax

continue

FirstAction=CameBackPrompt

~UserOption.operator

ThxAndMoreInfoBlurb_1

CallerUnpluggedCoax=true

~InteractionError

WaitGetPenForVoIPSteps

AskIfShouldReadVoIPStepsWithoutPen

~UserOption.help

Good

continue

yes

no

VoIPStep1of4

~InteractionError

AskIfShouldRepeatAllVoIPSteps

~UserOption.helpno

~InteractionError

VoIPStep2of4

~InteractionError.NoInput

VoIPStep3of4


VoIPStep4of4


yes

~UserOption.repeat

VoIPStepsEpilogue

VoIPStepsRepeatPreamble

NB: For CIP4.0 compatibility, in VoIPStepNofN activities, moved RepeatPrompts definitions to HelpPrompts; in all questions, changed RepeatThat grammar use to “help”; removed RepeatPrompts (5158.wav) from

AskIfShouldReadVoIPStepsWithoutPen, and (5164.wav) from AskIfShouldRepeatAllVoIPSteps. We should undo these changes when CIP4.1 is ubiquitous.

HasRouter

true


heardVoIPStepsheardVoIPSteps

true true


VoIPStepsPreamble_router

VoIPStep2of4_router

VoIPStepsPreamble

VoIPStep1of4

VoIPStep3of4


VoIPStep4of4

~InteractionError.NoInput~InteractionError.NoInputabort

operator

Motorola

Linksys

Toshiba

yesno

no

yes

yes

no

no

yes

yes

no

no

yes

yes

no

yes

no

no

yes

yes

yesno

ConfirmZoom noyes

ConfirmNetgear noyes

ConfirmCom21

ConfirmWebstar

ConfirmDLink

ConfirmAmbit

ConfirmBestdata

ConfirmCybersurfer

ConfirmToshiba

ConfirmTerayon

ConfirmRCA

ConfirmMotorola

ConfirmLinksys

NeedMakeModel

NeedMakeModel

NeedMakeModel

NeedMakeModel

NeedMakeModel

NeedMakeModel

NeedMakeModel

ConfirmCMXno

yes

netgear

linksys

motorola

rca

NeedMakeModelModemGroupNameBest dataBestdata

Zoom

CMX

DLink

Scientific Atlanta

Com 21

3Com

Netgear

Ambit

Cybersurfer

RCARCA NEW

rca

Terayonno

IsCMX_1 yes

no

modemtype=cmx LeftThree

modemtype=3com BottomTwo

ConfirmArris yes

noArris

modemtype

linksys netgear

WaitLinksysHaveCableLight

~UserOption.help

DoesLinksysHaveCableLight

no

continue

MistookRouter=true

continue

ThanksForChecking

WaitNetgearHaveStatusLight


DoesNetgearHaveStatusLightyes

no

Mistook Router!

modemtype

netgear

LinksysTooNetgearToo

UnplugThatModem

JustAMoment

ComputerCount=multiple

lightsAgainVisit=net

MistookRouter=true


modemtype

linksys


continue

yes

RouterBrand=linksys RouterBrand=netgear

If caller mistook router, we identify modem properly, then

power it down.

WaitGoFindModem_3 WaitGoFindModem_4

CallerConfirmedCabledataModemtrue

dlink

WaitDLinkHaveStatusLight


DoesDLinkHaveStatusLight

yes

no

RouterBrand=dlink

WaitGoFindModem_5

continue

MistookRouter=true


modemtype

dlink


DLinkToo

TopEnd_ModemDown

TopEnd_GetModem TopEnd_GetModem TopEnd_GetModem

- Hundreds of pages of call-flow (300-400 for typical call flow)

- Thousands of nodes

- Several thousands of prompts

- Hundreds of grammars

Call Flow as Software

Use software engineering principles to create and maintain

large call-flows

- Hierarchical structure- Reuse, libraries- Object oriented principles- Configurability- Source control

Call Flow as Software

Use software engineering principles to create and maintain

large call-flows

- Hierarchical structure- Reuse, libraries- Object oriented principles- Configurability- Source control


Symptom Identification with Statistical Spoken Language Modeling (SSLU)

SSLU toolsInternet Symptoms

0.00%2.00%4.00%6.00%8.00%

10.00%12.00%14.00%

inte

rnet

slow

secu

rity

oper

ator

intern

etse

tup

email

inte

rnet

cant

conn

ect

vide

o

mod

emlogo

n

inte

rnet

appo

intm

ent

hom

enet

wor

k

emailset

upinac

cess

iblesite

s

serv

ice

cable

ques

tion

billin

gvo

ip

emailcan

tsen

dorrec

ipad

dres

sinte

rnet

inte

rmitten

t

emailser

vice

chan

geac

coun

t

softw

are

multip

lesy

mto

ms

pare

ntalco

ntro

l

cras

hfro

zenc

ompu

tye

s

help

emailspa

mpo

pups

canc

elfirew

all

repe

at

emailw

ebm

ail

37 Unique Symptoms225,000 utterances+ 3,000 utterances/week


Video Symptoms

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

14.00%

16.00%

18.00%

Serv

iceN

oSer

vice

Cha

nnelMissing

BoxN

otW

orking

Pictur

eNoP

icture

Intern

et

Pictur

eSno

wyF

uzzy

Line

Prob

lem

Pictur

ePoo

rQua

lity

Ope

rator

Rem

oteN

otW

orking

Appo

intm

ent

Pictur

eMultip

lePicS

ymptom

HDTV

Pictur

eOther

Cha

nnelOther

BoxO

ther

DVR

Ord

erPa

yPer

View

Other

Serv

iceO

ther

BoxM

oxi

Rem

oteV

ague

Soun

dNoS

ound

BoxW

ontT

urnO

n

Ord

erOnD

eman

dOther

TVNoC

ableTV

Guide

NotW

orking

Pictur

eBlack

Rem

oteO

ther

Cha

nnelCan

tCha

nge

TVOther

Rem

oteP

rogr

amMyR

emote

Acco

untB

illing

Serv

iceM

ultip

leOut

Serv

iceV

ague

Acco

untS

ervice

Cha

nge

Ord

erOther

BoxV

ague

Soun

dPoo

rQua

lity

Serv

iceO

utag

e

PINPa

sswor

dPro

blem

Ord

erOnD

eman

dVag

ue

Acco

untO

ther

Soun

dVag

ue

Soun

dOther

Pictur

eVag

ue

Pictur

ePixelated

Pictur

eFro

zen

Phon

e

Ord

erVa

gue

Aske

dToC

all

TVW

ontT

urnO

n

Cha

nnelVa

gue

Pictur

eOther

Color

s

Acco

untV

ague

Ord

erPa

yPer

View

Vagu

e

Ord

erPI

NPa

sswor

d

Ord

erOnD

eman

dError

Erro

rOther

Clock

BoxM

ultip

leBo

xSym

ptom

s




Contender: Exploring Alternative Strategies

Cumulative average automation rate

14.00%

15.00%

16.00%

17.00%

18.00%

19.00%

20.00%

21.00%

10/26 10/27 10/28 10/29 10/30 10/31 11/1 11/2 11/3 11/4 11/5 11/6 11/7

WITH EXAMPLESOFFER CHOICESORIGINAL

0

5000

10000

15000

20000

25000

10/26 10/27 10/28 10/29 10/30 10/31 11/1 11/2 11/3 11/4 11/5 11/6 11/7

Number of calls


Conclusions

• The third generation of dialog systems is here• Problem Solving: not just form filling or transactions• The doctor analogy• The power of backend integration and data • Call-flow as software• Symptom Identification and SSLU• Exploration and Learning

advanced speech and language technology for complex...

Documents