chatbot history - university of texas at austingdurrett/courses/sp2019/lectures/lec...‣ topic...
TRANSCRIPT
![Page 1: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/1.jpg)
ChatbotHistory
![Page 2: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/2.jpg)
TuringTest(1950)‣ Imita:ongame:AandBarelockedinroomsandanswerC’sques:onsviatypewriter.BotharetryingtoactlikeB
A B
C
B B
trainedjudgeC trainedjudge
OriginalInterpreta:on: StandardInterpreta:on:
‣ Thetestisnot“doesthiscomputerseemhuman-liketorandompeoplewithawebbrowser?”
![Page 3: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/3.jpg)
ELIZA‣ Created1964-1966atMIT,heavilyscripted
‣ DOCTORscriptwasmostsuccessful:repeatsuser’sinput,asksinaneques:ons
Weizenbaum(1966)
![Page 4: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/4.jpg)
ELIZA
(.*)you(.*)me
WhydoyouthinkI$2you?
‣ Iden:fykeyword,iden:fycontext,applytransforma:onrule
‣ Veryli`leneedtogeneratenewcontent,butcanonlyhaveonetypeofconversa:on
My(.)(.*)
Whatelsecomestomindwhenyou thinkofyour$1?
![Page 5: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/5.jpg)
Cleverbot‣ Carpenter(1986),onlinesystembuiltin2006
‣ “Nearestneighbors”:humansaysstatementA,findahumanresponseinhuman-humanorhuman-computerchatstostatementA,repeatthat
User:Hitherecleverbot.Cleverbot:Howareyou?User:Doingokay.whatplansdoyouhavetoday?Cleverbot:Idorevision.User:Whatareyourevisingfor?Cleverbot:Mathshistory.User:Howoldareyou?Cleverbot:That'snoneofyourbusiness.User:Whatismathshistory?Cleverbot:Idon’tknow,farmingmaybe
‣ Canofengivesensibleanswers,butthebotdoesn’treallyimposehigh-level discoursestructure
![Page 6: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/6.jpg)
Data-DrivenApproaches‣ Cantreatasamachinetransla:onproblem:“translate”fromcurrentu`erancetonextone
‣ Filterthedata,usesta:s:calmeasurestopruneextractedphrasestogetbe`erperformance
Ri`eretal.(2011)
![Page 7: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/7.jpg)
Data-DrivenApproaches
Ri`eretal.(2011)
![Page 8: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/8.jpg)
NeuralChatbots
![Page 9: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/9.jpg)
Seq2seqmodels
Whatareyoudoing
I
<s>
am going home [STOP]
‣ Justlikeconven:onalMT,cantrainseq2seqmodelsforthistask
‣Whymightthismodelperformpoorly?Whatmightitbebadat?
‣ Hardtoevaluate:
![Page 10: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/10.jpg)
Sub:tlesDatadoyouwanttomeetyoursponsorforthelast10years?ofcourse!buthedoesn’twanttoseeme!
yeah,wewerejustgoingtohituptacobell.well,it'smypleasure.
andwherehadyoubeenjustbefore?i'dbeentothepalaceofthelegionofhonor,theartgallery.
‣ Howcana`en:onbeuseful?
![Page 11: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/11.jpg)
LackofDiversity
Lietal.(2016)
‣ Trainingtomaximizelikelihoodgivesasystemthatpreferscommonresponses:
![Page 12: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/12.jpg)
LackofDiversity
Lietal.(2016)
‣ Solu:on:mutualinforma:oncriterion;responseRshouldbepredic:veofuseru`eranceUaswell
‣Mutualinforma:on:
‣ Standardcondi:onallikelihood: logP (R|U)
log
P (R,U)
P (R)P (U)
= logP (R|U)� logP (R)
‣ logP(R)reflectsprobabili:esunderalanguagemodel
![Page 13: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/13.jpg)
LackofDiversity
Lietal.(2016)
![Page 14: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/14.jpg)
Specificity
Ko,Durre`,Li(2019)
Whatareyoudoing
I
<s>
don’t know [STOP]
speclevel=1(nonspecific)
‣Whentrainingthedecoder,condi:ononthespecificityoftheresponse
Idon’tknow=>speclevel1
‣ Trainaspecificityclassifieronlabeleddata
Goingtothestore=>speclevel3
![Page 15: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/15.jpg)
Specificity
Ko,Durre`,Li(2019)
Whatareyoudoing
I
<s>
don’t know [STOP]
speclevel=1(nonspecific)
Whatareyoudoing
Goingtothestore[STOP]
<s>
speclevel=4(specific)
‣ Attest:me,setthespecificitylevelhighertogetlessgenericresponses
![Page 16: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/16.jpg)
Specificity
‣ Canuseothermodelstotrytofixtheseissues.Butthefactsares:llallmadeup,eveniftheymakesense
Ko,Durre`,Li(2019)
![Page 17: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/17.jpg)
AgentPersonas‣ Howdeepcanaconversa:onbewithoutmoreseman:cgrounding?Basicfactsaren’tevenconsistent…
Lietal.(2016)Persona…‣ Givethebotapersona:setoffactsthatitcanatleastconsistentlyreporton
![Page 18: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/18.jpg)
PersonaChat
Zhangetal.(2018)
![Page 19: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/19.jpg)
PersonaChat
Zhangetal.(2018)
‣ Ranking:retrieveu`erancefromtrainingsettouseattest:me
![Page 20: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/20.jpg)
AlexaPrize
![Page 21: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/21.jpg)
AmazonAlexaPrize‣ Challenge:createabotthatusersinteractwithonaveragefor20minutes(viaAlexa,soturnsarekindofslow)
‣ “Alexa,let’schat”willgetyoutalkingtooneofthese
‣ $1Mprizeifyougetto20minutes.Only$500kwonin2017and2018sinceteamsgotto~10minutes
![Page 22: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/22.jpg)
GunrockSystem
Chen…ZhouYu(2018)
![Page 23: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/23.jpg)
GunrockSystem
Chen…ZhouYu(2018)
‣ NLU:sentencesegmenta:on,parsing,NER,coref,dialogueactpredic:on
‣ Detect:topicintents,lexicalintents,otherintents(tellusertoexit)
‣ Topicmodules(x11):
‣ Animals:retrieveanimaltriviawiththeredditAPI
‣ Holiday:whatholidaysisittoday,etc.
‣ Genera:on:templated
‣ Customdialogueflowforeach
![Page 24: Chatbot History - University of Texas at Austingdurrett/courses/sp2019/lectures/lec...‣ Topic modules (x11): ‣ Animals: retrieve animal trivia with the reddit API ‣ Holiday:](https://reader033.vdocuments.us/reader033/viewer/2022052002/601597c73ce7bd789816cf39/html5/thumbnails/24.jpg)
Takeaways
‣ State-of-the-artchatbotsareheavilyhand-engineered
‣ XiaoIce:MicrosofchatbotinChinese,100M+users,30billionturnstotal,averageuserinteracts60:mes/month
‣ Peopledoseemtoliketalkingtothem…?
‣ Neuralmethodsaremuchlessrobustandrequirelotsofcoercionrightnow,hardtogetdataforthedesiredUX
‣ Next:me:task-orientedsystems(Siri,etc.)