ManagingActive&HealthyAgeingwithServiceRobots
D5.7– RobotSemanticSentimentAnalysis
ProjectAcronym: MARIO
ProjectTitle: Managingactiveandhealthyagingwithuseofcaringservicerobots
ProjectNumber: 643808
Call: H2020-PHC-2014-single-stage
Topic: PHC-19-2014
TypeofAction: RIA
Ref. Ares(2017)3839883 - 31/07/2017
ManagingActive&HealthyAgeingwithServiceRobots
D5.7– RobotSemanticSentimentAnalysis_FinalversionWorkPackage WP5
DueDate M30
SubmissionDate 31/07/2017
StartDateoftheProject 01/02/2015
DurationoftheProject 36Months
Organisation ResponsibleofDeliverable CNR
Version 2.1
Status Final
Authorname(s) ValentinaPresutti (CNR)AldoGangemi (CNR)DiegoReforgiato Recupero (R2M)Domenico Pisanelli (CNR)PaoloCiancarini (CNR)AndreaGiovanniNuzzolese (CNR)LuigiAsprino (CNR)AlessandroRusso(CNR)
Reviewer(s) ChristosKouroupetroglou (CNET),André Freitas(R2M)
Nature ☐ R– Report☐ P– Prototype☐ D– Demonstrator☒ O– Other
DisseminationLevel ☒ PU– Public☐ CO– Confidential,onlyformembersoftheConsortium(includingtheCommission)☐ RE– RestrictedtoagroupspecifiedbytheConsortium(includingtheCommissionServices)
2
ManagingActive&HealthyAgeingwithServiceRobots
RevisionHistoryVersion Date Modifiedby Comments
0.1 16/06/2017 ValentinaPresutti (CNR) Firstdraft startingfromD5.3
0.2 19/06/2017 AlessandroRusso(CNR) Revisionandchangesonthedocumentstructure
0.3 22/06/2017 AlessandroRusso(CNR) Addedcontentonpolaritydetection
0.4 26/06/2017 AndreaGiovanniNuzzolese (CNR) Revisionandextensionofsemanticsentimentanalysiscontent
0.4.1 27/06/2017 AndreaGiovanniNuzzolese (CNR) Unifiednotationforfigureswithknowledgegraphs
0.5 05/07/2017 AlessandroRusso(CNR) Initialcontentonusecasescenario
0.5.1 06/07/2017 AlessandroRusso(CNR) ContentandfiguresforMyMemoriesscenario
0.5.2 10/07/2017 AndreaNuzzolese (CNR) Addedframe-basedexampleinusecasescenario
0.6 13/07/2017 AlessandroRusso(CNR) Revisionofdocumentoutlineandoverviewsections
1.0 16/07/2017 AlessandroRusso(CNR)Overallrevisionandminoreditstoimprovereadability;draftversionforinternalreview
continues
3
ManagingActive&HealthyAgeingwithServiceRobots
RevisionHistory(cont.)Version Date Modifiedby Comments
1.0 19/07/2017 AdamSantorelli (NUIG) Commentsandfeedbackonv1.0
1.0 21/07/2017 ChristosKouroupetroglou (CNET)AndréFreitas(PASSAU) Reviewer'sfeedbackonv1.0
1.1 25/07/2017 AlessandroRusso(CNR) Revisionandupdatesaccordingtoreviewer'sfeedback
2.0 27/07/2017 AlessandroRusso,LuigiAsprino,AndreaGiovanniNuzzolese (CNR)
Revisionoftheoveralldocumentwrtreviewers’comments;addedabbreviationslist;minorformattingedits;versionforcoordinator
2.0 28/07/2017 AislingDolan(NUIG)AlexandrosGkiokas (ORTELIO) Commentsandfeedbackonv2.0
2.1 28/07/2017 AlessandroRusso(CNR) Revisionwithadditionaledits;finalversion
4
ManagingActive&HealthyAgeingwithServiceRobots
Copyright © 2016, MARIO Consortium
The MARIO Consortium (http://www.mario-project.eu/) grants third partiesthe right to use and distribute all or parts of this document, provided that theMARIO project and the document are properly referenced.
THIS DOCUMENT IS PROVIDED BY THE COPYRIGHT HOLDERS ANDCONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES,INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OFMERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AREDISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER ORCONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITEDTO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANYTHEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THEUSE OF THIS DOCUMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCHDAMAGE.
5
ManagingActive&HealthyAgeingwithServiceRobots
ExecutiveSummary
Deliverable 5.7 describes the software components that were designed anddeveloped in the context of Task 5.3 (WP5) to provide the MARIO systemarchitecture and software framework with Sentiment Analysis capabilities.This version represents the final, consolidated version of Deliverable 5.3.
Specifically, the components presented here constitute MARIO’s SentimentAnalysis subsystem. The main component aims at providing semanticsentiment analysis capabilities, through a formal representation andevaluation of the sentiment expressed in text sentences. It is built as anextension of FRED (a machine reading component introduced in Deliverable5.6) and relies on novel resources developed in the context of this task.
6
ManagingActive&HealthyAgeingwithServiceRobots
TableofContentsExecutiveSummary 6ListofAcronymsandAbbreviations 8Introduction 9
WorkPackage5objectives 10Purposeandtargetgroupofthedeliverable 11Relationstootheractivitiesintheproject 12DocumentOutline 13AboutMARIO 14
OverviewonSentimentAnalysis 15MARIOSentimentAnalysisSubsystem 20
Polaritydetection 25Semanticsentimentanalysis 29
Opinionmodelontology 33Sentimentlexicalresources 39Frame-basedsentimentanalysis 42
RepresentativeUseCaseScenario 53SentimentanalysisintheMyMemoriesapp 54
Concludingremarks 67References 69
7
ManagingActive&HealthyAgeingwithServiceRobots
AcronymsandAbbreviations
8
API ApplicationProgrammingInterface
CGA ComprehensiveGeriatric Assessment
JSON JavaScriptObjectNotation
KB KnowledgeBase
MPI Multidimentional PrognosticIndex
MON MARIOOntologyNetwork
NLP NaturalLanguage Processing
NLU NaturalLanguage Understanding
OWL WebOntologyLanguage
PWD Person/PeoplewithDementia
RDF ResourceDescriptionFramework
REST Representationalstatetransfer
S2T SpeechtoText
ManagingActive&HealthyAgeingwithServiceRobots
Introduction
Naturallanguageistheprimarymeansthroughwhichpeoplewithdementia(PWD)caninteractwithMARIOrobots.Task5.3dealswithMARIO’scapabilitytoextractsentimentinformationfromnaturallanguagesentencesexpressedbyapersonwithdementia(PWD).TheabilitytoautomaticallyextractandcategorisesentimentinformationfromthetextualrepresentationofnaturallanguagesentencesenablesMARIOtoadjustandadaptitsbehaviouraccordingtothesentimentoropinionpotentiallyexpressedbyPWD.
Thisdeliverablepresentstheapproaches,algorithmsandsoftwarecomponentsthatenableMARIOtoperformsentimentanalysisoverthetextualrepresentationofspokennaturallanguageinput.Inparticular,MARIOisequippedwithsemanticsentimentanalysiscapabilities,whichexploitaframe-baseformalrepresentationofthetextualinputbyaugmentingtheNaturalLanguageProcessing(NLP)capabilitiesdiscussedinDeliverable5.6.
9
ManagingActive&HealthyAgeingwithServiceRobots
WorkPackage5Objectives
WP5aimsatdevelopingtheframeworkandtoolsthatallowMARIOrobotstointeractwithhumansandunderstandtheirneedsexpressedthroughspokennaturallanguage.Asfundamentalbuildingblocks,understandingcapabilitiesexploitmachinereading/listeningcomponentsandRDF/OWLontologiestofirstproduceandthenprocessaformalencodingofthetextualrepresentationofnaturallanguage.Assuch,themainobjectivesofWP5are:• todesignanddeveloptheMarioOntologyNetwork(MON)andKnowledgeBase;
• toprovideMARIOwiththeabilityoftransformingnaturallanguageintoaformalrepresentation,toenablereadingandlisteningcapabilitiesonthebasisofFRED;
• toprovideMARIOwiththecapabilityofrecognising,storingandreusingsentimentinformation,onthebasisofsemanticsentimentanalysistechniques.
10
ManagingActive&HealthyAgeingwithServiceRobots
PurposeandTargetGroupoftheDeliverable
ThisdeliverableaimsatdescribingthesoftwarecomponentsdesignedanddevelopedinTask5.3.Thesecomponentsprovidetherobotwithsentimentanalysiscapabilitiescomplementingtheabilitytoprocessandunderstandspokennaturallanguage.Theroleofsentimentanalysisinhuman-robotinteraction,withafocusonPWD,isconsidered.
Duetoitstechnicalnature,thedeliverableismainlytargetingresearches,practitionersanddevelopersinterestedinsentimentanalysistechniquesandalgorithms,andinparticularsemanticframe-basedtechniquesandtheirapplicationforservicecompanionrobots.Inadditiontothetechnicalaspects,concreteusecasesareconsidered,withtheaimofprovidinghealthexpertswithanunderstandingonhowthesetechniquescanimprovetheinteractionbetweenPWDandcompanionrobots.
11
ManagingActive&HealthyAgeingwithServiceRobots
RelationstootherActivitiesintheProject
ThisdeliverabledirectlyreliesontheresultsofTask5.1(seeD5.1)asfarasthebackgroundknowledgeandknowledgemodelsthatitusesareconcerned(aspartoftheMON– MARIOOntologyNetwork),anditisstronglyrelatedtotheactivitiescarriedoutinTask5.2concerningNaturalLanguageUnderstanding(seeD5.6).
TheoverallWP5receivesasinputtheuserandfunctionalrequirementsandthesystemarchitecturefromWP1,whileWP2providestheKompai robotandplatformwherethesoftwarecomponentsaredeployed.ThesecomponentsprovidesentimentanalysisservicestotheapplicationsandmodulesdevelopedinWPs3,4and6withspecificfocusoncapturingandrepresentingsentimentknowledge,inlinewiththeintegrationproceduresdefinedinWP7.
ValidationactivitiesinWP8providefeedbacktotheiterativedesignanddevelopmentprocessofthesoftwarecomponents,andcontributetotheirevolutionandrefinement.
12
ManagingActive&HealthyAgeingwithServiceRobots
DocumentOutline
Therestofthisdeliverableisorganisedinthreemainsections.Specifically:
• thefirstsectionprovidesanoverviewofsentimentanalysisanditsroleinsupportingtheinteractionwithPWD;
• thesecondsectionprovidesadescriptionofMARIO’sSentimentAnalysissubsystem,withafocusonthemodels,resourcesandalgorithmsthatsupportandenablesemanticsentimentanalysiscapabilities;
• thethirdsectionpresentsarepresentativeusecasescenariosfortheSentimentAnalysissubsystem,byoutlininghowitscapabilitiesareusedinthecontextoftheMyMemoriesapplicationforreminiscence.
13
ManagingActive&HealthyAgeingwithServiceRobots
AboutMARIO
MARIO addresses the difficult challenges of loneliness, isolation and dementia in olderpersons through innovative and multi-faceted inventions delivered by service robots.The effects of these conditions are severe and life-limiting. They burden individualsand societal support systems. Human intervention is costly but the severity can beprevented and/or mitigated by simple changes in self-perception and brain stimulationmediated by robots.
From this unique combination, clear advances are made in the use of semantic dataanalytics, personal interaction, and unique applications tailored to better connectolder persons to their care providers, community, own social circle and also to theirpersonal interests. Each objective is developed with a focus on loneliness, isolationand dementia. The impact centres on deep progress toward EU scientific and marketleadership in service robots and a user driven solution for this major societalchallenge. The competitive advantage is the ability to treat tough challengesappropriately. In addition, a clear path has been developed on how to bring MARIOsolutions to the end users through market deployment.
14
ManagingActive&HealthyAgeingwithServiceRobots
SentimentAnalysis
ManagingActive&HealthyAgeingwithServiceRobots
SentimentandEmotionswhenInteractingwithPWD
Theroleandimportanceofrecognisingthesentiment oremotionalstate ofPWDwheninteractingwiththemthroughaconversationalapproachislargelyrecognised.
IdentifytheemotionalstateoftheresponseHowisthispersonfeeling?Iftheyhavebeenabletospeak,whatdothewordsconvey?
http://www.scie.org.uk/dementia/after-diagnosis/communication/conversation.asp
Sometimestheemotionsbeingexpressedaremoreimportantthanwhatisbeingsaid.
http://www.alz.org/care/dementia-communication-tips.asp
16
ManagingActive&HealthyAgeingwithServiceRobots
SentimentAnalysis– Overview(1/2)
Sentimentanalysisconcernsthestudyofintelligentalgorithmscapableofautomaticallymining(i.e.,identifyingandcategorising)opinionsfromnaturallanguagecontent.
InthecontextofMARIO,weareinterestedinapplyingsentimentanalysistosentencesinnaturallanguagespokenbyPWD,especiallywhentheyareexpectedtoexpressfeedbackaboutsomerecentactivityortoconverseaboutevents,peopleandplacesthatcharacterisetheirlifehistoryandpastexperiences.
ThisinformationcanthenbestoredandusedinordertocontributetoapersonaliseduserexperienceandinfluenceMARIO’sdecisionmakingwheninteractingwiththeusers.
17
ManagingActive&HealthyAgeingwithServiceRobots
SentimentAnalysis– Overview(2/3)
TheinitialworkingandresearchhypothesiswasbasedontheassumptionthatsentimentanalysiscapabilitiescouldbemainlyusedtosupporttheComprehensiveGeriatricAssessment(CGA)andMultidimensionalPrognosticIndex(MPI)roboticmodules(investigatedinWP4).
AsdetailedinDeliverables4.3and5.6,thehuman-robotinteractionprocessrequiredtosupportCGAhastobebasedonaconversationalapproachdrivenbystandardisedclinicalquestionnaires.
Sincetheinitialvalidationactivitiesandon-the-fieldobservationsrelatedtothedevelopmentoftheCGAmodule,itemergedthatPWD’srepliestothequestionsdefinedintheassessmentquestionnairesdonotcarrysentimentinformation (mostofthequestionsassumeayes/noanswerorarebasedonamultiple-choicestructure),preventingsentimentanalysistechniquestobeeffectivelyusedandcontributetotheMPI.
18
ManagingActive&HealthyAgeingwithServiceRobots
SentimentAnalysis– Overview(3/3)
Asdetailedlaterinthisdeliverable,animmediateapplicationofsentimentanalysiscapabilitiesistotheMyMemories appforreminiscence(presentedinD3.3),whichenablesMARIOtointeractwithPWDtostimulateconversationandelicitmemories,bypromptingthemandaskingthemtoexpresstheirmood/feelings/opinion(e.g.,aboutpersonaleventsorpeople)afterlookingatapicturewithfewwordsorasentence,etc.
ThisinformationexpressedthroughspeechiscapturedbytheUnderstandingSubsystem andprocessedbytheSentimentAnalysismodule,soastoproducedataexpressingsentimentandemotionalinformationassociatedtospecificentitiesorevents.
AtalaterstageMARIOcanusethisinformationinordertodecidewhetherto,e.g.,showapictureagainortalkaboutaspecificpersonorpastevent,accordingtospecificgoalsandconversationalstrategiesitisimplementing.
19
ManagingActive&HealthyAgeingwithServiceRobots
MARIOSentimentAnalysisSubsystem
Components,servicesandcapabilities
ManagingActive&HealthyAgeingwithServiceRobots21
ArchitecturalReference
ReferringtoMARIOarchitecture,thistaskcontributestothe NaturalLanguageUnderstandingsubsystem(seeD5.6),byintroducingsentimentanalysismodulesandservices.
ManagingActive&HealthyAgeingwithServiceRobots
Approach andContributions
InordertoachievethegoalofthistaskandprovideMARIOwithmultilingualsentimentanalysiscapabilities,we:• adoptedamodular andservice-orienteddesign approach,tosupportmultipleapproacheshavingdifferentcapabilities;
• reusedanontologyforrepresentingopinions;• integratedsentimentlexicalresources intoMARIO’sbackgroundknowledge(relyingontheFramester resource[13,14],describedinDeliverable5.1);
• developedasentence-polarity evaluationmodule;• extendedandimplementedasemanticframe-basedsentimentanalysis algorithm.
22
ManagingActive&HealthyAgeingwithServiceRobots
SentimentAnalysisComponents
TheSentimentAnalysisSubsystemcomplements(andrelieson)thecapabilitiesprovidedbytheNaturalLanguageUnderstanding(NLU)Subsystem(describedinD5.6).
Itimplementstwomainstrategies,detailedinthenextslides,forcomputingsentiment andemotionalscores:1. simplesentimentpolarityanalysis
– itoperatesatasentencelevel,toidentifytheoverallpolarity (Negative-Neutral-Positive)ofagivensentence;
2. advancedsemanticsentimentanalysis– itoperatesonasemanticframe-basedrepresentation ofasentence,toidentifythe
sentimentexpressedbyanopinionholder onacertainentity ortopic;– itreliesontheMARIOKnowledgeBase(accessedviatheRESTAPIprovidedbyLizard;see
Deliverable5.1) andotherresourcesalreadyexploitedbytheNLUsubsystem,withtheadditionofsentimentlexicalresources;
– thiscapabilityisprovidedbyasoftwarecomponentaccessiblethroughaRESTservice1.
231 http://wit.istc.cnr.it/stlab-tools/sentilo/service
ManagingActive&HealthyAgeingwithServiceRobots
ComponentsOverview
24
• ThisfigurehighlightsthemaindependenciesandtheflowamongthecomponentsdevelopedinthistaskandothercomponentsintheMARIOarchitecture.
• TheSentimentAnalysissubsystemdirectlyinteractswithMARIOKnowledgeBaseviatheRESTAPIprovidedbyLizard (seeDeliverable5.1)forretrievingandstoringdata.
• Theothercomponentsaredevelopedinotherproject’stasksandWPs(T5.1,T5.2,WP3,WP4andWP6).
ManagingActive&HealthyAgeingwithServiceRobots
PolarityDetection
ManagingActive&HealthyAgeingwithServiceRobots
Sentence-basedPolarityDetection
Aimsatidentifyingtheoverall tonality/sentiment (reflectingspeaker’sattitude)expressedinasentence.• Itclassifiesagivensentence/textaccordingtoapolarityscale (typically,Negative-
Neutral-Positive),potentiallywitharankingscore orconfidencevalue.• Itmapstoaclassificationproblem,addressedwithMachineLearningtechniques;
– e.g.,NaïveBayes,MaximumEntropy(MaxEnt),SupportVectorMachines(SVM)orDeepNeuralNetworkclassifiers,exploitingsyntacticfeaturesextractedfromthesentences.
• ThepolarityclassificationstepisprecededbyNaturalLanguageProcessing(NLP)tasks:– basicNLPpipelineincludestokenization,sentencesplittingandsyntactic
parsing;– part-of-speech(POS)taggingandlemmatisationstepscanbeaddedtothe
pipeline.
26
ManagingActive&HealthyAgeingwithServiceRobots
PolarityDetectionModule
MARIO’spolaritydetectionmodule reliesontheSentimentAnalysiscapabilitiesoftheStanfordCoreNLP framework[1],which:• usesadeeplearningapproachwithRecursiveNeuralNetworks,trainedonthe
StanfordSentimentTreebankdataset[2];• isbasedonanunderlyingmodelthataimsatcapturingtheeffectsofcontrastive
conjunctionsaswellasnegation;• classifiesandscoresinputsentencesaccordingto5classesofsentiment:very
negative(-2),negative(-1),neutral(0),positive(1),andverypositive(2);• reliesonaNLPpre-processingpipelinewithtexttokenization,sentencesplitting
andsyntacticparsing.
27
Tokenization SentenceSplitting
SyntacticParsing
SentimentClassification
Polaritydetectionmodule
2verypositive
1positive
0neutral
-1negative
-2verynegative
inputtext
foreachsentence
ManagingActive&HealthyAgeingwithServiceRobots
PolarityDetection
Pros:+ pre-trainedalgorithmsandmodelsareavailableforthedifferentstepsof
theprocessingpipeline;+ goodforsentencesexpressingasingle,generalopinionorsentiment;+ simpletouseandinterprettheanalysisresults.
Limitations:- unabletoclearlydistinguishmultipleopinionsexpressedinasingle
sentenceoverdifferententities/topics• e.g.,thesentence“Thefoodwasgreat,buttheweatherwassobad!” isclassifiedasa
wholeasNegative,withnodistinctionaboutthedifferentopinionsexpressedaboutthefoodandtheweather
- doesnotexploitsemanticfeaturesoftheinput.
Beyondpolaritydetection:abilitytoidentifythesentiment expressedbyanopinionholder onacertainentity ortopic.
28
ManagingActive&HealthyAgeingwithServiceRobots
SemanticSentimentAnalysis
ManagingActive&HealthyAgeingwithServiceRobots
What’sanopinion
Anopinion canbedefinedasanintentionalstatementbysomebody(holder)onsomefact(topic)thatisexpressedwithapossiblesentiment.
ASentimentAnalysissystemshouldbeabletoextractandcharacteriseopinions byrecognisingtheattitude (positive,negativeorobjective)ofanopinionholder onacertaintopic,orbyevaluatingtheoveralltonality ofasentence.
30
ManagingActive&HealthyAgeingwithServiceRobots
Formaldefinition
ThegoalofSentimentAnalysisistodetectquintuples(ej,ajk,soijkl,hi,tl)fromunstructuredtext,whereanopinionisaquintuple[3,4]:
(ej,ajk,soijkl,hi,tl)
where:• ej isatargetentity;• ajk isanaspect/feature oftheentityej;• soijkl isthesentimentvalue oftheopinionfromopinionholderhi onaspectajk ofentityej attimetl.soijkl ispositive,negativeorneutral,orarating;
• hi isanopinionholder;• tl isthetime whentheopinionisexpressed.
31
ManagingActive&HealthyAgeingwithServiceRobots
SemanticsintoSentimentAnalysis
Traditionalapproacheshardlycopewithsubtlelinguisticforms,combinedandconcurrentpositive/negativeopinions,andimplicitjudgements.
Theliteratureshowsevidencethattheinclusionofsemanticfeatures insentimentanalysisalgorithmsimprovestheiroverallperformance,e.g.[5].
Linkeddata,ontologies,controlledvocabularies,andlexicalresourceshelpaggregatingtheconceptualandaffectiveinformationassociatedwithnaturallanguageopinions.
32
ManagingActive&HealthyAgeingwithServiceRobots
OpinionModelOntology
ManagingActive&HealthyAgeingwithServiceRobots
Opinionmodelontology (1/5)
Ouropinionmodel definesthemainconceptscharacterisinganopinion,andisusedforannotating thesemanticrepresentationofasentence,inordertoidentifyitsopinionholderandtopic.
34
ManagingActive&HealthyAgeingwithServiceRobots
Opinionmodelontology(2/5)
35
AsshowninthisFigure,wedefinethesemanticrepresentationofanopinionsentenceashavingtwoparts:(i) opiniontriggercontextand(ii) opinionatedcontext.
Theopiniontriggercontext isoptionalandidentifiesconceptsthatindicatethepresenceofanopinionexpressedinthesentence,anditsholder.Itiscomposedoftwoparts:theentitiesthatallowtheidentificationoftheopinionholders (i.e.holders),andtheeventse.g.,think,say,support,etc.,thatactastriggersofanopinion(i.e.opiniontriggers).
Theopinionatedcontextidentifiesconceptsthatexpressanopinion(possiblyincludingsentiments).Itiscomposedoftwoparts:theentitiesthatidentifytheopinion topics (i.e.topics),andthoseexpressingtheopinionanditspossibleassociatedsentiments(i.e.,theopinionfeatures).
Insomecases,termsthatactivateanopinioncanalsoconveytheopinionitselforapossiblesentiment,e.g.“approve”or“deny”.Whenthishappens,suchtermsplaytheroleofopiniontriggersaswellasthatofopinionfeatures.
ManagingActive&HealthyAgeingwithServiceRobots
Opinionmodelontology(3/5)
36
Asfortopics,wedistinguishbetweenmaintopicsandsub-topics.
Infact,atopiccanbeacomplexstructure.e.g.aneventorasituation,includingothercomplexstructures.ForthepurposeofSentimentAnalysis,itisimportanttodistinguishallmaintopics,thatarethedirecttargetsofanopinion,fromsub-topics,whichcouldbeindirecttargetsofanopinion.
Forexample,themaintopicoftheopinionsentence:“Annasaystheweatherwillbecomebeautiful”istheeventbecome,whileweather isasub-topic.
ManagingActive&HealthyAgeingwithServiceRobots
Opinionmodelontology(4/5)
37
Thefollowingaxiomsformalisetheconceptsofmaintopicandsub-topicbyusingastandarddescriptionlogicsyntax,directlytranslatableintoOWL(thelanguageusedforrepresentingallMARIOontologies; seeDeliverable5.1).
(MainTopic ⊔ SubTopic) ⊑ Topic
Topics(ofopinionsentences)canbeeithermaintopics(directtargetsoftheopinion)orsub-topics(indirecttargetsofanopinion).
(Topic ⊓ (∃involvedIn(dul:Situation ⊓ MainTopic))) ⊑ SubTopic
Whenamaintopicisasituation,itsinvolvedentitiesaresubtopics.
(Topic ⊓ (∃dependsOn(dul:Event ⊓ MainTopic))) ⊑ SubTopic
Whenamaintopicisanevent,entitiesthathaveadependencyrelationwithitaresub-topics.Examplesofdependencyrelationsare:participationintheevent(e.g.,havingaroleinit),
causalityrelations,temporalrelationse.g.,follows,precedes),etc.
ManagingActive&HealthyAgeingwithServiceRobots
Opinionmodelontology(5/5)
Followingtheopinionmodel,thesemanticrepresentationofthesentence“Annasaystheweatherwillbecomebeautiful”isdepictedintheFigureabove.
38
ManagingActive&HealthyAgeingwithServiceRobots
SentimentLexicalresources
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentlexicalresources (1/2)
Lexicalresourcesarekeytoannotate naturallanguagesentenceatthe“lexicallayer”withsentimentinformation.
Theyprovidetheterms (andpossiblyascore forthem)thatactastriggers ofanopinionorasopinionfeatures.
MARIOSemanticSentimentAnalysismodulereliesondifferentlexicalresources,listedinthenextslide.
40
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentlexicalresources (2/2)
• Sentic-Net [6]:apubliclyavailablesemanticandaaffectiveresourceforconcept-levelopinionandsentimentanalysis.SenticNetisbuiltbymeansofsenticcomputing,aparadigmthatexploitsbothAIandSemanticWebtechniquestobetterrecognise,interpret,andprocessnaturallanguageopinionsovertheWeb.WehavetransformedSentic-NettotheRDFformatandaligneditwithFramester(see D5.1)soastoexploititintheSentimentAnalysismodule.
• SentiWord-Net [7]:alexicalresourceforopinionmining.SentiWordNetassignstoeachsynsetofWordNetthreesentimentscores:positivity,negativity,objectivity.ThisresourceisintegratedwithinFramesterwiththesameapproachandaimastheintegrationofSenticNet.
• DepecheMood [8]:anemotionlexiconbuiltby harvestingcrowdsourcedaffectiveannotation fromasocialnewsnetwork.
• (arevisionof)theLevin’sclassificationofverbs [9]:inthisrevisionwehaveclassifiedfourclassesofopinionverbsthatimplythepresenceofanholder;wecallthemopiniontriggerverbs.TheyarealsoincludedinMARIObackgroundknowledgethroughalignmentwithFramester.
• Sentilo-Net [10]:aresourcethatannotatessemanticrolesofframessoastoenable theselectionofsubtopicsthatareindirectlyaffectedbyopinionsexpressedinasentence,aswellastheevaluationoftheirpolarity.Thisresourceiskeytoevaluatesentimentexpressedincomplexstructuressuchaseventsanddistinguishthedifferentsentimentsassociatedwiththeirparticipants.
41
ManagingActive&HealthyAgeingwithServiceRobots
Frame-basedsentimentanalysis
ManagingActive&HealthyAgeingwithServiceRobots
Frame-basedsentimentanalysis
ThismoduleisimplementedasanextensionofFRED,henceitexploitsadeepparsinganalysis ofasentenceanditsframe-basedrepresentation.
FREDaswellastheuseofBabelNet1 aspartofFramester (seeD5.1)guaranteetheapplicabilityofourmethodtobothItalianandEnglish,whicharethetargetlanguagesforMARIO.
Theframe-basedrepresentationofthesentence isfurtherannotatedwiththeopinionmodelontology(seeslides32-37)
Thecoreofthemoduleisasentimentpropagationalgorithm thatreliesontheSentilo-Netresource(seeslide40).Thealgorithmcomputesthesentimentscoreassociatedwitheachspecificidentifiedtopic inthesentencerepresentation,accordingtotherolethattheyplayintheirparticipatingframe.
Inthefollowingslidesweshowthroughanexamplehowthealgorithmworksandhowthedifferentlexicalsentimentresourcesareinvolved.
431 http://babelnet.org/
ManagingActive&HealthyAgeingwithServiceRobots 44
Thegraphaboverepresentsthesentence:“PeoplehopethatthePresidentwillbecondemnedbythejudges”
ThesentenceisfirstprocessedbyFRED (see D5.6), whichprovidesasoutputitsframe-basedgraphrepresentation.
ManagingActive&HealthyAgeingwithServiceRobots 45
Thegraphisthenpassedtotheframe-basedsentimentanalysismodule,thatannotates theresultinggraphwiththeopinionmodelontology.
Thisannotationisperformedbyidentifyingtriggeringverbs,opinionholders andopiniontopicsaccordingtorulesassociatedwiththerevisedLevin’sverbclassification[9].
Furthermore,thesemanticrolesidentifiedinthesentenceareannotated accordingtotheSentilo-Netresource,inordertoindicateforeachsub-topicwhetherandhowitisindirectlyaffectedbytheevent(e.g.,sentilo:playSentisitveRole).
ManagingActive&HealthyAgeingwithServiceRobots 46
Finally,basedontheSentilo-Net-relatedannotations,thescoresarepropagatedthroughthegraph inordertoassignthemtotheacutalentitiestheyarereferredtoandwiththecorrectsign.
Allsentimentfeatures (termscarryingasentimentpolarity)areidentifiedandtheirscoresrepresentedinthemodel.
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentpropagationalgorithm (1/4)
Itreliesonasub-algorithmnamedCombinedScorewhichassignsanindividualsentimentscore(ifapplicable)toeachelementinanopinionsentencegraph. Tothisaim,itreliesonSentiWordNetandSenticNet.
Thealgorithmassignsascoretoadjectivesandadverbsthatareidentifiedbydul:hasQuality relationvalues,andtoinstancesofdul:Event thatarerecognisedastriggerevents,i.e., identifiedbysentilo:hasOpinionTrigger relationvalues.
Wehaveinvestigatedandimplementedtwoalternativeapproachesforscoreselection:• thefirstapproachassignsascoreretrievedbyqueryingthepolarityattributeofaconceptinSenticNet;
• thesecondonecombinesSenticNetandSentiWordNetscores.
47
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentpropagationalgorithm (2/4)
FromasetofempiricalobservationsonusingthePropagationalgorithmwithdifferentapproaches,wenoticedthatthemethodthatcombinesthescoresfromthetworesourcesshowstobemorereliable.
Thisconfirmedourexpectationsbasedonthefollowingrationale:sometimes,SenticNetmissesascorevalueforarequiredconcept.Moreover,itprovidesonescoreperconceptwithoutdistinguishingitspossibledifferentnuances.Hence,SenticNetscoreapproximatesanaveragevalueforthescoresofallpossiblesenses,orpossiblyindicatesthemostprobableone.
Forthisreason,combiningtheSentiWordNetscoresofmostfrequentsensesandtheSenticNetscorecanprovideanappropriatebalancedvalue.Wehavedevisedasimpleheuristics forcomputingthiscombinedscore.
InthenextslidetheCombinedScorealgorithmissketched.
48
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentpropagationalgorithm(3/4)
49
CombinedScore Algorithm
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentpropagationalgorithm(4/4)
Givenanentity,identifiedasatopicofanopinion (eitheramainorsubtopic),wecomputeitssentimentscore bycombiningthescoresofallitsassociatedopinionfeatures (i.e.,valuesofdul:hasQualityrelations),whichareextractedfromtheRDFgraphrepresentingtheopinionatedsentence.
Ifatopicparticipatesinanevent orasituationoccurrence,wepropagate theirsentimentscorestoit,accordingtothesemanticsexpressedbytheframe-basedthematicrole(e.g.,vn.role:Agent)thatitplays,itssensitivenessandfactualimpactattributevalues.
50
ManagingActive&HealthyAgeingwithServiceRobots
Where:• sc(x) isthescoreofanentityx asprovidedbytheCombinedScorealgorithm;• qi(t) isanobjectvalueofatripletdul:hasQualityqi.Suchtriplesrepresentdirectopinion
features,i.e.adjectivesandadverbs,associatedwithentitiescomposingtheopinionsentence;• typej(t) isatypeoftexpressedintheRDFgraphbymeansofrdf:type triples;• cxtk(t) isacontextoft,ifany.Itcanbeeitherasituationoranevent,whichtparticipatesin;• truth(t) isatruthvalueassociatedwitht,wheretistypicallyaneventorsituationoccurrence,
oraquality.Ifitsvalueisfalse,itmeansthattheentityisnegated– forexample,inasentencesuchas‘‘Johnisnotagoodguy’’,aRDFtriplesituation_1
boxing:hasTruthValueboxing:False wouldbeincludedinthegraph,anditseffectwouldbetochangethesignofthesentimentscoreassignedtothefeaturegood;
• mod(t) isamarkedmodalityofatopict,ifany– forexample,inasentencesuchasIwouldlikeadog,anRDFrelationshipfred:like_1
boxing:hasModalityboxing:Necessarywouldbeincluded.Atthistime,thepropagationalgorithmdoesnotyetusethisinformation,butitsabstractmodel,thef function,includesit;
• trig(sent) isanopiniontriggerexpressioninthesentencecontainingt.51
ThesentimentscoreSCsentiment ofatopict canbedefinedasafunctionf definedas:
Sentimentscore
ManagingActive&HealthyAgeingwithServiceRobots
Sentimentpropagationalgorithmflowchart
52
ManagingActive&HealthyAgeingwithServiceRobots
RepresentativeUseCaseScenario
SentimentAnalysisintheMyMemoriesApplication
ManagingActive&HealthyAgeingwithServiceRobots
SentimentanalysisintheMyMemoriesApp
TheMyMemoriesapp isarepresentativeexampleofaMARIOapplicationthatcombinesthecapabilitiesofNLPmodulesandservices withthecapabilitiesofSentimentAnalysismodulesandservices,aspartofitslogicandinteraction/dialoguemanagementstrategy.• Apprequirements,characteristicsanddesignaredetailedinDeliverable3.3.• AppinteractionwithMARIO’sNLPmodulesanddialoguemanagementarepresentedinDeliverable5.6.
Inthefollowing,wedescribethepeculiaritiesofthisappintermsofSentimentAnalysisandtheirimpactontheapp’sdialoguemanagementapproach• forthesakeofself-containedness andreadabilityofthisDeliverable,themaincharacteristicsoftheappreportedinD5.6aresummarisedagain.
Researchoutcomesrelatedtothisappshavebeenpresentedatthe1stInternationalWorkshoponApplicationofSemanticWebtechnologiesinRobotics(AnSWeR),co-locatedwiththe14thExtendedSemanticWebConference(ESWC2017)inPortoroz,Slovenia[11].
54
ManagingActive&HealthyAgeingwithServiceRobots
MyMemoriesApplication
Enablestherobottoundertakeinteractiveandpersonalisedreminiscence sessionsthroughaconversationalapproachbasedonuser-specificknowledgeandmaterials.• KnowledgeBase(KB)support: userprofiles,family/social
relationships,lifeevents,taggedmediaobjects(e.g.,photographs).• interactionpatterns: prompttheuserwithfocusedmemory
triggers(verbalprompt+photograph).
55
User-specificKnowledge Base
Interaction Patterns, NLPand Dialogue Management
ManagingActive&HealthyAgeingwithServiceRobots
ArchitecturalReference
56
Managesreminiscencesessionsanddialoguestateandflow• interactionpatternandpromptselection• NLPservicesinvocationsandinterpretation
Parametricinteractionpatternsprovidedialogueblueprint• question/promptformulation• NLPinterpretationstrategy
ManagingActive&HealthyAgeingwithServiceRobots
InteractionPatterns
Theinteractionwiththeuserduringareminiscencesessioncanfollowtwodifferentconversationalapproaches,bothbasedonsystem-initiated dialoguefragmentsdefinedintheformofinteractionpatterns.1. Question-based:MARIOaskstheuserfocusedclosed-ended questions
relatedtotheimagecontextuallyshownasmemorytrigger– detailedinD5.6
2. Prompt-based:MARIOpromptstheuserwithopen-ended promptsorquestionsrelatedtotheimage:– promptsaimatstimulatingreminiscenceaboutpeople(e.g.,“Whatwasyou
sisterlikeasachild?Tellmemoreabouther!”),places(e.g.,“WhatwasitliketogrewupinLondon?”)andlifeevents(e.g.,“Tellmemoreaboutyourweddingday!”)
– sentimentanalysis capabilitiesenableMARIOtoidentifythepolarityorsentimentexpressedbythePwD andtoreplyandactaccordingly
57
ManagingActive&HealthyAgeingwithServiceRobots
Prompt-basedInteractionPatterns
58
Patternstructureandelements(JSON-encoded)
precondition
prompt
ifPositiveSay
ifNeutralSay
ifNegativeSay
ConstraintmappingtoqueriesovertheKB,definingunderwhichconditionsthepromptingquestioncanbeused.
MultilingualparametricprompttemplatestobeinstantiatedwithKBdata(entitiesandtheirpropertyvalues).
Multilingualparametricsystemutterancetemplatetobeinstantiatedifuser’sanswerhaspositive sentiment.
Multilingualparametricsystemutterancetemplatetobeinstantiatedifuser’sanswerhasneutral sentiment.
Multilingualparametricsystemutterancetemplatetobeinstantiatedifuser’sanswerhasnegative sentiment.
ManagingActive&HealthyAgeingwithServiceRobots
Prompt-basedInteractionProcess(1/2)
Foraninteractionpattern whoseapplicabilitypreconditions aresatisfied(wrttheKnowledgeBase,andinparticularforaspecificphotograph),thedialogueismanagedaccordingtothefollowingmainsteps:• thecorrespondingprompttemplate isinstantiatedandthepromptisissuedtothePWD;
• thetextualrepresentationofPWD’svocalinputisprocessedrelyingonthecapabilitiesoftheSentimentAnalysissubsystem;
• dependingonthesentimentandscoreidentifiedforPWD’sanswerinthesentimentanalysisstep,thecorrespondingutteranceisissuedbyMario,asdefinedintheinteractionpattern.
Theoverallprocessisgraphicallysummarisedinthefollowingslideandthenillustratedwithaconcreteexample.
59
ManagingActive&HealthyAgeingwithServiceRobots
Prompt-basedInteractionProcess(2/2)
60
ManagingActive&HealthyAgeingwithServiceRobots
Example
61
USER ANSWER PROCESSINGTask: sentimentanalysis (polaritydetection)overuser’sutterance
Possible outcomes• Positive ⟶ encouragetotellmore(“Thatsoundsnice!Tellmemoreaboutyourmarriage!”),
showotherpicsof sameevent• Neutral ⟶ proposal(“Tellmemoreifyoulike,oraskmetoshowyouanotherphoto!”)• Negative⟶ movetonextphoto
ManagingActive&HealthyAgeingwithServiceRobots
Example– Polaritydetection(1/2)
62
“Irememberitwasagreatday!”
http://nlp.stanford.edu:8080/sentiment/rntnDemo.html
ManagingActive&HealthyAgeingwithServiceRobots
Polaritydetection(1/2)
SimplepolaritydetectionenablesMARIOtoidentifyuser’sreactiontoaspecificpromptandimage.
Polarityinfoprovidedbythesentimentanalysismodulecanbe• usedtodrivehowMARIOproceedsinthereminiscencesession
– usedifferentutterancesandstrategiesfordifferentpolarities(asoutlinedinthepreviousslides:showempathyandasktotellmoreforapositivereaction,proposetomovetonextphotoforanegativereaction,etc.);
– selectnextinteractionpatterndependingonpolarity,forexamplebychoosingoravoidingphotos/promptsrelatedtothecurrentsubject(lifeevent,person,etc.);
• storedinrelationtotheimageandsubjectoftheprompt– reusedintheimage/patternselectionprocessinsubsequentreminiscencesessions(e.g.,
favouringmemorytriggersgeneratingpositivefeelings).
However,noinfocanbeextractedonspecifictopic(s)orentitiesaboutwhichthesentimentisexpressed(e.g.,theweddingdayitself,thewife,thepicture,etc.)• finer-grainedsentimentpolarityscoresareprovidedbyframe-basedsentimentanalysis.
63
ManagingActive&HealthyAgeingwithServiceRobots
Polaritydetection(2/2)
ThepolaritydetectionprocessisalsoinfluencedbythecharacteristicsofPWD’sutterancesandtheirprocessingbytheSpeechtoText(S2T)component(describedinD5.6).• IfanutterancecomingfromtheS2Tisrecognisedasasinglesentencebythesentencesplitstepin
theNLPpipeline,thepolarityofthesentenceisanalysedandevaluated,andMARIOreactsasoutlinedbefore(seeslides58-59).– Foragivenprompt,thePWDmayexpressdifferentopinionsisdifferentutterances,eachcorrespondingtoa
singlesentenceandfollowedbyMARIO’sreaction.– Eachsentenceisprocessedandanalysedindependently(e.g.,apositivereactionwillfirstleadMARIOto
encouragethePWDtotellmoreaboutthecurrentperson/event/place;asubsequentsentencewithnegativepolaritywillthenleadMARIOtosuggesttomovetoanotheritem).
• IfanutterancecomingfromtheS2TisrecognisedasmultiplesentencesbythesentencesplitstepintheNLPpipeline,thepolarityofeachsentenceisanalysedandevaluated.– Theoverallpolarityoftheutterancecanbeassessedbycombiningthepolarityscoresofthesentences(e.g.,
bycomputinganaveragesentimentscoreoverthenumberofsentences).– Again,noinfocanbeextractedonspecifictopic(s)orentitiesaboutwhich(potentiallydifferent)sentiments
areexpressed.– Forexample,anutterancewithapositivesentenceandanegativeonewithsimilarscoreswouldbe
consideredasneutral,regardlessofthementionedtopicsorentities;noinfoiscapturedonthepositiveandnegativeattitudesandthecorrespondingtopicsorentitiesthatcharacteriseeachsentence.
64
ManagingActive&HealthyAgeingwithServiceRobots
Example– Frame-basedanalysis(1/2)MARIOcanmodelthequestionbyusingtheframe-basedapproachenabledbyFRED.
65
Whatwasyourweddingdaylike?
ManagingActive&HealthyAgeingwithServiceRobots
Example– Frame-basedanalysis(2/2)Frame-basedrepresentationofthe user’s opinionsentence
66
Thereminiscenceassociatedwith thePatient’sweddingday
isrecognisedasapositivesituation(0.313polarityvalue)
“Irememberitwasagreatday!”
Apositivescore(0.564)is alsospecifically associated withtheentity representing theday
ManagingActive&HealthyAgeingwithServiceRobots
ConcludingRemarks(1/2)
• ThesoftwarecomponentspresentedinthisdocumentarepartoftheoverallMARIOframeworkdeployedon6Kompai-2robots usedforevaluationandvalidationactivitiesinthe3projectpilotsites:– GeriatricUnitatthehospitalCasaSollievo della Sofferenza,Italy;– NursingCentres attheNationalUniversityofIreland,Galway;– CommunitiesinStockport.
• Atthetimeofwriting,evaluationandvalidationactivitiesareundergoing(accordingtothetime-linedefinedinWP8).
• Thedataandfeedbackthatwillbecollectedduringthefinalvalidation period (Aug-Nov2017)willserveasabasisforassessing,indifferentsettings,thecapabilitiesoftheSentimentAnalysissubsystem
67
ManagingActive&HealthyAgeingwithServiceRobots
ConcludingRemarks(2/2)
• Whilesimplepolarity-basedsentimentanalysistechniquesarecurrentlyusedinmultipledomains,theframe-basedsemanticsentimentanalysisapproachinvestigatedinTask5.3representsamajorcontribution,inparticularforthetargeteddomain
• Ongoingresearchactivities,whosetimeframegoesbeyondWP5,aimtoinvestigatehowtheapproachcanbeextendedfor:– aspect-based sentimentanalysis,torecognisethesentiment
expressedonspecificaspects/featuresrelatedtoatopicorentity– emotionanalysis,todetectspecificemotionsoremotionalstates
beyondpolarity
68
ManagingActive&HealthyAgeingwithServiceRobots
References(1/2)
69
[1] StanfordCoreNLP Sentiment.Availableat:http://nlp.stanford.edu/sentiment/code.html[2] R.Socher,A.Perelygin,J.Wu,J.Chuang,C.Manning,A.Ng,andC.Potts."RecursiveDeepModels
forSemanticCompositionalityOveraSentimentTreebank".In:Proc.oftheConferenceonEmpiricalMethodsinNaturalLanguageProcessing(EMNLP2013),2013
[3] BingLiu.“SentimentAnalysisandSubjectivity”.HandbookofNaturalLanguageProcessing,2010[4] BingLiu.“SentimentAnalysisandOpinionMining”.Morgan&ClaypoolPublishers.May2012[5] H.Saif,Y.He,andH.Alani.“SemanticSentimentAnalysisofTwitter”.Boston,UA,pp.508–524,
Springer,2012[6] E.Cambria,C.Havasi,andA.Hussain,“SenticNet 2:Asemanticandaffectiveresourceforopinion
miningandsentimentanalysis”. In:Proc.FLAIRSConf.,2012.[7] A.Esuli,S.Baccianella,andF.Sebastiani,“SentiWordNet 3.0:Anenhancedlexical resourcefor
sentimentanalysisandopinionmining”. In:Proc.7thconf.Int.LanguageResourcesEvaluation,2010.
[8] DepecheMood.Availableat:https://github.com/marcoguerini/DepecheMood/releases[9] Levinopinion.Availableat:http://www.stlab.istc.cnr.it/documents/sentilo/levin-opinion.zip[10]Sentilo-Net.Availableat:http://www.stlab.istc.cnr.it/documents/sentilo/sentilonet.zip
ManagingActive&HealthyAgeingwithServiceRobots
References(2/2)
70
[11] L.Asprino,A.Gangemi,A.G.Nuzzolese,V.Presutti,A.Russo:“Knowledge-drivenSupportforReminiscenceonCompanionRobots”.In:Proceedingsof1stInternationalWorkshoponApplicationofSemanticWebtechnologiesinRobotics(AnSWeR 2017),Portoroz,Slovenia,2017
[12]A.Gangemi,V.Presutti,D.Reforgiato Recupero,A.G.Nuzzolese,F.Draicchio,M.Mongiovì:“SemanticWebMachineReadingwithFRED”.SemanticWebJournal,8(6),2017
[13]A.Gangemi,M.Alam,L.Asprino,V.Presutti,D.Reforgiato Recupero.“Framester:AWideCoverageLinguisticLinkedDataHub”.In:Proceedingsofthe20thInternationalConferenceonKnowledgeEngineeringandKnowledgeManagement(EKAW2016),pp.19-23,2016
[14]Framester - Agianthuboflinguisticlinkedopenresources.https://lipn.univ-paris13.fr/framester/