daily happiness recognition from mobile phone data
TRANSCRIPT
Happiness Recognitionfrom Mobile Phone Data
Andrey Bogomolov1 Bruno Lepri2 Fabio Pianesi2
1University of Trento,Via Sommarive, 5
I-38123 Povo - Trento, Italy
2Fondazione Bruno KesslerVia Sommarive, 18
I-38050 Povo - Trento, Italy
EmoPAR group meeting 2013-JUN-19, Trento, Italy.
1 / 49
Outline
IntroductionProblem StatementSource Data
Recognition ModelBasic FeaturesFinal Feature Space
Results
Limitations
Summary
2 / 49
Happiness as an emotional state – why is it important?
Your ideas?. . .
3 / 49
General Problem Statement
Inputs
I Pervasive technology data.I Multimodal data.
Outputs
I Emotion recognition.I Mood recognition.I Personality recognition.
4 / 49
Our Problem Statement
Inputs
I Smartphone call log.I Smartphone sms log.I Smartphone Bluetooth proximity hits.
Outputs
I Daily happiness recognition.I 3-classes: {not happy, neutral, happy}.
5 / 49
Data Collection
I 117 subjectsI dates: 21 February, 2010 – 16 July, 2011
Source Space Dataset: Living Laboratoryphone calls 33497
sms 22587Bluetooth hits 1460939
6 / 49
Happiness Data
Recorded Happiness Scores Density
0.0
0.2
0.4
0.6
0.8
0 2 4 6Score
Den
sity
7 / 49
Happiness Data
Descriptive Statisticsnumber of records 12991
mean 4.84standard deviation 1.26
median 5.00mean average deviation 1.48
min 1.00max 7.00
range 6.00skew -0.39
kurtosis -0.07
8 / 49
Happiness Data
Within-person and between-subject variance
0 1 2 3 4 5 6
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
density.default(x = r1[, 1])
Variance
Den
sity
Within−Person VarianceBetween−Person Variance
9 / 49
Feature Space
Basic Features
I General Phone UsageI DiversityI Active BehaviorsI Regularity
10 / 49
Feature Space
Basic Features: General Phone Usage1. Total Number of Calls (Outgoing+Incoming)2. Total Number of Incoming Calls3. Total Number of Outgoing Calls4. Total Number of Missed Calls5. Number of SMS received6. Number of SMS sent
11 / 49
Feature Space
Basic Features: Diversity7. Number of Unique Contacts Called8. Number of Unique Contacts who Called9. Number of Unique Contacts Communicated with (Incoming+Outgoing)10. Number of Unique Contacts Associated with Missed Calls11. Entropy of Call Contacts12. Call Contacts to Interactions Ratio13. Number of Unique Contacts SMS received from14. Number of Unique Contacts SMS sent to15. Entropy of SMS Contacts16. Sms Contacts to Interactions Ratio
12 / 49
Feature Space
Basic Features: Active Behaviors17. Percent Call During the Night18. Percent Call Initiated19. Sms response rate20. Sms response latency21. Percent SMS Initiated
13 / 49
Feature Space
Basic Features: Regularity22. Average Inter-event Time for Calls (time elapsed between two events)23. Average Inter-event Time for SMS (time elapsed between two events)24. Variance Inter-event Time for Calls (time elapsed between two events)25. Variance Inter-event Time for SMS (time elapsed between two events)
14 / 49
Feature Space
Proximity FeaturesGeneral Bluetooth Proximity1. Number of Bluetooth IDs2. Times most common Bluetooth ID is seen3. Bluetooth IDs accounting for n% of IDs seen4. Bluetooth IDs seen for more than k time slots5. Time interval for which a Bluetooth ID is seen6. Entropy of Bluetooth contactsDiversity7. Contacts to interactions ratioRegularity8. Average Bluetooth interactions inter-event time(time elapsed between two events)9. Variance of the Bluetooth interactions inter-event time(time elapsed between two events)
15 / 49
Feature Space Innovation
“Background Noise” Features
I a) peoples activity, as detected through their smartphonesI b) the weather conditions {humidity, wind speed, pressure,
total precipitation and visibility}I c) personality traits {“Big Five"}
Functional Innovation
I a) time domain – sliding window functionsI b) Miller-Madow correction for entropy calculation
HMM(θ) ≡ −p∑
i=1
θML,i log θML,i +m − 1
2N
16 / 49
Top-30 Features
-1 0 1 MeanDecreaseAccuracy MeanDecreaseGinimeanTemperature 0.0099 0.0033 0.0094 0.0082 154.8379
humidity 0.0047 -0.0033 0.0115 0.0074 149.7668pressure -0.0002 0.0008 0.0015 0.0011 149.0302
windSpeed 0.0028 0.0017 0.0051 0.0040 142.7727visibility 0.0024 -0.0009 0.0051 0.0034 120.8683
neuroticism 0.0690 0.0288 0.0399 0.0419 90.5721conscientiousness 0.0659 0.0480 0.0668 0.0627 90.2708
extraversion 0.0511 0.0357 0.0472 0.0454 76.9467openness 0.0656 0.0340 0.0406 0.0429 73.9181
totalPrecipitation 0.0007 -0.0000 0.0012 0.0009 73.5273agreeableness 0.0536 0.0282 0.0235 0.0289 70.7261
bluetoothQ95TimeForWhichIdSeen 0.0233 0.0120 0.0149 0.0155 22.4018bluetoothQ90TimeForWhichIdSeen 0.0161 0.0082 0.0141 0.0131 19.8364
smsRepliedEventsLatencyMedian 0.0093 0.0085 0.0096 0.0093 18.7719bluetoothIdsMoreThan04TimeSlotsSeen 0.0114 0.0066 0.0124 0.0110 15.6738
bluetoothMaxTimeForWhichIdSeen 0.0141 0.0072 0.0095 0.0097 15.2546bluetoothTotalEntropyMillerMadow 0.0058 0.0017 0.0035 0.0035 13.9000
bluetoothTotalEntropyShannon 0.0065 0.0009 0.0060 0.0050 13.1826callMeanInterEventTimePerDay -0.0003 -0.0000 0.0014 0.0009 13.1180
incomingAndOutgoingCallsPerDay -0.0000 -0.0002 0.0024 0.0015 12.3962bluetoothQ50TimeForWhichIdSeen 0.0104 0.0104 0.0091 0.0095 12.2270
callStandardDeviationInterEventTimePerDay 0.0004 -0.0005 0.0007 0.0004 10.1723bluetoothIdsMoreThan19TimeSlotsSeen 0.0087 0.0033 0.0089 0.0077 9.7388
incomingCallsPerDay 0.0003 -0.0005 0.0009 0.0005 9.6572outgoingContactsToInteractionsRatioPerDay 0.0005 -0.0004 0.0013 0.0009 9.2016
callsInitiatedRatioPerDay -0.0001 -0.0001 0.0014 0.0008 9.0245entropyMillerMadowCallsOutgoingWindow3Days 0.0001 -0.0008 0.0014 0.0008 8.7199
bluetoothIdsMoreThan09TimeSlotsSeen 0.0074 0.0046 0.0040 0.0046 8.6006bluetoothQ75TimeForWhichIdSeen 0.0027 0.0018 0.0032 0.0028 8.4454
outgoingCallsPerDay -0.0001 -0.0001 0.0012 0.0007 8.3368
17 / 49
Top-20 Features Correlation
agreeableness
bluetoothIdsMoreThan04TimeSlotsSeen
bluetoothMaxTimeForWhichIdSeen
bluetoothQ90TimeForWhichIdSeen
bluetoothQ95TimeForWhichIdSeen
bluetoothTotalEntropyMillerMadow
bluetoothTotalEntropyShannon
callMeanInterEventTimePerDay
conscientiousness
extraversion
humidity
incomingAndOutgoingCallsPerDay
meanTemperature
neuroticism
openness
pressure
smsRepliedEventsLatencyMedian
totalPrecipitation
visibility
windSpeed
agreeablenessbluetoothIdsMoreThan04TimeSlotsSeenbluetoothMaxTimeForWhichIdSeenbluetoothQ90TimeForWhichIdSeenbluetoothQ95TimeForWhichIdSeenbluetoothTotalEntropyMillerMadowbluetoothTotalEntropyShannoncallMeanInterEventTimePerDayconscientiousnessextraversionhumidityincomingAndOutgoingCallsPerDaymeanTemperatureneuroticismopennesspressuresmsRepliedEventsLatencyMediantotalPrecipitationvisibilitywindSpeedVar1
Var2
0.0
0.5
1.0value
18 / 49
Results
Final Classifier Performance Metrics ComparisonTraining set Test set
Accuracy 0.8081 0.8036Kappa 0.5879 0.5743
AccuracyLower 0.8004 0.7878AccuracyUpper 0.8156 0.8187
AccuracyNull 0.6415 0.6419AccuracyPValue 2.139e-303 8.826e-73McnemarPValue 5.647e-208 1.738e-57
19 / 49
Results
Final Classifier Confusion Matrix for Training Set-1 0 1
-1 782 119 750 153 1170 1451 600 903 6448
Final Classifier Confusion Matrix for Test Set-1 0 1
-1 197 30 140 34 274 371 152 243 1616
20 / 49
Results
What We Learnt: Final Model ROC curve
Specificity
Sen
sitiv
ity
0.0
0.2
0.4
0.6
0.8
1.0
1.0 0.8 0.6 0.4 0.2 0.0
AUC: 0.844
21 / 49
Limitations
Data
I Data loss not registered as NA’sI BatteryI Temporal resolution
Model
I Requires personality dataI Requires 1 week data collection periodI Not tested on diverse cultural groups
22 / 49
Summary
I Automatic recognition of people’s daily happiness frommobile phone data is feasible.
I Accuracy is approaching the results of multi-modalobtrusive methods.
I Future work should be focused on multi-step recognitionmodel development.
23 / 49
Thank you!{[email protected]}
24 / 49
References I
Aharony, Nadav, Wei Pan, Cory Ip, Inas Khayal, andAlex Pentland (2011). “Social fMRI: Investigating and shapingsocial mechanisms in the real world”. In: Pervasive and MobileComputing 7.6, pp. 643–659.Anguita, Davide, Alessandro Ghio, Luca Oneto, Xavier Parra,and Jorge L. Reyes-Ortiz (2012). “Human Activity Recognitionon Smartphones Using a Multiclass Hardware-Friendly SupportVector Machine”. In: IWAAL, pp. 216–223.Bickmore, Timothy W., Rosalind, and W. Picard (2005).“Establishing and Maintaining Long-Term Human-ComputerRelationships”. In: ACM Transactions on Computer HumanInteraction 12, pp. 293–327.Bishop, C. M. (1995). “Training with Noise is Equivalent toTikhonov Regularization”. In: Neural Computation 7.1,pp. 108–116.
25 / 49
References II
Both, Fiemke, Mark Hoogendoorn, Michel C. A. Klein, andJan Treur (2010). “Computational modeling and analysis of therole of physical activity in mood regulation and depression”. In:Proceedings of the 17th international conference on Neuralinformation processing: theory and algorithms - Volume Part I.ICONIP’10. Sydney, Australia: Springer-Verlag, pp. 270–281.Brave, Scott and Clifford Nass (2003). “The human-computerinteraction handbook”. In: ed. by Julie A. Jacko andAndrew Sears. Hillsdale, NJ, USA: L. Erlbaum Associates Inc.Chap. Emotion in human-computer interaction, pp. 81–96.Breiman, Leo (Oct. 2001). “Random Forests”. In: Mach. Learn.45.1, pp. 5–32.Broek, Egon L. (Jan. 2013). “Ubiquitous emotion-awarecomputing”. In: Personal Ubiquitous Comput. 17.1, pp. 53–67.Buhmann, Martin D. and M. D. Buhmann (2003). Radial BasisFunctions. New York, NY, USA: Cambridge University Press.
26 / 49
References III
Busso, Carlos, Zhigang Deng, Serdar Yildirim, Murtaza Bulut,Chul Min Lee, Abe Kazemzadeh, Sungbok Lee,Ulrich Neumann, and Shrikanth Narayanan (2004). “Analysis ofemotion recognition using facial expressions, speech andmultimodal information”. In: in Sixth International Conferenceon Multimodal Interfaces ICMI 2004. ACM Press, pp. 205–211.Campbell, Angus, Philip E. Converse, and Willard L Rodgers(1976). The quality of American life : perceptions, evaluations,and satisfactions / Angus Campbell, Philip E. Converse, WillardL. Rodgers. English. Russell Sage Foundation New York, xi,583 p. :Carlson, Charles R., Frank P. Gantz, and John C. Masters(1983). “Adults’ emotional states and recognition of emotion inyoung children”. English. In: Motivation and Emotion 7.1,pp. 81–101.
27 / 49
References IV
Carver, Charles S. and Michael F. Scheier (Jan. 1990). “Originsand Functions of Positive and Negative Affect: AControl-Process View”. In: Psychological Review 97.1,pp. 19–35.Chittaranjan, Gokul, Jan Blom, and Daniel Gatica-Perez (Mar.2013). “Mining large-scale smartphone data for personalitystudies”. In: Personal Ubiquitous Comput. 17.3, pp. 433–450.Cohen, Jacob (Apr. 1960). “A Coefficient of Agreement forNominal Scales”. In: Educational and PsychologicalMeasurement 20.1, pp. 37–46.Cohn, Jeffrey F. (2007). “Foundations of human computing:facial expression and emotion”. In: Proceedings of the ICMI2006 and IJCAI 2007 international conference on Artificalintelligence for human computing. ICMI’06/IJCAI’07. Banff,Canada: Springer-Verlag, pp. 1–16.
28 / 49
References VCosta, P T and R R McCrae (1992). Revised NEO personalityinventory (NEO PI-R) and neo five-factor inventory (NEO-FFI).Vol. 0. October. Psychological Assessment Resources,pp. 17–20.— (1995). “Domains and facets: hierarchical personalityassessment using the revised NEO personality inventory.” In:Journal of Personality Assessment 64.1, pp. 21–50.Costa, Paul T. and Robert R. McCrae (1980). “Influence ofextraversion and neuroticism on subjective well-being: Happyand unhappy people”. In: Journal of Personality and SocialPsychology 38 (4), pp. 668–678.Cowie, R., E. Douglas-Cowie, K. Karpouzis, G. Caridakis,M. Wallace, and S. Kollias (2008). “Recognition of EmotionalStates in Natural Human-Computer Interaction”. English. In:Multimodal User Interfaces. Ed. by Dimitrios Tzovaras. Signalsand Commmunication Technologies. Springer BerlinHeidelberg, pp. 119–153.
29 / 49
References VICowie, Roddy, Ellen Douglas-Cowie, Nicolas Tsapatsoulis,George Votsis, Stefanos Kollias, Winfried Fellenz, andJohn G Taylor (2001). “Emotion recognition in human-computerinteraction”. In: Signal Processing Magazine, IEEE 18.1,pp. 32–80.DeNeve, K.M. and H. Cooper (1998). “The happy personality: ameta-analysis of 137 personality traits and subjectivewell-being.” In: Psychol Bull 124.2, pp. 197–229.Denissen, Jaap J. A., Ligaya Butalid, Lars Penke, and MarcelA. G. Van Aken (2008). “The effects of weather on daily mood:A multilevel approach.” In: Emotion Researcher 8.5,pp. 662–667.Diener, Ed (1994). “Assessing subjective well-being: Progressand opportunities”. English. In: Social Indicators Research31.2, pp. 103–157.Diener, Ed and Martin E.P. Seligman (2002). “Very HappyPeople”. In: Psychological Science 13.1, pp. 81–84.
30 / 49
References VII
Dolan, R. J. (Nov. 2002). “Emotion, cognition, and behavior.”In: Science (New York, N.Y.) 298.5596.Dong, Wen, Bruno Lepri, and Alex Pentland (2011). “Modelingthe co-evolution of behaviors and social relationships usingmobile phone data”. In: MUM, pp. 134–143.Eagle, Nathan and Alex Pentland (2006). “Reality mining:sensing complex social systems”. In: Personal and UbiquitousComputing 10.4, pp. 255–268.Eagle, Nathan, Alex (Sandy) Pentland, and David Lazer (2009).“Inferring friendship network structure by using mobile phonedata”. In: Proceedings of the National Academy of Sciences106.36, pp. 15274–15278.Feldman, Lisa A (1995). “Valence focus and arousal focus:Individual differences in the structure of affective experience”.In: Journal of personality and social psychology 69,pp. 153–153.
31 / 49
References VIIIFUNF Open Sensing Framework. (Access date: 12-JUN-2013).http://www.funf.org.Garcia, Salvador and Francisco Herrera (Dec. 2008). “AnExtension on "Statistical Comparisons of Classifiers overMultiple Data Sets" for all Pairwise Comparisons”. In: Journalof Machine Learning Research 9, pp. 2677–2694.Gray, J. A. (1991). “Neural systems, emotion, and personality.”English. In: Neurobiology of learning, emotion, and affect /editor, John Madden IV. Raven Press New York, pp. 273–306.Greenberg, Ariel M., William G. Kennedy, and Nathan Bos,eds. (2013). Social Computing, Behavioral-Cultural Modelingand Prediction - 6th International Conference, SBP 2013,Washington, DC, USA, April 2-5, 2013. Proceedings.Vol. 7812. Lecture Notes in Computer Science. Springer.Gunes, H. and M. Pantic (2010). “Automatic, Dimensional andContinuous Emotion Recognition”. In: Int’l Journal of SyntheticEmotion 1.1, pp. 68–99.
32 / 49
References IX
Gunes, H., B. Schuller, M. Pantic, and R. Cowie (2011).“Emotion Representation, Analysis and Synthesis inContinuous Space: A Survey”. In: Proceedings of IEEEInternational Conference on Automatic Face and GestureRecognition (FG’11), EmoSPACE 2011 - 1st InternationalWorkshop on Emotion Synthesis, rePresentation, and Analysisin Continuous spacE. Santa Barbara, CA, USA, pp. 827–834.Hand, DavidJ. and RobertJ. Till (2001). “A SimpleGeneralisation of the Area Under the ROC Curve for MultipleClass Classification Problems”. English. In: Machine Learning45 (2), pp. 171–186.Hernandez, Javier, Mohammed E. Hoque, andRosalind W. Picard (2012). “Mood Meter: large-scale andlong-term smile monitoring system”. In: ACM SIGGRAPH 2012Emerging Technologies. SIGGRAPH ’12. Los Angeles,California: ACM, 15:1–15:1.
33 / 49
References XJohn, Oliver P. and Sanjay Srivastava (1999). “The Big Five traittaxonomy: History, measurement, and theoreticalperspectives”. In: Handbook of Personality: Theory andResearch. Ed. by Lawrence A. Pervin and Oliver P. John.Second. New York: Guilford Press, pp. 102–138.Kapoor, Ashish, Winslow Burleson, and Rosalind W. Picard(Aug. 2007). “Automatic prediction of frustration”. In: Int. J.Hum.-Comput. Stud. 65.8, pp. 724–736.Kapoor, Ashish and Rosalind W. Picard (2005). “Multimodalaffect recognition in learning environments”. In: Proceedings ofthe 13th annual ACM international conference on Multimedia.MULTIMEDIA ’05. Hilton, Singapore: ACM, pp. 677–682.Keller, Matthew C., Barbara L. Fredrickson, Oscar Ybarra,Stephane Cote, Kareem Johnson, Joe Mikels, Anne Conway,and Tor Wager (Sept. 2005). “A Warm Heart and a Clear Head:The Contingent Effects of Weather on Mood and Cognition”.In: Psychological Science 16.9, pp. 724–731.
34 / 49
References XI
Krzanowski, Wojtek J. (2005). “Misclassification Rates”. In:Encyclopedia of Statistics in Behavioral Science.Lane, Nicholas, Mashfiqui Mohammod, Mu Lin, Xiaochao Yang,Hong Lu, Shahid Ali, Afsaneh Doryab, Ethan Berke,Tanzeem Choudhury, and Andrew Campbell (Apr. 2012).“BeWell: A Smartphone Application to Monitor, Model andPromote Wellbeing”. In: IEEE.Lane, Nicholas D., Emiliano Miluzzo, Hong Lu, Daniel Peebles,Tanzeem Choudhury, and Andrew T. Campbell (Sept. 2010). “Asurvey of mobile phone sensing”. In: Comm. Mag. 48.9,pp. 140–150.Lee, Gary R. and Masako Ishii-Kuntz (Dec. 1987). “SocialInteraction, Loneliness, and Emotional Well-Being among theElderly”. In: Research on Aging 9.4, pp. 459–482.
35 / 49
References XII
Lester, Jonathan, Tanzeem Choudhury, and Gaetano Borriello(2006). “A Practical Approach to Recognizing PhysicalActivities”. In: Pervasive Computing. Ed. by KennethP. Fishkin,Bernt Schiele, Paddy Nixon, and Aaron Quigley. Vol. 3968.Lecture Notes in Computer Science. Springer BerlinHeidelberg, pp. 1–16.LEWIS, M.E. and J.M.E. HAVILAND-JONES (2004). Handbookof Emotions. Guilford Publications, Incorporated.Lin, Wei-Jiun and James J. Chen (Jan. 2013).“Class-imbalanced classifiers for high-dimensional data”. In:Briefings in Bioinformatics 14.1, pp. 13–26.Lyubomirsky, Sonja, Laura King, and Ed Diener (Nov. 2005).“The Benefits of Frequent Positive Affect: Does Happiness Leadto Success?” In: Psychological Bulletin 131.6, pp. 803–855.
36 / 49
References XIIILyubomirsky, Sonja, Chris Tkach, and Robin M. Dimatteo(2006). “WHAT ARE THE DIFFERENCES BETWEENHAPPINESS AND SELF-ESTEEM?” In: Social IndicatorsResearch 78, pp. 363–404.Ma, Yuanchao, Bin Xu, Yin Bai, Guodong Sun, and Run Zhu(2012). “Daily Mood Assessment Based on Mobile PhoneSensing”. In: Wearable and Implantable Body Sensor Networks(BSN), 2012 Ninth International Conference on, pp. 142–147.Madan, Anmol, Manuel Cebrián, David Lazer, andAlex Pentland (2010). “Social sensing for epidemiologicalbehavior change”. In: UbiComp, pp. 291–300.McCrae, Robert R. and Paul T. Costa (1991). “Adding Liebeund Arbeit: The Full Five-Factor Model and Well-Being”. In:Personality and Social Psychology Bulletin 17.2, pp. 227–232.Miller, G. (1955). “Note on the bias of information estimates”.In: Information theory in psychology: Problems and methods2.95, p. 100.
37 / 49
References XIV
Miller, Geoffrey (2012). “The Smartphone PsychologyManifesto”. In: Perspectives on Psychological Science 7.3,pp. 221–237.Montjoye, Yves-Alexandre, Jordi Quoidbach, Florent Robic, andAlex(Sandy) Pentland (2013a). “Predicting Personality UsingNovel Mobile Phone-Based Metrics”. In: Social Computing,Behavioral-Cultural Modeling and Prediction. Ed. byArielM. Greenberg, WilliamG. Kennedy, and NathanD. Bos.Vol. 7812. Lecture Notes in Computer Science. Springer BerlinHeidelberg, pp. 48–55.Montjoye, Yves-Alexandre de, Jordi Quoidbach, Florent Robic,and Alex Pentland (2013b). “Predicting Personality Using NovelMobile Phone-Based Metrics”. In: SBP. Ed. byAriel M. Greenberg, William G. Kennedy, and Nathan Bos.Vol. 7812. Lecture Notes in Computer Science. Springer,pp. 48–55.
38 / 49
References XVMoturu, Sai T., Inas Khayal, Nadav Aharony, Wei Pan, andAlex Pentland (2011). “Using Social Sensing to Understand theLinks between Sleep, Mood, and Sociability”. In:SocialCom/PASSAT, pp. 208–214.Muaremi, Amir, Bert Arnrich, and Gerhard Tröster (2012). “ASurvey on Measuring Happiness with Smart Phones”. In:Myers, David G. and Ed Diener (1995). “Who Is Happy?” In:Psychological Science 6.1, pp. 10–19.Myers, I.B. and P.B. Myers (1980). Gifts differing: understandingpersonality type. Davies-Black Pub.Nicolaou, M. A., H. Gunes, and M. Pantic (2011).“Output-Associative RVM Regression for Dimensional andContinuous Emotion Prediction”. In: Proceedings of IEEEInternational Conference on Automatic Face and GestureRecognition (FG’11). Santa Barbara, CA, USA, pp. 16–23.Nicu Sebe Erwin Bakker, Ira Cohen Theo Gevers andThomas Huang. Bimodal Emotion Recognition.
39 / 49
References XVI
Oikonomopoulos, A., I. Patras, M. Pantic, and N. Paragios(2007). “Trajectory-based Representation of Human Actions”.In: Lecture Notes in Artificial Intelligence, Special Volume onArtificial Intelligence for Human Computing. Ed. by T. Huang,A. Nijholt, A. Pentland, and M. Pantic. Vol. 4451, pp. 133–154.Paninski, Liam (June 2003). “Estimation of entropy and mutualinformation”. In: Neural Comput. 15.6, pp. 1191–1253.Pantic, M. (2005). “Affective Computing”. In: in TheEncyclopedia of Multimedia Technology and Networking, M.Pagani, Ed. Hershy, USA: Idea Group Reference. Vol. 1,pp. 8–14.— (2009a). “Affective Computing (revisited)”. In: in TheEncyclopedia of Multimedia Technology and Networking - 2ndEdition, M. Pagani, Ed., Hershy, USA: Idea Group Reference.Vol. 1, pp. 15–21.
40 / 49
References XVII
Pantic, M. (2009b). “Machine analysis of facial behaviour:Naturalistic and dynamic behaviour”. In: PhilosophicalTransactions of Royal Society B 364, pp. 3505–3513.Pantic, M., A. Pentland, A. Nijholt, and T. Huang (2007).“Human Computing and Machine Understanding of HumanBehavior: A Survey”. In: Lecture Notes in Artificial Intelligence,Special Volume on Artificial Intelligence for Human Computing.Ed. by T. Huang, A. Nijholt, A. Pentland, and M. Pantic.Vol. 4451, pp. 47–71.Pantic, M., R. Cowie, F. D’ericco, D. Heylen, M. Mehu,C. Pelachaud, I. Poggi, M. Schroder, and A. Vinciarelli (2011).“Social Signal Processing: The Research Agenda”. In: VisualAnalysis of Humans, pp. 511–538.Pentland, Alex and Andrew Liu (Jan. 1999). “Modeling andprediction of human behavior”. In: Neural Comput. 11.1,pp. 229–242.
41 / 49
References XVIII
Picard, Rosalind W. (1997). Affective computing. Cambridge,MA, USA: MIT Press.Quercia, Daniele (2013). Donât Worry, Be Happy: TheGeography of Happiness on Facebook. http://profzero.org/publications/websci13_dontworry.pdf.Quercia, Daniele, Jonathan Ellis, Licia Capra, andJon Crowcroft (2012). “Tracking "gross community happiness"from tweets”. In: CSCW, pp. 965–968.Rabbi, Mashfiqui, Shahid Ali, Tanzeem Choudhury, andEthan Berke (2011). “Passive and In-Situ assessment of mentaland physical well-being using mobile sensors”. In: Proceedingsof the 13th international conference on Ubiquitous computing.UbiComp ’11. Beijing, China: ACM, pp. 385–394.
42 / 49
References XIX
Rachuri, Kiran K., Mirco Musolesi, Cecilia Mascolo,Peter J. Rentfrow, Chris Longworth, and Andrius Aucinas(2010). “EmotionSense: a mobile phones based adaptiveplatform for experimental social psychology research”. In:UbiComp, pp. 281–290.Robert LiKamWa Yunxin Liu, Nicholas D. Lane Lin Zhong(2013). MoodScope: Building a Mood Sensor from SmartphoneUsage Patterns.http://niclane.org/pubs/mobisys_moodscope.pdf.Salah, Albert Ali and Bruno Lepri (2011). “Challenges ofHuman Behavior Understanding for Inducing BehavioralChange”. In: AmI Workshops, pp. 249–251.
43 / 49
References XXSalah, AlbertAli, Javier Ruiz-del Solar, Ãetin Meriçli, andPierre-Yves Oudeyer (2012). “Human Behavior Understandingfor Robotics”. In: Human Behavior Understanding. Ed. byAlbertAli Salah, Javier Ruiz-del Solar, Ãetin Meriçli, andPierre-Yves Oudeyer. Vol. 7559. Lecture Notes in ComputerScience. Springer Berlin Heidelberg, pp. 1–16.Sanderson C.A., Cantor N. (1999). “A life task perspective onpersonality coherence: Stability versus change in tasks, goals,strategies, and outcomes.” In: D. Cervone & Y. Shoda (Eds.),The Coherence of Personality: Social-cognitive Bases ofConsistency, Variability, and Organization. New York, NY, USA:Guilford Press, pp. 372–392.Schuller, B., R.J. Villar, G. Rigoll, and M. Lang (2005).“Meta-Classifiers in Acoustic and Linguistic FeatureFusion-Based Affect Recognition”. In: Acoustics, Speech, andSignal Processing, 2005. Proceedings. (ICASSP ’05). IEEEInternational Conference on. Vol. 1, pp. 325–328.
44 / 49
References XXI
Sebe, Nicu, Michael S. Lew, Ira Cohen, Ashutosh Garg, andThomas S. Huang (2002). “Emotion Recognition Using aCauchy Naive Bayes Classifier”. In: in Proc. ICPR, pp. 17–20.Soleymani, M. and M. Pantic (2013). “Emotionally Aware TV”.In: Proceedings of TVUX-2013: Workshop on Exploring andEnhancing the User Experience for TV at ACM CHI 2013.Paris, France.Soleymani, M., M. Pantic, and T. Pun (2012). “MultimodalEmotion Recognition in Response to Videos”. In: IEEE Trans.on Affective Computing 3.2. in press, pp. 211–223.Staiano, Jacopo, Bruno Lepri, Ramanathan Subramanian,Nicu Sebe, and Fabio Pianesi (2011). “Automatic modeling ofpersonality states in small group interactions”. In: Proceedingsof the 19th ACM international conference on Multimedia. MM’11. Scottsdale, Arizona, USA: ACM, pp. 989–992.
45 / 49
References XXII
Staiano, Jacopo, Bruno Lepri, Nadav Aharony, Fabio Pianesi,Nicu Sebe, and Alex Pentland (2012). “Friends don’t lie:inferring personality traits from social network structure”. In:Proceedings of the 2012 ACM Conference on UbiquitousComputing. ACM, pp. 321–330.Stehman, Stephen V. (1997). “Selecting and interpretingmeasures of thematic classification accuracy”. In: RemoteSensing of Environment 62.1, pp. 77–89.Track your Happiness Web Application. (Access date:12-JUN-2013). http://www.trackyourhappiness.org.Tsutsui, Yoshiro (2012). “Weather and Individual Happiness”.In: Weather, Climate, and Society 5.1, pp. 70–82.Unser, Michael (2000). “Sampling–50 years after Shannon”. In:Proceedings of the IEEE, pp. 569–587.Vapnik, Vladimir N. (1995). The nature of statistical learningtheory. New York, NY, USA: Springer-Verlag New York, Inc.
46 / 49
References XXIII
Vitters, Joar (2001). “Personality traits and subjectivewell-being: emotional stability, not extraversion, is probably theimportant predictor”. In: Personality and Individual Differences31.6, pp. 903 –914.Wagner, Johannes, Florian Lingenfelser, Elisabeth Andre,Jonghwa Kim, and Thurid Vogt (2011a). “Exploring FusionMethods for Multimodal Emotion Recognition with MissingData”. In: IEEE Transactions on Affective Computing 2.4,pp. 206–218.Wagner, Johannes, Elisabeth Andre, Florian Lingenfelser, andJonghwa Kim (Oct. 2011b). “Exploring Fusion Methods forMultimodal Emotion Recognition with Missing Data”. In: IEEETrans. Affect. Comput. 2.4, pp. 206–218.Wandishin, Matthew S. and Steven J. Mullen (2009). MulticlassROC Analysis. American Meteorological Society.
47 / 49
References XXIV
Wang, Yongjin and Ling Guan (2008). “Recognizing HumanEmotional State From Audiovisual Signals”. In: Multimedia,IEEE Transactions on 10.4, pp. 659–668.Zajonc, Robert B and Daniel N McIntosh (1992). “Emotionsresearch: Some promising questions and some questionablepromises”. In: Psychological Science, pp. 70–74.Zeng, Z., M. Pantic, and T. Huang (2009). “Emotion Recognitionbased on Multimodal Information”. In: Affective InformationProcessing. Ed. by T. Tan and J. Tao. Springer, pp. 241–266.Zeng, Z., M. Pantic, G. Roisman, and T. Huang (2009). “ASurvey of Affect Recognition Methods: Audio, Visual, andSpontaneous Expressions”. In: IEEE Transactions on PatternAnalysis and Machine Intelligence 31.1, pp. 39–58.
48 / 49
References XXV
Zeng, Zhihong, Jilin Tu, Ming Liu, Tong Zhang,Nicholas Rizzolo, Zhenqiu Zhang, Thomas S. Huang,Dan Roth, and Stephen Levinson (2004). “Bimodal HCI-relatedaffect recognition”. In: Proceedings of the 6th internationalconference on Multimodal interfaces, pp. 137–143.
49 / 49