extracting dimensions of interpersonal interactions and
TRANSCRIPT
EXTRACTING DIMENSIONS OF INTERPERSONAL
INTERACTIONS AND RELATIONSHIPS
Farzana Rashid
Dissertation Prepared for the Degree of
DOCTOR OF PHILOSOPHY
UNIVERSITY OF NORTH TEXAS
August 2020
APPROVED: Eduardo Blanco, Major Professor Rodney D. Nielsen, Committee Member Xiaohui Yuan, Committee Member Dirk Hovy, Committee Member Barrett Bryant, Chair of the Department
of Computer Science and Engineering
Hanchen Huang, Dean of the College of Engineering
Victor Prybutok, Dean of the Toulouse Graduate School
Rashid, Farzana. Extracting Dimensions of Interpersonal Interactions and
Relationships. Doctor of Philosophy (Computer Science and Engineering), August 2020,
87 pages, 26 tables, 2 appendices, 77 numbered references.
People interact with each other through natural language to express feelings,
thoughts, intentions, instructions etc. These interactions as a result form relationships.
Besides names of relationships like siblings, spouse, friends etc., a number of dimensions
(e.g. cooperative vs. competitive, temporary vs. enduring, equal vs. hierarchical etc.) can
also be used to capture the underlying properties of interpersonal interactions and
relationships. More fine-grained descriptors (e.g. angry, rude, nice, supportive etc.) can
also be used to indicate the reasons or social-acts behind the dimension cooperative vs.
competitive. The way people interact with others may also tell us about their personal
traits, which in turn may be indicative of their probable success in their future. The works
presented in the dissertation involve creating corpora with fine-grained descriptors of
interactions and relationships. We also described experiments and their results that
indicated that the processes of identifying the dimensions can be automated.
Copyright 2020
by
Farzana Rashid
ii
ACKNOWLEDGEMENTS
I would like to express my sincere gratitude to my wonderful advisor Dr. Eduardo
Blanco for the continuous support of my Ph.D study and related research. I am immensely
thankful for his patience, motivation, and immense knowledge. His guidance helped me in all
the time of research and writing of this dissertation. He is, without doubt, the best advisor
and mentor I could have ever had for my Ph.D study.
Besides my advisor, I would like to thank the rest of my committee: Dr. Rodney
Nielsen, Dr. Xiaohui Yuan, and Dr. Dirk Hovy, for their insightful comments, questions,
suggestions and encouragement. In particular, I appreciate Dr. Dirk Hovy for his guidance
on a major portion of my dissertation.
I thank my fellow labmates for the stimulating discussions, for motivating me during
my difficult times, and for all the fun we have had in the last few years. Also I thank all my
friends for their continuous support.
Last but not the least, I would like to thank my family: my dear husband, my father,
my son, my parents-in-law, my brothers and siblings-in-law for supporting and encouraging
me all through.
iii
TABLE OF CONTENTS
Page
ACKNOWLEDGEMENTS iii
LIST OF TABLES vi
CHAPTER 1 INTRODUCTION 1
CHAPTER 2 RESEARCH GOALS AND CONTRIBUTIONS 4
CHAPTER 3 RELATED WORK 6
3.1. Related Works in Social Sciences 6
3.2. Related Works in Computational Linguistics 8
3.3. Motivation from Related Works 12
CHAPTER 4 DIMENSIONS OF INTERPERSONAL INTERACTIONS, RELATIONSHIPS,
AND SOCIAL ACTS 14
4.1. Dimensions of Interpersonal Interactions and Relationships 14
4.1.1. Dimensions of Interactions 15
4.1.2. Dimensions of Relationships 16
4.2. Social Acts 16
4.2.1. Cooperative Social Acts 17
4.2.2. Competitive Social Acts 18
4.3. The Data Sets and the Annotations 19
4.4. How the Pairs of People were Chosen 20
4.4.1. For Ontonotes Corpus 20
4.4.2. For Friends Corpus 21
4.4.3. For Friends Extension 22
4.5. Annotations 23
4.5.1. Annotation Process for Ontonotes Corpus 23
4.5.2. Annotation Quality in Ontonotes Corpus 25
iv
4.5.3. Annotation Process for Friends Corpus 27
4.5.4. Annotation Quality in Friends Corpus 29
4.5.5. Annotation Process for Friends Extension 32
4.5.6. Annotation Quality in Friends Extension 33
4.6. Experiments and Results 44
4.6.1. Experiments with Ontonotes Corpus 45
4.6.2. Experiments with Friends Corpus 48
4.6.3. Experiments with Friends Extension 51
4.6.4. Error Analysis of Friends Extension 56
CHAPTER 5 DIMENSIONS OF INTERPERSONAL INTERACTIONS AND
RELATIONSHIPS IN CHAT FORUMS 58
5.1. Original Adansonia Dataset 59
5.2. Annotating Dimensions of Interpersonal Interactions and Relationships 59
5.3. Experiments and Results 62
5.3.1. SVM Setup 63
5.3.2. Multitask Learning (MTL) Setup 63
5.3.3. Results 64
5.3.4. Automatic Annotation 67
CHAPTER 6 CONCLUSION 68
6.1. Our Achievements 68
6.2. Applications and Future Directions 69
APPENDIX A RESULT CALCULATION 72
APPENDIX B PUBLICATIONS 77
REFERENCES 79
v
LIST OF TABLES
Page
Table 4.1. Basic corpus counts. We show the number of conversation turns,
interactions (i.e., one person referring to another one), unique
relationships (i.e., unique pairs of people who interact with each other),
and the pairs of people with most interactions in the Friends dataset. 22
Table 4.2. Annotation examples for pairs of people (x, y) from the Ontonotes data.
We refer to dimensions by their first descriptor (Section 4.1); 1 (-1)
indicates that the first (second) descriptor is true, and 0 that the value is
unknown. 24
Table 4.3. Inter-annotator agreement per dimension of interpersonal relationships in
the corpus derived from Ontonotes. κ values in the 0.60–0.80 range are
considered substantial, over 0.80 would be perfect [4]. 26
Table 4.4. Pearson correlations between dimensions of interpersonal relationships in
our corpus derived from the Ontonotes data. 27
Table 4.5. Annotation examples from the Friends dataset. We show examples of
contrasting values for selected dimensions. The first party is always the
speaker, and the second party is underlined. I stands for interactions, and
R for relationship. 30
Table 4.6. Inter-annotator agreement (raw agreement and Cohen’s κ) in the Friends
dataset. κ values between 0.6 and 0.8 indicate substantial agreement, κ
values over 0.8 indicate perfect agreement [4]. 32
Table 4.7. Pearson correlations between dimensions of interpersonal interactions and
relationships in the Friends dataset. 33
Table 4.8. Stastistics of each of the 24 episodes of the first season of the Friends
series showing the number of utterances and the most interesting pairs
with their number of interactions in parentheses. 34
Table 4.9. Number of interactions between the main six characters. For each pair, we
vi
show the total number of interactions (left), and the number of cooperative
and competitive interactions (between parentheses). We also show the
total number of cooperative and competitive interactions per character
with the other five main characters in the Friends extension dataset. 35
Table 4.10. Annotation examples from the Friends extension dataset. We show
examples of interactions categorized based on the social acts of the
dimension cooperative vs. competitive. The first party is the speaker, and
the second party is the person being spoken to. 35
Table 4.11. Inter-annotator agreements for dimensions and social Acts (raw agreement
and Cohen’s κ) in the Friends extension dataset. κ values between 0.6
and 0.8 indicate substantial agreement, κ values over 0.8 indicate nearly
perfect agreement [4]. 39
Table 4.12. Pearson correlations between pairs of dimensions of interactions and
dimensions in the Friends extension dataset. 42
Table 4.13. Distributions of cooperative and competitive social acts among the 4,257
interactions in the Friends extension dataset. 43
Table 4.14. Feature set used to determine dimensions of interpersonal relationships
between pairs of people (x, y). Verb features are extracted from the verb
of which either x or y is the subject, Person features are extracted from
x and y independently, and Persons features are extracted from x and y. 46
Table 4.15. Results obtained for all dimensions with several combinations of features
for the Ontonotes dataset. 49
Table 4.16. Results obtained for each dimension with the best combination of features
for all dimensions (Verb + Personx + Persony + Personx Persony,
boldfaced in Table 4.15). The last three columns under All contains
the weighted average of Precision (P), Recall (R) and F-mesure (F) of
the dimensions based on the distributions of the 3 labels. The last row
contains the weighted averages of each column. 50
vii
Table 4.17. Results obtained with the test set with several systems (average of all
dimensions) for the Friends dataset. Previous refers to the previous
conversation in which the same pair of people interacted not the
immediately previous turn). 52
Table 4.18. Results obtained per dimension with the best system (all features, Table
4.17). The results under All the weighted averages for all labels, recall
that the label distribution is biased (Figure 4.3). 52
Table 4.19. Results for predicting dimensions of interactions and relationships with
the Friends extension dataset. The bottom two systems use gold
dimensions as features thus results are unrealistic. 54
Table 4.20. Detailed results for predicting dimensions of interactions and relations
with the best-performing realistic system (BOW + BOW prev. +
sentiment + other in Table 4.19). 55
Table 4.21. Results for predicting social acts (cooperative and competitive). Systems
labeled+social acts prev. use gold social acts and thus results are
unrealistic. 55
Table 4.22. Detailed results for predicting social acts with the best-performing
realistic system (BOW + BOW prev. + sentiment + other in Table
4.21). The average shown is the weighted average based on the number of
instances in each label. 56
Table 5.1. Annotation examples of contrasting values for each dimension. Each chat
interaction is either directed to an individual or others in general. 60
Table 5.2. Inter-annotator agreements for dimensions. 0.6 ≤ κ ≤ 0.8: substantial
agreement, κ > 0.8: nearly perfect agreement [4]. 61
Table 5.3. Pearson correlations between pairs of dimensions of interactions (indicated
by the names’ initial letters). 62
Table 5.4. Results for predicting styles of interactions and three indicators of business
success. Each value is a weighted average over the different labels based
viii
on the number of instances falling in that label. 66
Table A.1. Results obtained for all dimensions with several combinations of features
for the Ontonotes Dataset. 73
Table A.2. Results obtained for each dimension with the best combination of features
for all dimensions (Verb + Personx + Persony + Personx Persony,
boldfaced in Table A.1). 74
Table A.3. Macro averages of the Precision, Recall and F-measure respectively over
the different labels for each dimension shown in Table A.2. 75
Table A.4. Macro averages of the Precision, Recall and F-measure respectively over
the different labels for each dimension shown in Table 4.18. 76
Table A.5. Macro averages of the Precision, Recall and F-measure respectively over
the different labels for each dimension shown in Table 4.20. 76
ix
CHAPTER 1
INTRODUCTION
According to the Oxford Dictionary, a relationship is how two or more concepts, ob-
jects, or people are connected or the state of being connected. When something or someone
is connected to something or someone in any way, they are related to each other (e.g., two
people from the same country are related to each other as fellow countrymen). If two people
are part of the same family, we say that they are relatives. If two objects, like windows
and doors are part of a house, we may say that there is a relationship between the two,
namely, they are components of the house. There are names for certain relationships, for
example, we have relationship names like born same country (for fellow countrymen),
kinship (for relatives), and same house (for members of the same house). There are re-
lationship names like married to or spouse, and child of or offsprings. An instance
of relationship married to is married to(Bill Clinton, Hillary Clinton). This instance
could be extracted from the following text sample: Bill Clinton married Hillary Clinton in
1975. As society is composed of humans, relationships between people, i.e., interpersonal
relationships, are of special interest. Humans are social beings that interact which each other
often. People interact in many different ways. They interact verbally, physically, through
body gestures, in written communication, etc. A series of these interactions define the rela-
tionship. The nature of the interactions between two people and their behavior during these
interactions determine how their relationship progresses. Certain interactions exerted on
each other can determine whether the relationship would continue or break. People interact
through natural language to express feelings, thoughts, intentions, instructions, etc. Their
words often express politeness or rudeness, i.e., the words can indicate the elemental prop-
erties or characteristics of the interactions. Interactions can have different characteristics,
for example, if two people are talking to each other sitting side by side, we can describe
their interaction to be a spatially near one. On the other hand, if a person talks to an-
other over a phone call, it is almost certainly the case that the other person is not at the
1
same premise, so here the interaction is spatially distant. Here we see that the communica-
tion device is helping us to understand whether a relationship is spatially distant or near.
Similarly, the content of an interaction, regardless of the communication device, gives hints
about the nature of the interaction. If one person is praising the other, the interaction is
likely a friendly one. On the other hand, if one person raises their voice and scolds the
other, their interaction is probably an unfriendly one. Several interactions, each of differ-
ent characteristics, thus add up and build relationships between two people. How people
build and maintain their relationships affects their lifestyles and well-being in the society
they live in [42, 25]. These relationships are the main connections in societal networks.
Behavioral and relationship scientists, thus, study people’s interactions and the resulting
relationships [57] to understand society and how it functions. When a customer at a grocery
shop shops there for the first time and starts talking to the salesperson there, the first few
interactions may be superficial ones. But if the customer shops at that shop regularly, then
the interactions build up a relationship with the salesperson that may turn intense as they
probably interact often. We say that if two people meet or come across each other frequently
they are intensely involved. So, here the frequency of interactions is helping us determine
the intensity of a relationship. So we can say that the behavior, media, or frequency can
be a few of many ways that can help us characterize certain interactions. Information ex-
traction is the automatic extraction of information from machine-readable sources that are
unstructured or maybe just semi-structured [47]. Relationship extraction or identification of
relations between entities is part of the Information extraction field. This involves extract-
ing relations like person works for organization, or works for(Bill, IBM) from the
sentence “Bill works for IBM.”) This simple but useful information is usually hidden within
the text, which might be easy for humans to understand but difficult for machines to inter-
pret. Information extraction techniques are improving, and extracting this otherwise hidden
information is getting better. Several applications such as question answering, information
retrieval, also text summarization would benefit from this information. Extracting interper-
sonal relationships would lead to understanding human nature and their behavior within
2
society. As a lot of data is now available online, processing these big data and automatically
getting useful information about interpersonal relationships would help social scientists to
progress their research on societal networks and bonding. Only assigning a relation type,
however, does not account for nuances in the relationship between two individuals. First,
a relationship can be characterized beyond a relation type. For example, people who are
coworkers may be spatially near or distant (working at the same or different offices), and
have an equal or hierarchical relationship (two software developers or a developer and the
CEO). Second, relationships are defined by multiple interactions, and the fine-grained char-
acteristics of interactions do not necessarily mirror the characteristics of the corresponding
relationship. For example, software developers having a cooperative professional relationship
may have a heated interaction in a meeting that does not affect the long-term professional
relationship. Similarly, the same software developers having a task-oriented relationship
may have occasionally pleasure oriented interactions (e.g., when they go out for drinks on
Fridays). This added information on top of the relations type explains the nature of the
relationship and will help more in understanding individuals interact with each other. How-
ever, the in-depth information is harder to extract than just the type of the relationship, as
it is not explicitly said with plain language. Extracting these fine-grained characteristics of
interactions and relationships is what we targeted to do. We are interested in knowing these
characteristics or dimensions, as we call them. The details will be explained in the upcoming
chapters.
3
CHAPTER 2
RESEARCH GOALS AND CONTRIBUTIONS
In the field of information extraction, one of the many subfields is relationship ex-
traction (RE). RE is a classic NLP problem. Given a sentence, RE aims to detect if there
exists a certain semantic relationship between two entities of interest [54]. Even though
our research goal falls under this subfield, we do not aim to extract named interpersonal
relations. Instead, we characterize the interactions and resulting relationship with their ele-
mental properties, also known as dimensions inspired by previous works. We are exploring
different domains of datasets, including Ontonotes, TV series transcript datasets, and online
chat conversations between 5000+ aspiring entrepreneurs to perform our tasks. Our major
milestones included choosing datasets, annotating the dimensions of interactions and rela-
tionships on pairs of people extracted from the dataset, and experimenting with models to
be able to classify these interactions and relationships. Our contributions are as follows:
(1) We chose a set of dimensions of interpersonal interactions and relationships with
inspiration from previous work. By analyzing several annotated instances, we real-
ized that a few of the dimensions applied to interactions and the others applied to
relationships. So, we and grouped the dimensions accordingly. We also defined a
number of the underlying social acts (or sub-types) of the dimension cooperative vs.
competitive.
(2) We targeted interactions and relationships between people as described as follows:
(a) We considered sentences that contained a pair of people connected by the action
of a verb. We found that samples of interpersonal interactions collected from
a wide variety of domains can be identified with different dimensions. We thus
conclude that relationships can be described in terms of dimensions universally.
(b) We considered dialogues or instances of speech where a person talks about or
mentions another person. We found that words mentioned by a person to refer
to another person include detailed information about the relationship between
4
the two.
(c) We considered dialogues or instances of speech where a person talks to another
person. We found that direct speech between two people also holds in-depth
information about the relationship between them.
(d) We considered chat interactions where a real young aspiring entrepreneur chats
with others through online platform discussing and sharing ideas about prob-
able business and funding opportunities. We found that dimensions can char-
acterize chat interactions. We also found that the dimension information can
help predict the ability of a person’s success in business.
(3) We formed large corpora by annotating the dimensions (of interactions and rela-
tionships) and social acts on data from different domains. High agreements in the
annotations indicated that it is possible to annotate the datasets reliably. Distri-
butions of labels differed depending on the type or domain of the datasets. This
difference indicated that certain values of the dimensions are predominant in cer-
tain domains. Correlation analysis for the dimensions and the different social acts
indicated that values certain dimensions are correlated to values of certain other
dimensions.
(4) We conducted Machine Learning experiments. The decent results of classification
showed that it is possible to learn the dimensions and the social acts with learning
methods simply based on language usage. Machine Learning models developed for
the purpose can be used as baseline models for future studies aimed to develop more
sophisticated models.
5
CHAPTER 3
RELATED WORK
As people are social beings, they constantly interact with each other and show certain
behaviors aimed towards each other. As many of their behaviors come only into play when
they interact with each other, behavioral analysis involves the study of interpersonal interac-
tions. The study of interpersonal interactions helps scientists understand how people behave
in certain situations, why they behave in certain ways, and what leads them to that [9]. The
trajectories of relationships that form due to the changes in the properties of interactions
are studied as well.
Just the way social scientists are interested in understanding relationships, compu-
tational linguists are interested in developing tools to extract explicit or implicit cues and
mentions of interpersonal relationships. They try to extract these from written text as well as
spoken words. Here, we discuss a few of the prior works in social cciences and computational
linguistics that are related to our work.
3.1. Related Works in Social Sciences
There is an entire discipline called relationship science, the main focus of which is
to study interpersonal relationships [57]. This discipline is composed of ideas from many
sub-disciplines within social, behavioral, psychology, and biological science. Relationship
scientists study human behavior and their effects on their daily lives and natural surround-
ings. They also study the impact of the exterior environments of the relationships on their
interior dynamics. They also study the effects of interpersonal relationships on cognition.
A relationship can be described in more detail with more elemental properties that
define its nature from different perspectives. These elemental properties of interpersonal re-
lationships that characterize the state of a relationship are called dimensions in social science
and have been studied for decades [71]. In this study, the goal is to understand how people
perceive relationships based on some fundamental characteristics, and they came up with a
number of dimensions. They conducted a questionnaire study to discover the fundamental
6
dimensions underlying people’s perceptions of interpersonal relationships. A number of hu-
man subjects rated 20 of their own interpersonal relationships and 25 typical relationships
on 25 bipolar scales. Their careful analysis of the data revealed 4 dimensions, which were
interpreted as cooperative and friendly vs. competitive and hostile, equal vs. unequal, intense
vs. superficial, and socio-emotional and informal vs. task-oriented and formal. These four
dimensions were studied against those obtained from studies of personality, person percep-
tion, and individual behavior in interpersonal situations. They found that these fundamental
dimensions were extremely stable. Our set of dimensions is inspired by the ideas of these
dimensions mentioned in this paper and a number of other studies discussed below.
Kelley (2013) [42] studies close personal relationships and their essential properties.
She aims to understand the close relationships to be able to systematically assess and classify
so that their functioning can be improved and developed. She discusses some fundamental
dimensions of dyadic relationships. She focuses on close dyadic relationships to understand
the interdependence and the dynamics of such relationships. In the process, she observes
how interpersonal behavior affects relationships in the long run. We also try to understand
interpersonal interactions and relationships to carry out annotation processes to build our
dataset. We do this with the aim of building a system that can automatically extract certain
dimensions of these interpersonal interactions. However, a few of such dimensions are also
discussed in the work by Kelley.
Deutsch [25] examines the connection between types of psychological interdependence
and psychological orientations. He establishes that distinctive psychological orientations
(composed of cognitive, motivational, and moral orientations) are associated with the dis-
tinctive types of interdependence in diverse social relationships in complex societies. He also
says that there is a bidirectional connection between psychological orientations and types
of interdependence, that is, a psychological orientation can induce or be induced by a given
type of interdependence. He also points out that cultural backgrounds of peoples, their per-
sonal histories, and their genetic endowments affect the way they use their orientations. His
hypothesis states that to cope with the psychological requirements of different types of social
7
relationships, people can customize the usage of their psychological orientations as required
in different situations. He does not attempt any computational model but just gives us a
view from social sciences.
Adamopoulos [2] says that there are underlying meanings behind every kind of inter-
personal behavior. In other words, there are reasons for certain kinds of behavior, and again
these behaviors lead to certain changes. He presents a theory of action construal with an
emphasis on the emergence of their social meaning. He reviews the theoretical framework
that generated models concerning the meaning of interpersonal behavior, which involves the
exchange of material and psychological resources. He also points out that these exchanges
are contingent upon many constraints that affect interpersonal interaction. He studied the
context of social resource theory and thus added to the area of interpersonal structure in gen-
eral. We also gather ideas of looking at relationships with elemental properties or dimensions
from this work.
3.2. Related Works in Computational Linguistics
Information extraction (IE) involves extracting structured information from machine-
readable documents. This is mostly done by processing human language texts by means of
natural language processing (NLP) or computational linguistics. Examples also include
content extraction out of images, audio, or video besides language expressed through text.
As information extraction is still pretty difficult, extraction is only doable on very narrow or
selected domains. Different kinds of information can be extracted using automated methods.
One kind of information that is often extracted is relationships between entities such
as people, organizations, and locations, hence the formation of the subfield relation extrac-
tion. A few competitions have served as evaluation benchmarks [30, 26, 44, 64], and include
interpersonal relationships such as business, spouse, and children. Aguilar et al. (2014)
[3] compare several evaluations, and automated approaches to relationship extraction– also
referred to as link prediction and knowledge base completion [76, 50, 70]. Open information
extraction [72] has emerged as an unsupervised domain-independent approach to extract re-
lationships. There is also a semi-supervised relation extraction system with large-scale word
8
clustering [63] and other systems like [24] which create multigraph representations of the
relationships extracted (and their quality). Regardless of details, all these previous efforts
extract explicit relationship types or names and do not attempt to characterize instances of
relationships with dimensions.
Besides extracting relationships per se, there have been efforts within computational
linguistics involving interpersonal relationships. Voskarides et al. [68] extract human-
readable descriptions of relationships in a knowledge graph by ranking sentences that justify
the relationships. Their method explains the relationship between two Wikipedia entities,
and evaluate their method on a dataset annotated, prepared, and made public by them. In
the aim to infer a human-readable description for the relationship between two entities, they
devise a ranking task as they automatically extract sentences from a corpus and rank them
based on how well they describe the relationships.
Iyyer et al. [40] propose an unsupervised algorithm to extract relationship trajec-
tories of fictional characters, i.e., how interpersonal relationships evolve in fictional stories.
Their system learns descriptors of events like marriage, murder, etc. and also those for in-
terpersonal (relationship) states like love, sadness, etc. They leverage neural networks to
achieve their learning goal. Moon and Qi [48] have taken a hybrid approach of combining
the features of unsupervised and supervised learning methods to extract relationships. Their
system identifies the main characters in novels and collects the sentences related to them for
extracting relationships. This kind of work can be applied in story summarization and anal-
ysis of the major characters in stories. Kokkinakis [43] does manual annotation of a small
sample of Swedish 19th and 20th-century prose with interpersonal relationships between
characters in six literary works. They inspect models for annotation on a larger scale, both
manually as well as automatically. Bajracharya et al. [6] carry out a rule-based approach
that identifies characters from stories and determines the family relationship among them.
There is work for learning character types, or personas from films [7], inferring the polarity
of relationships between people in narrative summaries [62] and extracting social network
from literally fictions [27]. We have used TV series transcript data to learn dimensions of
9
interpersonal relationships.
Bracewell et al. [12] introduced 9 social acts or moves (e.g., agreement, undermin-
ing) designed to characterize relationships between individuals exhibiting adversarial and
collegial behavior (similar to our cooperative vs. competitive dimension). The 9 acts are:
agreement (statements that a group member makes to indicate that he/she shares the same
view about something another member has said or done); challenge credibility (attempts
to discredit or raise doubt about another group members qualifications or abilities); dis-
agreement (statements a group member makes to indicate that he/she does not share the
same view about something another member has said or done); disrespect (inappropriate
statements that a group member makes to insult another member of the group); offer grati-
tude (a sincere expression of thanks that one group member makes to another); relationship
conflict (personal, heated disagreement between individuals); solidarity (statements that a
group member makes to strengthen the groups sense of community and unity); supportive
behavior (statements of personal support that one group member makes toward another);
and undermining (hostile expressions that a group member makes to damage the reputation
of another group member). These social acts were selected from literature in the fields of
psychology and organizational behavior. They presented the annotations of these nine social
acts for discourses communicated in Chinese and English. This work inspires us to further
characterize the dimension of cooperative vs. competitive into subcategories, similar to the
nine social acts.
Studies have been done to differentiate types of interaction based on students under-
lying motivation. They also investigated how different forms of interaction are related to
educational outcomes [77]. They grouped social interactions into three categories of under-
lying motivations for educational outcomes. One is interaction as a response to curricular
demands, interaction for broader educational purposes, and interaction for diverse experi-
ences. In their work, they did not categorize interactions with social acts as we did. They
also did not differentiate between cooperativeness or competitiveness. In our study, we con-
sider that social acts are the reasons behind a behavior or interaction for being cooperative
10
or competitive.
Also, there have been studies to see what kind of conversational interactions and what
factors activate frustration in a person [60]. Their model emphasizes the motives that drive
interpersonal behaviors and the negative effect (such as anger). One hundred twenty-nine
students participated in chat conversations. They studied how student-participants behave
when they (a) thought the interaction was interpersonal (with a human) or noninterper-
sonal (with a computer); (b) were assertive and nonassertive and (c) the replies given to the
participants contained dominating or non-dominating language. Participants showed more
interpersonal behaviors when they thought the other party was human. Assertive partici-
pants who interacted with a dominating and apparently human partner were more hostile
and had angry conversations. Even though this work did not work explicitly with social acts,
it can be compared with such as it studies the motives and outcomes of human behavior or
interaction based on certain environmental factors.
Politeness in online forums has been studied [23]. They annotate a new corpus
of requests for politeness. They use this corpus to evaluate aspects of politeness and to
identify politeness markers and contexts. Also, Danescu-Niculescu-Mizil et al. [22] study
how power differences affects language style in online communities. They showed that in
group discussions, power differentials between participants are subtly revealed by how much
one individual immediately echoes the linguistic style of the person they are responding to.
They create an analysis framework with which they study the conversational behavior to
reveal power difference. Prabhakaran and Rambow [53] present a classifier to detect power
relationships in email threads. Similarly, Gibert [28] explores how people in hierarchical
relationships communicate through email, and Bramsel et al. [13] focus on identifying power
relationships in social networks based purely on interpersonal communication of the members
of the network. Some have studied how the roles of Wikipedia editors affect their success
[45]. We similarly categorize relationships between people using hierarchical vs. equal as
one of the dimensions, among others. Business relevance and sentiment are annotated on
online chat interactions among aspiring entrepreneurs [67]. Unlike them, we use different
11
domains of dataset like TV series transcripts, news articles, TV interview transcripts, and
chat interactions.
3.3. Motivation from Related Works
The study of interpersonal communication or relation is a multidisciplinary activity,
and it includes the study of psychology, sociology, anthropology, medicine, social psychology,
psychiatry, clinical psychology, and facets of the language studies [55]. Sociologists, psychol-
ogists, etc. have studied human relationships to understand what people want or wish for,
what they aim to achieve, and how they act in different situations [19]. We can expect that
people in close relationships will reflect similar preferences in movies, books, music, etc. [19].
Thus behavioral science studies can be incorporated with relationship extraction to figuring
out how a person is going to behave in terms of his/her buying habits, travel habits, etc.
If it is possible to figure out whom a person is intimate with, it is likely that this
person would behave similarly as their close friends. This infomation can also be used to
build friends networks [1, 35, 27] by social media platforms. The information, in turn, can
be utilized by advertising to figure out what items the person may be interested in. Just the
way there is a need to consider subtypes of the generic association [10], there is a need to
consider the subtypes or elemental properties of interpersonal relationships that are called
the dimensions of interpersonal relationships. Characterizing interpersonal relationship is
a hard but important problem [34]. These dimensions have been studied for decades in
social sciences [71, 42, 25, 2, 58, 61]. Extracting just the named dyadic relationship is
already useful for applications like question answering [75], but these names only provide a
general understanding of the relationship between two people. But if the question-answering
models use the fine-grained dimensions of relationships, they can provide more information
about certain relationships. This information can be used to predict relationship trajectories
between people [40].
These days, robots are used to assist people in domestic works. They also assist
in engaging elders in conversations to reduce their loneliness. They also serve people with
autism and help in the physical rehabilitation of a variety of patients. For these activities,
12
robots need to learn new tasks, skills, and individual preferences that they can learn from
the natural languages used by the users in their natural interpersonal interactions [14]. So
the robots need to be more socially aware. With in-depth knowledge about interpersonal
relationships, more socially aware robots or chatbots can be built, as shown by Hameed [32].
In light of the above needs, we describe interpersonal relationships in terms of their
fine-grained dimensions and focused on how they can be extracted from natural language
text. We propose that rather than expressing relationships just with a name, we should
express them in terms of more descriptive properties called the dimensions. We showed that
these dimensions can be annotated by humans and can also be reliably predicted by models
trained on the language solely.
13
CHAPTER 4
DIMENSIONS OF INTERPERSONAL INTERACTIONS, RELATIONSHIPS, AND
SOCIAL ACTS
In this chapter, we start by discussing the details of the dimensions of interpersonal
interactions and relationships in section 4.1. Then we move on to discussing the social acts
in section 4.2. We then discuss the datasets used in three different projects we perform in
section 4.3. We also elaborate on and the annotations of these dimensions and social acts.
We also present analyses of these annotations. Then in Section 4.6, we present the details
on the experiments carried out in these three projects.
4.1. Dimensions of Interpersonal Interactions and Relationships
The elemental characteristics or properties of interactions or relationships are known
as their dimensions. The perfect set of dimensions of interpersonal interactions and relation-
ships is by no means agreed upon, and neither is the terminology to refer to what apparently
is the same dimension. For example, the terms dominance, submission, potency, autonomy,
and control are used to describe the distribution of power in a relationship [25]. The dimen-
sions we work with are primarily borrowed from previous works in social science, although
we add two new dimensions. Interestingly, the previous works which define these dimensions
do so from a theoretical point of view or after conducting experiments with subjects to reveal
how they perceive interpersonal relationships. The latter was done using multidimensional
scaling analysis after subjects compared 25 relationships, e.g., between a parent and child,
between business partners [71].
Social scientists have proposed additional dimensions, e.g., voluntary vs. involuntary,
public vs. private and licit vs. illicit [25], or self-benefiting vs. service-oriented [2]. We
discarded these additional dimensions because we discovered that they are not applicable to
most pairs of people we work with. We provide brief descriptions of the nine dimensions of
interpersonal interactions and relationships we work with here with examples in Sections 4.1.1
and 4.1.2. Note that these dimensions are not completely independent, for example, enduring
14
relationships are usually intense and intimate, and intense and pleasure oriented relationship
are almost always intimate. We also mention which dimensions apply to interactions and
which ones apply to relationships.
Here are the definitions of the nine dimensions that we worked with along with indi-
cations on whether they define interactions or relationships. First stated are the dimensions
of interactions and then are the dimensions of relationships. There also are references from
which these dimensions are inspired from.
4.1.1. Dimensions of Interactions
Here we describe the dimensions of Interactions.
Cooperative vs. Competitive: An interaction is cooperative if both the people involved
(a) have a common interest or goal, (b) like each other, (c) benefit from the relationship, or
(d) think alike or have similar views. Otherwise, it is competitive. We use this as a dimension
of Interactions. Inspiration of this dimension is taken from Wish et al. [71].
Active vs. Passive: An interaction is active if both people communicate directly.
Otherwise, the interaction is passive. For example, when John talks to Bill about Mary,
John and Bill have an active interaction, and John and Mary have a passive interaction. We
use this as a dimension of interactions. The inspiration for this dimension came from Kelley
[42].
Concurrent vs. Non-concurrent : An interaction is concurrent if both people are
involved in an event or action at the same time. Otherwise, the interaction is non-concurrent.
We use this as a dimension of interactions. We thought of this dimension to be appropriate
for our tasks.
Spatially Near vs. Distant : An interaction is spatially near (or near for short) if both
people are at the same location during the event that grants the interaction. Otherwise, the
interaction is spatially distant (or distant). We use this as a dimension of interactions. We
thought of this dimension to be appropriate for our tasks.
15
4.1.2. Dimensions of Relationships
Here we describe the dimensions of Relationships.
Equal vs. Hierarchical : A relationship is equal if both people (a) have the same social
status, (b) are at the same level in the power structure, (c) share similar responsibilities,
or (d) have the same role. Otherwise, the relationship is hierarchical. We use this as a
dimension of Relationships. The inspiration for this dimension also came from Wish et al.
[71].
Intense vs. Superficial : A relationship is intense if both people interact with each
other frequently, i.e., they are involved repeatedly. Otherwise, the relationship is superficial.
We use this as a dimension of Relationships. The inspiration for this dimension is also taken
from Wish et al. [71].
Pleasure vs. Task Oriented : A relationship is pleasure-oriented if both people interact
socially, and their relationship is not bound by professional rules or regulations. Otherwise,
the relationship is task-oriented. We use this as a dimension of Relationships. The inspiration
for this dimension is also taken from Wish et al. [71].
Intimate vs. Unintimate: A relationship is intimate if both people are emotionally
close and warm to each other. Otherwise, the relationship is unintimate. We use this as
a dimension of Relationships. The inspiration for this dimension came from Adamopolous
(2012) [2].
Temporary vs. Enduring : A relationship is temporary if it lasts less than a day. A
relationship is enduring if it lasts over a month. Otherwise (if it lasts more than a day and
less than a month), this dimension is undefined. We use this as a dimension of Relationships.
Inspiration of this dimension came from Deutsch (2011) [25].
4.2. Social Acts
Just like interactions and relationships can be characterized in terms of dimen-
sions, competitive and cooperative interactions can be further characterized with social acts
[8, 11, 12]. These social acts are like acts denoting the reasons behind the cooperativeness or
competitiveness of an interaction. Social acts influence future interactions and relationship
16
trajectories. For example, a mother may scold her son for not listening to her (competitive
interaction), and latter show support by praising him for doing his homework on time (coop-
erative interaction). Similarly, friends may mock each other (competitive interaction) while
their overall relationship remains in good terms because, for example, they mainly partici-
pate in excited and nice social acts. Note that social acts depend on individual interactions,
not on the long-term relationship between individuals.
Regarding social acts, we follow Bracewell et al. (2012a)[11], which note that coop-
erative and competitive interactions (they use adversarial and collegial) can be understood
by the underlying social act or pragmatic speech acts that signal a dialogue participants so-
cial intentions. We work with the 11 social acts defined below. Six of them are novel, and
five of them are borrowed from the work by Bracewell et al. (2012a)[11], which in turn are
inspired by other previous works [15, 41, 17, 51]. The novel social acts were defined after
pilot annotations.
4.2.1. Cooperative Social Acts
We consider five social acts for cooperative interactions:
• Agree applies when the first party agrees with the second party or shares the same
point of view [11, 12]. Utterances such as Yes, you are correct and Sure, why not?
are two examples showing agreement.
• Support applies when the first party praises, encourages or motivates the second
party. The first party usually mentions the second party’s qualities or appreciates
his efforts. For example, You are looking beautiful, You made us proud and Come
on, you can do it! show support. This social act is similar to supportive behavior
[17, 12].
• Excited applies when the first party shows enthusiasm and passion towards some-
thing the second party said. For example, Wow, I am thrilled to know that! and
That was fun! are utterances showing excitement.
• Nice applies when someone expresses sincere thanks, apologies, polite greeting, etc.
For instance, Thank you so much and Hello, how are you? are utterances showing
17
niceness.
• Normal is the default social act of cooperative interactions. It applies when the
first party states a fact, asks a question, and, in general, when no other social act
applies. For example, This is where I live and I wake up at 7 a.m. show the normal
social act.
4.2.2. Competitive Social Acts
We consider six social acts for competitive interactions:
• Complain applies when the first party accuses, denounces, or laments something to
the second party. The complain does not need to involve the second party. For
example, You did everything wrong and I hate my new job downtown are complain
social acts.
• Disagree applies when the first party indicates that they do not share the same
view about something said or done [11, 12]. The following utterance exemplifies
this social act: No, that’s not true!
• Disbelief applies when one doubts or discredits somebody’s knowledge, qualifica-
tions, abilities or credibility. For example, Are you sure? and I don’t think he
graduated high school indicates disbelief. This social act is similar to challenge cred-
ibility [51].
• Dissatisfied applies when the first party is disappointed or annoyed about an idea
shared or work done. For example, I expected better from you shows dissatisfaction.
• Mock applies when the first party jokes about, makes fun of, or says something
sarcastically about somebody to someone. The person being mocked does not need
to be the second party. Examples include someone saying to a short person Hi, big
guy! or asking whether somebody’s boyfriend eats chalk.
• Rude applies when behaving badly or rudely with someone. The first party may
also be scolding, rebuking, or warning someone. For example, I don’t care or Don’t
you ever go there, and other shows of disrespect fall under this social act. This is
similar to disrespect [11, 12].
18
4.3. The Data Sets and the Annotations
Our task involved annotating a large amount of data with the labels indicating the
dimensions of interpersonal interactions and relationships. Existing corpora annotating re-
lationships consider only selected interpersonal relationships and do not target dimensions
(Chapter 3). Our goal is to target dimensions of interpersonal relationships between any two
individuals, from weak links (e.g., journalists interviewing celebrities) to strong ties (e.g.,
close friends). So far, our annotation effort has taken place in three independent tasks, and
we have carried out detailed analyses of our resulting datasets independently. In those three
tasks, we have created:
(1) A corpus by first retrieving pairs of people from text data of mixed genres, and then
annotating dimensions for their relationships. We decided to add our annotations
to Ontonotes [38]. Doing so has several advantages. First, Ontonotes data contains
texts from several domains and genres (e.g., conversational telephone speech, TV
interview transcripts, weblogs, broadcast); thus, we not only work with newspa-
per articles. Second, Ontonotes includes part-of-speech tags, named entities, and
coreference chains, three annotation layers that allow us to streamline the corpus
creation process. For easy reference, we will call this resulting corpus, Ontonotes
corpus, from here on.
(2) A corpus by augmenting on an existing corpus of scripts from the TV show Friends
[20]. Instead of starting from scratch, we chose to do the annotation because anno-
tating dimensions of interactions and relationships requires a corpus in which the
same people interact several times. We work with the 24 episodes from Season 1
because:
(a) they contain a large number of conversation turns(9,168);
(b) involve many characters (42 characters speak at least 100 conservation turns);
(c) they include speaker information (i.e., we have access to who says what);
(d) and they include annotations linking each mention of people in each conversa-
tion turn to the actual person (the name of the person).
19
Beyond size, the main motivation to use this corpus is the last item above: starting
from scratch with another corpus of dialogues would require substantially greater
annotation effort. We refer the reader to the aforecited paper for details, but the
original corpus clusters mention to people such as guy, my brother, and emphhe
together with other mentions of the same person. The original corpus is publicly
available,1. For easy reference, we refer to the resulting corpus as the Friends corpus
from here on.
(3) Another corpus by annotating the dimensions of interactions and relations, and the
underlying social acts behind cooperative and competitive interactions on pairs of
people formed between a speaker and the person being spoken to. While looking for a
source corpus that contains many interactions between the same people, we thought
of reusing the same original corpus from the TV show Friends, as mentioned above
[20]. We work with twelve episodes of Season 1, which contain 4,257 interactions.
For easy reference, we refer to the resulting corpus as the Friends extension from
here on.
4.4. How the Pairs of People were Chosen
Before we could start our annotations, we required to select portions of the huge
original dataset as our candidates to be annotated. This curation resulted in a reasonable
size of the data that was easy to handle. The process involved choosing the appropriate
sentences based on some simple criteria for each of our corpora. Here we describe how the
appropriate sentences were chosen partly automatically and partly manually:
4.4.1. For Ontonotes Corpus
We retrieve pairs of people within each sentence in OntoNotes following four steps:
(1) Collect all instances of personal pronouns (part-of-speech tag PRP) I, he, and she
within the sentence.
(2) Collect all named entities PERSON within the sentences.
1https://github.com/emorynlp/character-mining
20
Figure 4.1. Frequencies of the top 20 most frequent verbs after retrieving
pairs of people from the Ontonotes dataset (Section 4.4.1). We discard verbs
with frequency <4, and randomly select up to 26 pairs per verb for a total of
1,048 pairs.
(3) Keep one mention per coreference chain, giving priority to named entities over
pronouns.
(4) Generate combinations of 2 elements from the union of the pronouns and named
entities subject to the following constraints: at least
(a) one is a PERSON named entity, and
(b) one is the nsubj (nominal subject syntactic dependency) of a verb.
Figure 4.1 shows the frequencies of the top 20 most frequent verbs after retrieving
pairs of people from the Ontonotes dataset.
4.4.2. For Friends Corpus
For the Friends corpus, we decided to pick pairs of people from the conversations
carried out by the characters in the 24 episodes of the first season of the TV series Friends.
To select pairs of people whose dimensions of interactions and relationships were to be
annotated, we collected all instances of somebody mentioning (or referring to) somebody
else in a conversation turn. We consider relationships between individuals who interact at
least once.
Table 4.1 shows basic counts per episode. We show the number of interactions, unique
relationships (i.e., interactions between unique pairs of people), and the pairs of people who
interact the most.
21
Dimension Agreement κ coefficient
Cooperative 86% 0.74
Equal 81% 0.63
Intense 84% 0.73
Pleasure Oriented 86% 0.70
Active 82% 0.59
Intimate 83% 0.68
Temporary 76% 0.61
Concurrent 84% 0.72
Spatially Near 80% 0.67
All 82% 0.68
Table 4.1. Basic corpus counts. We show the number of conversation turns,
interactions (i.e., one person referring to another one), unique relationships
(i.e., unique pairs of people who interact with each other), and the pairs of
people with most interactions in the Friends dataset.
4.4.3. For Friends Extension
Since the original corpus has speaker information, it is straightforward to determine
the first party of all interactions. The second party, however, is not readily available. We
manually annotated the second party (who is the first party talking to?) as a preliminary
step prior to annotating dimensions and social acts. Note that the second party is not always
the author of the next utterance. We annotated all conversation turns, and a single annotator
indicated the second party after watching the episode. There are three options: the second
party is an individual, more than one individual, or nobody. If it is an individual, the
annotator indicates the name. If it is more than one individual, the annotator chooses many
(11% of utterances). Finally, the annotator indicates self when somebody is actually saying
a monologue or thinking aloud and thus talking to nobody in particular (2% of instances).
22
69.9%
20.9%
26.4%
65.9%
27.4%
66.2%
14.4%
79.3%
58.4%
35.3%
Cooperative Equal Intense Pleasure Oriented Active
15%
79.3%80.1%
14.5%
46.4%
47.1%
40.4%
51.4%
1
-1
Intimate Temporary Concurrent Spatially Near
Figure 4.2. Label distribution per dimension of interpersonal relationships
from the Ontonotes data. The missing portion of each pie chart corresponds
to labels 0 and inv, which always amount to less than 5% each.
We consider as interactions all utterances except the ones in which the second party could
not be identified (all but self ), and these are the interactions that were annotated with
dimensions and social acts (Section 4.5.5). Table 4.9 shows how many interactions there are
for the main six characters.
4.5. Annotations
Here we discuss how we carried out the annotation processes on the three corpora, a
number of selected annotation examples from each of the corpus, annotation qualities and
label distributions.
4.5.1. Annotation Process for Ontonotes Corpus
After generating pairs, annotators determined values for each dimension of interper-
sonal interactions and relationships. As from this dataset, it was hard to acquire a series of
interactions by the same pair of people, a relationship between a pair of people was considered
to be formed only with one interaction (or sentence).
23
Sentence Coop
erati
ve
Equ
al
Inte
nse
Ple
asu
reO
rien
ted
Act
ive
Inti
mate
Tem
por
ary
Con
curr
ent
Sp
atia
lly
Nea
r
1 [Cheney]x [got]verb a telephone call from his democratic coun-
terpart [Joseph Lieberman]y wishing him a speedy recovery.
1 1 -1 -1 1 -1 1 1 -1
2 [. . . ] [I]x [interviewed]verb one of the nation’s top jockies
[Shane Sellers]y about the battle he waged everyday to con-
trol his weight.
1 -1 -1 -1 1 -1 1 1 1
3 [I]x have always remembered the encouragement which Mr.
[Yu Youren]y [gave]verb me as a young reporter. He said that
to be a fearless champion of social justice, as is expected of
a journalist, the [. . . ]
1 -1 1 -1 1 1 -1 1 1
4 One day, Dingxiang took the opportunity to again urge
him to change his ways [. . . ]. After this, [Zhang Sheng]x
[threw]verb out [Dingxiang]y, sold off the family possessions,
and spent his days living a life of dissipation.
-1 1 -1 1 1 -1 -1 1 1
Table 4.2. Annotation examples for pairs of people (x, y) from the Ontonotes
data. We refer to dimensions by their first descriptor (Section 4.1); 1 (-1)
indicates that the first (second) descriptor is true, and 0 that the value is
unknown.
The annotation interface showed the sentence from which the pair of people was
generated, and the previous and next sentence to provide some context. The pair of people
of interest were highlighted, but no additional information was shown (e.g., the verb of
which one person is the subject). Annotators assigned a value to each dimension based on
the relationship between the two individuals at the time the verbal event of which one of
the individuals is the subject takes place. They were trained to take into account context
24
(previous and next sentences), and to interpret the text as they normally would. Therefore,
they assign values using world knowledge that may not be explicitly stated in the text. For
example, two people talking on the phone would have a spatially distant interaction because
(most likely) they are not next to each other while talking.
During the first batch of annotations, we discovered that for a given pair of people,
dimensions sometimes could not be determined because (a) there is not enough evidence
in the text provided (i.e., sentence from which the pair was generated, previous and next
sentences) or (b) the pair is invalid and assigning dimensions is nonsensical. We use label
0 in the former case and inv in the latter. For example, in the sentence, He criticized Ken
Starr, the value for the dimension spatially near (vs. distant) was marked 0 as there is
not enough information to determine whether He and Ken Starr are at the same location
when criticized took place. We refer to dimensions by the first descriptor (as given in the
definitions in Section 4.1) and use 1 if the first descriptor of a dimension is true, and -1 if
the second descriptor is true. For example, label -1 applied to dimension temporary means
that the relationship is enduring. Figure 4.2 shows the label percentages of the different
dimensions in the Ontonotes data.
4.5.2. Annotation Quality in Ontonotes Corpus
Two graduate students did the annotations. They started annotating small batches
of pairs of people, and discussed disagreements with each other. After several iterations,
they annotated independently 10% of all pairs of people generated. Table 4.3 shows the
inter-annotator agreement per dimension of interpersonal relationships in the Ontonotes
Data. Overall, Cohen’s kappa coefficient is 0.68, and the coefficients range from 0.59 to
0.74, depending on the dimension. Note that kappa coefficients in the range 0.600.80 are
considered substantial [4]. Given these high agreements, the rest of the pairs were annotated
once.
Even though the annotations were done very carefully, we acknowledge that some
annotations are ambiguous. We will discuss a few examples from the Ontonotes data found
in Table 4.2. Sentences like, Sentence 1 and Sentence 2 encode COMMUNICATION re-
25
Dimension Agreement κ coefficient
Cooperative 86% 0.74
Equal 81% 0.63
Intense 84% 0.73
Pleasure Oriented 86% 0.70
Active 82% 0.59
Intimate 83% 0.68
Temporary 76% 0.61
Concurrent 84% 0.72
Spatially Near 80% 0.67
All 82% 0.68
Table 4.3. Inter-annotator agreement per dimension of interpersonal rela-
tionships in the corpus derived from Ontonotes. κ values in the 0.60–0.80
range are considered substantial, over 0.80 would be perfect [4].
lationships between two individuals, and both are cooperative, superficial, work-oriented,
active, unintimate, temporary, and concurrent. The values for two dimensions, however, are
different. Two counterparts, in Sentence 1, are at the same level in the power structure
(equal), but the interviewer and interviewee are not in Sentence 2. Similarly, talking on the
phone entails that the individuals are spatially distant in Sentence 1, but interviewing (most
likely) means that they were spatially near in Sentence 2. One could argue that 0 would
be a better label for spatially near in Sentence 2, but annotators interpreted that inter-
viewed refers to an in-person interview. In Sentence 3, the context describes one person (Yu
Youren) encouraging another one (I). Annotators indicate that this relationship, unlike the
ones in Sentences 1 and 2, is intense (frequent interactions), intimate (emotionally close),
and enduring (lasting over a month). These values are not explicitly stated, but they are un-
derstood given the long-lasting impact Yu Youren had on I. Finally, Sentence 4 exemplifies a
competitive relationship. The context describes a struggling relationship between Dingxiang
26
Coop
erat
ive
Equ
al
Inte
nse
Ple
asu
reO
r.
Act
ive
Inti
mat
e
Tem
por
ary
Con
curr
ent
Cooperative –
Equal -.06 –
Intense .24 .11 –
Pleasure Or. .14 .29 .39 –
Active .18 .22 .43 .25 –
Intimate .18 .17 .59 .62 .31 –
Temporary -.17 .10 -.56 -.43 -.27 -.61 –
Concurrent .17 .22 .31 .21 .74 .26 -.17 –
Spat. Near .21 .18 .30 .25 .68 .28 .19 .89
Table 4.4. Pearson correlations between dimensions of interpersonal rela-
tionships in our corpus derived from the Ontonotes data.
and Zhang Sheng. When the latter threw the former out, the relationship was superficial
and unintimate, but (most likely) existed for longer than a month (enduring).
Table 4.4 shows the correlations between the different dimensions. Not surprisingly,
some dimensions correlate with each other. For example, enduring relationships tend to
also be intense (0.56) and intimate (0.61), and concurrent relationships tend to be active
(0.74). The highest correlation is between concurrent and spatially near (0.89), indicating
that if two people participate in a common event at the same time, usually they are at the
same location. Note, however, that most correlations are low, and some dimensions (e.g.,
cooperative, equal) have low correlations (¡0.30) with all dimensions. This indicates that the
dimensions, as defined indicate some unique property of the interaction or relationship.
4.5.3. Annotation Process for Friends Corpus
We annotated for all the 24 episodes of the first season of Friends. Table 4.8 shows the
statistics of each of the 24 episodes of the first season of the Friends series showing the number
27
of utterances and the most interesting pairs with their number of interactions in parentheses.
The annotations were done one episode at a time. Annotators were presented with the
full transcript of the episode, including speaker information (who speaks what?) and the
names of the individuals mentioned in each conversation turn (who do speakers talk about?).
Annotators read each episode from the beginning and annotate dimensions of interactions
and relationships after each interaction. Regarding interactions, they were instructed to
annotate dimensions taking into account the language of the current conversation turn.
Regarding relationships, they were instructed to annotate dimensions taking into account
all previous conversation turns within the same episode. For example, if previous turns
state that characters Rachel and Monica are best friends, the relationship will continue to
be annotated intense even if an interaction does not indicate so (until a turn indicates that
they are not friends, if applicable).
We discovered during pilot annotations that the value for a dimension sometimes
could not be determined. For example, if the first interaction between Rachel and Monica
is Rachel: How are [you] Monica doing?, we cannot tell if the relationship is temporary or
enduring. We note, however, that all interaction after we find out that they are best friends
(as long as they remain best friends) will be annotated enduring. Just like earlier, we refer to
dimensions by the first descriptor as in the definitions in 4.1, and use 1 if the first descriptor
of a dimension is true, -1 if the second descriptor is true, and 0 if neither the first nor the
second descriptor can be chosen.
Table 4.5 shows the annotation examples from the Friends dataset. The interactions
in conversation turns 1 and 2 are competitive: Phoebe is ridiculing Paul by asking Monica
if he eats chalk, and Ross is confronting an unnamed woman. In turn (3), Monica refers to
Phoebe with affection (as the latter sleeps); thus the interaction is cooperative. Turns (4–6)
exemplify concurrent vs. nonconcurrent. In (4), Rachel is inquiring whether she can meet
Alan (Monica’s boyfriend), and Rachel and Alan are not involved in the same event (at this
point, the meeting may or may not happen). In (5–6), however, the speaker and second
party are involved directly in a communication event. In examples (4–6), the values for
28
active are the same as for concurrent. Examples 7–9 show values for the dimension intense.
These require some explanation, as additional information beyond the current conversation
turn is required (recall that dimensions of relationships are annotated taking into account
the previous turns within the same episode, Section 4.5.3). Dr. Franzblau is the doctor of
a friend’s ex-wife, so he and Monica have a superficial relationship (Turn 7). At the point
Turn (8) is spoken by Chandler, she and Aurora are strangers, so they have a superficial
relationship.
In (9), Joey talks about his mom, so the relationship is (intense). Example 10 shows
Rachel talking to her credit card company representative, so it is work oriented, whereas in
11 Monica is talking to her friend Rachel, so it is pleasure oriented.
Examples 12 and 15 are fairly straightforward: previous interactions reveal that Joey
is an adult (12), and Rachels’s talking to her father in 13. Example 14 is annotated equal,
as Chandler and Ross are friends based on previous interactions (the use of pal also helps).
In example 15, Rachel is talking to her old friends, so it is equal, and usage of you guys
helps in this one. Example 16 shows that Monica was expecting an invitation from Rachel,
thus indicating that they knew each other for long (enduring). Example 17 shows Rachel
talking to a waiter whom she has possibly met for the first time, so it is annotated temporary.
Example 18 exemplifies a distant interaction as it is said over the phone. Example 19 is a
direct interaction between Chandler and Joey, so it is near.
4.5.4. Annotation Quality in Friends Corpus
The annotations were done by two graduate students in computational linguistics.
First, they did pilot annotations to better define the dimensions. After several iterations,
both of them annotated three episodes (15% of all interactions). Table 4.6 show the anno-
tations agreement in this Friends dataset. Cohen’s range between 0.77 and 0.89, and most
(7 out of 9) are above 0.80, which is considered perfect agreement. And as values between
0.60 and 0.80 are considered substantial [4], the remaining episodes were annotated once.
Many of the dimensions we consider in this work are intuitively correlated. For exam-
ple, concurrent interactions must be active, and pleasure oriented interactions are probably
29
Speaker 2nd party Coop.
1: Wait, does he eat chalk? Phoebe Paul -1
2: Hey, hey, hey, that’s not the rule and you know it. Ross Woman -1
3: She is so peaceful. Monica Phoebe 1
Speaker 2nd party Conc.
4: Well, then can we meet him? Rachel Alan -1
5: Hey, buddy, what’s up. Chandler Alan 1
6: Ma, I am sorry. Joey Mrs. Tribbiani 1
Speaker 2nd party Itense
7: Um, has Dr. Franzblau been by? Rachel Dr. Franzblau -1
8: There’s a beautiful lady at eight, nine ten o’clock! Chandler Aurora -1
9: Ma, I am sorry. Joey Mrs. Tribbiani 1
Speaker 2nd party Pleas.
10: Could you please tell me what this is in reference to? Monica Visa -1
11: Uh, Rach, it’s the Visa card people. Monica Rachel 1
Speaker 2nd party Equal
12: Oh, ah- the kid has it. Joey the kid -1
13: Hey Dad, what’s up? Rachel Dad -1
14: Happy birthday, pal. Chandller Ross 1
15: So c’mon, you guys, tell me all the dirt! Rachel Leslie 1
Speaker 2nd party Temp.
16: Who wasn’t invited to the wedding. Monica Rachel -1
17: Can I get you some coffee? Waitress Rachel 1
Speaker 2nd party Near
18: Hi it-it’s Rachel. Rachel Mindy -1
19: Give it to me. Chandler Joey 1
Table 4.5. Annotation examples from the Friends dataset. We show exam-
ples of contrasting values for selected dimensions. The first party is always the
speaker, and the second party is underlined. I stands for interactions, and R
for relationship.30
also equal. We note, however, that interactions can be passive and concurrent, e.g., in (Mon-
ica talking to Joey)[He] (Paul) is just [a guy] (Paul) I am dating!, Monica and Paul have a
passive and concurrent interaction (they are dating, but they are not talking to each other).
Table 4.7 shows Pearson correlations between all dimensions of interactions and relationships.
Most correlations are under 0.3 (29 out Spatially Near vs. Spatially Distant (Interaction),
Intense vs. Superficial (Relationship), although some pairs do have high correlations. In
particular, active interactions tend to be both concurrent (0.81) and spatially near (0.71),
and spatially near interactions tend to be concurrent (0.89). Regarding relationships, inti-
mate correlates with intense (0.75), pleasure oriented with equal (0.50), and temporary with
both superficial (0.52) and intimate (0.54).
Figure 4.3 shows the label percentage distributions per dimension in the Friends
dataset. Regarding interactions, we note that (a) values for all dimensions can be determined
almost always (the percentages of 0 (unknown) are almost zero), and (b) the first descriptor
is much more common in all dimensions. These percentages do not represent the distribution
of interactions between people in general: the scripts of the TV show Friends mostly contain
conversation between friends. Regarding relationships, we observe a larger percentage of 0
(unknown) although values of all dimension can be determined most of the time (labels 1 and
-1, indicating that the first or second descriptor apply). Most dimensions are biased towards
1 (the only exception is temporary, as most relationships are enduring), especially pleasure
oriented and equal (91.2% and 84.8%). Again, these distributions would be different if we
worked with other sources of dialogue than the TV show Friends.
Figures 4.4 and 4.5 show the most salient words for dimensions spatially near and
intense respectively in the Friends dataset. We calculated salience using tf-idf [59]. In-
teractions containing derogatory words (e.g.pig,bugs,pretending,cheating) tend to be distant,
and near interactions contain mostly neutral and nicer words such as friends, sweatheart and
please. We also note that cognitive verbs and nouns (e.g.,thinking,figured (out), looking(into),
cause), as well as important events (birth-day,thanksgiving) and slang usage (e.g.,whad-dya)
signal intense relationships.
31
Dimension κ coefficient
Cooperative (Int.) 0.82
Cooperative (Rel.) 0.81
Equal 0.85
Intense 0.80
Pleasure Oriented 0.77
Active (Int.) 0.89
Active (Rel.) 0.63
Intimate 0.85
Temporary 0.78
Concurrent 0.84
Spatially Near 0.87
All 0.81
Table 4.6. Inter-annotator agreement (raw agreement and Cohen’s κ) in the
Friends dataset. κ values between 0.6 and 0.8 indicate substantial agreement,
κ values over 0.8 indicate perfect agreement [4].
4.5.5. Annotation Process for Friends Extension
After determining the pairs of people (speaker speaking to someone) as discussed in
Section 4.4.3, two annotators carried out the annotation processes. First, they annotated
dimensions of interactions and relationships. Second, they annotated the social acts for
cooperative and competitive interactions. Both annotation phases were done one episode at
a time.
In the first phase, they read the transcript of each episode sequentially and annotated
(a) dimensions of interactions (cooperative or competitive) taking into account only the cur-
rent interaction, and (b) dimensions of the corresponding relationship (equal or hierarchical,
intense or superficial, intimate or unintimate, and temporary or enduring) taking into ac-
32
Coop
erat
ive(
Int)
Coop
erat
ive(
Rel
)
Equ
al
Inte
nse
Ple
asu
reO
r.
Act
ive(
Int)
Act
ive(
Rel
)
Inti
mat
e
Tem
por
ary
Con
curr
ent
Cooperative(Int) –
Cooperative(Rel) .39 –
Equal -.01 -.02 –
Intense .08 .38 .28 –
Pleasure Or. -.05 -.05 .50 .13 –
Active(Int) .06 .22 .08 .28 .01 –
Active(Rel) .02 .14 .14 .29 .04 .30 –
Intimate .12 .43 .32 .75 .28 .28 .30 –
Temporary -.02 -.14 -.24 -.52 -.27 -.33 -.29 -.54 –
Concurrent .02 .18 .06 .24 .08 .81 .32 .26 -.31 –
Spat. Near .01 .16 .05 .22 .08 .71 .25 .22 -.28 .89
Table 4.7. Pearson correlations between dimensions of interpersonal inter-
actions and relationships in the Friends dataset.
count all interactions between the two characters. From here on, we refer to dimensions by
their first descriptor, and use (a) 1 or −1 if the first or the second descriptor applies, and 0
if neither descriptor applies.
In the second phase, annotators annotated the social acts of the 99.4% of interactions
that are either cooperative or competitive. This process was done one interaction at a time.
4.5.6. Annotation Quality in Friends Extension
Two graduate students in computational linguistics carried out the annotations. First,
they did pilot annotations to better define the dimensions and social acts (Section ??). After
several preliminary iterations, both of them annotated one episode (approximately 10% of
all interactions). Table 4.11 presents the inter-annotator agreements for the dimensions.
33
Episode Interactions Interesting pair(interactions)
1 143 Monica-Rachel (23), Monica-Paul (21)
2 105 Rachel-Barry (14), Carol-Ross (12)
3 98 Joey-Chandler (12), Monica-Alan (10)
4 64 Monica-Rachel (15), Chandler-Ross (12)
5 76 Cahndler-Janice (14), Ross-Rachel (13)
6 82 Chandler-Aurora (32)
7 76 Ross-Rachel (13), Joey-Ross (12)
8 64 Monica-Mrs. Geller (14)
9 75 Monica-Rachel (13), Ross-Susan (10)
10 69 Phoebe-David (22)
11 159 Chandler-Ross (22), Ross-Mrs. Bing (21)
12 109 Phoebe-Paolo (14), Phoebe-Rachel (12)
13 120 Joey- Mr. Tribbiani (21), Joey-Mrs. Tribbiani
14 239 Chandler-Janice (28), Ross-Carol (23)
15 78 Joey-Ross (15)
16 95 Chandler-Nina (15)
17 109 Monica-Rachel (28), Phoebe-Ursula
18 42 Ross-Rachel (9)
19 29 Ross-Rachel(9)
20 111 Rachel-Mindy (30), Rachel-Barry (17)
21 85 Monica-Fake Monica (23)
22 92 Phoebe-Chandler (32), Monica-Ethan (13)
23 104 Ross-Susan (19)
24 107 Ross-Rachel (30)
Table 4.8. Stastistics of each of the 24 episodes of the first season of the
Friends series showing the number of utterances and the most interesting pairs
with their number of interactions in parentheses.
34
(Ro)ss (M)onica (R)achel (C)handler (J)oey (P)hoebe Coop. Comp.
Ro * 104 (87,17) 151 (143,8) 42 (34,8) 93 (72,21) 39 (32,7) 368 61
M 90 (65,23) * 119 (88,31) 37 (26,11) 79 (51,28) 60 (49,11) 279 104
Ra 105 (101,4) 105 (74,31) * 42 (34,8) 12 (9,2) 74 (66,8) 284 53
C 110 (68,40) 47 (28,19) 40 (32,8) * 89 (57,31) 35 (30,4) 215 102
J 134 (98,36) 92 (68,23) 23 (19,4) 73 (56,17) * 12 (11,1) 252 81
P 41 (38,3) 93 (66,24) 100 (91,9) 37 (27,10) 26 (20,6) * 242 52
Table 4.9. Number of interactions between the main six characters. For
each pair, we show the total number of interactions (left), and the number of
cooperative and competitive interactions (between parentheses). We also show
the total number of cooperative and competitive interactions per character
with the other five main characters in the Friends extension dataset.
Speaker 2nd party Social Act (Coop.)
1: Oh, by the way, great service tonight. Ross Rachel appreciate
2: You should feel great about yourself! Monica Rachel encouarge
3: Look at you, you are so big I can’t believe it! Rachel Leslie excited
4: How was it with your friends ? Monica Rachel nice
Speaker 2nd party Social Act (Comp.)
5: You have got waaay too much time. Chandler Joey complain
6: No, you don’t. Monica Ross disagree
7: Paul, is it? Chandler Alan mock
8: Be Murky. Phoebe Ross scold
Table 4.10. Annotation examples from the Friends extension dataset. We
show examples of interactions categorized based on the social acts of the di-
mension cooperative vs. competitive. The first party is the speaker, and the
second party is the person being spoken to.
35
Inte
ract
ions
80.8%
16.5%
72.6%
27.1%
77.7%
22.3%
80.2%
19.5%
1
-1
Cooperative Active Concurrent Spatially Near
Rel
atio
nsh
ips
84.8%
11.5%
57.5%
21.3%
91.2%7.2%
58.2%
24.6%
27.6%
60.6%
Equal Intense Pleasure Oriented Intimate Temporary
Figure 4.3. Label percentages per dimension in the Friends dataset. The
missing portion of pie charts correspond to label 0 (unknown).
Cohen’s κ range between 0.82 and 0.94 (top block). The agreements (Cohen’s κ) for social
acts range between 0.78 and 0.89 (bottom block). κ values between 0.60 and 0.80 are
considered substantial [4], and above 0.80 are considered nearly perfect. Based on these
agreements, the remainder of episodes were annotated once.
Table 4.10 annotation examples from the Friends extension dataset of social acts of
the dimension cooperative vs. competitive. In (1) Ross is appreciating what Rachel has done
as a waiter, so he is appreciating her work. In (2) Monica is encouraging Rachel by saying
some good words about her. Rachel shows her excitement about her friend’s pregnancy in
(3), and in (4) Monica is being nice with Rachel by asking how she spent time with her
friends. All these are examples of cooperative interactions with their social acts.
Examples 5–8 show competitive interactions with their social acts. In (5) Chandler
complains about Joey when he sees him doing meaningless acts wasting his time. In (6)
Monica straightforwardly disagreed with her brother. In (7) Chandler is mocking his friend’s
new boyfriend, and in (8) Phoebe is just scolding her friend.
The screenshots of actual scenes of the Friends series are shown with the annotations
36
Spatially Near vs. Spatially Distant (Interaction)
Figure 4.4. Most salient words (calculated with tf-idf) for dimension spa-
tially near in the Friends dataset.
37
Intense vs. Superficial (Relationship)
Figure 4.5. Most salient words (calculated with tf-idf) for dimension intense
in the Friends dataset.
38
raw κ
Dim
ensi
on
s
Cooperative (I) 96.7% 0.90
Equal (R) 97.3% 0.94
Intense (R) 94.3% 0.87
Intimate (R) 90.4% 0.82
Temporary (R) 92.6% 0.83
S.A
cts
Cooperative 91.7% 0.89
Competitve 90.4% 0.78
Table 4.11. Inter-annotator agreements for dimensions and social Acts (raw
agreement and Cohen’s κ) in the Friends extension dataset. κ values between
0.6 and 0.8 indicate substantial agreement, κ values over 0.8 indicate nearly
perfect agreement [4].
of the cooperative and competitive social acts respectively in figures 4.6 and 4.7. 4.6 exem-
plifies cooperative interactions. In the top-left scene in this figure, Ross encourages his sister
Monica to go on a date with the guy who she had wanted to date for a long time. This is
an example of the support social act. The top-right scene exemplifies the social act normal :
Ross telling his friends what he is supposed to do to assemble a piece of furniture. In the
scene at the bottom-left, Rachel expresses her excitement about receiving her first paycheck.
Finally, in the bottom-right scene, Monica is apologizing to her boyfriend and is being nice
to him.
The 4.7 exemplifies competitive interactions. In the top-left scene, Rachel is rudely
talking to her father over the phone and showing anger. In the top-right scene, Phoebe is
mocking Monica by asking her whether her boyfriend eats chalk. In the scene at the bottom-
left, Monica complains about Rachel saying (and forgetting) that she had the keys. Finally,
in the bottom-right scene, Rachel disagrees with what Monica had complained about.
Figure 4.8 shows the Label percentages per dimension in the Friends extension
dataset. Except for the dimension temporary, the first descriptor is much more common
39
Figure 4.6. Scenes from Friends exemplifying cooperative interactions as
well as their social acts. It shows examples of support, normal, nice and
excited social acts (clockwise).
in all dimensions. In other words, most interactions are cooperative (79.4%), and most rela-
tionships are equal (88.1%), intense (67.6%), intimate (72.7%) and enduring (75.8%). These
distributions are not representative of interactions in general, they are biased because of the
corpus we work with.
Table 4.12 shows the Pearson correlations between pairs of dimensions of interactions
and dimensions in the Friends extension dataset. Most correlations are below 0.53, and
cooperative is not correlated with any. The highest correlation is between intense and in-
timate, which is to be expected. Not surprisingly, intense and intimate relationships tend
to be enduring (inversely correlated with temporary), but the correlations are 0.45 and 0.53
40
Figure 4.7. Scenes from Friends exemplifying competitive interactions as
well as their social acts. It shows examples of rude, mock, disagree and com-
plain social acts (clockwise).
respective and thus they do not capture exactly the same properties.
Table 4.13 shows the distribution of social acts. Regarding social acts for cooperative
interactions, normal is the most frequent (80.5%), and the remaining range from 3.1% (agree)
to 9.7% (nice). The social acts for competitive interactions are more uniformly distributed.
We observe a clear minority (disbelief, 1.2%), but the remaining 5 social acts range from
16.2% to 26.3%.
Figure 4.9 shows the most salient words of four social acts, where salience is calculated
using tf-idf [59]. Adjectives such as unbelievable, amazing and fascinating usually indicate
excited social acts. Not surprisingly, positive words such as okay, please and goodnight indi-
41
Figure 4.8. Label percentages per dimension in the Friends extension
dataset. -1, 1 and 0 represent first descriptor, second descriptor and unknown
respectively.
Coop
erat
ive
(I)
Equ
al(R
)
Inte
nse
(R)
Inti
mat
e(R
)
Equal (R) 0.02 -
Intense (R) -0.04 0.40 -
Intimate (R) -0.05 0.38 0.80 -
Temporary (R) 0.01 -0.22 -0.45 -0.53
Table 4.12. Pearson correlations between pairs of dimensions of interactions
and dimensions in the Friends extension dataset.
42
Percentage
Coop
erat
ive
Agree 3.1%
Excited 3.3%
Nice 9.7%
Normal 80.5%
Support 3.4%
Com
pet
itiv
eComplain 19.4%
Disagree 19.2%
Disbelief 1.2%
Dissatisfaction 26.3%
Mock 16.2%
Rude 17.6%
Table 4.13. Distributions of cooperative and competitive social acts among
the 4,257 interactions in the Friends extension dataset.
excited–cooperative nice–cooperative rude–competitive mock–competitve
Figure 4.9. Most salient words (calculated with tf-idf) for four social acts:
excited and nice (cooperative interactions), and rude and mock (competitive
interactions) in the Friends extension dataset.
cate nice social acts. More interestingly, rude social acts include utterances about (quitting)
smoking and unfair situations. Finally, mocking social acts include derogatory nicknames
(e.g., schhteve for Steve), physical imperfections (e.g., hump) and negative adjectives (e.g.,
wrong, brutal). Figure 4.10 shows which social acts follow each cooperative social act. Figure
43
agree and excited
nice and normal
support
Figure 4.10. Charts showing which social acts follow each cooperative social act.
4.11 shows which social acts follow each competitive social act.
4.6. Experiments and Results
Machine learning experiments were done with the data from both the corpus to see if
it is possible to learn the dimensions automatically or not. We conducted experiments using
standard supervised machine learning.
44
complain and disagree
disbelief and dissatisfaction
mock and rude
Figure 4.11. Charts showing which social acts follow each competitive social act.
4.6.1. Experiments with Ontonotes Corpus
In the experiments with the data from the Ontonotes corpus, each pair of people
become an instance, and we split instances into training (80%) and test (20%). As a learning
algorithm, we use SVM with RBF kernel as implemented in scikit-learn [52]. We report
results in the test set after tuning the SVM parameters (C and ) using 10-fold cross-validation
with the training set. More specifically, we train one classifier per dimension, and experiment
with all instances but the ones annotated inv. Thus, each classifier predicts 3 labels: 1(the
45
Feature Description
Verb
word, tag Word form and POS tag of the verb
dep out Outgoing syntactic dependency from verb
deps in Flags indicating incoming syntactic dependencies to the verb
lex name Name of the WordNet lexical file of the verb
token before Word form and POS tag of the token before the verb
token after Word form and POS tag of the token after the verb
Person
words, tags Concatenation of word forms and POS tags
type Whether the person is a pronoun or named entity
dep out Outgoing syntactic dependency
distance verb Number of tokens between the person and the verb
first token Word form and POS tag of the first token in the person
last token Word form and POS tag of the last token in the person
token before Word form and POS tag of the token before the person
token after Word form and POS tag of the token after the person
Personx Persony
direction Flag indicating whether x occurs before or after y
type Whether x and y are PERSON NEs or pronoun
Table 4.14. Feature set used to determine dimensions of interpersonal rela-
tionships between pairs of people (x, y). Verb features are extracted from the
verb of which either x or y is the subject, Person features are extracted from
x and y independently, and Persons features are extracted from x and y.
first descriptor applies), -1(the second descriptor applies), and 0 (neither descriptor applies).
The features we work with for Ontonotes corpus are summarized in Table 4.14. Most
features are standard. We refer to the pair of people as x and y. Verb features capture in-
formation about the verb to which x or y attach. We include words and part-of-speech tags
(verb, and tokens before and after), the name of the WordNet lexical file to which the verb
belongs, and dependencies. The inclusion of the lexical file gives general information about
46
a specific verb and inform about a broad category in which the verb falls under. This is
considered to be helpful because verbs of similar kind can depict similar relationship dimen-
sions. We extract inward and outward dependencies of the verb to get strctural knowledge of
the sentence. Person features are extracted from x and y independently, and consists mostly
of words and part-of-speech tags. We also included a flag indicating whether the person is
a pronoun or named entity (type), and the number of tokens between the person and the
verb (distance-verb). Personx Persony features capture information of both x and y . They
capture (a) whether x occurs before or after y in the sentence, and (b) whether they are
both named entities or one is a pronoun and the other one a named entity (type feature).
As we collect word forms of the names and verbs and their POS tags, the dependency of the
verbs to capture the structure information, bag-of-words features were not collected.
We present overall Ontonotes corpus results (averages of the classifiers for each di-
mension) using the majority baseline and with several feature combinations in Table 4.15.
Then, we present detailed results per dimension with the best feature combination in Table
4.16. We only present results obtained in the test set. The table shows results obtained for
each dimension with the best combination of features for all dimensions (Verb + Personx
+ Persony + Personx Persony, boldfaced in Table 4.15). The last three columns under All
contains the weighted average of Precision (P), Recall (R) and F-measure (F) of the di-
mensions based on the distributions of the three labels. The last row contains the weighted
averages of each column. An explanation of how the values are calculated is provided in the
appendix.
The majority baseline obtains 0.53 average F-measure. Recall that we build a classifier
per dimension, thus the combination of the nine majority-baseline classifiers predict two
labels: 1 (0.55 F-measure) and -1 (0.64 F-measure). Models trained with any combination
of features outperform the majority baseline, but they never learn to predict label 0. Since
0 occurs between 0.86% and 4.3% depending on the dimension (Section 4.4, this limitation
does not affect overall performance substantially. Verb features alone yield a 0.65 average
F- measure (1: 0.64, -1: 0.65). Adding features derived from x (Verb + Persony) improves
47
performance (0.71 average F-measure), and adding features derived from y (Verb + Persony)
slightly improves performance (0.67 average F-measure). In both cases, -1 is predicted
more accurately than 1 (0.78 vs. 0.69 and 0.74 vs. 0.67). Finally, adding all features
(Verb + Personx +Persony+ Personx Persony) yields the best results (average F-measure:
0.72), although by a minimal margin with respect to Verb + Personx. Detailed Results
per Dimension. Table 4.16 presents results per dimension with the best overall combination
of features (Verb+Personx+Persony+ Personx Persony). All dimensions obtain overall F-
measures between 0.65 and 0.83 (last column). Results per label are heavily biased towards
the most frequent label per dimension (Figure 4.2), although it is the case that the models
we experiment with predict both 1 and -1 for all dimensions. As stated above, none of them
predict 0, but this limitation does not substantially penalize overall performance because
of the low frequency of this dimensions are similar (0.67 vs. 0.76 and 0.76 vs. 0.58), and
the labels are distributed relatively evenly in our corpus (40.4% vs. 51.4% and. 58.4% vs.
35.3%). Finally, F-measures per label with other dimensions are biased towards the most
frequent label. For example, only 15% of all pairs of people have an enduring relationship,
and the F-measure for 1 with dimension temporary is quite high.
The comparison of the results with different feature sets reveals that only knowing
about the verb that connects both the people involved is not enough; we need some lexical
information about the words indicating the people as well. Knowledge about where they
verbs are placed in the sentence, and how many words are there in the sentence contribute to
useful information that leads us to better results. So we may deem that to indicate different
values of these dimensions, we possibly arrange the words in the sentence differently. In
other words, the linguistics of a sentence has something to do with the value of the dimension
administered.
4.6.2. Experiments with Friends Corpus
With the data from the Friends corpus, we experimented with SVM classifiers with
RBF kernel to predict dimensions of interactions and relationships. We divided the 24
episodes into train (episodes 120) and test (2124), and trained one classifier per dimension
48
Features Label P R F
Majority
Baseline
1 0.43 0.76 0.55
0 0.00 0.00 0.00
-1 0.57 0.59 0.64
Avg. 0.45 0.65 0.53
Verb
1 0.62 0.72 0.64
0 0.00 0.00 0.00
-1 0.71 0.78 0.65
Avg. 0.62 0.71 0.65
Verb, Personx
1 0.68 0.75 0.69
0 0.00 0.00 0.00
-1 0.78 0.79 0.78
Avg. 0.70 0.74 0.71
Verb, Persony
1 0.64 0.72 0.67
0 0.00 0.00 0.00
-1 0.74 0.76 0.74
Avg. 0.66 0.70 0.67
Verb, Personx,
Persony,
Personx Persony
1 0.69 0.74 0.70
0 0.00 0.00 0.00
-1 0.77 0.80 0.76
Avg. 0.71 0.76 0.72
Table 4.15. Results obtained for all dimensions with several combinations
of features for the Ontonotes dataset.
using scikit-learn [52]. Each classifier is trained with three labels: 1 (1st descriptor), -1 (2nd
descriptor), and 0 (unknown). The SVM parameters (C and ) were tuned using 10-fold
cross-validation with the train split, and results are reported using the test split. Note that
different pairs of people interact more or less in each episode (4.8). Thus, the classifiers
49
Dimension1 (1st desc.) 0 (unk.) -1 (2nd desc.) All
P R F P R F P R F P R F
Cooperative 0.73 0.96 0.83 0.00 0.00 0.00 0.60 0.19 0.29 0.66 0.72 0.65
Equal 0.56 0.10 0.17 0.00 0.00 0.00 0.74 0.97 0.84 0.68 0.74 0.66
Intense 0.39 0.30 0.34 0.00 0.00 0.00 0.78 0.85 0.82 0.67 0.71 0.69
Pleasure 0.40 0.28 0.33 0.00 0.00 0.00 0.87 0.93 0.90 0.79 0.82 0.80
Active 0.69 0.85 0.76 0.00 0.00 0.00 0.68 0.51 0.58 0.67 0.69 0.67
Intimate 0.44 0.17 0.24 0.00 0.00 0.00 0.88 0.98 0.93 0.81 0.86 0.83
Temporary 0.85 0.96 0.91 0.00 0.00 0.00 0.33 0.10 0.16 0.77 0.83 0.79
Concurrent 0.72 0.80 0.76 0.00 0.00 0.00 0.77 0.75 0.76 0.71 0.74 0.73
Spat. Near 0.66 0.68 0.67 0.00 0.00 0.00 0.73 0.79 0.76 0.66 0.70 0.68
Average 0.69 0.74 0.70 0.00 0.00 0.00 0.77 0.76 0.77 0.71 0.76 0.72
Table 4.16. Results obtained for each dimension with the best combination
of features for all dimensions (Verb + Personx + Persony + Personx Persony,
boldfaced in Table 4.15). The last three columns under All contains the
weighted average of Precision (P), Recall (R) and F-mesure (F) of the di-
mensions based on the distributions of the 3 labels. The last row contains the
weighted averages of each column.
are grounded on general language usage and not modeling who talks and who is talked
about. We also experimented with LSTMs taking as input the current conversation turn
and previous turns, but do not report results because SVM classifiers yielded better results.
For the Friends corpus we use a combination of features extracted directly from
the conversation turn, sentiment lexica, and context. Specifically, we extract (a) the first
word in the conversation turn,(b) bag-of-words features (binary flags and tf-idf scores), and
(c) the root verb, and flags indicating the presence of exclamation, question marks and
negation cues from (Morante and Daelemans, 2012) [49]. Regarding sentiment, we extract
flags indicating whether the turn has a positive, negative or neutral word in the list by
50
Hamilton et al. (2016)[33], the sentiment score of the turn (summation of sentiment scores
per token over the number of tokens in the turn), and a flag indicating whether the turn
contains a negative word from the list by Hu and Liu (2004)[39]. Regarding context, we
extract bag-of-words features from the previous conversation turn in which the same people
interact (not necessarily the preceding turn). The BOW features gave information about all
the words contained in the sentence. The information about the presence of punctuation
symbols like (?, !) were also included as these symbols set the tone of an interaction.
Similarly, the sentiment of the sentence was thought to be indicative of the value of the
dimension cooperative vs. competitive. The presence or absence of negative words was also
thought to be affecting the learning of the same dimension. But as we are considering a lot
more dimensions than the mentioned one, the average result does not seem to be affected
much with these features. Context information was hoped to facilitate the learning.
Table 4.19 shows the overall results (average of all dimensions) obtained with the
majority baseline and several feature combinations with Friends corpus. All feature com-
binations outperform the baseline. Sentiment features are not beneficial, leading to the
conclusion that sentiment does not correlate with dimensions of interactions and relation-
ships between people. This may look surprising at first sight, but recall that our dimensions
capture much more than if two people get along. Finally, the bag-of-words features from
the previous turn in which the same people interacted bring a small improvement (F: 0.70
vs. 0.72). We show results per dimension for the best feature combination (all) in Table
4.18. Despite the label distributions are biased shown in Figure 4.3, the system predicts
most labels for most dimensions except the very biased ones (cooperative, equal, and pleasure
oriented). Note that 0 (unknown) does not allow us to determine the value of a dimension,
and the low results with this label are not a concern.
4.6.3. Experiments with Friends Extension
We use SVM classifiers with RBF kernel to predict dimensions of interactions and
relationships as well as the social acts of cooperative and competitive interactions. We divided
the entire corpus (12 episodes) into stratified train and test splits, and use the implementation
51
P R F
Majority baseline 0.54 0.73 0.62
SVM with first word 0.62 0.71 0.65
+ BOW 0.66 0.73 0.67
+ sentiment 0.65 0.72 0.66
+ other 0.70 0.75 0.70
+ BOW previous 0.72 0.76 0.72
Table 4.17. Results obtained with the test set with several systems (average
of all dimensions) for the Friends dataset. Previous refers to the previous
conversation in which the same pair of people interacted not the immediately
previous turn).
1 (1st desc.) 0 (Unk) -1 (2nd desc.) All
P R F P R F P R F P R F
Inte
ract
ion
Cooperative 0.83 0.96 0.89 0.00 0.00 0.00 0.31 0.08 0.13 0.73 0.80 0.75
Active 0.92 0.90 0.92 n/a n/a n/a 0.75 0.76 0.75 0.87 0.87 0.87
Concurrent 0.92 0.92 0.92 n/a n/a n/a 0.70 0.69 0.70 0.87 0.87 0.87
Spat. Near 0.89 0.94 0.91 n/a n/a n/a 0.69 0.53 0.60 0.85 0.86 0.85
Rel
atio
nsh
ip
Equal 0.86 0.95 0.90 0.00 0.00 0.00 0.15 0.07 0.10 0.76 0.83 0.79
Intense 0.70 0.81 0.75 0.28 0.11 0.16 0.38 0.46 0.41 0.56 0.60 0.57
Pleasure Or. 0.82 1.00 0.90 0.00 0.00 0.00 1.00 0.02 0.04 0.82 0.83 0.75
Intimate 0.61 0.85 0.71 0.26 0.05 0.09 0.42 0.36 0.39 0.48 0.56 0.49
Temporary 0.53 0.42 0.47 0.67 0.17 0.27 0.62 0.85 0.72 0.60 0.60 0.57
Table 4.18. Results obtained per dimension with the best system (all fea-
tures, Table 4.17). The results under All the weighted averages for all labels,
recall that the label distribution is biased (Figure 4.3).
52
in scikit-learn [52] to train the classifiers. We tune SVM hyperparameters (C and γ) using
10-fold cross-validation with the train split. Regarding dimensions, we train one classifier
per dimension and all of them predict three labels: Regarding social acts, we train one for
cooperative social acts (5 labels: agree, excited, nice, support and normal) and another one
for competitive social acts (6 labels: complain, disagree, disbelief, dissatisfaction, mock and
rude).
For the Friends corpus we extract features from the utterance, context, and sentiment
lexica. The feature set does not include information about the two parties involved in an
interaction in order to rely only on language usage. The features set is rather simple and
includes the following. First, we extract the first word in an utterance (first). Second, we
extract bag-of-words representations (binary flags and tf-idf scores) of the utterance. Third,
we include the same bag-of-words representations for the previous utterance (BOW prev.).
Fourth, we extract features from sentiment lexica. Specifically, we extract flags indicating
whether the turn has a positive, negative or neutral word in the list by Hamilton et al.
(2016)[33], the sentiment score of the utterance (summation of sentiment scores per token
over the number of tokens in the utterance), and a flag indicating whether the utterance
contains a negative word from the list by Hu and Liu (2004) [39]. Fifth, we extract other
features, which include (a) the root verb (b) binary flags indicating the presence of exclama-
tion, question marks, and negation cues from [49]. The justification behind using the above
features is that they worked well in the Friends project. As we are dealing with very similar
data from the same original dataset in this project as well, we stuck with the same choices
of features.
Classifiers trained with the features above yield results in a realistic scenario. We
also report results using as features gold values for dimensions of the previous and current
interaction (all but the dimension being predicted), as well as the social acts of the previous
utterance. This was done to include context information regarding dimensions and social
acts. It is seen that when someone interacts with another cooperatively in one interaction, it
is likely that the next interaction would be cooperative as well. Or at least there is a common
53
P R F
Majority baseline 0.59 0.76 0.66
first + BOW 0.70 0.75 0.70
+ BOW prev. + sentiment + other 0.72 0.77 0.72
+ dims. previous interaction 0.75 0.77 0.75
+ other dims. current interaction 0.85 0.86 0.85
Table 4.19. Results for predicting dimensions of interactions and relation-
ships with the Friends extension dataset. The bottom two systems use gold
dimensions as features thus results are unrealistic.
pattern. Thus including the values of dimensions or social acts of previous sentences improved
the results considerably. These results are not realistic but show that classifiers are far from
perfect even in this unrealistic scenario.
Table 4.19 shows the results for predicting dimensions of interactions and relation-
ships (weighted averages) with the Friends extension dataset. Any combination of features
outperforms the majority baseline (F1: 0.66). Using as features the first word and bag-
of-words representations for the utterance brings the F1 to 0.70, and including all features
yields 0.72 F1. The results including gold annotations for all dimensions of the previous
interaction (F1: 0.75) and other dimensions of the current interaction (F1: 0.85) indicate,
unsurprisingly, that this information is beneficial, although results are far from perfect. Ta-
ble 4.20 shows the results for predicting dimensions of interactions and relations with the
best-performing realistic system.
Table 4.21 shows the overall results for predicting social acts (weighted average).
Recall that social acts are predicted only for interaction that are either cooperative or com-
petitive. This time, the features capturing the first work in the utterance and bag-of-words
representations bring substantial improvements with respect to the majority baseline with
both cooperative (F1: 0.71 vs. 0.77) and competitive (F1: 0.11 vs. 0.38) social acts. Adding
the additional features, however, does not bring improvements in results. The results with
54
1 (1st desc.) 0 (unk.) -1 (2nd desc.) All
P R F P R F P R F P R F
I Cooperative 0.81 0.94 0.87 0.00 0.00 0.00 0.43 0.17 0.24 0.73 0.78 0.74
R
Equal 0.90 0.96 0.93 0.50 0.16 0.24 0.33 0.16 0.21 0.83 0.87 0.85
Intense 0.71 0.96 0.81 0.38 0.05 0.09 0.64 0.21 0.32 0.66 0.70 0.63
Intimate 0.76 0.96 0.85 0.38 0.05 0.10 0.53 0.19 0.28 0.69 0.74 0.68
Temporary 0.37 0.08 0.13 0.80 0.06 0.11 0.77 0.97 0.86 0.70 0.75 0.68
Table 4.20. Detailed results for predicting dimensions of interactions and
relations with the best-performing realistic system (BOW + BOW prev. +
sentiment + other in Table 4.19).
P R F
Coop
er.
Majority baseline .64 .80 .71
first + BOW .75 .81 .77
+BOW prev.+sent.+ other .76 .81 .77
+social acts prev. .79 .82 .80
Com
pet
.
Majority baseline .07 .26 .11
first + BOW .38 .39 .38
+BOW prev.+sent.+other .40 .41 .39
+social acts prev. .55 .53 .52
Table 4.21. Results for predicting social acts (cooperative and competitive).
Systems labeled+social acts prev. use gold social acts and thus results are
unrealistic.
the feature set that includes gold social act information of the previous utterance have large
improvements with competitive social acts (F1: 0.39 vs. 0.52), and minor improvements
with cooperative social acts (F1: 0.77). Table 4.22 shows detailed results per social act.
55
P R F
Coop
erat
ive
Agree 0.25 0.05 0.08
Excited 0.43 0.13 0.20
Nice 0.55 0.33 0.42
Normal 0.84 0.96 0.89
Support 0.50 0.12 0.19
Average 0.76 0.81 0.77C
omp
etit
ive
Complain 0.38 0.33 0.35
Disagree 0.57 0.50 0.53
Disbelief 0.00 0.00 0.00
Dissatisfaction 0.38 0.60 0.46
Mock 0.39 0.33 0.36
Rude 0.33 0.19 0.24
Average 0.40 0.41 0.39
Table 4.22. Detailed results for predicting social acts with the best-
performing realistic system (BOW + BOW prev. + sentiment + other in
Table 4.21). The average shown is the weighted average based on the number
of instances in each label.
Just like with dimensions, these results demonstrate that the task is challenging.
4.6.4. Error Analysis of Friends Extension
The classifiers for predicting social acts are far from perfect. In this section, we
illustrate common errors made by the classifiers. The majority of cooperative interactions
are a normal social act (80.5%, Table 4.13), and unsurprisingly, our classifier wrongly predicts
many interactions as normal. One example is the utterance I’m having a son! said by Ross.
He utters the words with sheer joy and excitement, and the annotators had identified it to be
excited. But the classifier predicts it as normal, as it fails to capture the excitement usually
56
expressed through tone of voice. Another example in which the classifier fails to capture
the excitement is utterance Hey!, exclaimed by Chandler when he just got to know that his
friend Ross was becoming a father. As the only word spoken in the utterance (Hey) is often
used as a form of greetings, it is wrongly preicted as being nice instead of excited. Another
similar example is Paolo excitedly greeting his girlfriend by saying Oh, hi sweetie, but the
utterance being automatically identified as nice again. In Ok, ok, How about if we split it?,
Joey is normally suggesting something to his friend (to split a bill). The classifier wrongly
predicts agreeing social act, most likely because of the word ok appears three times.
There is generally a fine difference between the social acts dissatisfaction and com-
plain. By saying That-that was like the worst breakup in history!, even though Monica is
rather showing her dissatisfaction with her friend about his skills at breaking up, the classi-
fier predicts as complain social act. On the other hand, Chandler is rather complainig about
his friend Phoebe in Wait a minute, wait a minute, I see where this is going, you’re gonna
ask him to New Year’s, aren’t you?. But this utterance was misclassified as rude. Another
example of a wrong prediction is the utterance You’re gonna break the pact. The system
wrongly predicts rude for this complain social act, most likely because of the negative verb
break.
57
CHAPTER 5
DIMENSIONS OF INTERPERSONAL INTERACTIONS AND RELATIONSHIPS IN
CHAT FORUMS
People are social beings who communicate their feelings, emotions, thoughts, ideas,
etc. with each other through verbal and non-verbal interactions. People build relationships
based on these interactions, and these relationships, in turn, help them create and maintain
networks of peers. Peers in a network cooperate with each other, help each other to learn,
exchange ideas, but also compete for resources [67]. Peer networks are particularly important
for innovation and entrepreneurship [29] as an active exchange of ideas occurs there.
People are usually assumed to be altruistic in networks like online social forums.
They cooperate with and help one another with answers, advice, and ideas. The motivations
behind helping a peer include, but are not limited to, getting pure pleasure from helping,
self-advancement, building a reputation, developing relationships, or sheer entertainment
[65].
When people interact with each other, their interactions vary along various commu-
nicative styles or dimensions of interactions (as mentioned earlier in this document) such
as showing cooperativeness, equality, business orientation, etc. [56]. Varying these dimen-
sions or styles provides tools to achieve communicative goals. For example, someone who
is trying to build a reputation, will tend to use a more cooperative style. Someone who
simply tries to be helpful may use more words of advice in their interactions. The usage of
certain relationship-establishing styles is more prevalent in certain personalities [21] and in
certain settings. Also, people with a certain personality trait interact with others in a simi-
lar expected manner in similar situations [61]. Business-oriented people communicate more
independence, tolerance of ambiguity, risk-taking propensity, innovativeness, and leadership
qualities [69].
The impact of these dimensions or styles is therefore, an important factor in text
analysis. However, due to its complex, decentralized nature (cooperativeness is more than
58
just the use of a few keywords, but includes a whole inventory of communicative tools),
these dimensions have been studied very little in NLP. Part of the reason is the lack of
adequate corpora. We provide another such corpus and report encouraging results for the
above dimensions.
5.1. Original Adansonia Dataset
Our ultimate goal is to predict the communicative styles and strategies of young
people in online peer networks of aspiring entrepreneurs. The initial corpus is a result
of a large-scale social science experiment that involved around 5000 entrepreneurs (mostly
aspiring) from 49 African countries [67]. Those entrepreneurs, after completing an online
business course, interacted with their peers, within groups of sixty, through an Internet
platform for about two months and a half, resulting in about 140,000 chat interactions.
Besides the chat interactions, the original dataset also contains background informa-
tion about the speakers (country of origin, educational background, age, gender, etc.) All
these participants also submitted business proposals that were then evaluated to assess their
success.
The original experimental setup was designed to assess how communication among
peers mutually affects them. [67], therefore, already applied NLP techniques for semantic
analysis of the interactions. They also manually annotated other indicators, i.e., business-
relatedness, sentiment, and audience (one or several people) on a subset of 10k sentences,
and then predicted them for all remaining instances in the online interactions. This dataset,
therefore, provides a perfect starting point for our goals.
The first step to address our goal involves annotating styles of speech on a number of
interactions among people. We use the subset of the data with previous manual annotations,
and added our own annotations to further enrich the data.
5.2. Annotating Dimensions of Interpersonal Interactions and Relationships
We sample around 5500 chat interactions which were previously annotated for business-
relatedness, sentiment, and audience, and annotate the four dimensions of interpersonal in-
59
Cooperative
1: Mobile Webshop is a very good concept. Cooperative
2: You have not done anything yet. Competitive
Motivational
3: @NAME1234 well said @NAME456 start small and dream big..welldone Motivational
4: I meant to say voting contest to be precise. Neutral
Equal
5: Wishing you a very wonderful weekend. Equal
6: Happy to engage you on this.. Hierarchical
Advice
7: Think about it. Advice
8: This is cool Sunday. Neutral
Table 5.1. Annotation examples of contrasting values for each dimension.
Each chat interaction is either directed to an individual or others in general.
teractions and relationships:
(1) Cooperativeness indicating the friendliness shown towards the target audience, with
label values cooperative, competitive, and neutral.
(2) Advice indicating whether the interaction contains any words of advice with label
values advice and neutral.
(3) Motivational indicating whether the interaction contains any words of motivation,
with label values motivation and neutral [77].
(4) Equality indicating whether there is a display of hierarchy between the speaker and
the receiver, with label values equal and hierarchical.
For all dimensions, unknown is used whenever it is hard to determine any of the other values
from context.
60
Three graduate students with experience in NLP tasks annotated the data. After a
training and practice session, each of the annotators annotated their part of around 2,100
chat interactions. 502 of these were shared among all three annotators, so we can compute
agreement measures. We obtain the most probable labels for the shared portion using MACE
[37]. We summarize the agreements scores (raw agreement: 83%; averaged pairwise Cohen’s
κ: 0.53; Krippendorff’s alpha: 0.52) in Table 5.2. The average MACE competence score of
these annotators is 0.53.
raw κ Krippendorff’s α
Cooperative 76% 0.58 0.57
Motivational 91% 0.74 0.74
Equal 77% 0.33 0.34
Advice 91% 0.60 0.60
Average 83% 0.53 0.52
Table 5.2. Inter-annotator agreements for dimensions. 0.6 ≤ κ ≤ 0.8: sub-
stantial agreement, κ > 0.8: nearly perfect agreement [4].
Table 5.3 shows the Pearson’s correlations between pairs of the dimensions of interac-
tions and previous annotations. This indicates that Motivational diemsnions are usually also
Cooperative (0.61), give Advice (0.56) and are Equal (0.54). Interestingly, many Business-
related interactions are not very Cooperative (0.42).
For cooperativeness, 42.3% are labeled cooperative, 50.3% are neutral and only 2.14%
are labeled competitive. For the motivational style, 14.1% are motivational and 81.2% are
neutral. For the advice style, 9.2% are advice and 85.9% are neutral. For the equality style,
77.3% are equal and 8.8% are hierarchical.
Table 5.1 shows a number of actual chat interactions from the dataset with different
values for the dimensions annotated. In interaction (1), the praising is considered a cooper-
ative response, whereas in (2) the speaker is chiding someone, indicating a competitiveness.
61
S C M E A
B -0.17 -0.42 -0.28 -0.27 0.01
S – 0.29 -0.01 -0.04 -0.10
C – 0.61 0.37 0.40
M – 0.54 0.56
E – 0.26
Table 5.3. Pearson correlations between pairs of dimensions of interactions
(indicated by the names’ initial letters).
The praise in (3) is motivational. Example (4) does not really communicate any motivation,
so it is labeled as neutral for this style. Example (5) is just a greeting and does not indicate if
anybody is displaying hierarchy over anyone else, so it is equal. Example (6) shows that the
speaker instructs someone on how to behave (hierarchical). In (7), the speaker is advising
someone to think about a matter whereas (8) is just another neutral statement.
5.3. Experiments and Results
People with certain personalities tend to show certain values for the dimensions of
their interactions with others [21]. People with business-oriented mind are expected to
show more keenness and gain more success in business. Business-oriented people are usually
more independent, more tolerant, do not fear a risk, tend to innovate more often, and are
good leaders. [69]. We want to see if there is some correlation between the dimensions of
interactions displayed by people and their ability to succeed in business. We want to predict
the four different dimensions of interactions (cooperative, motivational, advice, equality),
as well as three subsequent indicators of business success: (1) whether the person owns
a business (has business), (2) whether someone has ever owned a business (business
ever) and (3)whether they have submitted a business proposal to compete for a funding
opportunity to start a business (business proposal). The dataset we annotated came with
annotations of these three business indicators, making our work easier.
62
We compare (1) an SVM classifier with RBF kernel to predict both the dimensions of
interactions and relationships as well as the business success indicators and (2) a Multitask
Learning (MTL) Convolutional Neural Network to predict the business success indicators.
We divide our annotated dataset into 80-20 stratified train-test splits for predicting
the dimensions. For the business success prediction, we use the first 500 instances as test
and the rest as train.
5.3.1. SVM Setup
We use the SVM implementation in scikit-learn [52] and tune the hyperparameters
(C and γ) using 10-fold cross-validation within the train split. We train one classifier per
style and per business success indicator to predict the different labels.
Feature Set. We extract features from the chat interactions and sentiment lexica. The
feature set relies only on language usage. We extract the first word in a chat interaction,
the bag-of-words representations (binary flags and tf-idf scores) of the chat interaction, and
features from sentiment lexica. Specifically, we extract flags indicating whether the turn
has a positive, negative or neutral word in the list by [33], the sentiment score of the chat
interaction (summation of sentiment scores per token over the number of tokens), and a
flag indicating whether the interaction contains a negative word from the list by [39]. We
also extract other features, which include (a) the root verb (b) binary flags indicating the
presence of exclamation, question marks and negation cues from [49].
5.3.2. Multitask Learning (MTL) Setup
Transfer learning is an approach to machine learning which uses what has been learned
from past related tasks to facilitate learning a new task. Transfer learning thus leverages
previous experience to learn novel, but related concepts more efficiently [73]. Multitask
Learning is a variation of transfer learning that one task learns better by utilizing the in-
formation found in the training signals of other related tasks. The task of interest (target
task) and the related tasks (auxiliary or extra tasks) are learned in parallel, utilizing a
shared representation. Each task can help in learning other tasks. In a single task learning,
63
information that would have been normally used as features, would rather be treated as
outputs instead in a multitask setup. Multitask Learning usually improves generalization
performance. Multitask Learning can be used with different learning algorithms and in dif-
ferent domains[18]. In our tasks of learning 3 business success indicators (has business,
business ever or business proposal), we leverage multitask learning by simultaneously
learning related auxiliary tasks, in our case, the dimensions of interactions (business, sen-
timent, cooperativeness, motivational, advice, equality). We use a standard Convolutional
Neural Network over word-embeddings, with one output per task. We preprocess the data
(convert to lowercase, removed URLs and stop-words, converted numbers to 0’s etc.) and
learn a skip-gram embeddings model [46] trained for 50 epochs. We use an embedding size
of 512, choosing a power of 2 for memory efficiency.
In the CNN, the input layer has the word indices of the text, converted via the embed-
ding matrix into a matrix of words x embeddings. We convolve two parallel channels with
max-pooling layers, and convolutional window sizes 4 and 8 over the input. The two window
sizes account for both short and relatively long patterns in the texts. In both channels, the
initial number of filters is 128 for the first convolution, and 256 in the second one. We join
the output of the convolutional channels and pass it through an attention mechanism [5, 66]
to emphasize the weight of any meaningful pattern recognized by the convolutions. We use
the implementation of [74]. The output consists of 7 independent, fully-connected layers for
the predictions, respectively in the form of discrete labels for classification of one of the busi-
ness success indicators of a person (has business, business ever or business proposal)
as the target task, and the dimensions of interactions (business, sentiment, cooperativeness,
motivational, advice, equality) as the auxiliary tasks. Figure 5.1 shows the architecture of
such a multitask learning network. We trained one model per business success indicator.
5.3.3. Results
Table 5.4 compares the results of the different systems to predict the dimensions of
interactions as well as the business success indicators. Our SVM model does much better
than majority baseline for all the dimensions of interactions (F-measures = 0.77 vs. 0.34,
64
words x embeddings
Convolutional max-pooling layer window: 4, filters: 128
Convolutional max-pooling layer window: 8, filters: 128
Convolutional max-pooling layer window: 2, filters: 256
Convolutional max-pooling layer window: 4, filters: 256
Attention mechanismSize: 512
Fully connected
layer
Fully connected
layer
Fully connected
layer
Fully connected
layer
Fully connected
layer
Fully connected
layer
Fully connected
layer
Output 1 Output 2 Output 3 Output 4 Output 5 Output 6 Output 7
Architecture of the
Multi-task Learning set-up
Input
Output
Figure 5.1. Multitask Learning Architecture.
0.89 vs. 0.73, 0.78 vs. 0.67 and 0.86 vs. 0.79). For the business success indicators, either the
SVM (F-measures = 0.59, 0.38 and 0.64) or the MTL (F-measures = 0.63, 0.51 and 0.65)
model outperforms the majority baseline.
The above results indicate that an SVM model can be used to predict the dimen-
sions, and can thus be used for automatic annotations of the dimensions for the remaining.
The results we got are decent but not excellent. So far, we have used features based on
language only. We could incorporate other information as features as well. Extralinguistic
factors like demographic information (for example, age, gender information) can improve
the classification if included as features[36]. Table 5.3 shows correlations between certain
dimensions. Including one dimension as a feature for predicting another correlated dimen-
sion will improve the results. The Adansonia dataset is rich with other information like
65
Model P R F
Coop.majority 0.25 0.50 0.34
SVM 0.77 0.77 0.77
Motivationmajority 0.66 0.81 0.73
SVM 0.90 0.90 0.89
Equalmajority 0.60 0.77 0.67
SVM 0.78 0.81 0.78
Advicemajority 0.74 0.86 0.79
SVM 0.86 0.88 0.86
has
business
majority 0.51 0.71 0.59
SVM 0.58 0.68 0.59
MTL 0.61 0.66 0.63
business
ever
majority 0.20 0.44 0.27
SVM 0.54 0.47 0.38
MTL 0.52 0.51 0.51
business
pro-
posal
majority 0.57 0.75 0.64
SVM 0.66 0.69 0.67
MTL 0.65 0.75 0.65
Table 5.4. Results for predicting styles of interactions and three indicators
of business success. Each value is a weighted average over the different labels
based on the number of instances falling in that label.
the educational background of a participant, their geographical background, language pro-
ficiency, etc. We can carry out a correlation analysis on these background information and
the business success indicators of the participants. If there are high correlations between any
of these background information and the business success indicators, to improve our results,
66
we can either include the information as features in the SVM models or as auxiliary tasks in
the multitask learning setup. The multitask learning setup for a particular business success
indicator can also incorporate other business success indicators as well as the demographic
information as auxiliary tasks.
5.3.4. Automatic Annotation
The manually annotated data discussed in Section ?? was used to train an SVM
model with feature sets and results, as discussed in Section 5.3.1. This SVM model was used
to automatically annotate the rest of the Adansonia data. Here we will discuss the label
distributions of the automatically annotated data. For the dimension cooperativeness, 34.9%
are cooperative and 59.8% are neutral and only 5.15% are competitive. For the dimension
motivational, 10.7% are motivational and 85.9% are neutral. For the dimension advice, 5.5%
are advice and 91.7% are neutral . For the dimension Equality, 86.6% are equal and 9.7%
are hierarchical.
67
CHAPTER 6
CONCLUSION
6.1. Our Achievements
In this dissertation, we have presented a set of nine dimensions of interpersonal in-
teractions and relationships, including dimensions with a long tradition in social science and
new ones. These dimensions allowed us to differentiate the core characteristics of interactions
and relationships between two individuals. For example, people that communicate may be
spatially near or spatially distant (asking questions in class vs. chatting online), and have
a pleasure-oriented or work-oriented relationship (somebody wishing good luck to a friend
vs. interviewer and interviewee). These dimensions are applicable to any interaction or
relationship regardless of the underlying name relationship type (e.g., siblings, friends,
doctor-patient).
We have put annotation effort on annotating these dimensions on data from a mixed
domain of news articles, TV interview transcripts. We also augmented our dimension an-
notation on TV series transcripts. We got an idea of how the values of the dimensions can
change. Our annotations show that assigning values to dimensions can be done reliably with
substantial Cohen’s Kappa values for inter-annotator agreements. Useful dimension values
(1 and -1 labels) are assigned to dimensions in most pairs of people. Experimental results
following a standard supervised machine-learning approach show that assigning values to di-
mensions can be automated ( 0.72 F-measures for both the previous works). We believe that
extracting dimensions of interpersonal interactions and relationships complements previous
efforts that extract relationships.
We have also studied values of dimensions for pairs of people defined slightly differ-
ently than the previous works, where we consider the speaker and the person being spoken
to forming the pairs of people for whom we will annotate dimensions of interactions and
relationships. We also use social acts to further characterize cooperative and competitive
interactions (five and six acts respectively). We work with dialogue transcripts of the first
68
season of the popular TV show Friends, this time with the first 12 episodes. It is analyzed
to see if dimensions and social acts are intuitive to humans by calculating Cohen’s Kappa
for agreements. Beyond numerical analysis, we also discuss salient words for social acts and
the label distribution of both dimensions and social acts. We try simple SVM trained with
bag-of-words features and other features derived from sentiment to confirm if learning is
possible or not. We also experiment with features from gold annotations of the previous and
current utterance (other dimensions and social acts) to see whether that improves results or
not.
We also will study how young aspiring entrepreneurs exchange messages with each
other on online platforms as well as in person. Annotation projects are conducted to annotate
interaction dimensions like equality, coopertiveness, and other interaction styles indicating
motivation and advice on about 5000 chat interactions. SVM models are trained and tested
on these annotated data to see if these dimensions or styles can be reliably predicted or not.
The rest of the data from a large corpus of 140,000 data are then annotated with these values.
The manually annotated data is also used in a multitask learning set-up to predict business
success indicators denoting whether a user has applied for a business funding, whether the
user already has a business, and whether they have ever been involved in business.
6.2. Applications and Future Directions
Two people usually have a constant named relationship during their lifetime or a
good portion of it, but their interpersonal interactions can be described with different values
of the dimensions at different points in time. Extraction of these values can help us build
trajectories of relationships between pairs of people over a course of time [40]. Works like
ours can be used to analyze characters of novels or other literary works. Also, a similar
analysis can be used to analyze historical characters and their relationships with others
around them. It can help predict how people will act in certain circumstances. Political
figures can also be analyzed similarly, and their moves may be predicted. Extracting in-depth
information about people’s relationships will inform us more about people of historical or
political importance. Question Answering about people will be benefitted from this. Search
69
Engines and Information Retrieval systems can incorporate our idea.
We also found that business success can be predicted using information extracted
from their spoken words and the values of the dimensions of their relationship. Other
background information about these people can farther be added in a controlled fashion
to see if they contribute to the learning of business success. This also gives us hope that
business success, it may be possible to learn other traits like success in politics, success in
academia, job, etc. This can help people to know what qualities or practices they should
work on to ensure success in life. People can avoid events or behavioral characteristics that
show a correlation with failure. Previously, cluster analyses on the behavioral profiles of
pairs of people indicated that there are three friendship types, and the friendship types they
are involved in tell us how adjusted they are in the society [31].
An aggregate of people work as a system, and the ones involved are similar and
share similar fates, choices, course of life, etc. [16]. The values of the dimensions can
be used to cluster people into different groups based on how they maintain relationships
with people around them. This will give us an idea of the system they belong to, which
will then reveal a lot about the people’s behavioral characteristics. We can predict their
lifestyle choices. This may give us a lead to what products the person may be interested
in. The clustering technique may be used to predict the socio-economic background of
the users. This can further be used to predict their financial ability. This will be helpful
information for advertising companies and other businesses so that the right products can be
advertised to them. As the saying goes, ”birds of the same feather flock together”, looking
at whom a person mingles with will also give an idea about what they like and can afford.
Recommendation systems for movies, music, etc. can also utilize the information to predict
what a person may be interested in watching or listening to. Behavioral analysis can also
help predict whom a person is comfortable with, whom they can be friends with. This can
be used by friend recommendation systems in social media, match-making sites, etc. If a
person is mostly friendly, intensely involved for a long time with another person, are in close
proximity most of the time or frequently, it is likely that these two people may enjoy similar
70
entertainment, similar food, may be interested in similar services, similar or closeby stores,
similar restaurants, etc. They may plan to spend their holidays together in the same places.
Thus, airline and travel companies can use information about relationships between people
to suggest trip offers better.
71
APPENDIX A
RESULT CALCULATION
72
Features Label P R F
Majority
Baseline
1 0.43 0.76 0.55
0 0.00 0.00 0.00
-1 0.57 0.59 0.64
Avg. 0.45 0.65 0.53
Verb
1 0.62 0.72 0.64
0 0.00 0.00 0.00
-1 0.71 0.78 0.65
Avg. 0.62 0.71 0.65
Verb, Personx
1 0.68 0.75 0.69
0 0.00 0.00 0.00
-1 0.78 0.79 0.78
Avg. 0.70 0.74 0.71
Verb, Persony
1 0.64 0.72 0.67
0 0.00 0.00 0.00
-1 0.74 0.76 0.74
Avg. 0.66 0.70 0.67
Verb, Personx,
Persony,
Personx Persony
1 0.69 0.74 0.70
0 0.00 0.00 0.00
-1 0.77 0.80 0.76
Avg. 0.71 0.76 0.72
Table A.1. Results obtained for all dimensions with several combinations of
features for the Ontonotes Dataset.
Table A.1 shows the results for different feature combinations for experiments on the
Ontonotes data (same as Table 4.15). Table A.2 shows the detailed results obtained for each
of the dimensions with the best feature combinations, that is the result bolded in A.1 (similar
to Table 4.16). In Table A.2 for each row with a dimension name, we show the precision
73
Dimension1 (1st desc.) 0 (unk.) -1 (2nd desc.) All
P R F P R F P R F P R F
Cooperative 0.73 0.96 0.83 0.00 0.00 0.00 0.60 0.19 0.29 0.66 0.72 0.65
Equal 0.56 0.10 0.17 0.00 0.00 0.00 0.74 0.97 0.84 0.68 0.74 0.66
Intense 0.39 0.30 0.34 0.00 0.00 0.00 0.78 0.85 0.82 0.67 0.71 0.69
Pleasure 0.40 0.28 0.33 0.00 0.00 0.00 0.87 0.93 0.90 0.79 0.82 0.80
Active 0.69 0.85 0.76 0.00 0.00 0.00 0.68 0.51 0.58 0.67 0.69 0.67
Intimate 0.44 0.17 0.24 0.00 0.00 0.00 0.88 0.98 0.93 0.81 0.86 0.83
Temporary 0.85 0.96 0.91 0.00 0.00 0.00 0.33 0.10 0.16 0.77 0.83 0.79
Concurrent 0.72 0.80 0.76 0.00 0.00 0.00 0.77 0.75 0.76 0.71 0.74 0.73
Spat. Near 0.66 0.68 0.67 0.00 0.00 0.00 0.73 0.79 0.76 0.66 0.70 0.68
Average 0.69 0.74 0.70 0.00 0.00 0.00 0.77 0.76 0.77 0.71 0.76 0.72
Table A.2. Results obtained for each dimension with the best combination
of features for all dimensions (Verb + Personx + Persony + Personx Persony,
boldfaced in Table A.1).
(P), recall (R) and F-measures (F) of the three labels 1 (first descriptor), 0 (unknown) and
-1 (second descriptor) respectively. The last three columns show the weighted averages of
these P, R, and F values based on the number of instances falling under the different labels
respectively for each dimension. The F-measure in the last column is not simply the harmonic
mean of the Precision and Recall values in the previous two columns, rather the weighted
averages of the F-measures obtained from each of the labels 1, 0 and -1. Thus in some of
the cases, it is seen that the F-measure in the last column does not always fall between
the Precision and Recall values found in the previous two columns. In the second last row,
for each column, we show the weighted averages of that column based on the number of
instances falling under each dimension.
As the weighted average calculation may create confusions about the values, we are
also showing the macro averages of the P, R and F values. Tables A.3, A.4 and A.5 show
74
DimensionMacro Average
P R F
Cooperative 0.44 0.38 0.41
Equal 0.43 0.35 0.39
Intense 0.39 0.38 0.38
Pleasure 0.42 0.40 0.41
Active 0.45 0.45 0.45
Intimate 0.44 0.38 0.41
Temporary 0.39 0.35 0.37
Concurrent 0.50 0.52 0.51
Near 0.46 0.49 0.47
Average 0.44 0.41 0.42
Table A.3. Macro averages of the Precision, Recall and F-measure respec-
tively over the different labels for each dimension shown in Table A.2.
the corresponding macro averages of the values found in Tables A.2, 4.18 and 4.20. The
macro averages of the P and R are calculated as straigth-forward unweighted averages over
the labels for each dimension. The F for each dimension is the harmonic mean of these
calculated average P and the R values. Here the values are considerably lower as the values
from the labels which had low values and had very few number of instances (label 0) are
also included without a weight incorporated.
75
DimensionMacro Average
P R F
Cooperative 0.38 0.35 0.46
Active 0.84 0.83 0.83
Concurrent 0.81 0.80 0.81
Spat. Near 0.79 0.76 0.77
Equal 0.33 0.34 0.33
Intense 0.45 0.46 0.45
Pleasure 0.60 0.34 0.44
Intimate 0.43 0.42 0.42
Temporary 0.60 0.48 0.54
Table A.4. Macro averages of the Precision, Recall and F-measure respec-
tively over the different labels for each dimension shown in Table 4.18.
DimensionMacro Average
P R F
Cooperative 0.41 0.37 0.37
Equal 0.57 0.42 0.48
Intense 0.58 0.42 0.49
Intimate 0.56 0.42 0.48
Temporary 0.65 0.64 0.64
Table A.5. Macro averages of the Precision, Recall and F-measure respec-
tively over the different labels for each dimension shown in Table 4.20.
76
APPENDIX B
PUBLICATIONS
77
The research presented in this dissertation has resulted in the following publications:
(1) Farzana Rashid and Eduardo Blanco. 2017. Dimensions of Interpersonal Relation-
ships: Corpus and Experiments. In Proceedings of the 2017 Conference on Empirical
Methods in Natural Language Processing (EMNLP). Copenhagen, Denmark.
(2) Farzana Rashid and Eduardo Blanco. 2018. Characterizing Interactions and Re-
lationships between People. In Proceedings of the 2018 Conference on Empirical
Methods in Natural Language Processing (EMNLP). Brussels, Belgium.
78
REFERENCES
[1] Sinnet: Social interaction network extractor from text, The Companion Volume of the
Proceedings of IJCNLP 2013: System Demonstrations, 2013, pp. 33–36.
[2] John Adamopoulos, The emergence of social meaning: A theory of action construal,
pp. 255–272, Springer New York, New York, NY, 2012.
[3] Jacqueline Aguilar, Charley Beller, Paul McNamee, Benjamin Van Durme, Stephanie
Strassel, Zhiyi Song, and Joe Ellis, A comparison of the events and relations across
ace, ere, tac-kbp, and framenet annotation standards, Proceedings of the Second Work-
shop on EVENTS: Definition, Detection, Coreference, and Representation (Baltimore,
Maryland, USA), Association for Computational Linguistics, June 2014, pp. 45–53.
[4] Ron Artstein and Massimo Poesio, Inter-coder agreement for computational linguistics,
Comput. Linguist. 34 (2008), no. 4, 555–596.
[5] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio, Neural machine translation
by jointly learning to align and translate, 2014, cite arxiv:1409.0473Comment: Accepted
at ICLR 2015 as oral presentation.
[6] A. Bajracharya, S. Shrestha, S. Upadhyaya, B. Shrawan, and S. Shakya, Automated
characters recognition and family relationship extraction from stories, 2018 8th Interna-
tional Conference on Cloud Computing, Data Science Engineering (Confluence), 2018,
pp. 14–15.
[7] David Bamman, Brendan OConnor, and Noah A Smith, Learning latent personas of
film characters, Proceedings of the 51st Annual Meeting of the Association for Compu-
tational Linguistics (Volume 1: Long Papers), 2013, pp. 352–361.
[8] Emily M. Bender, Jonathan T Morgan, Meghan Oxley, Mark Zachary, Brian Hutchin-
son, Alex Marin, Bin Zhang, and Mari Ostendorf, Annotating social acts: Authority
claims and alignment movesin wikipedia talk pages, Proceedings of the Workshop on
Language in Social Media, June 2011, pp. 48–57.
79
[9] Bernard E. Whitley, Mary E. Kite, and Heather L. Adams, Principles of research in
behavioral science, Routledge, Ney York, 2018.
[10] Eduardo Blanco and Dan Moldovan, Finding associations between people.
[11] David B. Bracewell, Marc T. Tomlinson, Mary Brunson, Jesse Plymale, Jiajun
Bracewell, and Daniel Boerger, Annotation of adversarial and collegial social actions
in discourse, Proceedings of the Sixth Linguistic Annotation Workshop, 2012, pp. 184–
192.
[12] David B. Bracewell, Marc T. Tomlinson, and Hui Wang, Identification of social acts in
dialogue, Proceedings of COLING 2012: Technical Papers, 2012, pp. 375–390.
[13] Philip Bramsen, Martha Escobar-Molano, Ami Patel, and Rafael Alonso, Extracting so-
cial power relationships from natural language, Proceedings of the 49th Annual Meeting
of the ACL, 2011, pp. 773–782.
[14] Cynthia Breazeal, Role of expressive behaviour for robots that learn from people, Philo-
sophical Transactions of the Royal Society B: Biological Sciences 364 (2009), no. 1535,
3527–3538.
[15] B Brewer and W Gardner, Who is this we? levels of collective identity and self repre-
sentations, 71(1) (1996), 83–93.
[16] Donald T Campbell, Common fate, similarity, and other indices of the status of aggre-
gates of persons as social entities, Behavioral science 3 (1958), no. 1, 14–25.
[17] M. A. Campion, E. M. Papper, and G. J. Medsker, Relations between work team char-
acteristics and effectiveness: A replication and extension., (1996), 429–452.
[18] Rich Caruana, Multitask learning, Machine Learning 28 (1997).
[19] Snigdha Chaturvedi, Structured approaches for exploring interpersonal relationships in
natural language text, Ph.D. thesis, 01 2016.
[20] Yu-Hsin Chen and Jinho D. Choi, Character identification on multiparty conversation:
Identifying mentions of characters in tv shows, Proceedings of the 17th Annual Meeting
of the Special Interest Group on Discourse and Dialogue, 2016, pp. 90–100.
80
[21] Kevin H. C. Cheng, Further linguistic markers of personality: The way we say things
matters, 2011.
[22] Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg, Echoes
of power: Language effects and power differences in social interaction, Proceedings of
WWW, WWW ’12, ACM, 2012, pp. 699–708.
[23] Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and
Christopher Potts, A computational approach to politeness with application to social
factors, Proceedings of the ACL, August 2013, pp. 250–259.
[24] Lea Deleris, Francesca Bonin, Elizabeth Daly, Stephane Deparis, Yufang Hou, Charles
Jochim, Yassine Lassoued, and Killian Levacher, Know who your Friends are: Un-
derstanding social connections from unstructured text, Proceedings of the 2018 Confer-
ence of the North American Chapter of the Association for Computational Linguistics:
Demonstrations (New Orleans, Louisiana), Association for Computational Linguistics,
June 2018, pp. 76–80.
[25] Morton Deutsch, Interdependence and psychological orientation, Conflict, Interdepen-
dence, and Justice, Springer, 2011, pp. 247–271.
[26] George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie
Strassel, and Ralph Weischedel, The automatic content extraction (ace) program tasks,
data, and evaluation, Proceedings of LREC, May 2004.
[27] David K Elson, Nicholas Dames, and Kathleen R McKeown, Extracting social networks
from literary fiction, Proceedings of the 48th annual meeting of the association for
computational linguistics, Association for Computational Linguistics, 2010, pp. 138–
147.
[28] Eric Gilbert, Phrases that signal workplace hierarchy, Proceedings of the ACM 2012
Conference on Computer Supported Cooperative Work (New York, NY, USA), CSCW
’12, ACM, 2012, pp. 1037–1046.
[29] Juanita Gonzalez-Uribe and Michael Leatherbee, The Effects of Business Accelerators
81
on Venture Performance: Evidence from Start-Up Chile, The Review of Financial Stud-
ies 31 (2017), no. 4, 1566–1603.
[30] Ralph Grishman and Beth Sundheim, Message understanding conference-6: A brief
history, Proceedings of the 16th Conference on Computational Linguistics - Volume
1 (Stroudsburg, PA, USA), COLING ’96, Association for Computational Linguistics,
1996, pp. 466–471.
[31] Berna Guroglu, Cornelis FM Van Lieshout, Gerbert JT Haselager, and Ron HJ Scholte,
Similarity and complementarity of behavioral profiles of friendship types and types of
friends: Friendships and psychosocial adjustment, Journal of Research on Adolescence
17 (2007), no. 2, 357–386.
[32] I. A. Hameed, Using natural language processing (nlp) for designing socially intelligent
robots, 2016 Joint IEEE International Conference on Development and Learning and
Epigenetic Robotics (ICDL-EpiRob), 2016, pp. 268–269.
[33] William L. Hamilton, Kevin Clark, Jure Leskovec, and Dan Jurafsky, Inducing domain-
specific sentiment lexicons from unlabeled corpora, Proceedings of EMNLP, November
2016, pp. 595–605.
[34] J. Hauffa, G. Bossert, N. Richter, F. Wolf, N. Liesenfeld, and G. Groh, Beyond foaf:
Challenges in characterizing social relations, 2011 IEEE Third International Conference
on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on
Social Computing, 2011, pp. 790–795.
[35] Hua He, Denilson Barbosa, and Grzegorz Kondrak, Identification of speakers in novels,
Proceedings of the 51st Annual Meeting of the Association for Computational Linguis-
tics (Volume 1: Long Papers), 2013, pp. 1312–1320.
[36] Dirk Hovy, Demographic factors improve classification performance, Proceedings of the
53rd Annual Meeting of the Association for Computational Linguistics and the 7th In-
ternational Joint Conference on Natural Language Processing (Volume 1: Long Papers)
(Beijing, China), Association for Computational Linguistics, July 2015, pp. 752–762.
[37] Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy, Learning whom
82
to trust with MACE, Proceedings of the 2013 Conference of the North American Chap-
ter of the Association for Computational Linguistics: Human Language Technologies
(Atlanta, Georgia), Association for Computational Linguistics, June 2013, pp. 1120–
1130.
[38] Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph
Weischedel, OntoNotes: the 90% Solution, NAACL ’06: Proceedings of the Human
Language Technology Conference of the NAACL, Companion Volume: Short Papers on
XX (Morristown, NJ, USA), Association for Computational Linguistics, 2006, pp. 57–
60.
[39] Minqing Hu and Bing Liu, Mining and summarizing customer reviews, Proceedings of
the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data
Mining (New York, NY, USA), KDD ’04, ACM, 2004, pp. 168–177.
[40] Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal
Daume III, Feuding families and former friends: Unsupervised learning for dynamic
fictional relationships, Proceedings of NAACL, June 2016, pp. 1534–1544.
[41] K. A. Jehn and E. A. Mannix, The dynamic nature of conflict: A longitudinal study of
intragroup conflict and group performance, 44(2) (2001), 238.
[42] Harold H Kelley, Personal relationships: Their structures and processes, Psychology
Press, 2013.
[43] Dimitrios Kokkinakis, Annotation of interpersonal relations in swedish prose fiction.,
Proceedings of the 3rd Workshop on Annotation of Corpora for Research in the Hu-
manities (ACRH-3). Sofia, Bulgaria., 2013, pp. 37–47.
[44] Seth Kulick, Ann Bies, and Justin Mott, Inter-annotator agreement for ere annotation,
Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference,
and Representation (Baltimore, Maryland, USA), Association for Computational Lin-
guistics, June 2014, pp. 21–25.
[45] K. Maki, M. Yoder, Y. Jo, and C. Ros, Roles and success in wikipedia talk pages:
83
Identifying latent patterns of behavior, Proceedings of the Eighth International Joint
Conference on Natural Language Processing, November 2017, pp. 1026–1035.
[46] Tomas Mikolov, Scott Wen-tau Yih, and Geoffrey Zweig, Linguistic regularities in con-
tinuous space word representations, Proceedings of the 2013 Conference of the North
American Chapter of the Association for Computational Linguistics: Human Lan-
guage Technologies (NAACL-HLT-2013), Association for Computational Linguistics,
May 2013.
[47] Nikola Milosevic, A multi-layered approach to information extraction from tables in
biomedical documents, 2018.
[48] Sangwoo Moon and Hairong Qi, A hybrid feature extraction framework based on risk
minimization and independence maximization, 06 2009, pp. 2141–2144.
[49] Roser Morante and Walter Daelemans, Conandoyle-neg: Annotation of negation cues
and their scope in conan doyle stories, Proceedings of LREC, 2012, ACL Anthology
Identifier: L12-1077, pp. 1563–1568.
[50] Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson, Stranse: a novel em-
bedding model of entities and relationships in knowledge bases, Proceedings of the 2016
Conference of the North American Chapter of the Association for Computational Lin-
guistics: Human Language Technologies (San Diego, California), Association for Com-
putational Linguistics, June 2016, pp. 460–466.
[51] D. A. Owens and R. I. Sutton, Status contests in meetings: Negotiating the informal
order., (2001), 299–316.
[52] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel,
P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau,
M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Machine learning in Python,
Journal of Machine Learning Research 12 (2011), 2825–2830.
[53] Vinodkumar Prabhakaran and Owen Rambow, Predicting power relations between par-
ticipants in written dialog from a single thread, Proceedings of the ACL, 2014, pp. 339–
344.
84
[54] Yanjun Qi, Bing Bai, Xia Ning, and Pavel Kuksa, Systems and methods for semi-
supervised relationship extraction, October 28 2014, US Patent 8,874,432.
[55] S Ramaraju et al., Psychological perspectives on interpersonal communication, Journal
of Arts, Science & Commerce 3 (2012), no. 4, 68–73.
[56] Farzana Rashid and Eduardo Blanco, Characterizing interactions and relationships be-
tween people, Proceedings of EMNLP, 2018, pp. 4395–4404.
[57] Harry T Reis, W Andrew Collins, and Ellen Berscheid, The relationship context of
human behavior and development., Psychological bulletin 126 (2000), no. 6, 844.
[58] David R Schmitt, Interpersonal relations: Cooperation and competition, Journal of the
Experimental Analysis of Behavior 42 (1984), no. 3, 377–383.
[59] Hinrich Schutze, Christopher D Manning, and Prabhakar Raghavan, Introduction to
information retrieval, vol. 39, Cambridge University Press, 2008.
[60] Nicole Shechtman and Leonard M. Horowitz, Interpersonal and noninterpersonal in-
teractions, interpersonal motives, and the effect of frustrated motives, Personality and
Social Psychology Bulletin 32 (2006), no. 8, 1126–1139, PMID: 16861315.
[61] Mark Snyder and Arthur A Stukas Jr, Interpersonal processes: The interplay of cog-
nitive, motivational, and behavioral activities in social interaction, Annual review of
psychology 50 (1999), no. 1, 273–303.
[62] Shashank Srivastava, Snigdha Chaturvedi, and Tom Michael Mitchell, Inferring inter-
personal relations in narrative summaries, AAAI, 2015.
[63] Ang Sun, Ralph Grishman, and Satoshi Sekine, Semi-supervised relation extraction with
large-scale word clustering, Proceedings of the 49th Annual Meeting of the Associa-
tion for Computational Linguistics: Human Language Technologies (Portland, Oregon,
USA), Association for Computational Linguistics, June 2011, pp. 521–529.
[64] Mihai Surdeanu and Ji Heng, Overview of the english slot filling track at the tac2014
knowledge base population evaluation, Proceedings of the TAC-KBP 2014 Workshop,
2014.
85
[65] Yla Tausczik and James Pennebaker, Participation in an online mathematics commu-
nity: Differentiating motivations to add, 02 2012, pp. 207–216.
[66] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N.
Gomez, Lukasz Kaiser, and Illia Polosukhin, Attention is all you need, CoRR
abs/1706.03762 (2017).
[67] Fernando Vega-Redondo, Paolo Pin, Diego Ubfal, Cristiana Benedetti-Fasil, Charles
Brummitt, Gaia Rubera, Dirk Hovy, and Tommaso Fornaciari, Peer Networks and En-
trepreneurship: a Pan-African RCT, Working Papers 648, IGIER (Innocenzo Gasparini
Institute for Economic Research), Bocconi University, 2019.
[68] Nikos Voskarides, Edgar Meij, Manos Tsagkias, Maarten de Rijke, and Wouter
Weerkamp, Learning to explain entity relationships in knowledge graphs, Proceedings
of ACL, July 2015, pp. 564–574.
[69] Stephanie Wagener, Marjan Gorgievski, and Serge Rijsdijk, Businessman or host? in-
dividual differences between entrepreneurs and small business owners in the hospitality
industry, The Service Industries Journal 30 (2010), no. 9, 1513–1527.
[70] Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and
Dekang Lin, Knowledge base completion via search-based question answering, Proceed-
ings of the 23rd International Conference on World Wide Web (New York, NY, USA),
WWW ’14, ACM, 2014, pp. 515–526.
[71] Myron Wish, Morton Deutsch, and Susan J Kaplan, Perceived dimensions of interper-
sonal relations., Journal of Personality and Social Psychology 33 (1976), no. 4, 409.
[72] Fei Wu and Daniel S. Weld, Open information extraction using wikipedia, Proceedings of
the 48th Annual Meeting of the Association for Computational Linguistics (Stroudsburg,
PA, USA), ACL ’10, Association for Computational Linguistics, 2010, pp. 118–127.
[73] Liu Yang, Steve Hanneke, and Jaime Carbonell, A theory of transfer learning with
applications to active learning, Machine Learning 90 (2013).
[74] Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy, Hier-
archical attention networks for document classification, Proceedings of the 2016 Confer-
86
ence of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies (San Diego, California), Association for Computational
Linguistics, June 2016, pp. 1480–1489.
[75] Xuchen Yao and Benjamin Van Durme, Information extraction over structured data:
Question answering with freebase, Proceedings of the 52nd Annual Meeting of the Asso-
ciation for Computational Linguistics (Volume 1: Long Papers) (Baltimore, Maryland),
Association for Computational Linguistics, June 2014, pp. 956–966.
[76] Xiaofeng Yu and Wai Lam, Jointly identifying entities and extracting relations in en-
cyclopedia text via a graphical model approach, Proceedings of the 23rd International
Conference on Computational Linguistics: Posters (Stroudsburg, PA, USA), COLING
’10, Association for Computational Linguistics, 2010, pp. 1399–1407.
[77] Hsun yu Chan and Xueli Wang, Interact for what? the relationship between inter-
personal interaction based on motivation and educational outcomes among students in
manufacturing programs at two-year technical colleges, Community College Review 44
(2016), no. 1, 26–48.
87