extracting dimensions of interpersonal interactions and

97
EXTRACTING DIMENSIONS OF INTERPERSONAL INTERACTIONS AND RELATIONSHIPS Farzana Rashid Dissertation Prepared for the Degree of DOCTOR OF PHILOSOPHY UNIVERSITY OF NORTH TEXAS August 2020 APPROVED: Eduardo Blanco, Major Professor Rodney D. Nielsen, Committee Member Xiaohui Yuan, Committee Member Dirk Hovy, Committee Member Barrett Bryant, Chair of the Department of Computer Science and Engineering Hanchen Huang, Dean of the College of Engineering Victor Prybutok, Dean of the Toulouse Graduate School

Upload: others

Post on 11-Jun-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Extracting Dimensions of Interpersonal Interactions and

EXTRACTING DIMENSIONS OF INTERPERSONAL

INTERACTIONS AND RELATIONSHIPS

Farzana Rashid

Dissertation Prepared for the Degree of

DOCTOR OF PHILOSOPHY

UNIVERSITY OF NORTH TEXAS

August 2020

APPROVED: Eduardo Blanco, Major Professor Rodney D. Nielsen, Committee Member Xiaohui Yuan, Committee Member Dirk Hovy, Committee Member Barrett Bryant, Chair of the Department

of Computer Science and Engineering

Hanchen Huang, Dean of the College of Engineering

Victor Prybutok, Dean of the Toulouse Graduate School

Page 2: Extracting Dimensions of Interpersonal Interactions and

Rashid, Farzana. Extracting Dimensions of Interpersonal Interactions and

Relationships. Doctor of Philosophy (Computer Science and Engineering), August 2020,

87 pages, 26 tables, 2 appendices, 77 numbered references.

People interact with each other through natural language to express feelings,

thoughts, intentions, instructions etc. These interactions as a result form relationships.

Besides names of relationships like siblings, spouse, friends etc., a number of dimensions

(e.g. cooperative vs. competitive, temporary vs. enduring, equal vs. hierarchical etc.) can

also be used to capture the underlying properties of interpersonal interactions and

relationships. More fine-grained descriptors (e.g. angry, rude, nice, supportive etc.) can

also be used to indicate the reasons or social-acts behind the dimension cooperative vs.

competitive. The way people interact with others may also tell us about their personal

traits, which in turn may be indicative of their probable success in their future. The works

presented in the dissertation involve creating corpora with fine-grained descriptors of

interactions and relationships. We also described experiments and their results that

indicated that the processes of identifying the dimensions can be automated.

Page 3: Extracting Dimensions of Interpersonal Interactions and

Copyright 2020

by

Farzana Rashid

ii

Page 4: Extracting Dimensions of Interpersonal Interactions and

ACKNOWLEDGEMENTS

I would like to express my sincere gratitude to my wonderful advisor Dr. Eduardo

Blanco for the continuous support of my Ph.D study and related research. I am immensely

thankful for his patience, motivation, and immense knowledge. His guidance helped me in all

the time of research and writing of this dissertation. He is, without doubt, the best advisor

and mentor I could have ever had for my Ph.D study.

Besides my advisor, I would like to thank the rest of my committee: Dr. Rodney

Nielsen, Dr. Xiaohui Yuan, and Dr. Dirk Hovy, for their insightful comments, questions,

suggestions and encouragement. In particular, I appreciate Dr. Dirk Hovy for his guidance

on a major portion of my dissertation.

I thank my fellow labmates for the stimulating discussions, for motivating me during

my difficult times, and for all the fun we have had in the last few years. Also I thank all my

friends for their continuous support.

Last but not the least, I would like to thank my family: my dear husband, my father,

my son, my parents-in-law, my brothers and siblings-in-law for supporting and encouraging

me all through.

iii

Page 5: Extracting Dimensions of Interpersonal Interactions and

TABLE OF CONTENTS

Page

ACKNOWLEDGEMENTS iii

LIST OF TABLES vi

CHAPTER 1 INTRODUCTION 1

CHAPTER 2 RESEARCH GOALS AND CONTRIBUTIONS 4

CHAPTER 3 RELATED WORK 6

3.1. Related Works in Social Sciences 6

3.2. Related Works in Computational Linguistics 8

3.3. Motivation from Related Works 12

CHAPTER 4 DIMENSIONS OF INTERPERSONAL INTERACTIONS, RELATIONSHIPS,

AND SOCIAL ACTS 14

4.1. Dimensions of Interpersonal Interactions and Relationships 14

4.1.1. Dimensions of Interactions 15

4.1.2. Dimensions of Relationships 16

4.2. Social Acts 16

4.2.1. Cooperative Social Acts 17

4.2.2. Competitive Social Acts 18

4.3. The Data Sets and the Annotations 19

4.4. How the Pairs of People were Chosen 20

4.4.1. For Ontonotes Corpus 20

4.4.2. For Friends Corpus 21

4.4.3. For Friends Extension 22

4.5. Annotations 23

4.5.1. Annotation Process for Ontonotes Corpus 23

4.5.2. Annotation Quality in Ontonotes Corpus 25

iv

Page 6: Extracting Dimensions of Interpersonal Interactions and

4.5.3. Annotation Process for Friends Corpus 27

4.5.4. Annotation Quality in Friends Corpus 29

4.5.5. Annotation Process for Friends Extension 32

4.5.6. Annotation Quality in Friends Extension 33

4.6. Experiments and Results 44

4.6.1. Experiments with Ontonotes Corpus 45

4.6.2. Experiments with Friends Corpus 48

4.6.3. Experiments with Friends Extension 51

4.6.4. Error Analysis of Friends Extension 56

CHAPTER 5 DIMENSIONS OF INTERPERSONAL INTERACTIONS AND

RELATIONSHIPS IN CHAT FORUMS 58

5.1. Original Adansonia Dataset 59

5.2. Annotating Dimensions of Interpersonal Interactions and Relationships 59

5.3. Experiments and Results 62

5.3.1. SVM Setup 63

5.3.2. Multitask Learning (MTL) Setup 63

5.3.3. Results 64

5.3.4. Automatic Annotation 67

CHAPTER 6 CONCLUSION 68

6.1. Our Achievements 68

6.2. Applications and Future Directions 69

APPENDIX A RESULT CALCULATION 72

APPENDIX B PUBLICATIONS 77

REFERENCES 79

v

Page 7: Extracting Dimensions of Interpersonal Interactions and

LIST OF TABLES

Page

Table 4.1. Basic corpus counts. We show the number of conversation turns,

interactions (i.e., one person referring to another one), unique

relationships (i.e., unique pairs of people who interact with each other),

and the pairs of people with most interactions in the Friends dataset. 22

Table 4.2. Annotation examples for pairs of people (x, y) from the Ontonotes data.

We refer to dimensions by their first descriptor (Section 4.1); 1 (-1)

indicates that the first (second) descriptor is true, and 0 that the value is

unknown. 24

Table 4.3. Inter-annotator agreement per dimension of interpersonal relationships in

the corpus derived from Ontonotes. κ values in the 0.60–0.80 range are

considered substantial, over 0.80 would be perfect [4]. 26

Table 4.4. Pearson correlations between dimensions of interpersonal relationships in

our corpus derived from the Ontonotes data. 27

Table 4.5. Annotation examples from the Friends dataset. We show examples of

contrasting values for selected dimensions. The first party is always the

speaker, and the second party is underlined. I stands for interactions, and

R for relationship. 30

Table 4.6. Inter-annotator agreement (raw agreement and Cohen’s κ) in the Friends

dataset. κ values between 0.6 and 0.8 indicate substantial agreement, κ

values over 0.8 indicate perfect agreement [4]. 32

Table 4.7. Pearson correlations between dimensions of interpersonal interactions and

relationships in the Friends dataset. 33

Table 4.8. Stastistics of each of the 24 episodes of the first season of the Friends

series showing the number of utterances and the most interesting pairs

with their number of interactions in parentheses. 34

Table 4.9. Number of interactions between the main six characters. For each pair, we

vi

Page 8: Extracting Dimensions of Interpersonal Interactions and

show the total number of interactions (left), and the number of cooperative

and competitive interactions (between parentheses). We also show the

total number of cooperative and competitive interactions per character

with the other five main characters in the Friends extension dataset. 35

Table 4.10. Annotation examples from the Friends extension dataset. We show

examples of interactions categorized based on the social acts of the

dimension cooperative vs. competitive. The first party is the speaker, and

the second party is the person being spoken to. 35

Table 4.11. Inter-annotator agreements for dimensions and social Acts (raw agreement

and Cohen’s κ) in the Friends extension dataset. κ values between 0.6

and 0.8 indicate substantial agreement, κ values over 0.8 indicate nearly

perfect agreement [4]. 39

Table 4.12. Pearson correlations between pairs of dimensions of interactions and

dimensions in the Friends extension dataset. 42

Table 4.13. Distributions of cooperative and competitive social acts among the 4,257

interactions in the Friends extension dataset. 43

Table 4.14. Feature set used to determine dimensions of interpersonal relationships

between pairs of people (x, y). Verb features are extracted from the verb

of which either x or y is the subject, Person features are extracted from

x and y independently, and Persons features are extracted from x and y. 46

Table 4.15. Results obtained for all dimensions with several combinations of features

for the Ontonotes dataset. 49

Table 4.16. Results obtained for each dimension with the best combination of features

for all dimensions (Verb + Personx + Persony + Personx Persony,

boldfaced in Table 4.15). The last three columns under All contains

the weighted average of Precision (P), Recall (R) and F-mesure (F) of

the dimensions based on the distributions of the 3 labels. The last row

contains the weighted averages of each column. 50

vii

Page 9: Extracting Dimensions of Interpersonal Interactions and

Table 4.17. Results obtained with the test set with several systems (average of all

dimensions) for the Friends dataset. Previous refers to the previous

conversation in which the same pair of people interacted not the

immediately previous turn). 52

Table 4.18. Results obtained per dimension with the best system (all features, Table

4.17). The results under All the weighted averages for all labels, recall

that the label distribution is biased (Figure 4.3). 52

Table 4.19. Results for predicting dimensions of interactions and relationships with

the Friends extension dataset. The bottom two systems use gold

dimensions as features thus results are unrealistic. 54

Table 4.20. Detailed results for predicting dimensions of interactions and relations

with the best-performing realistic system (BOW + BOW prev. +

sentiment + other in Table 4.19). 55

Table 4.21. Results for predicting social acts (cooperative and competitive). Systems

labeled+social acts prev. use gold social acts and thus results are

unrealistic. 55

Table 4.22. Detailed results for predicting social acts with the best-performing

realistic system (BOW + BOW prev. + sentiment + other in Table

4.21). The average shown is the weighted average based on the number of

instances in each label. 56

Table 5.1. Annotation examples of contrasting values for each dimension. Each chat

interaction is either directed to an individual or others in general. 60

Table 5.2. Inter-annotator agreements for dimensions. 0.6 ≤ κ ≤ 0.8: substantial

agreement, κ > 0.8: nearly perfect agreement [4]. 61

Table 5.3. Pearson correlations between pairs of dimensions of interactions (indicated

by the names’ initial letters). 62

Table 5.4. Results for predicting styles of interactions and three indicators of business

success. Each value is a weighted average over the different labels based

viii

Page 10: Extracting Dimensions of Interpersonal Interactions and

on the number of instances falling in that label. 66

Table A.1. Results obtained for all dimensions with several combinations of features

for the Ontonotes Dataset. 73

Table A.2. Results obtained for each dimension with the best combination of features

for all dimensions (Verb + Personx + Persony + Personx Persony,

boldfaced in Table A.1). 74

Table A.3. Macro averages of the Precision, Recall and F-measure respectively over

the different labels for each dimension shown in Table A.2. 75

Table A.4. Macro averages of the Precision, Recall and F-measure respectively over

the different labels for each dimension shown in Table 4.18. 76

Table A.5. Macro averages of the Precision, Recall and F-measure respectively over

the different labels for each dimension shown in Table 4.20. 76

ix

Page 11: Extracting Dimensions of Interpersonal Interactions and

CHAPTER 1

INTRODUCTION

According to the Oxford Dictionary, a relationship is how two or more concepts, ob-

jects, or people are connected or the state of being connected. When something or someone

is connected to something or someone in any way, they are related to each other (e.g., two

people from the same country are related to each other as fellow countrymen). If two people

are part of the same family, we say that they are relatives. If two objects, like windows

and doors are part of a house, we may say that there is a relationship between the two,

namely, they are components of the house. There are names for certain relationships, for

example, we have relationship names like born same country (for fellow countrymen),

kinship (for relatives), and same house (for members of the same house). There are re-

lationship names like married to or spouse, and child of or offsprings. An instance

of relationship married to is married to(Bill Clinton, Hillary Clinton). This instance

could be extracted from the following text sample: Bill Clinton married Hillary Clinton in

1975. As society is composed of humans, relationships between people, i.e., interpersonal

relationships, are of special interest. Humans are social beings that interact which each other

often. People interact in many different ways. They interact verbally, physically, through

body gestures, in written communication, etc. A series of these interactions define the rela-

tionship. The nature of the interactions between two people and their behavior during these

interactions determine how their relationship progresses. Certain interactions exerted on

each other can determine whether the relationship would continue or break. People interact

through natural language to express feelings, thoughts, intentions, instructions, etc. Their

words often express politeness or rudeness, i.e., the words can indicate the elemental prop-

erties or characteristics of the interactions. Interactions can have different characteristics,

for example, if two people are talking to each other sitting side by side, we can describe

their interaction to be a spatially near one. On the other hand, if a person talks to an-

other over a phone call, it is almost certainly the case that the other person is not at the

1

Page 12: Extracting Dimensions of Interpersonal Interactions and

same premise, so here the interaction is spatially distant. Here we see that the communica-

tion device is helping us to understand whether a relationship is spatially distant or near.

Similarly, the content of an interaction, regardless of the communication device, gives hints

about the nature of the interaction. If one person is praising the other, the interaction is

likely a friendly one. On the other hand, if one person raises their voice and scolds the

other, their interaction is probably an unfriendly one. Several interactions, each of differ-

ent characteristics, thus add up and build relationships between two people. How people

build and maintain their relationships affects their lifestyles and well-being in the society

they live in [42, 25]. These relationships are the main connections in societal networks.

Behavioral and relationship scientists, thus, study people’s interactions and the resulting

relationships [57] to understand society and how it functions. When a customer at a grocery

shop shops there for the first time and starts talking to the salesperson there, the first few

interactions may be superficial ones. But if the customer shops at that shop regularly, then

the interactions build up a relationship with the salesperson that may turn intense as they

probably interact often. We say that if two people meet or come across each other frequently

they are intensely involved. So, here the frequency of interactions is helping us determine

the intensity of a relationship. So we can say that the behavior, media, or frequency can

be a few of many ways that can help us characterize certain interactions. Information ex-

traction is the automatic extraction of information from machine-readable sources that are

unstructured or maybe just semi-structured [47]. Relationship extraction or identification of

relations between entities is part of the Information extraction field. This involves extract-

ing relations like person works for organization, or works for(Bill, IBM) from the

sentence “Bill works for IBM.”) This simple but useful information is usually hidden within

the text, which might be easy for humans to understand but difficult for machines to inter-

pret. Information extraction techniques are improving, and extracting this otherwise hidden

information is getting better. Several applications such as question answering, information

retrieval, also text summarization would benefit from this information. Extracting interper-

sonal relationships would lead to understanding human nature and their behavior within

2

Page 13: Extracting Dimensions of Interpersonal Interactions and

society. As a lot of data is now available online, processing these big data and automatically

getting useful information about interpersonal relationships would help social scientists to

progress their research on societal networks and bonding. Only assigning a relation type,

however, does not account for nuances in the relationship between two individuals. First,

a relationship can be characterized beyond a relation type. For example, people who are

coworkers may be spatially near or distant (working at the same or different offices), and

have an equal or hierarchical relationship (two software developers or a developer and the

CEO). Second, relationships are defined by multiple interactions, and the fine-grained char-

acteristics of interactions do not necessarily mirror the characteristics of the corresponding

relationship. For example, software developers having a cooperative professional relationship

may have a heated interaction in a meeting that does not affect the long-term professional

relationship. Similarly, the same software developers having a task-oriented relationship

may have occasionally pleasure oriented interactions (e.g., when they go out for drinks on

Fridays). This added information on top of the relations type explains the nature of the

relationship and will help more in understanding individuals interact with each other. How-

ever, the in-depth information is harder to extract than just the type of the relationship, as

it is not explicitly said with plain language. Extracting these fine-grained characteristics of

interactions and relationships is what we targeted to do. We are interested in knowing these

characteristics or dimensions, as we call them. The details will be explained in the upcoming

chapters.

3

Page 14: Extracting Dimensions of Interpersonal Interactions and

CHAPTER 2

RESEARCH GOALS AND CONTRIBUTIONS

In the field of information extraction, one of the many subfields is relationship ex-

traction (RE). RE is a classic NLP problem. Given a sentence, RE aims to detect if there

exists a certain semantic relationship between two entities of interest [54]. Even though

our research goal falls under this subfield, we do not aim to extract named interpersonal

relations. Instead, we characterize the interactions and resulting relationship with their ele-

mental properties, also known as dimensions inspired by previous works. We are exploring

different domains of datasets, including Ontonotes, TV series transcript datasets, and online

chat conversations between 5000+ aspiring entrepreneurs to perform our tasks. Our major

milestones included choosing datasets, annotating the dimensions of interactions and rela-

tionships on pairs of people extracted from the dataset, and experimenting with models to

be able to classify these interactions and relationships. Our contributions are as follows:

(1) We chose a set of dimensions of interpersonal interactions and relationships with

inspiration from previous work. By analyzing several annotated instances, we real-

ized that a few of the dimensions applied to interactions and the others applied to

relationships. So, we and grouped the dimensions accordingly. We also defined a

number of the underlying social acts (or sub-types) of the dimension cooperative vs.

competitive.

(2) We targeted interactions and relationships between people as described as follows:

(a) We considered sentences that contained a pair of people connected by the action

of a verb. We found that samples of interpersonal interactions collected from

a wide variety of domains can be identified with different dimensions. We thus

conclude that relationships can be described in terms of dimensions universally.

(b) We considered dialogues or instances of speech where a person talks about or

mentions another person. We found that words mentioned by a person to refer

to another person include detailed information about the relationship between

4

Page 15: Extracting Dimensions of Interpersonal Interactions and

the two.

(c) We considered dialogues or instances of speech where a person talks to another

person. We found that direct speech between two people also holds in-depth

information about the relationship between them.

(d) We considered chat interactions where a real young aspiring entrepreneur chats

with others through online platform discussing and sharing ideas about prob-

able business and funding opportunities. We found that dimensions can char-

acterize chat interactions. We also found that the dimension information can

help predict the ability of a person’s success in business.

(3) We formed large corpora by annotating the dimensions (of interactions and rela-

tionships) and social acts on data from different domains. High agreements in the

annotations indicated that it is possible to annotate the datasets reliably. Distri-

butions of labels differed depending on the type or domain of the datasets. This

difference indicated that certain values of the dimensions are predominant in cer-

tain domains. Correlation analysis for the dimensions and the different social acts

indicated that values certain dimensions are correlated to values of certain other

dimensions.

(4) We conducted Machine Learning experiments. The decent results of classification

showed that it is possible to learn the dimensions and the social acts with learning

methods simply based on language usage. Machine Learning models developed for

the purpose can be used as baseline models for future studies aimed to develop more

sophisticated models.

5

Page 16: Extracting Dimensions of Interpersonal Interactions and

CHAPTER 3

RELATED WORK

As people are social beings, they constantly interact with each other and show certain

behaviors aimed towards each other. As many of their behaviors come only into play when

they interact with each other, behavioral analysis involves the study of interpersonal interac-

tions. The study of interpersonal interactions helps scientists understand how people behave

in certain situations, why they behave in certain ways, and what leads them to that [9]. The

trajectories of relationships that form due to the changes in the properties of interactions

are studied as well.

Just the way social scientists are interested in understanding relationships, compu-

tational linguists are interested in developing tools to extract explicit or implicit cues and

mentions of interpersonal relationships. They try to extract these from written text as well as

spoken words. Here, we discuss a few of the prior works in social cciences and computational

linguistics that are related to our work.

3.1. Related Works in Social Sciences

There is an entire discipline called relationship science, the main focus of which is

to study interpersonal relationships [57]. This discipline is composed of ideas from many

sub-disciplines within social, behavioral, psychology, and biological science. Relationship

scientists study human behavior and their effects on their daily lives and natural surround-

ings. They also study the impact of the exterior environments of the relationships on their

interior dynamics. They also study the effects of interpersonal relationships on cognition.

A relationship can be described in more detail with more elemental properties that

define its nature from different perspectives. These elemental properties of interpersonal re-

lationships that characterize the state of a relationship are called dimensions in social science

and have been studied for decades [71]. In this study, the goal is to understand how people

perceive relationships based on some fundamental characteristics, and they came up with a

number of dimensions. They conducted a questionnaire study to discover the fundamental

6

Page 17: Extracting Dimensions of Interpersonal Interactions and

dimensions underlying people’s perceptions of interpersonal relationships. A number of hu-

man subjects rated 20 of their own interpersonal relationships and 25 typical relationships

on 25 bipolar scales. Their careful analysis of the data revealed 4 dimensions, which were

interpreted as cooperative and friendly vs. competitive and hostile, equal vs. unequal, intense

vs. superficial, and socio-emotional and informal vs. task-oriented and formal. These four

dimensions were studied against those obtained from studies of personality, person percep-

tion, and individual behavior in interpersonal situations. They found that these fundamental

dimensions were extremely stable. Our set of dimensions is inspired by the ideas of these

dimensions mentioned in this paper and a number of other studies discussed below.

Kelley (2013) [42] studies close personal relationships and their essential properties.

She aims to understand the close relationships to be able to systematically assess and classify

so that their functioning can be improved and developed. She discusses some fundamental

dimensions of dyadic relationships. She focuses on close dyadic relationships to understand

the interdependence and the dynamics of such relationships. In the process, she observes

how interpersonal behavior affects relationships in the long run. We also try to understand

interpersonal interactions and relationships to carry out annotation processes to build our

dataset. We do this with the aim of building a system that can automatically extract certain

dimensions of these interpersonal interactions. However, a few of such dimensions are also

discussed in the work by Kelley.

Deutsch [25] examines the connection between types of psychological interdependence

and psychological orientations. He establishes that distinctive psychological orientations

(composed of cognitive, motivational, and moral orientations) are associated with the dis-

tinctive types of interdependence in diverse social relationships in complex societies. He also

says that there is a bidirectional connection between psychological orientations and types

of interdependence, that is, a psychological orientation can induce or be induced by a given

type of interdependence. He also points out that cultural backgrounds of peoples, their per-

sonal histories, and their genetic endowments affect the way they use their orientations. His

hypothesis states that to cope with the psychological requirements of different types of social

7

Page 18: Extracting Dimensions of Interpersonal Interactions and

relationships, people can customize the usage of their psychological orientations as required

in different situations. He does not attempt any computational model but just gives us a

view from social sciences.

Adamopoulos [2] says that there are underlying meanings behind every kind of inter-

personal behavior. In other words, there are reasons for certain kinds of behavior, and again

these behaviors lead to certain changes. He presents a theory of action construal with an

emphasis on the emergence of their social meaning. He reviews the theoretical framework

that generated models concerning the meaning of interpersonal behavior, which involves the

exchange of material and psychological resources. He also points out that these exchanges

are contingent upon many constraints that affect interpersonal interaction. He studied the

context of social resource theory and thus added to the area of interpersonal structure in gen-

eral. We also gather ideas of looking at relationships with elemental properties or dimensions

from this work.

3.2. Related Works in Computational Linguistics

Information extraction (IE) involves extracting structured information from machine-

readable documents. This is mostly done by processing human language texts by means of

natural language processing (NLP) or computational linguistics. Examples also include

content extraction out of images, audio, or video besides language expressed through text.

As information extraction is still pretty difficult, extraction is only doable on very narrow or

selected domains. Different kinds of information can be extracted using automated methods.

One kind of information that is often extracted is relationships between entities such

as people, organizations, and locations, hence the formation of the subfield relation extrac-

tion. A few competitions have served as evaluation benchmarks [30, 26, 44, 64], and include

interpersonal relationships such as business, spouse, and children. Aguilar et al. (2014)

[3] compare several evaluations, and automated approaches to relationship extraction– also

referred to as link prediction and knowledge base completion [76, 50, 70]. Open information

extraction [72] has emerged as an unsupervised domain-independent approach to extract re-

lationships. There is also a semi-supervised relation extraction system with large-scale word

8

Page 19: Extracting Dimensions of Interpersonal Interactions and

clustering [63] and other systems like [24] which create multigraph representations of the

relationships extracted (and their quality). Regardless of details, all these previous efforts

extract explicit relationship types or names and do not attempt to characterize instances of

relationships with dimensions.

Besides extracting relationships per se, there have been efforts within computational

linguistics involving interpersonal relationships. Voskarides et al. [68] extract human-

readable descriptions of relationships in a knowledge graph by ranking sentences that justify

the relationships. Their method explains the relationship between two Wikipedia entities,

and evaluate their method on a dataset annotated, prepared, and made public by them. In

the aim to infer a human-readable description for the relationship between two entities, they

devise a ranking task as they automatically extract sentences from a corpus and rank them

based on how well they describe the relationships.

Iyyer et al. [40] propose an unsupervised algorithm to extract relationship trajec-

tories of fictional characters, i.e., how interpersonal relationships evolve in fictional stories.

Their system learns descriptors of events like marriage, murder, etc. and also those for in-

terpersonal (relationship) states like love, sadness, etc. They leverage neural networks to

achieve their learning goal. Moon and Qi [48] have taken a hybrid approach of combining

the features of unsupervised and supervised learning methods to extract relationships. Their

system identifies the main characters in novels and collects the sentences related to them for

extracting relationships. This kind of work can be applied in story summarization and anal-

ysis of the major characters in stories. Kokkinakis [43] does manual annotation of a small

sample of Swedish 19th and 20th-century prose with interpersonal relationships between

characters in six literary works. They inspect models for annotation on a larger scale, both

manually as well as automatically. Bajracharya et al. [6] carry out a rule-based approach

that identifies characters from stories and determines the family relationship among them.

There is work for learning character types, or personas from films [7], inferring the polarity

of relationships between people in narrative summaries [62] and extracting social network

from literally fictions [27]. We have used TV series transcript data to learn dimensions of

9

Page 20: Extracting Dimensions of Interpersonal Interactions and

interpersonal relationships.

Bracewell et al. [12] introduced 9 social acts or moves (e.g., agreement, undermin-

ing) designed to characterize relationships between individuals exhibiting adversarial and

collegial behavior (similar to our cooperative vs. competitive dimension). The 9 acts are:

agreement (statements that a group member makes to indicate that he/she shares the same

view about something another member has said or done); challenge credibility (attempts

to discredit or raise doubt about another group members qualifications or abilities); dis-

agreement (statements a group member makes to indicate that he/she does not share the

same view about something another member has said or done); disrespect (inappropriate

statements that a group member makes to insult another member of the group); offer grati-

tude (a sincere expression of thanks that one group member makes to another); relationship

conflict (personal, heated disagreement between individuals); solidarity (statements that a

group member makes to strengthen the groups sense of community and unity); supportive

behavior (statements of personal support that one group member makes toward another);

and undermining (hostile expressions that a group member makes to damage the reputation

of another group member). These social acts were selected from literature in the fields of

psychology and organizational behavior. They presented the annotations of these nine social

acts for discourses communicated in Chinese and English. This work inspires us to further

characterize the dimension of cooperative vs. competitive into subcategories, similar to the

nine social acts.

Studies have been done to differentiate types of interaction based on students under-

lying motivation. They also investigated how different forms of interaction are related to

educational outcomes [77]. They grouped social interactions into three categories of under-

lying motivations for educational outcomes. One is interaction as a response to curricular

demands, interaction for broader educational purposes, and interaction for diverse experi-

ences. In their work, they did not categorize interactions with social acts as we did. They

also did not differentiate between cooperativeness or competitiveness. In our study, we con-

sider that social acts are the reasons behind a behavior or interaction for being cooperative

10

Page 21: Extracting Dimensions of Interpersonal Interactions and

or competitive.

Also, there have been studies to see what kind of conversational interactions and what

factors activate frustration in a person [60]. Their model emphasizes the motives that drive

interpersonal behaviors and the negative effect (such as anger). One hundred twenty-nine

students participated in chat conversations. They studied how student-participants behave

when they (a) thought the interaction was interpersonal (with a human) or noninterper-

sonal (with a computer); (b) were assertive and nonassertive and (c) the replies given to the

participants contained dominating or non-dominating language. Participants showed more

interpersonal behaviors when they thought the other party was human. Assertive partici-

pants who interacted with a dominating and apparently human partner were more hostile

and had angry conversations. Even though this work did not work explicitly with social acts,

it can be compared with such as it studies the motives and outcomes of human behavior or

interaction based on certain environmental factors.

Politeness in online forums has been studied [23]. They annotate a new corpus

of requests for politeness. They use this corpus to evaluate aspects of politeness and to

identify politeness markers and contexts. Also, Danescu-Niculescu-Mizil et al. [22] study

how power differences affects language style in online communities. They showed that in

group discussions, power differentials between participants are subtly revealed by how much

one individual immediately echoes the linguistic style of the person they are responding to.

They create an analysis framework with which they study the conversational behavior to

reveal power difference. Prabhakaran and Rambow [53] present a classifier to detect power

relationships in email threads. Similarly, Gibert [28] explores how people in hierarchical

relationships communicate through email, and Bramsel et al. [13] focus on identifying power

relationships in social networks based purely on interpersonal communication of the members

of the network. Some have studied how the roles of Wikipedia editors affect their success

[45]. We similarly categorize relationships between people using hierarchical vs. equal as

one of the dimensions, among others. Business relevance and sentiment are annotated on

online chat interactions among aspiring entrepreneurs [67]. Unlike them, we use different

11

Page 22: Extracting Dimensions of Interpersonal Interactions and

domains of dataset like TV series transcripts, news articles, TV interview transcripts, and

chat interactions.

3.3. Motivation from Related Works

The study of interpersonal communication or relation is a multidisciplinary activity,

and it includes the study of psychology, sociology, anthropology, medicine, social psychology,

psychiatry, clinical psychology, and facets of the language studies [55]. Sociologists, psychol-

ogists, etc. have studied human relationships to understand what people want or wish for,

what they aim to achieve, and how they act in different situations [19]. We can expect that

people in close relationships will reflect similar preferences in movies, books, music, etc. [19].

Thus behavioral science studies can be incorporated with relationship extraction to figuring

out how a person is going to behave in terms of his/her buying habits, travel habits, etc.

If it is possible to figure out whom a person is intimate with, it is likely that this

person would behave similarly as their close friends. This infomation can also be used to

build friends networks [1, 35, 27] by social media platforms. The information, in turn, can

be utilized by advertising to figure out what items the person may be interested in. Just the

way there is a need to consider subtypes of the generic association [10], there is a need to

consider the subtypes or elemental properties of interpersonal relationships that are called

the dimensions of interpersonal relationships. Characterizing interpersonal relationship is

a hard but important problem [34]. These dimensions have been studied for decades in

social sciences [71, 42, 25, 2, 58, 61]. Extracting just the named dyadic relationship is

already useful for applications like question answering [75], but these names only provide a

general understanding of the relationship between two people. But if the question-answering

models use the fine-grained dimensions of relationships, they can provide more information

about certain relationships. This information can be used to predict relationship trajectories

between people [40].

These days, robots are used to assist people in domestic works. They also assist

in engaging elders in conversations to reduce their loneliness. They also serve people with

autism and help in the physical rehabilitation of a variety of patients. For these activities,

12

Page 23: Extracting Dimensions of Interpersonal Interactions and

robots need to learn new tasks, skills, and individual preferences that they can learn from

the natural languages used by the users in their natural interpersonal interactions [14]. So

the robots need to be more socially aware. With in-depth knowledge about interpersonal

relationships, more socially aware robots or chatbots can be built, as shown by Hameed [32].

In light of the above needs, we describe interpersonal relationships in terms of their

fine-grained dimensions and focused on how they can be extracted from natural language

text. We propose that rather than expressing relationships just with a name, we should

express them in terms of more descriptive properties called the dimensions. We showed that

these dimensions can be annotated by humans and can also be reliably predicted by models

trained on the language solely.

13

Page 24: Extracting Dimensions of Interpersonal Interactions and

CHAPTER 4

DIMENSIONS OF INTERPERSONAL INTERACTIONS, RELATIONSHIPS, AND

SOCIAL ACTS

In this chapter, we start by discussing the details of the dimensions of interpersonal

interactions and relationships in section 4.1. Then we move on to discussing the social acts

in section 4.2. We then discuss the datasets used in three different projects we perform in

section 4.3. We also elaborate on and the annotations of these dimensions and social acts.

We also present analyses of these annotations. Then in Section 4.6, we present the details

on the experiments carried out in these three projects.

4.1. Dimensions of Interpersonal Interactions and Relationships

The elemental characteristics or properties of interactions or relationships are known

as their dimensions. The perfect set of dimensions of interpersonal interactions and relation-

ships is by no means agreed upon, and neither is the terminology to refer to what apparently

is the same dimension. For example, the terms dominance, submission, potency, autonomy,

and control are used to describe the distribution of power in a relationship [25]. The dimen-

sions we work with are primarily borrowed from previous works in social science, although

we add two new dimensions. Interestingly, the previous works which define these dimensions

do so from a theoretical point of view or after conducting experiments with subjects to reveal

how they perceive interpersonal relationships. The latter was done using multidimensional

scaling analysis after subjects compared 25 relationships, e.g., between a parent and child,

between business partners [71].

Social scientists have proposed additional dimensions, e.g., voluntary vs. involuntary,

public vs. private and licit vs. illicit [25], or self-benefiting vs. service-oriented [2]. We

discarded these additional dimensions because we discovered that they are not applicable to

most pairs of people we work with. We provide brief descriptions of the nine dimensions of

interpersonal interactions and relationships we work with here with examples in Sections 4.1.1

and 4.1.2. Note that these dimensions are not completely independent, for example, enduring

14

Page 25: Extracting Dimensions of Interpersonal Interactions and

relationships are usually intense and intimate, and intense and pleasure oriented relationship

are almost always intimate. We also mention which dimensions apply to interactions and

which ones apply to relationships.

Here are the definitions of the nine dimensions that we worked with along with indi-

cations on whether they define interactions or relationships. First stated are the dimensions

of interactions and then are the dimensions of relationships. There also are references from

which these dimensions are inspired from.

4.1.1. Dimensions of Interactions

Here we describe the dimensions of Interactions.

Cooperative vs. Competitive: An interaction is cooperative if both the people involved

(a) have a common interest or goal, (b) like each other, (c) benefit from the relationship, or

(d) think alike or have similar views. Otherwise, it is competitive. We use this as a dimension

of Interactions. Inspiration of this dimension is taken from Wish et al. [71].

Active vs. Passive: An interaction is active if both people communicate directly.

Otherwise, the interaction is passive. For example, when John talks to Bill about Mary,

John and Bill have an active interaction, and John and Mary have a passive interaction. We

use this as a dimension of interactions. The inspiration for this dimension came from Kelley

[42].

Concurrent vs. Non-concurrent : An interaction is concurrent if both people are

involved in an event or action at the same time. Otherwise, the interaction is non-concurrent.

We use this as a dimension of interactions. We thought of this dimension to be appropriate

for our tasks.

Spatially Near vs. Distant : An interaction is spatially near (or near for short) if both

people are at the same location during the event that grants the interaction. Otherwise, the

interaction is spatially distant (or distant). We use this as a dimension of interactions. We

thought of this dimension to be appropriate for our tasks.

15

Page 26: Extracting Dimensions of Interpersonal Interactions and

4.1.2. Dimensions of Relationships

Here we describe the dimensions of Relationships.

Equal vs. Hierarchical : A relationship is equal if both people (a) have the same social

status, (b) are at the same level in the power structure, (c) share similar responsibilities,

or (d) have the same role. Otherwise, the relationship is hierarchical. We use this as a

dimension of Relationships. The inspiration for this dimension also came from Wish et al.

[71].

Intense vs. Superficial : A relationship is intense if both people interact with each

other frequently, i.e., they are involved repeatedly. Otherwise, the relationship is superficial.

We use this as a dimension of Relationships. The inspiration for this dimension is also taken

from Wish et al. [71].

Pleasure vs. Task Oriented : A relationship is pleasure-oriented if both people interact

socially, and their relationship is not bound by professional rules or regulations. Otherwise,

the relationship is task-oriented. We use this as a dimension of Relationships. The inspiration

for this dimension is also taken from Wish et al. [71].

Intimate vs. Unintimate: A relationship is intimate if both people are emotionally

close and warm to each other. Otherwise, the relationship is unintimate. We use this as

a dimension of Relationships. The inspiration for this dimension came from Adamopolous

(2012) [2].

Temporary vs. Enduring : A relationship is temporary if it lasts less than a day. A

relationship is enduring if it lasts over a month. Otherwise (if it lasts more than a day and

less than a month), this dimension is undefined. We use this as a dimension of Relationships.

Inspiration of this dimension came from Deutsch (2011) [25].

4.2. Social Acts

Just like interactions and relationships can be characterized in terms of dimen-

sions, competitive and cooperative interactions can be further characterized with social acts

[8, 11, 12]. These social acts are like acts denoting the reasons behind the cooperativeness or

competitiveness of an interaction. Social acts influence future interactions and relationship

16

Page 27: Extracting Dimensions of Interpersonal Interactions and

trajectories. For example, a mother may scold her son for not listening to her (competitive

interaction), and latter show support by praising him for doing his homework on time (coop-

erative interaction). Similarly, friends may mock each other (competitive interaction) while

their overall relationship remains in good terms because, for example, they mainly partici-

pate in excited and nice social acts. Note that social acts depend on individual interactions,

not on the long-term relationship between individuals.

Regarding social acts, we follow Bracewell et al. (2012a)[11], which note that coop-

erative and competitive interactions (they use adversarial and collegial) can be understood

by the underlying social act or pragmatic speech acts that signal a dialogue participants so-

cial intentions. We work with the 11 social acts defined below. Six of them are novel, and

five of them are borrowed from the work by Bracewell et al. (2012a)[11], which in turn are

inspired by other previous works [15, 41, 17, 51]. The novel social acts were defined after

pilot annotations.

4.2.1. Cooperative Social Acts

We consider five social acts for cooperative interactions:

• Agree applies when the first party agrees with the second party or shares the same

point of view [11, 12]. Utterances such as Yes, you are correct and Sure, why not?

are two examples showing agreement.

• Support applies when the first party praises, encourages or motivates the second

party. The first party usually mentions the second party’s qualities or appreciates

his efforts. For example, You are looking beautiful, You made us proud and Come

on, you can do it! show support. This social act is similar to supportive behavior

[17, 12].

• Excited applies when the first party shows enthusiasm and passion towards some-

thing the second party said. For example, Wow, I am thrilled to know that! and

That was fun! are utterances showing excitement.

• Nice applies when someone expresses sincere thanks, apologies, polite greeting, etc.

For instance, Thank you so much and Hello, how are you? are utterances showing

17

Page 28: Extracting Dimensions of Interpersonal Interactions and

niceness.

• Normal is the default social act of cooperative interactions. It applies when the

first party states a fact, asks a question, and, in general, when no other social act

applies. For example, This is where I live and I wake up at 7 a.m. show the normal

social act.

4.2.2. Competitive Social Acts

We consider six social acts for competitive interactions:

• Complain applies when the first party accuses, denounces, or laments something to

the second party. The complain does not need to involve the second party. For

example, You did everything wrong and I hate my new job downtown are complain

social acts.

• Disagree applies when the first party indicates that they do not share the same

view about something said or done [11, 12]. The following utterance exemplifies

this social act: No, that’s not true!

• Disbelief applies when one doubts or discredits somebody’s knowledge, qualifica-

tions, abilities or credibility. For example, Are you sure? and I don’t think he

graduated high school indicates disbelief. This social act is similar to challenge cred-

ibility [51].

• Dissatisfied applies when the first party is disappointed or annoyed about an idea

shared or work done. For example, I expected better from you shows dissatisfaction.

• Mock applies when the first party jokes about, makes fun of, or says something

sarcastically about somebody to someone. The person being mocked does not need

to be the second party. Examples include someone saying to a short person Hi, big

guy! or asking whether somebody’s boyfriend eats chalk.

• Rude applies when behaving badly or rudely with someone. The first party may

also be scolding, rebuking, or warning someone. For example, I don’t care or Don’t

you ever go there, and other shows of disrespect fall under this social act. This is

similar to disrespect [11, 12].

18

Page 29: Extracting Dimensions of Interpersonal Interactions and

4.3. The Data Sets and the Annotations

Our task involved annotating a large amount of data with the labels indicating the

dimensions of interpersonal interactions and relationships. Existing corpora annotating re-

lationships consider only selected interpersonal relationships and do not target dimensions

(Chapter 3). Our goal is to target dimensions of interpersonal relationships between any two

individuals, from weak links (e.g., journalists interviewing celebrities) to strong ties (e.g.,

close friends). So far, our annotation effort has taken place in three independent tasks, and

we have carried out detailed analyses of our resulting datasets independently. In those three

tasks, we have created:

(1) A corpus by first retrieving pairs of people from text data of mixed genres, and then

annotating dimensions for their relationships. We decided to add our annotations

to Ontonotes [38]. Doing so has several advantages. First, Ontonotes data contains

texts from several domains and genres (e.g., conversational telephone speech, TV

interview transcripts, weblogs, broadcast); thus, we not only work with newspa-

per articles. Second, Ontonotes includes part-of-speech tags, named entities, and

coreference chains, three annotation layers that allow us to streamline the corpus

creation process. For easy reference, we will call this resulting corpus, Ontonotes

corpus, from here on.

(2) A corpus by augmenting on an existing corpus of scripts from the TV show Friends

[20]. Instead of starting from scratch, we chose to do the annotation because anno-

tating dimensions of interactions and relationships requires a corpus in which the

same people interact several times. We work with the 24 episodes from Season 1

because:

(a) they contain a large number of conversation turns(9,168);

(b) involve many characters (42 characters speak at least 100 conservation turns);

(c) they include speaker information (i.e., we have access to who says what);

(d) and they include annotations linking each mention of people in each conversa-

tion turn to the actual person (the name of the person).

19

Page 30: Extracting Dimensions of Interpersonal Interactions and

Beyond size, the main motivation to use this corpus is the last item above: starting

from scratch with another corpus of dialogues would require substantially greater

annotation effort. We refer the reader to the aforecited paper for details, but the

original corpus clusters mention to people such as guy, my brother, and emphhe

together with other mentions of the same person. The original corpus is publicly

available,1. For easy reference, we refer to the resulting corpus as the Friends corpus

from here on.

(3) Another corpus by annotating the dimensions of interactions and relations, and the

underlying social acts behind cooperative and competitive interactions on pairs of

people formed between a speaker and the person being spoken to. While looking for a

source corpus that contains many interactions between the same people, we thought

of reusing the same original corpus from the TV show Friends, as mentioned above

[20]. We work with twelve episodes of Season 1, which contain 4,257 interactions.

For easy reference, we refer to the resulting corpus as the Friends extension from

here on.

4.4. How the Pairs of People were Chosen

Before we could start our annotations, we required to select portions of the huge

original dataset as our candidates to be annotated. This curation resulted in a reasonable

size of the data that was easy to handle. The process involved choosing the appropriate

sentences based on some simple criteria for each of our corpora. Here we describe how the

appropriate sentences were chosen partly automatically and partly manually:

4.4.1. For Ontonotes Corpus

We retrieve pairs of people within each sentence in OntoNotes following four steps:

(1) Collect all instances of personal pronouns (part-of-speech tag PRP) I, he, and she

within the sentence.

(2) Collect all named entities PERSON within the sentences.

1https://github.com/emorynlp/character-mining

20

Page 31: Extracting Dimensions of Interpersonal Interactions and

Figure 4.1. Frequencies of the top 20 most frequent verbs after retrieving

pairs of people from the Ontonotes dataset (Section 4.4.1). We discard verbs

with frequency <4, and randomly select up to 26 pairs per verb for a total of

1,048 pairs.

(3) Keep one mention per coreference chain, giving priority to named entities over

pronouns.

(4) Generate combinations of 2 elements from the union of the pronouns and named

entities subject to the following constraints: at least

(a) one is a PERSON named entity, and

(b) one is the nsubj (nominal subject syntactic dependency) of a verb.

Figure 4.1 shows the frequencies of the top 20 most frequent verbs after retrieving

pairs of people from the Ontonotes dataset.

4.4.2. For Friends Corpus

For the Friends corpus, we decided to pick pairs of people from the conversations

carried out by the characters in the 24 episodes of the first season of the TV series Friends.

To select pairs of people whose dimensions of interactions and relationships were to be

annotated, we collected all instances of somebody mentioning (or referring to) somebody

else in a conversation turn. We consider relationships between individuals who interact at

least once.

Table 4.1 shows basic counts per episode. We show the number of interactions, unique

relationships (i.e., interactions between unique pairs of people), and the pairs of people who

interact the most.

21

Page 32: Extracting Dimensions of Interpersonal Interactions and

Dimension Agreement κ coefficient

Cooperative 86% 0.74

Equal 81% 0.63

Intense 84% 0.73

Pleasure Oriented 86% 0.70

Active 82% 0.59

Intimate 83% 0.68

Temporary 76% 0.61

Concurrent 84% 0.72

Spatially Near 80% 0.67

All 82% 0.68

Table 4.1. Basic corpus counts. We show the number of conversation turns,

interactions (i.e., one person referring to another one), unique relationships

(i.e., unique pairs of people who interact with each other), and the pairs of

people with most interactions in the Friends dataset.

4.4.3. For Friends Extension

Since the original corpus has speaker information, it is straightforward to determine

the first party of all interactions. The second party, however, is not readily available. We

manually annotated the second party (who is the first party talking to?) as a preliminary

step prior to annotating dimensions and social acts. Note that the second party is not always

the author of the next utterance. We annotated all conversation turns, and a single annotator

indicated the second party after watching the episode. There are three options: the second

party is an individual, more than one individual, or nobody. If it is an individual, the

annotator indicates the name. If it is more than one individual, the annotator chooses many

(11% of utterances). Finally, the annotator indicates self when somebody is actually saying

a monologue or thinking aloud and thus talking to nobody in particular (2% of instances).

22

Page 33: Extracting Dimensions of Interpersonal Interactions and

69.9%

20.9%

26.4%

65.9%

27.4%

66.2%

14.4%

79.3%

58.4%

35.3%

Cooperative Equal Intense Pleasure Oriented Active

15%

79.3%80.1%

14.5%

46.4%

47.1%

40.4%

51.4%

1

-1

Intimate Temporary Concurrent Spatially Near

Figure 4.2. Label distribution per dimension of interpersonal relationships

from the Ontonotes data. The missing portion of each pie chart corresponds

to labels 0 and inv, which always amount to less than 5% each.

We consider as interactions all utterances except the ones in which the second party could

not be identified (all but self ), and these are the interactions that were annotated with

dimensions and social acts (Section 4.5.5). Table 4.9 shows how many interactions there are

for the main six characters.

4.5. Annotations

Here we discuss how we carried out the annotation processes on the three corpora, a

number of selected annotation examples from each of the corpus, annotation qualities and

label distributions.

4.5.1. Annotation Process for Ontonotes Corpus

After generating pairs, annotators determined values for each dimension of interper-

sonal interactions and relationships. As from this dataset, it was hard to acquire a series of

interactions by the same pair of people, a relationship between a pair of people was considered

to be formed only with one interaction (or sentence).

23

Page 34: Extracting Dimensions of Interpersonal Interactions and

Sentence Coop

erati

ve

Equ

al

Inte

nse

Ple

asu

reO

rien

ted

Act

ive

Inti

mate

Tem

por

ary

Con

curr

ent

Sp

atia

lly

Nea

r

1 [Cheney]x [got]verb a telephone call from his democratic coun-

terpart [Joseph Lieberman]y wishing him a speedy recovery.

1 1 -1 -1 1 -1 1 1 -1

2 [. . . ] [I]x [interviewed]verb one of the nation’s top jockies

[Shane Sellers]y about the battle he waged everyday to con-

trol his weight.

1 -1 -1 -1 1 -1 1 1 1

3 [I]x have always remembered the encouragement which Mr.

[Yu Youren]y [gave]verb me as a young reporter. He said that

to be a fearless champion of social justice, as is expected of

a journalist, the [. . . ]

1 -1 1 -1 1 1 -1 1 1

4 One day, Dingxiang took the opportunity to again urge

him to change his ways [. . . ]. After this, [Zhang Sheng]x

[threw]verb out [Dingxiang]y, sold off the family possessions,

and spent his days living a life of dissipation.

-1 1 -1 1 1 -1 -1 1 1

Table 4.2. Annotation examples for pairs of people (x, y) from the Ontonotes

data. We refer to dimensions by their first descriptor (Section 4.1); 1 (-1)

indicates that the first (second) descriptor is true, and 0 that the value is

unknown.

The annotation interface showed the sentence from which the pair of people was

generated, and the previous and next sentence to provide some context. The pair of people

of interest were highlighted, but no additional information was shown (e.g., the verb of

which one person is the subject). Annotators assigned a value to each dimension based on

the relationship between the two individuals at the time the verbal event of which one of

the individuals is the subject takes place. They were trained to take into account context

24

Page 35: Extracting Dimensions of Interpersonal Interactions and

(previous and next sentences), and to interpret the text as they normally would. Therefore,

they assign values using world knowledge that may not be explicitly stated in the text. For

example, two people talking on the phone would have a spatially distant interaction because

(most likely) they are not next to each other while talking.

During the first batch of annotations, we discovered that for a given pair of people,

dimensions sometimes could not be determined because (a) there is not enough evidence

in the text provided (i.e., sentence from which the pair was generated, previous and next

sentences) or (b) the pair is invalid and assigning dimensions is nonsensical. We use label

0 in the former case and inv in the latter. For example, in the sentence, He criticized Ken

Starr, the value for the dimension spatially near (vs. distant) was marked 0 as there is

not enough information to determine whether He and Ken Starr are at the same location

when criticized took place. We refer to dimensions by the first descriptor (as given in the

definitions in Section 4.1) and use 1 if the first descriptor of a dimension is true, and -1 if

the second descriptor is true. For example, label -1 applied to dimension temporary means

that the relationship is enduring. Figure 4.2 shows the label percentages of the different

dimensions in the Ontonotes data.

4.5.2. Annotation Quality in Ontonotes Corpus

Two graduate students did the annotations. They started annotating small batches

of pairs of people, and discussed disagreements with each other. After several iterations,

they annotated independently 10% of all pairs of people generated. Table 4.3 shows the

inter-annotator agreement per dimension of interpersonal relationships in the Ontonotes

Data. Overall, Cohen’s kappa coefficient is 0.68, and the coefficients range from 0.59 to

0.74, depending on the dimension. Note that kappa coefficients in the range 0.600.80 are

considered substantial [4]. Given these high agreements, the rest of the pairs were annotated

once.

Even though the annotations were done very carefully, we acknowledge that some

annotations are ambiguous. We will discuss a few examples from the Ontonotes data found

in Table 4.2. Sentences like, Sentence 1 and Sentence 2 encode COMMUNICATION re-

25

Page 36: Extracting Dimensions of Interpersonal Interactions and

Dimension Agreement κ coefficient

Cooperative 86% 0.74

Equal 81% 0.63

Intense 84% 0.73

Pleasure Oriented 86% 0.70

Active 82% 0.59

Intimate 83% 0.68

Temporary 76% 0.61

Concurrent 84% 0.72

Spatially Near 80% 0.67

All 82% 0.68

Table 4.3. Inter-annotator agreement per dimension of interpersonal rela-

tionships in the corpus derived from Ontonotes. κ values in the 0.60–0.80

range are considered substantial, over 0.80 would be perfect [4].

lationships between two individuals, and both are cooperative, superficial, work-oriented,

active, unintimate, temporary, and concurrent. The values for two dimensions, however, are

different. Two counterparts, in Sentence 1, are at the same level in the power structure

(equal), but the interviewer and interviewee are not in Sentence 2. Similarly, talking on the

phone entails that the individuals are spatially distant in Sentence 1, but interviewing (most

likely) means that they were spatially near in Sentence 2. One could argue that 0 would

be a better label for spatially near in Sentence 2, but annotators interpreted that inter-

viewed refers to an in-person interview. In Sentence 3, the context describes one person (Yu

Youren) encouraging another one (I). Annotators indicate that this relationship, unlike the

ones in Sentences 1 and 2, is intense (frequent interactions), intimate (emotionally close),

and enduring (lasting over a month). These values are not explicitly stated, but they are un-

derstood given the long-lasting impact Yu Youren had on I. Finally, Sentence 4 exemplifies a

competitive relationship. The context describes a struggling relationship between Dingxiang

26

Page 37: Extracting Dimensions of Interpersonal Interactions and

Coop

erat

ive

Equ

al

Inte

nse

Ple

asu

reO

r.

Act

ive

Inti

mat

e

Tem

por

ary

Con

curr

ent

Cooperative –

Equal -.06 –

Intense .24 .11 –

Pleasure Or. .14 .29 .39 –

Active .18 .22 .43 .25 –

Intimate .18 .17 .59 .62 .31 –

Temporary -.17 .10 -.56 -.43 -.27 -.61 –

Concurrent .17 .22 .31 .21 .74 .26 -.17 –

Spat. Near .21 .18 .30 .25 .68 .28 .19 .89

Table 4.4. Pearson correlations between dimensions of interpersonal rela-

tionships in our corpus derived from the Ontonotes data.

and Zhang Sheng. When the latter threw the former out, the relationship was superficial

and unintimate, but (most likely) existed for longer than a month (enduring).

Table 4.4 shows the correlations between the different dimensions. Not surprisingly,

some dimensions correlate with each other. For example, enduring relationships tend to

also be intense (0.56) and intimate (0.61), and concurrent relationships tend to be active

(0.74). The highest correlation is between concurrent and spatially near (0.89), indicating

that if two people participate in a common event at the same time, usually they are at the

same location. Note, however, that most correlations are low, and some dimensions (e.g.,

cooperative, equal) have low correlations (¡0.30) with all dimensions. This indicates that the

dimensions, as defined indicate some unique property of the interaction or relationship.

4.5.3. Annotation Process for Friends Corpus

We annotated for all the 24 episodes of the first season of Friends. Table 4.8 shows the

statistics of each of the 24 episodes of the first season of the Friends series showing the number

27

Page 38: Extracting Dimensions of Interpersonal Interactions and

of utterances and the most interesting pairs with their number of interactions in parentheses.

The annotations were done one episode at a time. Annotators were presented with the

full transcript of the episode, including speaker information (who speaks what?) and the

names of the individuals mentioned in each conversation turn (who do speakers talk about?).

Annotators read each episode from the beginning and annotate dimensions of interactions

and relationships after each interaction. Regarding interactions, they were instructed to

annotate dimensions taking into account the language of the current conversation turn.

Regarding relationships, they were instructed to annotate dimensions taking into account

all previous conversation turns within the same episode. For example, if previous turns

state that characters Rachel and Monica are best friends, the relationship will continue to

be annotated intense even if an interaction does not indicate so (until a turn indicates that

they are not friends, if applicable).

We discovered during pilot annotations that the value for a dimension sometimes

could not be determined. For example, if the first interaction between Rachel and Monica

is Rachel: How are [you] Monica doing?, we cannot tell if the relationship is temporary or

enduring. We note, however, that all interaction after we find out that they are best friends

(as long as they remain best friends) will be annotated enduring. Just like earlier, we refer to

dimensions by the first descriptor as in the definitions in 4.1, and use 1 if the first descriptor

of a dimension is true, -1 if the second descriptor is true, and 0 if neither the first nor the

second descriptor can be chosen.

Table 4.5 shows the annotation examples from the Friends dataset. The interactions

in conversation turns 1 and 2 are competitive: Phoebe is ridiculing Paul by asking Monica

if he eats chalk, and Ross is confronting an unnamed woman. In turn (3), Monica refers to

Phoebe with affection (as the latter sleeps); thus the interaction is cooperative. Turns (4–6)

exemplify concurrent vs. nonconcurrent. In (4), Rachel is inquiring whether she can meet

Alan (Monica’s boyfriend), and Rachel and Alan are not involved in the same event (at this

point, the meeting may or may not happen). In (5–6), however, the speaker and second

party are involved directly in a communication event. In examples (4–6), the values for

28

Page 39: Extracting Dimensions of Interpersonal Interactions and

active are the same as for concurrent. Examples 7–9 show values for the dimension intense.

These require some explanation, as additional information beyond the current conversation

turn is required (recall that dimensions of relationships are annotated taking into account

the previous turns within the same episode, Section 4.5.3). Dr. Franzblau is the doctor of

a friend’s ex-wife, so he and Monica have a superficial relationship (Turn 7). At the point

Turn (8) is spoken by Chandler, she and Aurora are strangers, so they have a superficial

relationship.

In (9), Joey talks about his mom, so the relationship is (intense). Example 10 shows

Rachel talking to her credit card company representative, so it is work oriented, whereas in

11 Monica is talking to her friend Rachel, so it is pleasure oriented.

Examples 12 and 15 are fairly straightforward: previous interactions reveal that Joey

is an adult (12), and Rachels’s talking to her father in 13. Example 14 is annotated equal,

as Chandler and Ross are friends based on previous interactions (the use of pal also helps).

In example 15, Rachel is talking to her old friends, so it is equal, and usage of you guys

helps in this one. Example 16 shows that Monica was expecting an invitation from Rachel,

thus indicating that they knew each other for long (enduring). Example 17 shows Rachel

talking to a waiter whom she has possibly met for the first time, so it is annotated temporary.

Example 18 exemplifies a distant interaction as it is said over the phone. Example 19 is a

direct interaction between Chandler and Joey, so it is near.

4.5.4. Annotation Quality in Friends Corpus

The annotations were done by two graduate students in computational linguistics.

First, they did pilot annotations to better define the dimensions. After several iterations,

both of them annotated three episodes (15% of all interactions). Table 4.6 show the anno-

tations agreement in this Friends dataset. Cohen’s range between 0.77 and 0.89, and most

(7 out of 9) are above 0.80, which is considered perfect agreement. And as values between

0.60 and 0.80 are considered substantial [4], the remaining episodes were annotated once.

Many of the dimensions we consider in this work are intuitively correlated. For exam-

ple, concurrent interactions must be active, and pleasure oriented interactions are probably

29

Page 40: Extracting Dimensions of Interpersonal Interactions and

Speaker 2nd party Coop.

1: Wait, does he eat chalk? Phoebe Paul -1

2: Hey, hey, hey, that’s not the rule and you know it. Ross Woman -1

3: She is so peaceful. Monica Phoebe 1

Speaker 2nd party Conc.

4: Well, then can we meet him? Rachel Alan -1

5: Hey, buddy, what’s up. Chandler Alan 1

6: Ma, I am sorry. Joey Mrs. Tribbiani 1

Speaker 2nd party Itense

7: Um, has Dr. Franzblau been by? Rachel Dr. Franzblau -1

8: There’s a beautiful lady at eight, nine ten o’clock! Chandler Aurora -1

9: Ma, I am sorry. Joey Mrs. Tribbiani 1

Speaker 2nd party Pleas.

10: Could you please tell me what this is in reference to? Monica Visa -1

11: Uh, Rach, it’s the Visa card people. Monica Rachel 1

Speaker 2nd party Equal

12: Oh, ah- the kid has it. Joey the kid -1

13: Hey Dad, what’s up? Rachel Dad -1

14: Happy birthday, pal. Chandller Ross 1

15: So c’mon, you guys, tell me all the dirt! Rachel Leslie 1

Speaker 2nd party Temp.

16: Who wasn’t invited to the wedding. Monica Rachel -1

17: Can I get you some coffee? Waitress Rachel 1

Speaker 2nd party Near

18: Hi it-it’s Rachel. Rachel Mindy -1

19: Give it to me. Chandler Joey 1

Table 4.5. Annotation examples from the Friends dataset. We show exam-

ples of contrasting values for selected dimensions. The first party is always the

speaker, and the second party is underlined. I stands for interactions, and R

for relationship.30

Page 41: Extracting Dimensions of Interpersonal Interactions and

also equal. We note, however, that interactions can be passive and concurrent, e.g., in (Mon-

ica talking to Joey)[He] (Paul) is just [a guy] (Paul) I am dating!, Monica and Paul have a

passive and concurrent interaction (they are dating, but they are not talking to each other).

Table 4.7 shows Pearson correlations between all dimensions of interactions and relationships.

Most correlations are under 0.3 (29 out Spatially Near vs. Spatially Distant (Interaction),

Intense vs. Superficial (Relationship), although some pairs do have high correlations. In

particular, active interactions tend to be both concurrent (0.81) and spatially near (0.71),

and spatially near interactions tend to be concurrent (0.89). Regarding relationships, inti-

mate correlates with intense (0.75), pleasure oriented with equal (0.50), and temporary with

both superficial (0.52) and intimate (0.54).

Figure 4.3 shows the label percentage distributions per dimension in the Friends

dataset. Regarding interactions, we note that (a) values for all dimensions can be determined

almost always (the percentages of 0 (unknown) are almost zero), and (b) the first descriptor

is much more common in all dimensions. These percentages do not represent the distribution

of interactions between people in general: the scripts of the TV show Friends mostly contain

conversation between friends. Regarding relationships, we observe a larger percentage of 0

(unknown) although values of all dimension can be determined most of the time (labels 1 and

-1, indicating that the first or second descriptor apply). Most dimensions are biased towards

1 (the only exception is temporary, as most relationships are enduring), especially pleasure

oriented and equal (91.2% and 84.8%). Again, these distributions would be different if we

worked with other sources of dialogue than the TV show Friends.

Figures 4.4 and 4.5 show the most salient words for dimensions spatially near and

intense respectively in the Friends dataset. We calculated salience using tf-idf [59]. In-

teractions containing derogatory words (e.g.pig,bugs,pretending,cheating) tend to be distant,

and near interactions contain mostly neutral and nicer words such as friends, sweatheart and

please. We also note that cognitive verbs and nouns (e.g.,thinking,figured (out), looking(into),

cause), as well as important events (birth-day,thanksgiving) and slang usage (e.g.,whad-dya)

signal intense relationships.

31

Page 42: Extracting Dimensions of Interpersonal Interactions and

Dimension κ coefficient

Cooperative (Int.) 0.82

Cooperative (Rel.) 0.81

Equal 0.85

Intense 0.80

Pleasure Oriented 0.77

Active (Int.) 0.89

Active (Rel.) 0.63

Intimate 0.85

Temporary 0.78

Concurrent 0.84

Spatially Near 0.87

All 0.81

Table 4.6. Inter-annotator agreement (raw agreement and Cohen’s κ) in the

Friends dataset. κ values between 0.6 and 0.8 indicate substantial agreement,

κ values over 0.8 indicate perfect agreement [4].

4.5.5. Annotation Process for Friends Extension

After determining the pairs of people (speaker speaking to someone) as discussed in

Section 4.4.3, two annotators carried out the annotation processes. First, they annotated

dimensions of interactions and relationships. Second, they annotated the social acts for

cooperative and competitive interactions. Both annotation phases were done one episode at

a time.

In the first phase, they read the transcript of each episode sequentially and annotated

(a) dimensions of interactions (cooperative or competitive) taking into account only the cur-

rent interaction, and (b) dimensions of the corresponding relationship (equal or hierarchical,

intense or superficial, intimate or unintimate, and temporary or enduring) taking into ac-

32

Page 43: Extracting Dimensions of Interpersonal Interactions and

Coop

erat

ive(

Int)

Coop

erat

ive(

Rel

)

Equ

al

Inte

nse

Ple

asu

reO

r.

Act

ive(

Int)

Act

ive(

Rel

)

Inti

mat

e

Tem

por

ary

Con

curr

ent

Cooperative(Int) –

Cooperative(Rel) .39 –

Equal -.01 -.02 –

Intense .08 .38 .28 –

Pleasure Or. -.05 -.05 .50 .13 –

Active(Int) .06 .22 .08 .28 .01 –

Active(Rel) .02 .14 .14 .29 .04 .30 –

Intimate .12 .43 .32 .75 .28 .28 .30 –

Temporary -.02 -.14 -.24 -.52 -.27 -.33 -.29 -.54 –

Concurrent .02 .18 .06 .24 .08 .81 .32 .26 -.31 –

Spat. Near .01 .16 .05 .22 .08 .71 .25 .22 -.28 .89

Table 4.7. Pearson correlations between dimensions of interpersonal inter-

actions and relationships in the Friends dataset.

count all interactions between the two characters. From here on, we refer to dimensions by

their first descriptor, and use (a) 1 or −1 if the first or the second descriptor applies, and 0

if neither descriptor applies.

In the second phase, annotators annotated the social acts of the 99.4% of interactions

that are either cooperative or competitive. This process was done one interaction at a time.

4.5.6. Annotation Quality in Friends Extension

Two graduate students in computational linguistics carried out the annotations. First,

they did pilot annotations to better define the dimensions and social acts (Section ??). After

several preliminary iterations, both of them annotated one episode (approximately 10% of

all interactions). Table 4.11 presents the inter-annotator agreements for the dimensions.

33

Page 44: Extracting Dimensions of Interpersonal Interactions and

Episode Interactions Interesting pair(interactions)

1 143 Monica-Rachel (23), Monica-Paul (21)

2 105 Rachel-Barry (14), Carol-Ross (12)

3 98 Joey-Chandler (12), Monica-Alan (10)

4 64 Monica-Rachel (15), Chandler-Ross (12)

5 76 Cahndler-Janice (14), Ross-Rachel (13)

6 82 Chandler-Aurora (32)

7 76 Ross-Rachel (13), Joey-Ross (12)

8 64 Monica-Mrs. Geller (14)

9 75 Monica-Rachel (13), Ross-Susan (10)

10 69 Phoebe-David (22)

11 159 Chandler-Ross (22), Ross-Mrs. Bing (21)

12 109 Phoebe-Paolo (14), Phoebe-Rachel (12)

13 120 Joey- Mr. Tribbiani (21), Joey-Mrs. Tribbiani

14 239 Chandler-Janice (28), Ross-Carol (23)

15 78 Joey-Ross (15)

16 95 Chandler-Nina (15)

17 109 Monica-Rachel (28), Phoebe-Ursula

18 42 Ross-Rachel (9)

19 29 Ross-Rachel(9)

20 111 Rachel-Mindy (30), Rachel-Barry (17)

21 85 Monica-Fake Monica (23)

22 92 Phoebe-Chandler (32), Monica-Ethan (13)

23 104 Ross-Susan (19)

24 107 Ross-Rachel (30)

Table 4.8. Stastistics of each of the 24 episodes of the first season of the

Friends series showing the number of utterances and the most interesting pairs

with their number of interactions in parentheses.

34

Page 45: Extracting Dimensions of Interpersonal Interactions and

(Ro)ss (M)onica (R)achel (C)handler (J)oey (P)hoebe Coop. Comp.

Ro * 104 (87,17) 151 (143,8) 42 (34,8) 93 (72,21) 39 (32,7) 368 61

M 90 (65,23) * 119 (88,31) 37 (26,11) 79 (51,28) 60 (49,11) 279 104

Ra 105 (101,4) 105 (74,31) * 42 (34,8) 12 (9,2) 74 (66,8) 284 53

C 110 (68,40) 47 (28,19) 40 (32,8) * 89 (57,31) 35 (30,4) 215 102

J 134 (98,36) 92 (68,23) 23 (19,4) 73 (56,17) * 12 (11,1) 252 81

P 41 (38,3) 93 (66,24) 100 (91,9) 37 (27,10) 26 (20,6) * 242 52

Table 4.9. Number of interactions between the main six characters. For

each pair, we show the total number of interactions (left), and the number of

cooperative and competitive interactions (between parentheses). We also show

the total number of cooperative and competitive interactions per character

with the other five main characters in the Friends extension dataset.

Speaker 2nd party Social Act (Coop.)

1: Oh, by the way, great service tonight. Ross Rachel appreciate

2: You should feel great about yourself! Monica Rachel encouarge

3: Look at you, you are so big I can’t believe it! Rachel Leslie excited

4: How was it with your friends ? Monica Rachel nice

Speaker 2nd party Social Act (Comp.)

5: You have got waaay too much time. Chandler Joey complain

6: No, you don’t. Monica Ross disagree

7: Paul, is it? Chandler Alan mock

8: Be Murky. Phoebe Ross scold

Table 4.10. Annotation examples from the Friends extension dataset. We

show examples of interactions categorized based on the social acts of the di-

mension cooperative vs. competitive. The first party is the speaker, and the

second party is the person being spoken to.

35

Page 46: Extracting Dimensions of Interpersonal Interactions and

Inte

ract

ions

80.8%

16.5%

72.6%

27.1%

77.7%

22.3%

80.2%

19.5%

1

-1

Cooperative Active Concurrent Spatially Near

Rel

atio

nsh

ips

84.8%

11.5%

57.5%

21.3%

91.2%7.2%

58.2%

24.6%

27.6%

60.6%

Equal Intense Pleasure Oriented Intimate Temporary

Figure 4.3. Label percentages per dimension in the Friends dataset. The

missing portion of pie charts correspond to label 0 (unknown).

Cohen’s κ range between 0.82 and 0.94 (top block). The agreements (Cohen’s κ) for social

acts range between 0.78 and 0.89 (bottom block). κ values between 0.60 and 0.80 are

considered substantial [4], and above 0.80 are considered nearly perfect. Based on these

agreements, the remainder of episodes were annotated once.

Table 4.10 annotation examples from the Friends extension dataset of social acts of

the dimension cooperative vs. competitive. In (1) Ross is appreciating what Rachel has done

as a waiter, so he is appreciating her work. In (2) Monica is encouraging Rachel by saying

some good words about her. Rachel shows her excitement about her friend’s pregnancy in

(3), and in (4) Monica is being nice with Rachel by asking how she spent time with her

friends. All these are examples of cooperative interactions with their social acts.

Examples 5–8 show competitive interactions with their social acts. In (5) Chandler

complains about Joey when he sees him doing meaningless acts wasting his time. In (6)

Monica straightforwardly disagreed with her brother. In (7) Chandler is mocking his friend’s

new boyfriend, and in (8) Phoebe is just scolding her friend.

The screenshots of actual scenes of the Friends series are shown with the annotations

36

Page 47: Extracting Dimensions of Interpersonal Interactions and

Spatially Near vs. Spatially Distant (Interaction)

Figure 4.4. Most salient words (calculated with tf-idf) for dimension spa-

tially near in the Friends dataset.

37

Page 48: Extracting Dimensions of Interpersonal Interactions and

Intense vs. Superficial (Relationship)

Figure 4.5. Most salient words (calculated with tf-idf) for dimension intense

in the Friends dataset.

38

Page 49: Extracting Dimensions of Interpersonal Interactions and

raw κ

Dim

ensi

on

s

Cooperative (I) 96.7% 0.90

Equal (R) 97.3% 0.94

Intense (R) 94.3% 0.87

Intimate (R) 90.4% 0.82

Temporary (R) 92.6% 0.83

S.A

cts

Cooperative 91.7% 0.89

Competitve 90.4% 0.78

Table 4.11. Inter-annotator agreements for dimensions and social Acts (raw

agreement and Cohen’s κ) in the Friends extension dataset. κ values between

0.6 and 0.8 indicate substantial agreement, κ values over 0.8 indicate nearly

perfect agreement [4].

of the cooperative and competitive social acts respectively in figures 4.6 and 4.7. 4.6 exem-

plifies cooperative interactions. In the top-left scene in this figure, Ross encourages his sister

Monica to go on a date with the guy who she had wanted to date for a long time. This is

an example of the support social act. The top-right scene exemplifies the social act normal :

Ross telling his friends what he is supposed to do to assemble a piece of furniture. In the

scene at the bottom-left, Rachel expresses her excitement about receiving her first paycheck.

Finally, in the bottom-right scene, Monica is apologizing to her boyfriend and is being nice

to him.

The 4.7 exemplifies competitive interactions. In the top-left scene, Rachel is rudely

talking to her father over the phone and showing anger. In the top-right scene, Phoebe is

mocking Monica by asking her whether her boyfriend eats chalk. In the scene at the bottom-

left, Monica complains about Rachel saying (and forgetting) that she had the keys. Finally,

in the bottom-right scene, Rachel disagrees with what Monica had complained about.

Figure 4.8 shows the Label percentages per dimension in the Friends extension

dataset. Except for the dimension temporary, the first descriptor is much more common

39

Page 50: Extracting Dimensions of Interpersonal Interactions and

Figure 4.6. Scenes from Friends exemplifying cooperative interactions as

well as their social acts. It shows examples of support, normal, nice and

excited social acts (clockwise).

in all dimensions. In other words, most interactions are cooperative (79.4%), and most rela-

tionships are equal (88.1%), intense (67.6%), intimate (72.7%) and enduring (75.8%). These

distributions are not representative of interactions in general, they are biased because of the

corpus we work with.

Table 4.12 shows the Pearson correlations between pairs of dimensions of interactions

and dimensions in the Friends extension dataset. Most correlations are below 0.53, and

cooperative is not correlated with any. The highest correlation is between intense and in-

timate, which is to be expected. Not surprisingly, intense and intimate relationships tend

to be enduring (inversely correlated with temporary), but the correlations are 0.45 and 0.53

40

Page 51: Extracting Dimensions of Interpersonal Interactions and

Figure 4.7. Scenes from Friends exemplifying competitive interactions as

well as their social acts. It shows examples of rude, mock, disagree and com-

plain social acts (clockwise).

respective and thus they do not capture exactly the same properties.

Table 4.13 shows the distribution of social acts. Regarding social acts for cooperative

interactions, normal is the most frequent (80.5%), and the remaining range from 3.1% (agree)

to 9.7% (nice). The social acts for competitive interactions are more uniformly distributed.

We observe a clear minority (disbelief, 1.2%), but the remaining 5 social acts range from

16.2% to 26.3%.

Figure 4.9 shows the most salient words of four social acts, where salience is calculated

using tf-idf [59]. Adjectives such as unbelievable, amazing and fascinating usually indicate

excited social acts. Not surprisingly, positive words such as okay, please and goodnight indi-

41

Page 52: Extracting Dimensions of Interpersonal Interactions and

Figure 4.8. Label percentages per dimension in the Friends extension

dataset. -1, 1 and 0 represent first descriptor, second descriptor and unknown

respectively.

Coop

erat

ive

(I)

Equ

al(R

)

Inte

nse

(R)

Inti

mat

e(R

)

Equal (R) 0.02 -

Intense (R) -0.04 0.40 -

Intimate (R) -0.05 0.38 0.80 -

Temporary (R) 0.01 -0.22 -0.45 -0.53

Table 4.12. Pearson correlations between pairs of dimensions of interactions

and dimensions in the Friends extension dataset.

42

Page 53: Extracting Dimensions of Interpersonal Interactions and

Percentage

Coop

erat

ive

Agree 3.1%

Excited 3.3%

Nice 9.7%

Normal 80.5%

Support 3.4%

Com

pet

itiv

eComplain 19.4%

Disagree 19.2%

Disbelief 1.2%

Dissatisfaction 26.3%

Mock 16.2%

Rude 17.6%

Table 4.13. Distributions of cooperative and competitive social acts among

the 4,257 interactions in the Friends extension dataset.

excited–cooperative nice–cooperative rude–competitive mock–competitve

Figure 4.9. Most salient words (calculated with tf-idf) for four social acts:

excited and nice (cooperative interactions), and rude and mock (competitive

interactions) in the Friends extension dataset.

cate nice social acts. More interestingly, rude social acts include utterances about (quitting)

smoking and unfair situations. Finally, mocking social acts include derogatory nicknames

(e.g., schhteve for Steve), physical imperfections (e.g., hump) and negative adjectives (e.g.,

wrong, brutal). Figure 4.10 shows which social acts follow each cooperative social act. Figure

43

Page 54: Extracting Dimensions of Interpersonal Interactions and

agree and excited

nice and normal

support

Figure 4.10. Charts showing which social acts follow each cooperative social act.

4.11 shows which social acts follow each competitive social act.

4.6. Experiments and Results

Machine learning experiments were done with the data from both the corpus to see if

it is possible to learn the dimensions automatically or not. We conducted experiments using

standard supervised machine learning.

44

Page 55: Extracting Dimensions of Interpersonal Interactions and

complain and disagree

disbelief and dissatisfaction

mock and rude

Figure 4.11. Charts showing which social acts follow each competitive social act.

4.6.1. Experiments with Ontonotes Corpus

In the experiments with the data from the Ontonotes corpus, each pair of people

become an instance, and we split instances into training (80%) and test (20%). As a learning

algorithm, we use SVM with RBF kernel as implemented in scikit-learn [52]. We report

results in the test set after tuning the SVM parameters (C and ) using 10-fold cross-validation

with the training set. More specifically, we train one classifier per dimension, and experiment

with all instances but the ones annotated inv. Thus, each classifier predicts 3 labels: 1(the

45

Page 56: Extracting Dimensions of Interpersonal Interactions and

Feature Description

Verb

word, tag Word form and POS tag of the verb

dep out Outgoing syntactic dependency from verb

deps in Flags indicating incoming syntactic dependencies to the verb

lex name Name of the WordNet lexical file of the verb

token before Word form and POS tag of the token before the verb

token after Word form and POS tag of the token after the verb

Person

words, tags Concatenation of word forms and POS tags

type Whether the person is a pronoun or named entity

dep out Outgoing syntactic dependency

distance verb Number of tokens between the person and the verb

first token Word form and POS tag of the first token in the person

last token Word form and POS tag of the last token in the person

token before Word form and POS tag of the token before the person

token after Word form and POS tag of the token after the person

Personx Persony

direction Flag indicating whether x occurs before or after y

type Whether x and y are PERSON NEs or pronoun

Table 4.14. Feature set used to determine dimensions of interpersonal rela-

tionships between pairs of people (x, y). Verb features are extracted from the

verb of which either x or y is the subject, Person features are extracted from

x and y independently, and Persons features are extracted from x and y.

first descriptor applies), -1(the second descriptor applies), and 0 (neither descriptor applies).

The features we work with for Ontonotes corpus are summarized in Table 4.14. Most

features are standard. We refer to the pair of people as x and y. Verb features capture in-

formation about the verb to which x or y attach. We include words and part-of-speech tags

(verb, and tokens before and after), the name of the WordNet lexical file to which the verb

belongs, and dependencies. The inclusion of the lexical file gives general information about

46

Page 57: Extracting Dimensions of Interpersonal Interactions and

a specific verb and inform about a broad category in which the verb falls under. This is

considered to be helpful because verbs of similar kind can depict similar relationship dimen-

sions. We extract inward and outward dependencies of the verb to get strctural knowledge of

the sentence. Person features are extracted from x and y independently, and consists mostly

of words and part-of-speech tags. We also included a flag indicating whether the person is

a pronoun or named entity (type), and the number of tokens between the person and the

verb (distance-verb). Personx Persony features capture information of both x and y . They

capture (a) whether x occurs before or after y in the sentence, and (b) whether they are

both named entities or one is a pronoun and the other one a named entity (type feature).

As we collect word forms of the names and verbs and their POS tags, the dependency of the

verbs to capture the structure information, bag-of-words features were not collected.

We present overall Ontonotes corpus results (averages of the classifiers for each di-

mension) using the majority baseline and with several feature combinations in Table 4.15.

Then, we present detailed results per dimension with the best feature combination in Table

4.16. We only present results obtained in the test set. The table shows results obtained for

each dimension with the best combination of features for all dimensions (Verb + Personx

+ Persony + Personx Persony, boldfaced in Table 4.15). The last three columns under All

contains the weighted average of Precision (P), Recall (R) and F-measure (F) of the di-

mensions based on the distributions of the three labels. The last row contains the weighted

averages of each column. An explanation of how the values are calculated is provided in the

appendix.

The majority baseline obtains 0.53 average F-measure. Recall that we build a classifier

per dimension, thus the combination of the nine majority-baseline classifiers predict two

labels: 1 (0.55 F-measure) and -1 (0.64 F-measure). Models trained with any combination

of features outperform the majority baseline, but they never learn to predict label 0. Since

0 occurs between 0.86% and 4.3% depending on the dimension (Section 4.4, this limitation

does not affect overall performance substantially. Verb features alone yield a 0.65 average

F- measure (1: 0.64, -1: 0.65). Adding features derived from x (Verb + Persony) improves

47

Page 58: Extracting Dimensions of Interpersonal Interactions and

performance (0.71 average F-measure), and adding features derived from y (Verb + Persony)

slightly improves performance (0.67 average F-measure). In both cases, -1 is predicted

more accurately than 1 (0.78 vs. 0.69 and 0.74 vs. 0.67). Finally, adding all features

(Verb + Personx +Persony+ Personx Persony) yields the best results (average F-measure:

0.72), although by a minimal margin with respect to Verb + Personx. Detailed Results

per Dimension. Table 4.16 presents results per dimension with the best overall combination

of features (Verb+Personx+Persony+ Personx Persony). All dimensions obtain overall F-

measures between 0.65 and 0.83 (last column). Results per label are heavily biased towards

the most frequent label per dimension (Figure 4.2), although it is the case that the models

we experiment with predict both 1 and -1 for all dimensions. As stated above, none of them

predict 0, but this limitation does not substantially penalize overall performance because

of the low frequency of this dimensions are similar (0.67 vs. 0.76 and 0.76 vs. 0.58), and

the labels are distributed relatively evenly in our corpus (40.4% vs. 51.4% and. 58.4% vs.

35.3%). Finally, F-measures per label with other dimensions are biased towards the most

frequent label. For example, only 15% of all pairs of people have an enduring relationship,

and the F-measure for 1 with dimension temporary is quite high.

The comparison of the results with different feature sets reveals that only knowing

about the verb that connects both the people involved is not enough; we need some lexical

information about the words indicating the people as well. Knowledge about where they

verbs are placed in the sentence, and how many words are there in the sentence contribute to

useful information that leads us to better results. So we may deem that to indicate different

values of these dimensions, we possibly arrange the words in the sentence differently. In

other words, the linguistics of a sentence has something to do with the value of the dimension

administered.

4.6.2. Experiments with Friends Corpus

With the data from the Friends corpus, we experimented with SVM classifiers with

RBF kernel to predict dimensions of interactions and relationships. We divided the 24

episodes into train (episodes 120) and test (2124), and trained one classifier per dimension

48

Page 59: Extracting Dimensions of Interpersonal Interactions and

Features Label P R F

Majority

Baseline

1 0.43 0.76 0.55

0 0.00 0.00 0.00

-1 0.57 0.59 0.64

Avg. 0.45 0.65 0.53

Verb

1 0.62 0.72 0.64

0 0.00 0.00 0.00

-1 0.71 0.78 0.65

Avg. 0.62 0.71 0.65

Verb, Personx

1 0.68 0.75 0.69

0 0.00 0.00 0.00

-1 0.78 0.79 0.78

Avg. 0.70 0.74 0.71

Verb, Persony

1 0.64 0.72 0.67

0 0.00 0.00 0.00

-1 0.74 0.76 0.74

Avg. 0.66 0.70 0.67

Verb, Personx,

Persony,

Personx Persony

1 0.69 0.74 0.70

0 0.00 0.00 0.00

-1 0.77 0.80 0.76

Avg. 0.71 0.76 0.72

Table 4.15. Results obtained for all dimensions with several combinations

of features for the Ontonotes dataset.

using scikit-learn [52]. Each classifier is trained with three labels: 1 (1st descriptor), -1 (2nd

descriptor), and 0 (unknown). The SVM parameters (C and ) were tuned using 10-fold

cross-validation with the train split, and results are reported using the test split. Note that

different pairs of people interact more or less in each episode (4.8). Thus, the classifiers

49

Page 60: Extracting Dimensions of Interpersonal Interactions and

Dimension1 (1st desc.) 0 (unk.) -1 (2nd desc.) All

P R F P R F P R F P R F

Cooperative 0.73 0.96 0.83 0.00 0.00 0.00 0.60 0.19 0.29 0.66 0.72 0.65

Equal 0.56 0.10 0.17 0.00 0.00 0.00 0.74 0.97 0.84 0.68 0.74 0.66

Intense 0.39 0.30 0.34 0.00 0.00 0.00 0.78 0.85 0.82 0.67 0.71 0.69

Pleasure 0.40 0.28 0.33 0.00 0.00 0.00 0.87 0.93 0.90 0.79 0.82 0.80

Active 0.69 0.85 0.76 0.00 0.00 0.00 0.68 0.51 0.58 0.67 0.69 0.67

Intimate 0.44 0.17 0.24 0.00 0.00 0.00 0.88 0.98 0.93 0.81 0.86 0.83

Temporary 0.85 0.96 0.91 0.00 0.00 0.00 0.33 0.10 0.16 0.77 0.83 0.79

Concurrent 0.72 0.80 0.76 0.00 0.00 0.00 0.77 0.75 0.76 0.71 0.74 0.73

Spat. Near 0.66 0.68 0.67 0.00 0.00 0.00 0.73 0.79 0.76 0.66 0.70 0.68

Average 0.69 0.74 0.70 0.00 0.00 0.00 0.77 0.76 0.77 0.71 0.76 0.72

Table 4.16. Results obtained for each dimension with the best combination

of features for all dimensions (Verb + Personx + Persony + Personx Persony,

boldfaced in Table 4.15). The last three columns under All contains the

weighted average of Precision (P), Recall (R) and F-mesure (F) of the di-

mensions based on the distributions of the 3 labels. The last row contains the

weighted averages of each column.

are grounded on general language usage and not modeling who talks and who is talked

about. We also experimented with LSTMs taking as input the current conversation turn

and previous turns, but do not report results because SVM classifiers yielded better results.

For the Friends corpus we use a combination of features extracted directly from

the conversation turn, sentiment lexica, and context. Specifically, we extract (a) the first

word in the conversation turn,(b) bag-of-words features (binary flags and tf-idf scores), and

(c) the root verb, and flags indicating the presence of exclamation, question marks and

negation cues from (Morante and Daelemans, 2012) [49]. Regarding sentiment, we extract

flags indicating whether the turn has a positive, negative or neutral word in the list by

50

Page 61: Extracting Dimensions of Interpersonal Interactions and

Hamilton et al. (2016)[33], the sentiment score of the turn (summation of sentiment scores

per token over the number of tokens in the turn), and a flag indicating whether the turn

contains a negative word from the list by Hu and Liu (2004)[39]. Regarding context, we

extract bag-of-words features from the previous conversation turn in which the same people

interact (not necessarily the preceding turn). The BOW features gave information about all

the words contained in the sentence. The information about the presence of punctuation

symbols like (?, !) were also included as these symbols set the tone of an interaction.

Similarly, the sentiment of the sentence was thought to be indicative of the value of the

dimension cooperative vs. competitive. The presence or absence of negative words was also

thought to be affecting the learning of the same dimension. But as we are considering a lot

more dimensions than the mentioned one, the average result does not seem to be affected

much with these features. Context information was hoped to facilitate the learning.

Table 4.19 shows the overall results (average of all dimensions) obtained with the

majority baseline and several feature combinations with Friends corpus. All feature com-

binations outperform the baseline. Sentiment features are not beneficial, leading to the

conclusion that sentiment does not correlate with dimensions of interactions and relation-

ships between people. This may look surprising at first sight, but recall that our dimensions

capture much more than if two people get along. Finally, the bag-of-words features from

the previous turn in which the same people interacted bring a small improvement (F: 0.70

vs. 0.72). We show results per dimension for the best feature combination (all) in Table

4.18. Despite the label distributions are biased shown in Figure 4.3, the system predicts

most labels for most dimensions except the very biased ones (cooperative, equal, and pleasure

oriented). Note that 0 (unknown) does not allow us to determine the value of a dimension,

and the low results with this label are not a concern.

4.6.3. Experiments with Friends Extension

We use SVM classifiers with RBF kernel to predict dimensions of interactions and

relationships as well as the social acts of cooperative and competitive interactions. We divided

the entire corpus (12 episodes) into stratified train and test splits, and use the implementation

51

Page 62: Extracting Dimensions of Interpersonal Interactions and

P R F

Majority baseline 0.54 0.73 0.62

SVM with first word 0.62 0.71 0.65

+ BOW 0.66 0.73 0.67

+ sentiment 0.65 0.72 0.66

+ other 0.70 0.75 0.70

+ BOW previous 0.72 0.76 0.72

Table 4.17. Results obtained with the test set with several systems (average

of all dimensions) for the Friends dataset. Previous refers to the previous

conversation in which the same pair of people interacted not the immediately

previous turn).

1 (1st desc.) 0 (Unk) -1 (2nd desc.) All

P R F P R F P R F P R F

Inte

ract

ion

Cooperative 0.83 0.96 0.89 0.00 0.00 0.00 0.31 0.08 0.13 0.73 0.80 0.75

Active 0.92 0.90 0.92 n/a n/a n/a 0.75 0.76 0.75 0.87 0.87 0.87

Concurrent 0.92 0.92 0.92 n/a n/a n/a 0.70 0.69 0.70 0.87 0.87 0.87

Spat. Near 0.89 0.94 0.91 n/a n/a n/a 0.69 0.53 0.60 0.85 0.86 0.85

Rel

atio

nsh

ip

Equal 0.86 0.95 0.90 0.00 0.00 0.00 0.15 0.07 0.10 0.76 0.83 0.79

Intense 0.70 0.81 0.75 0.28 0.11 0.16 0.38 0.46 0.41 0.56 0.60 0.57

Pleasure Or. 0.82 1.00 0.90 0.00 0.00 0.00 1.00 0.02 0.04 0.82 0.83 0.75

Intimate 0.61 0.85 0.71 0.26 0.05 0.09 0.42 0.36 0.39 0.48 0.56 0.49

Temporary 0.53 0.42 0.47 0.67 0.17 0.27 0.62 0.85 0.72 0.60 0.60 0.57

Table 4.18. Results obtained per dimension with the best system (all fea-

tures, Table 4.17). The results under All the weighted averages for all labels,

recall that the label distribution is biased (Figure 4.3).

52

Page 63: Extracting Dimensions of Interpersonal Interactions and

in scikit-learn [52] to train the classifiers. We tune SVM hyperparameters (C and γ) using

10-fold cross-validation with the train split. Regarding dimensions, we train one classifier

per dimension and all of them predict three labels: Regarding social acts, we train one for

cooperative social acts (5 labels: agree, excited, nice, support and normal) and another one

for competitive social acts (6 labels: complain, disagree, disbelief, dissatisfaction, mock and

rude).

For the Friends corpus we extract features from the utterance, context, and sentiment

lexica. The feature set does not include information about the two parties involved in an

interaction in order to rely only on language usage. The features set is rather simple and

includes the following. First, we extract the first word in an utterance (first). Second, we

extract bag-of-words representations (binary flags and tf-idf scores) of the utterance. Third,

we include the same bag-of-words representations for the previous utterance (BOW prev.).

Fourth, we extract features from sentiment lexica. Specifically, we extract flags indicating

whether the turn has a positive, negative or neutral word in the list by Hamilton et al.

(2016)[33], the sentiment score of the utterance (summation of sentiment scores per token

over the number of tokens in the utterance), and a flag indicating whether the utterance

contains a negative word from the list by Hu and Liu (2004) [39]. Fifth, we extract other

features, which include (a) the root verb (b) binary flags indicating the presence of exclama-

tion, question marks, and negation cues from [49]. The justification behind using the above

features is that they worked well in the Friends project. As we are dealing with very similar

data from the same original dataset in this project as well, we stuck with the same choices

of features.

Classifiers trained with the features above yield results in a realistic scenario. We

also report results using as features gold values for dimensions of the previous and current

interaction (all but the dimension being predicted), as well as the social acts of the previous

utterance. This was done to include context information regarding dimensions and social

acts. It is seen that when someone interacts with another cooperatively in one interaction, it

is likely that the next interaction would be cooperative as well. Or at least there is a common

53

Page 64: Extracting Dimensions of Interpersonal Interactions and

P R F

Majority baseline 0.59 0.76 0.66

first + BOW 0.70 0.75 0.70

+ BOW prev. + sentiment + other 0.72 0.77 0.72

+ dims. previous interaction 0.75 0.77 0.75

+ other dims. current interaction 0.85 0.86 0.85

Table 4.19. Results for predicting dimensions of interactions and relation-

ships with the Friends extension dataset. The bottom two systems use gold

dimensions as features thus results are unrealistic.

pattern. Thus including the values of dimensions or social acts of previous sentences improved

the results considerably. These results are not realistic but show that classifiers are far from

perfect even in this unrealistic scenario.

Table 4.19 shows the results for predicting dimensions of interactions and relation-

ships (weighted averages) with the Friends extension dataset. Any combination of features

outperforms the majority baseline (F1: 0.66). Using as features the first word and bag-

of-words representations for the utterance brings the F1 to 0.70, and including all features

yields 0.72 F1. The results including gold annotations for all dimensions of the previous

interaction (F1: 0.75) and other dimensions of the current interaction (F1: 0.85) indicate,

unsurprisingly, that this information is beneficial, although results are far from perfect. Ta-

ble 4.20 shows the results for predicting dimensions of interactions and relations with the

best-performing realistic system.

Table 4.21 shows the overall results for predicting social acts (weighted average).

Recall that social acts are predicted only for interaction that are either cooperative or com-

petitive. This time, the features capturing the first work in the utterance and bag-of-words

representations bring substantial improvements with respect to the majority baseline with

both cooperative (F1: 0.71 vs. 0.77) and competitive (F1: 0.11 vs. 0.38) social acts. Adding

the additional features, however, does not bring improvements in results. The results with

54

Page 65: Extracting Dimensions of Interpersonal Interactions and

1 (1st desc.) 0 (unk.) -1 (2nd desc.) All

P R F P R F P R F P R F

I Cooperative 0.81 0.94 0.87 0.00 0.00 0.00 0.43 0.17 0.24 0.73 0.78 0.74

R

Equal 0.90 0.96 0.93 0.50 0.16 0.24 0.33 0.16 0.21 0.83 0.87 0.85

Intense 0.71 0.96 0.81 0.38 0.05 0.09 0.64 0.21 0.32 0.66 0.70 0.63

Intimate 0.76 0.96 0.85 0.38 0.05 0.10 0.53 0.19 0.28 0.69 0.74 0.68

Temporary 0.37 0.08 0.13 0.80 0.06 0.11 0.77 0.97 0.86 0.70 0.75 0.68

Table 4.20. Detailed results for predicting dimensions of interactions and

relations with the best-performing realistic system (BOW + BOW prev. +

sentiment + other in Table 4.19).

P R F

Coop

er.

Majority baseline .64 .80 .71

first + BOW .75 .81 .77

+BOW prev.+sent.+ other .76 .81 .77

+social acts prev. .79 .82 .80

Com

pet

.

Majority baseline .07 .26 .11

first + BOW .38 .39 .38

+BOW prev.+sent.+other .40 .41 .39

+social acts prev. .55 .53 .52

Table 4.21. Results for predicting social acts (cooperative and competitive).

Systems labeled+social acts prev. use gold social acts and thus results are

unrealistic.

the feature set that includes gold social act information of the previous utterance have large

improvements with competitive social acts (F1: 0.39 vs. 0.52), and minor improvements

with cooperative social acts (F1: 0.77). Table 4.22 shows detailed results per social act.

55

Page 66: Extracting Dimensions of Interpersonal Interactions and

P R F

Coop

erat

ive

Agree 0.25 0.05 0.08

Excited 0.43 0.13 0.20

Nice 0.55 0.33 0.42

Normal 0.84 0.96 0.89

Support 0.50 0.12 0.19

Average 0.76 0.81 0.77C

omp

etit

ive

Complain 0.38 0.33 0.35

Disagree 0.57 0.50 0.53

Disbelief 0.00 0.00 0.00

Dissatisfaction 0.38 0.60 0.46

Mock 0.39 0.33 0.36

Rude 0.33 0.19 0.24

Average 0.40 0.41 0.39

Table 4.22. Detailed results for predicting social acts with the best-

performing realistic system (BOW + BOW prev. + sentiment + other in

Table 4.21). The average shown is the weighted average based on the number

of instances in each label.

Just like with dimensions, these results demonstrate that the task is challenging.

4.6.4. Error Analysis of Friends Extension

The classifiers for predicting social acts are far from perfect. In this section, we

illustrate common errors made by the classifiers. The majority of cooperative interactions

are a normal social act (80.5%, Table 4.13), and unsurprisingly, our classifier wrongly predicts

many interactions as normal. One example is the utterance I’m having a son! said by Ross.

He utters the words with sheer joy and excitement, and the annotators had identified it to be

excited. But the classifier predicts it as normal, as it fails to capture the excitement usually

56

Page 67: Extracting Dimensions of Interpersonal Interactions and

expressed through tone of voice. Another example in which the classifier fails to capture

the excitement is utterance Hey!, exclaimed by Chandler when he just got to know that his

friend Ross was becoming a father. As the only word spoken in the utterance (Hey) is often

used as a form of greetings, it is wrongly preicted as being nice instead of excited. Another

similar example is Paolo excitedly greeting his girlfriend by saying Oh, hi sweetie, but the

utterance being automatically identified as nice again. In Ok, ok, How about if we split it?,

Joey is normally suggesting something to his friend (to split a bill). The classifier wrongly

predicts agreeing social act, most likely because of the word ok appears three times.

There is generally a fine difference between the social acts dissatisfaction and com-

plain. By saying That-that was like the worst breakup in history!, even though Monica is

rather showing her dissatisfaction with her friend about his skills at breaking up, the classi-

fier predicts as complain social act. On the other hand, Chandler is rather complainig about

his friend Phoebe in Wait a minute, wait a minute, I see where this is going, you’re gonna

ask him to New Year’s, aren’t you?. But this utterance was misclassified as rude. Another

example of a wrong prediction is the utterance You’re gonna break the pact. The system

wrongly predicts rude for this complain social act, most likely because of the negative verb

break.

57

Page 68: Extracting Dimensions of Interpersonal Interactions and

CHAPTER 5

DIMENSIONS OF INTERPERSONAL INTERACTIONS AND RELATIONSHIPS IN

CHAT FORUMS

People are social beings who communicate their feelings, emotions, thoughts, ideas,

etc. with each other through verbal and non-verbal interactions. People build relationships

based on these interactions, and these relationships, in turn, help them create and maintain

networks of peers. Peers in a network cooperate with each other, help each other to learn,

exchange ideas, but also compete for resources [67]. Peer networks are particularly important

for innovation and entrepreneurship [29] as an active exchange of ideas occurs there.

People are usually assumed to be altruistic in networks like online social forums.

They cooperate with and help one another with answers, advice, and ideas. The motivations

behind helping a peer include, but are not limited to, getting pure pleasure from helping,

self-advancement, building a reputation, developing relationships, or sheer entertainment

[65].

When people interact with each other, their interactions vary along various commu-

nicative styles or dimensions of interactions (as mentioned earlier in this document) such

as showing cooperativeness, equality, business orientation, etc. [56]. Varying these dimen-

sions or styles provides tools to achieve communicative goals. For example, someone who

is trying to build a reputation, will tend to use a more cooperative style. Someone who

simply tries to be helpful may use more words of advice in their interactions. The usage of

certain relationship-establishing styles is more prevalent in certain personalities [21] and in

certain settings. Also, people with a certain personality trait interact with others in a simi-

lar expected manner in similar situations [61]. Business-oriented people communicate more

independence, tolerance of ambiguity, risk-taking propensity, innovativeness, and leadership

qualities [69].

The impact of these dimensions or styles is therefore, an important factor in text

analysis. However, due to its complex, decentralized nature (cooperativeness is more than

58

Page 69: Extracting Dimensions of Interpersonal Interactions and

just the use of a few keywords, but includes a whole inventory of communicative tools),

these dimensions have been studied very little in NLP. Part of the reason is the lack of

adequate corpora. We provide another such corpus and report encouraging results for the

above dimensions.

5.1. Original Adansonia Dataset

Our ultimate goal is to predict the communicative styles and strategies of young

people in online peer networks of aspiring entrepreneurs. The initial corpus is a result

of a large-scale social science experiment that involved around 5000 entrepreneurs (mostly

aspiring) from 49 African countries [67]. Those entrepreneurs, after completing an online

business course, interacted with their peers, within groups of sixty, through an Internet

platform for about two months and a half, resulting in about 140,000 chat interactions.

Besides the chat interactions, the original dataset also contains background informa-

tion about the speakers (country of origin, educational background, age, gender, etc.) All

these participants also submitted business proposals that were then evaluated to assess their

success.

The original experimental setup was designed to assess how communication among

peers mutually affects them. [67], therefore, already applied NLP techniques for semantic

analysis of the interactions. They also manually annotated other indicators, i.e., business-

relatedness, sentiment, and audience (one or several people) on a subset of 10k sentences,

and then predicted them for all remaining instances in the online interactions. This dataset,

therefore, provides a perfect starting point for our goals.

The first step to address our goal involves annotating styles of speech on a number of

interactions among people. We use the subset of the data with previous manual annotations,

and added our own annotations to further enrich the data.

5.2. Annotating Dimensions of Interpersonal Interactions and Relationships

We sample around 5500 chat interactions which were previously annotated for business-

relatedness, sentiment, and audience, and annotate the four dimensions of interpersonal in-

59

Page 70: Extracting Dimensions of Interpersonal Interactions and

Cooperative

1: Mobile Webshop is a very good concept. Cooperative

2: You have not done anything yet. Competitive

Motivational

3: @NAME1234 well said @NAME456 start small and dream big..welldone Motivational

4: I meant to say voting contest to be precise. Neutral

Equal

5: Wishing you a very wonderful weekend. Equal

6: Happy to engage you on this.. Hierarchical

Advice

7: Think about it. Advice

8: This is cool Sunday. Neutral

Table 5.1. Annotation examples of contrasting values for each dimension.

Each chat interaction is either directed to an individual or others in general.

teractions and relationships:

(1) Cooperativeness indicating the friendliness shown towards the target audience, with

label values cooperative, competitive, and neutral.

(2) Advice indicating whether the interaction contains any words of advice with label

values advice and neutral.

(3) Motivational indicating whether the interaction contains any words of motivation,

with label values motivation and neutral [77].

(4) Equality indicating whether there is a display of hierarchy between the speaker and

the receiver, with label values equal and hierarchical.

For all dimensions, unknown is used whenever it is hard to determine any of the other values

from context.

60

Page 71: Extracting Dimensions of Interpersonal Interactions and

Three graduate students with experience in NLP tasks annotated the data. After a

training and practice session, each of the annotators annotated their part of around 2,100

chat interactions. 502 of these were shared among all three annotators, so we can compute

agreement measures. We obtain the most probable labels for the shared portion using MACE

[37]. We summarize the agreements scores (raw agreement: 83%; averaged pairwise Cohen’s

κ: 0.53; Krippendorff’s alpha: 0.52) in Table 5.2. The average MACE competence score of

these annotators is 0.53.

raw κ Krippendorff’s α

Cooperative 76% 0.58 0.57

Motivational 91% 0.74 0.74

Equal 77% 0.33 0.34

Advice 91% 0.60 0.60

Average 83% 0.53 0.52

Table 5.2. Inter-annotator agreements for dimensions. 0.6 ≤ κ ≤ 0.8: sub-

stantial agreement, κ > 0.8: nearly perfect agreement [4].

Table 5.3 shows the Pearson’s correlations between pairs of the dimensions of interac-

tions and previous annotations. This indicates that Motivational diemsnions are usually also

Cooperative (0.61), give Advice (0.56) and are Equal (0.54). Interestingly, many Business-

related interactions are not very Cooperative (0.42).

For cooperativeness, 42.3% are labeled cooperative, 50.3% are neutral and only 2.14%

are labeled competitive. For the motivational style, 14.1% are motivational and 81.2% are

neutral. For the advice style, 9.2% are advice and 85.9% are neutral. For the equality style,

77.3% are equal and 8.8% are hierarchical.

Table 5.1 shows a number of actual chat interactions from the dataset with different

values for the dimensions annotated. In interaction (1), the praising is considered a cooper-

ative response, whereas in (2) the speaker is chiding someone, indicating a competitiveness.

61

Page 72: Extracting Dimensions of Interpersonal Interactions and

S C M E A

B -0.17 -0.42 -0.28 -0.27 0.01

S – 0.29 -0.01 -0.04 -0.10

C – 0.61 0.37 0.40

M – 0.54 0.56

E – 0.26

Table 5.3. Pearson correlations between pairs of dimensions of interactions

(indicated by the names’ initial letters).

The praise in (3) is motivational. Example (4) does not really communicate any motivation,

so it is labeled as neutral for this style. Example (5) is just a greeting and does not indicate if

anybody is displaying hierarchy over anyone else, so it is equal. Example (6) shows that the

speaker instructs someone on how to behave (hierarchical). In (7), the speaker is advising

someone to think about a matter whereas (8) is just another neutral statement.

5.3. Experiments and Results

People with certain personalities tend to show certain values for the dimensions of

their interactions with others [21]. People with business-oriented mind are expected to

show more keenness and gain more success in business. Business-oriented people are usually

more independent, more tolerant, do not fear a risk, tend to innovate more often, and are

good leaders. [69]. We want to see if there is some correlation between the dimensions of

interactions displayed by people and their ability to succeed in business. We want to predict

the four different dimensions of interactions (cooperative, motivational, advice, equality),

as well as three subsequent indicators of business success: (1) whether the person owns

a business (has business), (2) whether someone has ever owned a business (business

ever) and (3)whether they have submitted a business proposal to compete for a funding

opportunity to start a business (business proposal). The dataset we annotated came with

annotations of these three business indicators, making our work easier.

62

Page 73: Extracting Dimensions of Interpersonal Interactions and

We compare (1) an SVM classifier with RBF kernel to predict both the dimensions of

interactions and relationships as well as the business success indicators and (2) a Multitask

Learning (MTL) Convolutional Neural Network to predict the business success indicators.

We divide our annotated dataset into 80-20 stratified train-test splits for predicting

the dimensions. For the business success prediction, we use the first 500 instances as test

and the rest as train.

5.3.1. SVM Setup

We use the SVM implementation in scikit-learn [52] and tune the hyperparameters

(C and γ) using 10-fold cross-validation within the train split. We train one classifier per

style and per business success indicator to predict the different labels.

Feature Set. We extract features from the chat interactions and sentiment lexica. The

feature set relies only on language usage. We extract the first word in a chat interaction,

the bag-of-words representations (binary flags and tf-idf scores) of the chat interaction, and

features from sentiment lexica. Specifically, we extract flags indicating whether the turn

has a positive, negative or neutral word in the list by [33], the sentiment score of the chat

interaction (summation of sentiment scores per token over the number of tokens), and a

flag indicating whether the interaction contains a negative word from the list by [39]. We

also extract other features, which include (a) the root verb (b) binary flags indicating the

presence of exclamation, question marks and negation cues from [49].

5.3.2. Multitask Learning (MTL) Setup

Transfer learning is an approach to machine learning which uses what has been learned

from past related tasks to facilitate learning a new task. Transfer learning thus leverages

previous experience to learn novel, but related concepts more efficiently [73]. Multitask

Learning is a variation of transfer learning that one task learns better by utilizing the in-

formation found in the training signals of other related tasks. The task of interest (target

task) and the related tasks (auxiliary or extra tasks) are learned in parallel, utilizing a

shared representation. Each task can help in learning other tasks. In a single task learning,

63

Page 74: Extracting Dimensions of Interpersonal Interactions and

information that would have been normally used as features, would rather be treated as

outputs instead in a multitask setup. Multitask Learning usually improves generalization

performance. Multitask Learning can be used with different learning algorithms and in dif-

ferent domains[18]. In our tasks of learning 3 business success indicators (has business,

business ever or business proposal), we leverage multitask learning by simultaneously

learning related auxiliary tasks, in our case, the dimensions of interactions (business, sen-

timent, cooperativeness, motivational, advice, equality). We use a standard Convolutional

Neural Network over word-embeddings, with one output per task. We preprocess the data

(convert to lowercase, removed URLs and stop-words, converted numbers to 0’s etc.) and

learn a skip-gram embeddings model [46] trained for 50 epochs. We use an embedding size

of 512, choosing a power of 2 for memory efficiency.

In the CNN, the input layer has the word indices of the text, converted via the embed-

ding matrix into a matrix of words x embeddings. We convolve two parallel channels with

max-pooling layers, and convolutional window sizes 4 and 8 over the input. The two window

sizes account for both short and relatively long patterns in the texts. In both channels, the

initial number of filters is 128 for the first convolution, and 256 in the second one. We join

the output of the convolutional channels and pass it through an attention mechanism [5, 66]

to emphasize the weight of any meaningful pattern recognized by the convolutions. We use

the implementation of [74]. The output consists of 7 independent, fully-connected layers for

the predictions, respectively in the form of discrete labels for classification of one of the busi-

ness success indicators of a person (has business, business ever or business proposal)

as the target task, and the dimensions of interactions (business, sentiment, cooperativeness,

motivational, advice, equality) as the auxiliary tasks. Figure 5.1 shows the architecture of

such a multitask learning network. We trained one model per business success indicator.

5.3.3. Results

Table 5.4 compares the results of the different systems to predict the dimensions of

interactions as well as the business success indicators. Our SVM model does much better

than majority baseline for all the dimensions of interactions (F-measures = 0.77 vs. 0.34,

64

Page 75: Extracting Dimensions of Interpersonal Interactions and

words x embeddings

Convolutional max-pooling layer window: 4, filters: 128

Convolutional max-pooling layer window: 8, filters: 128

Convolutional max-pooling layer window: 2, filters: 256

Convolutional max-pooling layer window: 4, filters: 256

Attention mechanismSize: 512

Fully connected

layer

Fully connected

layer

Fully connected

layer

Fully connected

layer

Fully connected

layer

Fully connected

layer

Fully connected

layer

Output 1 Output 2 Output 3 Output 4 Output 5 Output 6 Output 7

Architecture of the

Multi-task Learning set-up

Input

Output

Figure 5.1. Multitask Learning Architecture.

0.89 vs. 0.73, 0.78 vs. 0.67 and 0.86 vs. 0.79). For the business success indicators, either the

SVM (F-measures = 0.59, 0.38 and 0.64) or the MTL (F-measures = 0.63, 0.51 and 0.65)

model outperforms the majority baseline.

The above results indicate that an SVM model can be used to predict the dimen-

sions, and can thus be used for automatic annotations of the dimensions for the remaining.

The results we got are decent but not excellent. So far, we have used features based on

language only. We could incorporate other information as features as well. Extralinguistic

factors like demographic information (for example, age, gender information) can improve

the classification if included as features[36]. Table 5.3 shows correlations between certain

dimensions. Including one dimension as a feature for predicting another correlated dimen-

sion will improve the results. The Adansonia dataset is rich with other information like

65

Page 76: Extracting Dimensions of Interpersonal Interactions and

Model P R F

Coop.majority 0.25 0.50 0.34

SVM 0.77 0.77 0.77

Motivationmajority 0.66 0.81 0.73

SVM 0.90 0.90 0.89

Equalmajority 0.60 0.77 0.67

SVM 0.78 0.81 0.78

Advicemajority 0.74 0.86 0.79

SVM 0.86 0.88 0.86

has

business

majority 0.51 0.71 0.59

SVM 0.58 0.68 0.59

MTL 0.61 0.66 0.63

business

ever

majority 0.20 0.44 0.27

SVM 0.54 0.47 0.38

MTL 0.52 0.51 0.51

business

pro-

posal

majority 0.57 0.75 0.64

SVM 0.66 0.69 0.67

MTL 0.65 0.75 0.65

Table 5.4. Results for predicting styles of interactions and three indicators

of business success. Each value is a weighted average over the different labels

based on the number of instances falling in that label.

the educational background of a participant, their geographical background, language pro-

ficiency, etc. We can carry out a correlation analysis on these background information and

the business success indicators of the participants. If there are high correlations between any

of these background information and the business success indicators, to improve our results,

66

Page 77: Extracting Dimensions of Interpersonal Interactions and

we can either include the information as features in the SVM models or as auxiliary tasks in

the multitask learning setup. The multitask learning setup for a particular business success

indicator can also incorporate other business success indicators as well as the demographic

information as auxiliary tasks.

5.3.4. Automatic Annotation

The manually annotated data discussed in Section ?? was used to train an SVM

model with feature sets and results, as discussed in Section 5.3.1. This SVM model was used

to automatically annotate the rest of the Adansonia data. Here we will discuss the label

distributions of the automatically annotated data. For the dimension cooperativeness, 34.9%

are cooperative and 59.8% are neutral and only 5.15% are competitive. For the dimension

motivational, 10.7% are motivational and 85.9% are neutral. For the dimension advice, 5.5%

are advice and 91.7% are neutral . For the dimension Equality, 86.6% are equal and 9.7%

are hierarchical.

67

Page 78: Extracting Dimensions of Interpersonal Interactions and

CHAPTER 6

CONCLUSION

6.1. Our Achievements

In this dissertation, we have presented a set of nine dimensions of interpersonal in-

teractions and relationships, including dimensions with a long tradition in social science and

new ones. These dimensions allowed us to differentiate the core characteristics of interactions

and relationships between two individuals. For example, people that communicate may be

spatially near or spatially distant (asking questions in class vs. chatting online), and have

a pleasure-oriented or work-oriented relationship (somebody wishing good luck to a friend

vs. interviewer and interviewee). These dimensions are applicable to any interaction or

relationship regardless of the underlying name relationship type (e.g., siblings, friends,

doctor-patient).

We have put annotation effort on annotating these dimensions on data from a mixed

domain of news articles, TV interview transcripts. We also augmented our dimension an-

notation on TV series transcripts. We got an idea of how the values of the dimensions can

change. Our annotations show that assigning values to dimensions can be done reliably with

substantial Cohen’s Kappa values for inter-annotator agreements. Useful dimension values

(1 and -1 labels) are assigned to dimensions in most pairs of people. Experimental results

following a standard supervised machine-learning approach show that assigning values to di-

mensions can be automated ( 0.72 F-measures for both the previous works). We believe that

extracting dimensions of interpersonal interactions and relationships complements previous

efforts that extract relationships.

We have also studied values of dimensions for pairs of people defined slightly differ-

ently than the previous works, where we consider the speaker and the person being spoken

to forming the pairs of people for whom we will annotate dimensions of interactions and

relationships. We also use social acts to further characterize cooperative and competitive

interactions (five and six acts respectively). We work with dialogue transcripts of the first

68

Page 79: Extracting Dimensions of Interpersonal Interactions and

season of the popular TV show Friends, this time with the first 12 episodes. It is analyzed

to see if dimensions and social acts are intuitive to humans by calculating Cohen’s Kappa

for agreements. Beyond numerical analysis, we also discuss salient words for social acts and

the label distribution of both dimensions and social acts. We try simple SVM trained with

bag-of-words features and other features derived from sentiment to confirm if learning is

possible or not. We also experiment with features from gold annotations of the previous and

current utterance (other dimensions and social acts) to see whether that improves results or

not.

We also will study how young aspiring entrepreneurs exchange messages with each

other on online platforms as well as in person. Annotation projects are conducted to annotate

interaction dimensions like equality, coopertiveness, and other interaction styles indicating

motivation and advice on about 5000 chat interactions. SVM models are trained and tested

on these annotated data to see if these dimensions or styles can be reliably predicted or not.

The rest of the data from a large corpus of 140,000 data are then annotated with these values.

The manually annotated data is also used in a multitask learning set-up to predict business

success indicators denoting whether a user has applied for a business funding, whether the

user already has a business, and whether they have ever been involved in business.

6.2. Applications and Future Directions

Two people usually have a constant named relationship during their lifetime or a

good portion of it, but their interpersonal interactions can be described with different values

of the dimensions at different points in time. Extraction of these values can help us build

trajectories of relationships between pairs of people over a course of time [40]. Works like

ours can be used to analyze characters of novels or other literary works. Also, a similar

analysis can be used to analyze historical characters and their relationships with others

around them. It can help predict how people will act in certain circumstances. Political

figures can also be analyzed similarly, and their moves may be predicted. Extracting in-depth

information about people’s relationships will inform us more about people of historical or

political importance. Question Answering about people will be benefitted from this. Search

69

Page 80: Extracting Dimensions of Interpersonal Interactions and

Engines and Information Retrieval systems can incorporate our idea.

We also found that business success can be predicted using information extracted

from their spoken words and the values of the dimensions of their relationship. Other

background information about these people can farther be added in a controlled fashion

to see if they contribute to the learning of business success. This also gives us hope that

business success, it may be possible to learn other traits like success in politics, success in

academia, job, etc. This can help people to know what qualities or practices they should

work on to ensure success in life. People can avoid events or behavioral characteristics that

show a correlation with failure. Previously, cluster analyses on the behavioral profiles of

pairs of people indicated that there are three friendship types, and the friendship types they

are involved in tell us how adjusted they are in the society [31].

An aggregate of people work as a system, and the ones involved are similar and

share similar fates, choices, course of life, etc. [16]. The values of the dimensions can

be used to cluster people into different groups based on how they maintain relationships

with people around them. This will give us an idea of the system they belong to, which

will then reveal a lot about the people’s behavioral characteristics. We can predict their

lifestyle choices. This may give us a lead to what products the person may be interested

in. The clustering technique may be used to predict the socio-economic background of

the users. This can further be used to predict their financial ability. This will be helpful

information for advertising companies and other businesses so that the right products can be

advertised to them. As the saying goes, ”birds of the same feather flock together”, looking

at whom a person mingles with will also give an idea about what they like and can afford.

Recommendation systems for movies, music, etc. can also utilize the information to predict

what a person may be interested in watching or listening to. Behavioral analysis can also

help predict whom a person is comfortable with, whom they can be friends with. This can

be used by friend recommendation systems in social media, match-making sites, etc. If a

person is mostly friendly, intensely involved for a long time with another person, are in close

proximity most of the time or frequently, it is likely that these two people may enjoy similar

70

Page 81: Extracting Dimensions of Interpersonal Interactions and

entertainment, similar food, may be interested in similar services, similar or closeby stores,

similar restaurants, etc. They may plan to spend their holidays together in the same places.

Thus, airline and travel companies can use information about relationships between people

to suggest trip offers better.

71

Page 82: Extracting Dimensions of Interpersonal Interactions and

APPENDIX A

RESULT CALCULATION

72

Page 83: Extracting Dimensions of Interpersonal Interactions and

Features Label P R F

Majority

Baseline

1 0.43 0.76 0.55

0 0.00 0.00 0.00

-1 0.57 0.59 0.64

Avg. 0.45 0.65 0.53

Verb

1 0.62 0.72 0.64

0 0.00 0.00 0.00

-1 0.71 0.78 0.65

Avg. 0.62 0.71 0.65

Verb, Personx

1 0.68 0.75 0.69

0 0.00 0.00 0.00

-1 0.78 0.79 0.78

Avg. 0.70 0.74 0.71

Verb, Persony

1 0.64 0.72 0.67

0 0.00 0.00 0.00

-1 0.74 0.76 0.74

Avg. 0.66 0.70 0.67

Verb, Personx,

Persony,

Personx Persony

1 0.69 0.74 0.70

0 0.00 0.00 0.00

-1 0.77 0.80 0.76

Avg. 0.71 0.76 0.72

Table A.1. Results obtained for all dimensions with several combinations of

features for the Ontonotes Dataset.

Table A.1 shows the results for different feature combinations for experiments on the

Ontonotes data (same as Table 4.15). Table A.2 shows the detailed results obtained for each

of the dimensions with the best feature combinations, that is the result bolded in A.1 (similar

to Table 4.16). In Table A.2 for each row with a dimension name, we show the precision

73

Page 84: Extracting Dimensions of Interpersonal Interactions and

Dimension1 (1st desc.) 0 (unk.) -1 (2nd desc.) All

P R F P R F P R F P R F

Cooperative 0.73 0.96 0.83 0.00 0.00 0.00 0.60 0.19 0.29 0.66 0.72 0.65

Equal 0.56 0.10 0.17 0.00 0.00 0.00 0.74 0.97 0.84 0.68 0.74 0.66

Intense 0.39 0.30 0.34 0.00 0.00 0.00 0.78 0.85 0.82 0.67 0.71 0.69

Pleasure 0.40 0.28 0.33 0.00 0.00 0.00 0.87 0.93 0.90 0.79 0.82 0.80

Active 0.69 0.85 0.76 0.00 0.00 0.00 0.68 0.51 0.58 0.67 0.69 0.67

Intimate 0.44 0.17 0.24 0.00 0.00 0.00 0.88 0.98 0.93 0.81 0.86 0.83

Temporary 0.85 0.96 0.91 0.00 0.00 0.00 0.33 0.10 0.16 0.77 0.83 0.79

Concurrent 0.72 0.80 0.76 0.00 0.00 0.00 0.77 0.75 0.76 0.71 0.74 0.73

Spat. Near 0.66 0.68 0.67 0.00 0.00 0.00 0.73 0.79 0.76 0.66 0.70 0.68

Average 0.69 0.74 0.70 0.00 0.00 0.00 0.77 0.76 0.77 0.71 0.76 0.72

Table A.2. Results obtained for each dimension with the best combination

of features for all dimensions (Verb + Personx + Persony + Personx Persony,

boldfaced in Table A.1).

(P), recall (R) and F-measures (F) of the three labels 1 (first descriptor), 0 (unknown) and

-1 (second descriptor) respectively. The last three columns show the weighted averages of

these P, R, and F values based on the number of instances falling under the different labels

respectively for each dimension. The F-measure in the last column is not simply the harmonic

mean of the Precision and Recall values in the previous two columns, rather the weighted

averages of the F-measures obtained from each of the labels 1, 0 and -1. Thus in some of

the cases, it is seen that the F-measure in the last column does not always fall between

the Precision and Recall values found in the previous two columns. In the second last row,

for each column, we show the weighted averages of that column based on the number of

instances falling under each dimension.

As the weighted average calculation may create confusions about the values, we are

also showing the macro averages of the P, R and F values. Tables A.3, A.4 and A.5 show

74

Page 85: Extracting Dimensions of Interpersonal Interactions and

DimensionMacro Average

P R F

Cooperative 0.44 0.38 0.41

Equal 0.43 0.35 0.39

Intense 0.39 0.38 0.38

Pleasure 0.42 0.40 0.41

Active 0.45 0.45 0.45

Intimate 0.44 0.38 0.41

Temporary 0.39 0.35 0.37

Concurrent 0.50 0.52 0.51

Near 0.46 0.49 0.47

Average 0.44 0.41 0.42

Table A.3. Macro averages of the Precision, Recall and F-measure respec-

tively over the different labels for each dimension shown in Table A.2.

the corresponding macro averages of the values found in Tables A.2, 4.18 and 4.20. The

macro averages of the P and R are calculated as straigth-forward unweighted averages over

the labels for each dimension. The F for each dimension is the harmonic mean of these

calculated average P and the R values. Here the values are considerably lower as the values

from the labels which had low values and had very few number of instances (label 0) are

also included without a weight incorporated.

75

Page 86: Extracting Dimensions of Interpersonal Interactions and

DimensionMacro Average

P R F

Cooperative 0.38 0.35 0.46

Active 0.84 0.83 0.83

Concurrent 0.81 0.80 0.81

Spat. Near 0.79 0.76 0.77

Equal 0.33 0.34 0.33

Intense 0.45 0.46 0.45

Pleasure 0.60 0.34 0.44

Intimate 0.43 0.42 0.42

Temporary 0.60 0.48 0.54

Table A.4. Macro averages of the Precision, Recall and F-measure respec-

tively over the different labels for each dimension shown in Table 4.18.

DimensionMacro Average

P R F

Cooperative 0.41 0.37 0.37

Equal 0.57 0.42 0.48

Intense 0.58 0.42 0.49

Intimate 0.56 0.42 0.48

Temporary 0.65 0.64 0.64

Table A.5. Macro averages of the Precision, Recall and F-measure respec-

tively over the different labels for each dimension shown in Table 4.20.

76

Page 87: Extracting Dimensions of Interpersonal Interactions and

APPENDIX B

PUBLICATIONS

77

Page 88: Extracting Dimensions of Interpersonal Interactions and

The research presented in this dissertation has resulted in the following publications:

(1) Farzana Rashid and Eduardo Blanco. 2017. Dimensions of Interpersonal Relation-

ships: Corpus and Experiments. In Proceedings of the 2017 Conference on Empirical

Methods in Natural Language Processing (EMNLP). Copenhagen, Denmark.

(2) Farzana Rashid and Eduardo Blanco. 2018. Characterizing Interactions and Re-

lationships between People. In Proceedings of the 2018 Conference on Empirical

Methods in Natural Language Processing (EMNLP). Brussels, Belgium.

78

Page 89: Extracting Dimensions of Interpersonal Interactions and

REFERENCES

[1] Sinnet: Social interaction network extractor from text, The Companion Volume of the

Proceedings of IJCNLP 2013: System Demonstrations, 2013, pp. 33–36.

[2] John Adamopoulos, The emergence of social meaning: A theory of action construal,

pp. 255–272, Springer New York, New York, NY, 2012.

[3] Jacqueline Aguilar, Charley Beller, Paul McNamee, Benjamin Van Durme, Stephanie

Strassel, Zhiyi Song, and Joe Ellis, A comparison of the events and relations across

ace, ere, tac-kbp, and framenet annotation standards, Proceedings of the Second Work-

shop on EVENTS: Definition, Detection, Coreference, and Representation (Baltimore,

Maryland, USA), Association for Computational Linguistics, June 2014, pp. 45–53.

[4] Ron Artstein and Massimo Poesio, Inter-coder agreement for computational linguistics,

Comput. Linguist. 34 (2008), no. 4, 555–596.

[5] Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio, Neural machine translation

by jointly learning to align and translate, 2014, cite arxiv:1409.0473Comment: Accepted

at ICLR 2015 as oral presentation.

[6] A. Bajracharya, S. Shrestha, S. Upadhyaya, B. Shrawan, and S. Shakya, Automated

characters recognition and family relationship extraction from stories, 2018 8th Interna-

tional Conference on Cloud Computing, Data Science Engineering (Confluence), 2018,

pp. 14–15.

[7] David Bamman, Brendan OConnor, and Noah A Smith, Learning latent personas of

film characters, Proceedings of the 51st Annual Meeting of the Association for Compu-

tational Linguistics (Volume 1: Long Papers), 2013, pp. 352–361.

[8] Emily M. Bender, Jonathan T Morgan, Meghan Oxley, Mark Zachary, Brian Hutchin-

son, Alex Marin, Bin Zhang, and Mari Ostendorf, Annotating social acts: Authority

claims and alignment movesin wikipedia talk pages, Proceedings of the Workshop on

Language in Social Media, June 2011, pp. 48–57.

79

Page 90: Extracting Dimensions of Interpersonal Interactions and

[9] Bernard E. Whitley, Mary E. Kite, and Heather L. Adams, Principles of research in

behavioral science, Routledge, Ney York, 2018.

[10] Eduardo Blanco and Dan Moldovan, Finding associations between people.

[11] David B. Bracewell, Marc T. Tomlinson, Mary Brunson, Jesse Plymale, Jiajun

Bracewell, and Daniel Boerger, Annotation of adversarial and collegial social actions

in discourse, Proceedings of the Sixth Linguistic Annotation Workshop, 2012, pp. 184–

192.

[12] David B. Bracewell, Marc T. Tomlinson, and Hui Wang, Identification of social acts in

dialogue, Proceedings of COLING 2012: Technical Papers, 2012, pp. 375–390.

[13] Philip Bramsen, Martha Escobar-Molano, Ami Patel, and Rafael Alonso, Extracting so-

cial power relationships from natural language, Proceedings of the 49th Annual Meeting

of the ACL, 2011, pp. 773–782.

[14] Cynthia Breazeal, Role of expressive behaviour for robots that learn from people, Philo-

sophical Transactions of the Royal Society B: Biological Sciences 364 (2009), no. 1535,

3527–3538.

[15] B Brewer and W Gardner, Who is this we? levels of collective identity and self repre-

sentations, 71(1) (1996), 83–93.

[16] Donald T Campbell, Common fate, similarity, and other indices of the status of aggre-

gates of persons as social entities, Behavioral science 3 (1958), no. 1, 14–25.

[17] M. A. Campion, E. M. Papper, and G. J. Medsker, Relations between work team char-

acteristics and effectiveness: A replication and extension., (1996), 429–452.

[18] Rich Caruana, Multitask learning, Machine Learning 28 (1997).

[19] Snigdha Chaturvedi, Structured approaches for exploring interpersonal relationships in

natural language text, Ph.D. thesis, 01 2016.

[20] Yu-Hsin Chen and Jinho D. Choi, Character identification on multiparty conversation:

Identifying mentions of characters in tv shows, Proceedings of the 17th Annual Meeting

of the Special Interest Group on Discourse and Dialogue, 2016, pp. 90–100.

80

Page 91: Extracting Dimensions of Interpersonal Interactions and

[21] Kevin H. C. Cheng, Further linguistic markers of personality: The way we say things

matters, 2011.

[22] Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg, Echoes

of power: Language effects and power differences in social interaction, Proceedings of

WWW, WWW ’12, ACM, 2012, pp. 699–708.

[23] Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and

Christopher Potts, A computational approach to politeness with application to social

factors, Proceedings of the ACL, August 2013, pp. 250–259.

[24] Lea Deleris, Francesca Bonin, Elizabeth Daly, Stephane Deparis, Yufang Hou, Charles

Jochim, Yassine Lassoued, and Killian Levacher, Know who your Friends are: Un-

derstanding social connections from unstructured text, Proceedings of the 2018 Confer-

ence of the North American Chapter of the Association for Computational Linguistics:

Demonstrations (New Orleans, Louisiana), Association for Computational Linguistics,

June 2018, pp. 76–80.

[25] Morton Deutsch, Interdependence and psychological orientation, Conflict, Interdepen-

dence, and Justice, Springer, 2011, pp. 247–271.

[26] George Doddington, Alexis Mitchell, Mark Przybocki, Lance Ramshaw, Stephanie

Strassel, and Ralph Weischedel, The automatic content extraction (ace) program tasks,

data, and evaluation, Proceedings of LREC, May 2004.

[27] David K Elson, Nicholas Dames, and Kathleen R McKeown, Extracting social networks

from literary fiction, Proceedings of the 48th annual meeting of the association for

computational linguistics, Association for Computational Linguistics, 2010, pp. 138–

147.

[28] Eric Gilbert, Phrases that signal workplace hierarchy, Proceedings of the ACM 2012

Conference on Computer Supported Cooperative Work (New York, NY, USA), CSCW

’12, ACM, 2012, pp. 1037–1046.

[29] Juanita Gonzalez-Uribe and Michael Leatherbee, The Effects of Business Accelerators

81

Page 92: Extracting Dimensions of Interpersonal Interactions and

on Venture Performance: Evidence from Start-Up Chile, The Review of Financial Stud-

ies 31 (2017), no. 4, 1566–1603.

[30] Ralph Grishman and Beth Sundheim, Message understanding conference-6: A brief

history, Proceedings of the 16th Conference on Computational Linguistics - Volume

1 (Stroudsburg, PA, USA), COLING ’96, Association for Computational Linguistics,

1996, pp. 466–471.

[31] Berna Guroglu, Cornelis FM Van Lieshout, Gerbert JT Haselager, and Ron HJ Scholte,

Similarity and complementarity of behavioral profiles of friendship types and types of

friends: Friendships and psychosocial adjustment, Journal of Research on Adolescence

17 (2007), no. 2, 357–386.

[32] I. A. Hameed, Using natural language processing (nlp) for designing socially intelligent

robots, 2016 Joint IEEE International Conference on Development and Learning and

Epigenetic Robotics (ICDL-EpiRob), 2016, pp. 268–269.

[33] William L. Hamilton, Kevin Clark, Jure Leskovec, and Dan Jurafsky, Inducing domain-

specific sentiment lexicons from unlabeled corpora, Proceedings of EMNLP, November

2016, pp. 595–605.

[34] J. Hauffa, G. Bossert, N. Richter, F. Wolf, N. Liesenfeld, and G. Groh, Beyond foaf:

Challenges in characterizing social relations, 2011 IEEE Third International Conference

on Privacy, Security, Risk and Trust and 2011 IEEE Third International Conference on

Social Computing, 2011, pp. 790–795.

[35] Hua He, Denilson Barbosa, and Grzegorz Kondrak, Identification of speakers in novels,

Proceedings of the 51st Annual Meeting of the Association for Computational Linguis-

tics (Volume 1: Long Papers), 2013, pp. 1312–1320.

[36] Dirk Hovy, Demographic factors improve classification performance, Proceedings of the

53rd Annual Meeting of the Association for Computational Linguistics and the 7th In-

ternational Joint Conference on Natural Language Processing (Volume 1: Long Papers)

(Beijing, China), Association for Computational Linguistics, July 2015, pp. 752–762.

[37] Dirk Hovy, Taylor Berg-Kirkpatrick, Ashish Vaswani, and Eduard Hovy, Learning whom

82

Page 93: Extracting Dimensions of Interpersonal Interactions and

to trust with MACE, Proceedings of the 2013 Conference of the North American Chap-

ter of the Association for Computational Linguistics: Human Language Technologies

(Atlanta, Georgia), Association for Computational Linguistics, June 2013, pp. 1120–

1130.

[38] Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph

Weischedel, OntoNotes: the 90% Solution, NAACL ’06: Proceedings of the Human

Language Technology Conference of the NAACL, Companion Volume: Short Papers on

XX (Morristown, NJ, USA), Association for Computational Linguistics, 2006, pp. 57–

60.

[39] Minqing Hu and Bing Liu, Mining and summarizing customer reviews, Proceedings of

the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data

Mining (New York, NY, USA), KDD ’04, ACM, 2004, pp. 168–177.

[40] Mohit Iyyer, Anupam Guha, Snigdha Chaturvedi, Jordan Boyd-Graber, and Hal

Daume III, Feuding families and former friends: Unsupervised learning for dynamic

fictional relationships, Proceedings of NAACL, June 2016, pp. 1534–1544.

[41] K. A. Jehn and E. A. Mannix, The dynamic nature of conflict: A longitudinal study of

intragroup conflict and group performance, 44(2) (2001), 238.

[42] Harold H Kelley, Personal relationships: Their structures and processes, Psychology

Press, 2013.

[43] Dimitrios Kokkinakis, Annotation of interpersonal relations in swedish prose fiction.,

Proceedings of the 3rd Workshop on Annotation of Corpora for Research in the Hu-

manities (ACRH-3). Sofia, Bulgaria., 2013, pp. 37–47.

[44] Seth Kulick, Ann Bies, and Justin Mott, Inter-annotator agreement for ere annotation,

Proceedings of the Second Workshop on EVENTS: Definition, Detection, Coreference,

and Representation (Baltimore, Maryland, USA), Association for Computational Lin-

guistics, June 2014, pp. 21–25.

[45] K. Maki, M. Yoder, Y. Jo, and C. Ros, Roles and success in wikipedia talk pages:

83

Page 94: Extracting Dimensions of Interpersonal Interactions and

Identifying latent patterns of behavior, Proceedings of the Eighth International Joint

Conference on Natural Language Processing, November 2017, pp. 1026–1035.

[46] Tomas Mikolov, Scott Wen-tau Yih, and Geoffrey Zweig, Linguistic regularities in con-

tinuous space word representations, Proceedings of the 2013 Conference of the North

American Chapter of the Association for Computational Linguistics: Human Lan-

guage Technologies (NAACL-HLT-2013), Association for Computational Linguistics,

May 2013.

[47] Nikola Milosevic, A multi-layered approach to information extraction from tables in

biomedical documents, 2018.

[48] Sangwoo Moon and Hairong Qi, A hybrid feature extraction framework based on risk

minimization and independence maximization, 06 2009, pp. 2141–2144.

[49] Roser Morante and Walter Daelemans, Conandoyle-neg: Annotation of negation cues

and their scope in conan doyle stories, Proceedings of LREC, 2012, ACL Anthology

Identifier: L12-1077, pp. 1563–1568.

[50] Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, and Mark Johnson, Stranse: a novel em-

bedding model of entities and relationships in knowledge bases, Proceedings of the 2016

Conference of the North American Chapter of the Association for Computational Lin-

guistics: Human Language Technologies (San Diego, California), Association for Com-

putational Linguistics, June 2016, pp. 460–466.

[51] D. A. Owens and R. I. Sutton, Status contests in meetings: Negotiating the informal

order., (2001), 299–316.

[52] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel,

P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau,

M. Brucher, M. Perrot, and E. Duchesnay, Scikit-learn: Machine learning in Python,

Journal of Machine Learning Research 12 (2011), 2825–2830.

[53] Vinodkumar Prabhakaran and Owen Rambow, Predicting power relations between par-

ticipants in written dialog from a single thread, Proceedings of the ACL, 2014, pp. 339–

344.

84

Page 95: Extracting Dimensions of Interpersonal Interactions and

[54] Yanjun Qi, Bing Bai, Xia Ning, and Pavel Kuksa, Systems and methods for semi-

supervised relationship extraction, October 28 2014, US Patent 8,874,432.

[55] S Ramaraju et al., Psychological perspectives on interpersonal communication, Journal

of Arts, Science & Commerce 3 (2012), no. 4, 68–73.

[56] Farzana Rashid and Eduardo Blanco, Characterizing interactions and relationships be-

tween people, Proceedings of EMNLP, 2018, pp. 4395–4404.

[57] Harry T Reis, W Andrew Collins, and Ellen Berscheid, The relationship context of

human behavior and development., Psychological bulletin 126 (2000), no. 6, 844.

[58] David R Schmitt, Interpersonal relations: Cooperation and competition, Journal of the

Experimental Analysis of Behavior 42 (1984), no. 3, 377–383.

[59] Hinrich Schutze, Christopher D Manning, and Prabhakar Raghavan, Introduction to

information retrieval, vol. 39, Cambridge University Press, 2008.

[60] Nicole Shechtman and Leonard M. Horowitz, Interpersonal and noninterpersonal in-

teractions, interpersonal motives, and the effect of frustrated motives, Personality and

Social Psychology Bulletin 32 (2006), no. 8, 1126–1139, PMID: 16861315.

[61] Mark Snyder and Arthur A Stukas Jr, Interpersonal processes: The interplay of cog-

nitive, motivational, and behavioral activities in social interaction, Annual review of

psychology 50 (1999), no. 1, 273–303.

[62] Shashank Srivastava, Snigdha Chaturvedi, and Tom Michael Mitchell, Inferring inter-

personal relations in narrative summaries, AAAI, 2015.

[63] Ang Sun, Ralph Grishman, and Satoshi Sekine, Semi-supervised relation extraction with

large-scale word clustering, Proceedings of the 49th Annual Meeting of the Associa-

tion for Computational Linguistics: Human Language Technologies (Portland, Oregon,

USA), Association for Computational Linguistics, June 2011, pp. 521–529.

[64] Mihai Surdeanu and Ji Heng, Overview of the english slot filling track at the tac2014

knowledge base population evaluation, Proceedings of the TAC-KBP 2014 Workshop,

2014.

85

Page 96: Extracting Dimensions of Interpersonal Interactions and

[65] Yla Tausczik and James Pennebaker, Participation in an online mathematics commu-

nity: Differentiating motivations to add, 02 2012, pp. 207–216.

[66] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N.

Gomez, Lukasz Kaiser, and Illia Polosukhin, Attention is all you need, CoRR

abs/1706.03762 (2017).

[67] Fernando Vega-Redondo, Paolo Pin, Diego Ubfal, Cristiana Benedetti-Fasil, Charles

Brummitt, Gaia Rubera, Dirk Hovy, and Tommaso Fornaciari, Peer Networks and En-

trepreneurship: a Pan-African RCT, Working Papers 648, IGIER (Innocenzo Gasparini

Institute for Economic Research), Bocconi University, 2019.

[68] Nikos Voskarides, Edgar Meij, Manos Tsagkias, Maarten de Rijke, and Wouter

Weerkamp, Learning to explain entity relationships in knowledge graphs, Proceedings

of ACL, July 2015, pp. 564–574.

[69] Stephanie Wagener, Marjan Gorgievski, and Serge Rijsdijk, Businessman or host? in-

dividual differences between entrepreneurs and small business owners in the hospitality

industry, The Service Industries Journal 30 (2010), no. 9, 1513–1527.

[70] Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and

Dekang Lin, Knowledge base completion via search-based question answering, Proceed-

ings of the 23rd International Conference on World Wide Web (New York, NY, USA),

WWW ’14, ACM, 2014, pp. 515–526.

[71] Myron Wish, Morton Deutsch, and Susan J Kaplan, Perceived dimensions of interper-

sonal relations., Journal of Personality and Social Psychology 33 (1976), no. 4, 409.

[72] Fei Wu and Daniel S. Weld, Open information extraction using wikipedia, Proceedings of

the 48th Annual Meeting of the Association for Computational Linguistics (Stroudsburg,

PA, USA), ACL ’10, Association for Computational Linguistics, 2010, pp. 118–127.

[73] Liu Yang, Steve Hanneke, and Jaime Carbonell, A theory of transfer learning with

applications to active learning, Machine Learning 90 (2013).

[74] Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy, Hier-

archical attention networks for document classification, Proceedings of the 2016 Confer-

86

Page 97: Extracting Dimensions of Interpersonal Interactions and

ence of the North American Chapter of the Association for Computational Linguistics:

Human Language Technologies (San Diego, California), Association for Computational

Linguistics, June 2016, pp. 1480–1489.

[75] Xuchen Yao and Benjamin Van Durme, Information extraction over structured data:

Question answering with freebase, Proceedings of the 52nd Annual Meeting of the Asso-

ciation for Computational Linguistics (Volume 1: Long Papers) (Baltimore, Maryland),

Association for Computational Linguistics, June 2014, pp. 956–966.

[76] Xiaofeng Yu and Wai Lam, Jointly identifying entities and extracting relations in en-

cyclopedia text via a graphical model approach, Proceedings of the 23rd International

Conference on Computational Linguistics: Posters (Stroudsburg, PA, USA), COLING

’10, Association for Computational Linguistics, 2010, pp. 1399–1407.

[77] Hsun yu Chan and Xueli Wang, Interact for what? the relationship between inter-

personal interaction based on motivation and educational outcomes among students in

manufacturing programs at two-year technical colleges, Community College Review 44

(2016), no. 1, 26–48.

87