Creating and Using a Correlated Corpora to Glean
Communicative Commonalities
Jade Goldstein-Stewart Kerri A. Goodwin Roberta E. Sabin Ransom K. Winder
U.S. Dept. of Defense Loyola College Loyola College MITRE Corporation
May 30, 2008 LREC 2
Outline• Motivation
• Corpora collection
• General Corpora Characteristics– Word count– Readability
• Future directions
May 30, 2008 LREC 3
Motivation
• How do computer-mediated communication genres differ from traditional genres?
email interview
blog essay
chat discussion
• How consistent are communicative features across genres for a single individual?
• If such commonalities exist, how can they be utilized for document classification?
May 30, 2008 LREC 4
Email sample (2E1S3)
I do not feel that gender discrimination is a problem in the United States at the moment. My supervisor at my current job is a woman, and everyone respects her the same as the owner of the company, who is a man. I think this issue was more prevalent earlier last century. In these modern times, it really is not an issue in my opinion.
May 30, 2008 LREC 5
Blog sample (2B1S2)
While gender discrimination is something that should always be avoided ideally, there are some problems I have with the issue in general. As the discussion starter states, discrimination because of sex is defined as adverse action against another person, that would not have occurred had the person been of another sex.
May 30, 2008 LREC 6
Chat sample (2C1S1)
– Are there a lot of issues like this in the news, because to me generder discrimination is a thing of the past
– Aren't men found to be naturally more apt in certain fields, and women in others?
– Did any of you experienece any personal discrimination at your jobs, or witness it or anything?
– I definitely agree with that– Unless one person decides another person is not right for a
job solely based on gender, I don't believe it is discrimination
May 30, 2008 LREC 7
Aim: Collect a correlated corpora of
text samples
• Including both computer-mediated and not c-m • Including both individual and interactive, spoken
and text• Across 6 genres:
– email, essay, interview (phone) – blog, chat, discussion
• From the same individuals• On 6 distinct topics
May 30, 2008 LREC 8
Corpora CollectionSeptember 2006 through November 2007
Participants
• All college students, aged 18-29• 12 students in pilot study• 21 participants completed both Phase 1 (email, essay,
interview) & Phase 2 (blog, chat, discussion) • 10M/11W• 18 Caucasian/3 African-American • all had English as the primary language spoken at home
May 30, 2008 LREC 9
Topics• Piloted via individual interviews with a separate
group• Selected for
– production of expression– comfort of participates for the topic
• Topics: 1. Catholic Church2. Gay Marriage3. Iraq War4. Legalization of Marijuana5. Privacy as a U.S. Citizen6. Gender Discrimination
• Each introduced via a “starter” question
May 30, 2008 LREC 10
Other Design Issues
• Individual instructions standardized• Environments controlled
– In-house email system– Single discussion leader and phone interviewer– Relaxed discussion and interview setting– Chat sessions “gently” moderated
• Ordering of genres and topics controlled• Group membership randomized
– gender balance 2M/2W
May 30, 2008 LREC 11
All .txt files produced
• Interviews and Discussions transcribed– by trained psychology students– punctuation inserted– non-fluencies preserved
• Discussion and Chat dismembered to individual files
• Multiple blog entries combined to a single file
May 30, 2008 LREC 12
Resulting Corpora
•Blogs entries were combined into single files.
The 21 fully parallel corpora were used in this paper.
Limitations: size, homogeneity of subjects, non-spontaneity of discourse
Totals Emails Essays Interviews Blogs* Chat Discussion All180 180 186 132 132 132 942
21 fully parallel copora 126 126 126 126 126 126 756
From Same Individuals
May 30, 2008 LREC 13
General Corpora Characteristics
• Word Count– by topic – by genre – by gender of communicant
• Readability: Flesch reading ease & Flesch-Kincaid grade level– by topic – by genre – by gender of author
May 30, 2008 LREC 14
Word Count • No main effect for gender• No main effect for topic• Significant topic x gender interaction for Church and Discrimination
500
550
600
650
700
Church Gay Iraq Marij Privacy Discrim
TOPIC
ME
AN
WO
RD
CO
UN
T
Men
Women
Combined
May 30, 2008 LREC 15
Word Count (con’t)• Significant Main Effect for genre• Discussion had highest word counts• Direct communication produced higher word counts
100
300
500
700
900
1100
1300
E mailE s say Inter B log C hat D is c
G enre
Me
an
Wo
rd C
ou
nt
Men
W omen
C ombined
May 30, 2008 LREC 16
Readability• No significant main effect for gender• Significant main effect for genre
– Discussion and interview had highest reading ease– Main effect for topic
65
67
69
71
73
75
77
79
81
Church Gay Iraq Marij Privacy SexDis
TOPIC
ME
AN
FL
ES
CH
RE
AD
ING
EA
SE
Men
Women
Combined
May 30, 2008 LREC 17
Readability (con’t)• reading ease of conversational genres high• reading ease of non-conversational genres low
0
2
4
6
8
10
12
C hurc h G ay Iraq Marijuana P rivac y S exDis c
TO P IC
GR
AD
E L
EV
EL
E mail
E s s ay
Interview
B log
C hat
Dis c
May 30, 2008 LREC 18
Future Possibilities
• additional features for genderID, authorship • sentence complexity• cohesion of text
• feature change across time within a topic• classification by topic order
• classification by genre
• conversational dynamics in chat vs. discussion
May 30, 2008 LREC 19
Thank you.
Questions?
www.cs.loyola.edu/~res