a collusion-resistant automation scheme for social moderation systems
DESCRIPTION
For current Web 2.0 services, manual examination of user uploaded content is normally required to ensure its legitimacy and appropriateness, which is a substantial burden to service providers. To reduce labor costs and the delays caused by content censoring, social moderation has been proposed as a front-line mechanism, whereby user moderators are encouraged to examine content before system moderation is required. Given the immerse amount of new content added to the Web each day, there is a need for automation schemes to facilitate rear system moderation. This kind of mechanism is expected to automatically summarize reports from user moderators and ban misbehaving users or remove inappropriate content whenever possible. However, the accuracy of such schemes may be reduced by collusion attacks, where some work together to mislead the automatic summarization in order to obtain shared benefits. In this paper, we propose a collusion-resistant automation scheme for social moderation systems. Because some user moderators may collude and dishonestly claim that a user misbehaves, our scheme detects whether an accusation from a user moderator is fair or malicious based on the structure of mutual accusations of all users in the system. Through simulations we show that collusion attacks are likely to succeed if an intuitive countbased automation scheme is used. The proposed scheme, which is based on the community structure of the user accusation graph, achieves a decent performance in most scenarios.TRANSCRIPT
A Collusion-Resistant Automation Scheme for Social Moderation Systems
Jing-Kai Lou12, Kuan-Ta Chen1, Chin-Laung Lei2
Institute of Information Science, Academia Sinica1
Department of Electrical Engineering, National Taiwan University2
CCNC
The Rise of User-Generated Content
• eMarketer projects that the number of US UGC creators will rise to 108 million in 2012, from 77 million in 2007.
2009/1/12 2CCNC 2009 / Jing‐Kai Lou
Inappropriate UGC
• While most UGC creators behave responsibly, a minority of creators may upload inappropriate content, such like• pictures that violate copyright laws• splatter movies• …
• Content censorship is essential for Web 2.0 services
• One Solution: • hiring lots of official moderators• But, such high labor cost is a great burden to the service provider
• Another Solution:Social moderation has been proposed to solve the content censorship problem
2009/1/12 3CCNC 2009 / Jing‐Kai Lou
Social Moderation System
• A user-assist moderation• Every user is a reviewer!
BloggerBloggerAlbumAlbum VideoVideo
????
??!?!?
OO XXXX
Official moderators inspect and evaluate what your reportYou report/accuse
while browsing
XX
2009/1/12 4CCNC 2009 / Jing‐Kai Lou
Is Social Moderation good enough?
• Advantages of social moderation system:1. Fewer official moderators2. Detecting inappropriate content quickly
• BUT, the number of the reports is still large.• Even 1% uploading photos in Flickr are problematic,
there are about 43,200 reports each day.
• Can we help?
2009/1/12 5CCNC 2009 / Jing‐Kai Lou
Social Moderation Automation
• This is our motivation for proposing social moderation automation, which automatically summarizes the reports submitted by users.
• A preprocess:For eliminating manual inspection by official moderators as much as possible.
2009/1/12 CCNC 2009 / Jing‐Kai Lou 6
There is an intuitive way…
• Count-based Scheme identifies misbehaving users by considering the number of accusations (reports).
37473
2009/1/12 7CCNC 2009 / Jing‐Kai Lou
These photos are accused no more than (N =20) users
These photos are accused more than (N =20) users
However, there are many colluders…2009/1/12 8CCNC 2009 / Jing‐Kai Lou
Not All Users Are Trustable
•• While most users report responsibly, While most users report responsibly, colluders colluders report fake results report fake results to gain some benefits.to gain some benefits.
•• CountedCounted--based scheme may misidentify!based scheme may misidentify!
2009/1/12 9CCNC 2009 / Jing‐Kai Lou
Research Question
• CAN we automatically infers which accusations (reports) are fair or malicious?
• Need a better automation scheme to deal with collusion attacks
2009/1/12 10CCNC 2009 / Jing‐Kai Lou
Our Scheme
• Community-based scheme analyzes the accusation relations between the accusing users and accused users.
• Based on the derived information, the scheme infers whether the accusations are fair or malicious; that is, it distinguishes users that genuinely misbehave from victims of collusion attacks.
2009/1/12 CCNC 2009 / Jing‐Kai Lou 11
Our Contributions
• The evaluation results show that our scheme
– Achieves accuracy rate higher than 90%– Prevents at least 90% victims from collusion
attacks
2009/1/12 12CCNC 2009 / Jing‐Kai Lou
Accusation Relation
• Accusation Relation(R): a subset of A x A, A := {reporters, UGC creators}
• E.g. 5 users in this system, namely U1, U2, U3, U4, U5• Accusation Relation Matrix(M):
– U1 accuses (reports) U2– U2 accuses U4– U3 accuses U2 & U5– U4 accuses none– U5 accuses U2
User 1 2 3 4 5
1 0 1 0 0 0
2 0 0 0 1 0
3 0 1 0 0 1
4 0 0 0 0 0
5 0 1 0 0 0
2009/1/12 13CCNC 2009 / Jing‐Kai Lou
Accusing Graph
• Input for our community-based scheme• Accusing Graph(G):
– An undirected bipartite graph G(A+B, E)– A: {accusing identity of users}– B: {accused identity of users}
2009/1/12 CCNC 2009 / Jing‐Kai Lou 14
User 1 2 3 4 5
1 0 1 0 0 02 0 0 0 1 03 0 1 0 0 14 0 0 0 0 05 0 1 0 0 0
Meanings of Nodes
• Colluders• Careless accuser• Honest accuser• User doesn’t accuse
2009/1/12 15
C
H
V
L
O
M
L
M
• Victim• Unfortunate user• Misbehaving user• Low-abiding user
Identity of accusing userIdentity of accusing user Identity of accused userIdentity of accused user
CCNC 2009 / Jing‐Kai Lou
Accusing Community
• Adopting Girvan-Newman Algorithm to detect the communitiesand the inter-community edges
2009/1/12 16
Inter-community edge
CCNC 2009 / Jing‐Kai Lou
Inter-community edge
• Property 1: It is unlikely that an inter-community edge is an accusing edge between a colluder and a victim.
• Property 2: It is unlikely that an inter-community edge is an accusing edge between a careless accuser and an unfortunate user.
• Property 3: An inter-community edge most likely is an accusing edge between an honest accuser and a misbehaving user.
2009/1/12 17CCNC 2009 / Jing‐Kai Lou
Features for each User
• inter-community edges fair accusations• Base on the inter-community edges, we design
features for nodes– Incoming Accusation, IA(k) = 2,– Outgoing Accusation, OA(k) = 5
a
kb
c
2009/1/12 18
W = {a, b, c}CCNC 2009 / Jing‐Kai Lou
Clustering (IA, OA) pairs
Incoming Accusation
Out
goin
g A
ccus
atio
n
Unpicked unfortunat users
Unpicked misbehaving users
Unpicked victims
Picked Unfortunate users
Picked misbehaving users
Picked victims
192009/1/12 CCNC 2009 / Jing‐Kai Lou
Picked
Unpicked
Algorithm
1. Partitioning accusing graph into communities.
2. Computing the feature pair (IA, OA) of each user
3. Clustering based on their (IA, OA) pairs, and label users in the cluster with larger (IA, OA) as misbehaving users.
2009/1/12 20CCNC 2009 / Jing‐Kai Lou
Simulation Setup
• We use simulations to evaluate the performance of our scheme in detecting real misbehaving users in a social moderation system.
• Simulation Assumption:1. A honest user should only accuses users that
definitely misbehave.2. A colluder accuses victims.3. All users including colluders have a probability of
making an accusation by mistake.2009/1/12 21CCNC 2009 / Jing‐Kai Lou
Evaluation Metric
• What we care is, False Negative– Misidentifying victims as misbehaving users
• Collusion Resistance
2009/1/12 22CCNC 2009 / Jing‐Kai Lou
Effect of #(Misbehaving users)
Our m
ethod
Count-based m
ethod
2009/1/12 23CCNC 2009 / Jing‐Kai Lou
Conclusion
• We propose a community-based scheme based on the community structure of an accusing graph.
• The results show that the collusion resistance of our scheme is around 90%.
• We believe that collusion-resistant schemes will play an important role in the design of social moderation systems for Web 2.0 services
2009/1/12 24CCNC 2009 / Jing‐Kai Lou
Thank you for your listening
Q&A