fighting spam using social gate keepers
TRANSCRIPT
Fighting spam using Social GateKeepers
Clement Robert, Hein Min Htike & Sijo Emmanuel
INTRODUCTION● Increase in Spam Emails
○ 65% in 2005 to 89.1% in 2010○ 338 Billion USD in 2013 of cost
● Common mitigation techniques○ Spam filtering in user’s inbox
■ Has additional cost (bandwidth, infrastructure)○ Content-Based Filtering
■ False Positive and False Negatives issues
● Solutions○ Trust Based (eg:RE)*
■ People send/receive e mails from immediate neighbourhood ■ Issue: E-Mails coming from outside this circle still need to be verified by common filters
○ Credit Based ■ Introduced by Ostra*
● But, all the participants needed to adopt this system.
*RE: a whitelisting technique that is claimed to minimize the false positive rate*Ostra: a technique Leveraging trust to thwart unwanted communication
LENS(LEveraging social Networking and trust to prevent Spam transmission)
● Like RE ○ Deployed by small group of users○ Reaches outside of FoF
● FoF (Friends of Friends)○ People in the same community. (Friends or Friends of Friends)○ Possibility to send e mail from each other in the network
● Introduction to Gate Keepers (to be discussed)○ Users outside the Network who’re legitimate and authentic○ Those gate keepers help in connecting to users outside the network of F and FoF.
AGENDA● Objectives● Related Work ● LENS Architecture● Prototyping and Security Measures● Evaluation and conclusion● Q & A
OBJECTIVES
● Understand Gatekeepers used in LENS● Understand LENS architecture● Evaluation of LENS using two larger social Networks
○ Facebook○ Flickr
● Understand the results of the evaluation
RELATED WORK“Unwanted and Unsolıcıted e maıls”
● SpamAssassin○ Rely on content filtering○ But spammers are clever to follow filtering rules!
● Blacklisting & whitelisting of senders ○ Blacklisting became ineffective because spammers can create new addresses○ Whitelisting is effective but, does not allow unsolicited e mails
● Spammers want to pass the adverts○ Block payment gateways!
● Social Filters (IP based filters)○ Spammers can use Dynamic IPs
● Trust and Reputation Systems○ MailRank, SNARE .. but all have more increased false positive rate.
LENS Architecture - I● Focus on accepting emails from legitimate users.
○ Not reliant on detecting spams. ○ By selection of legitimate users
■ community members in close social circle of a user■ Socially distant legitimate and authentic users (GKs) in Mail Servers (MS)
● Four main components: ○ Community Formation ○ Trust Management○ GateKeeper Selection○ Spam Report Handler
● Run on Mail Server.
LENS Architecture - II● Ran on MS alongside MTA and
SMTP server● All functionalities are transparent
○ Exceptions: ■ Users can control his community■ Can report spams
● MSs running LENS assumed to have extended verification certificate from CA.○ To prove server’s legitimacy
Community Formation - I● Two levels
○ Friends of users and their friends-of-friends (FoF)■ Adding friend means “User A trusts User B not to send spam email and vice versa”■ Same as adding FoF into community
● Selective process● Involves human involvement (preserve privacy and prevent unnecessary
addition).
● User can receive all the messages from his community directly into his mailbox.
Community Formation - II● Forming Community - 2-step process
○ Adding friend■ User can request anyone in the email network for friendship. ■ Once added, an entry is made in CommList (Community List) with userID and label “F”.
● CommList - maintained for every user, contains entries of community users, accessible by MS or user only
○ Adding FoF■ User can suggest two of his friends to add each other to their communities as FoF
■ Once added, an entry is made in CommList (Community List) with userID and label “FoF” and userID of mutual friend.
Community Formation - III
Trust Management - I● LENS maintain trust rating (TR) for each user on the MS.
○ Assign rating to each user○ Use these ratings to decide legitimate user or not.
● Four types of users○ Trusted user (TU) - TR >= L (legitimate rating threshold) + unique identites○ Legitimate user (LU) - TR >= L ○ New user (NU) - 0 < TR < L○ Illegitimate user (IU) - spammers with negative TR
● Two methods of TR assignment ○ Direct TR - manual assignment by admins○ Automated TR
Trust Management - II● Direct TR - manual assignment of TR by admins
○ Priority over other methods○ Override existing TRs
● Automated TR - automatic assignment of TR by algorithm○ Feature of spammers -> one way communication (always sending out)
■ Abuse this by spammers sending emails to themselves○ Perform user voting
■ If legitimate user A sends B email, it means A trusts B, (vote for B once for the first email).■ When the vote count reaches threshold, the voted user becomes legitimated
● Offline and online modes○ Offline mode - mining existing server logs○ Online mode - live email data stream
● After becoming Legitimate user, need to pass identity test to become trusted user○ Bind unique identity to him (e.g. mobile number, challenge response)
GK Selection - I● User transparent process (no awareness of being GK, no control)
○ Select legitimate users with good reputation as GK■ Use them to vouch for senders outside the community of recipient■ GKs Only vouch for senders in their community
○ Generate keys for recipient and GK respectively ■ GK’s key used by GK’s MS to issue vouchers for GK’s community members, to email to R■ R’s key used to verify the vouchers
● 3 stages of GK selection○ Stage 1 - GK selection in adjacent community
■ Request: R’s MS request FoFs of community (boundary users) for good GK suggestion.
■ Suggestion: FoF’s MS suggest user (with largest number of friends outside R’s Network) from FoF’s friends. -> R chooses set of GKs with best coverage.
■ Verification of Legitimacy: very important step (more info later)
GK Selection - II● 3 stages of GK selection
○ Stage 2 - GK selection beyond adjacent communities ■ R’s MS request selected GKs’ MSs to find GKs in their adjacent communities as in stage 1.■ LENS stops at stage 2, but can expand the reachability beyond this.
○ Stage 3 - GK selection for new communication ■ Announcement: announce sender to R that wants to communicate. ■ Verification of Legitimacy: verify S is not spammer
● R adds S as GK. S’s MS issues vouchers to it’s community, they all can communicate with R now.
Verification of Legitimacy● Important part of GK selection process.
○ Enables R to verify that GK is legitimate and authentic
● Two steps of legitimacy verification○ Server authentication - to verify GK’s MS’s legitimacy○ TR of user - ensure that selected GK is not spammer
● Result of verification: RSA based public and secret keys○ PK is shared with R, SK used to vouch for entire community member of GK
Spam Report Handler● Handles spam reports
○ Only reports from TU are weighted. ■ To prevent spammers reporting non-spammers as spammers
● When user receives spam, reports the sender as spammer● Report handler then register report against the TR of reporter● Once handler receives enough reports from distinct trusted users, give - TR to
spammer. ● If spammer is not local user, add him to revocation list.● Remove spammer’s associated entries in CommList,PKList and VoucherList
LENS Prototype ● Developed during SMTP transactions for email processing
○ Integrated with Mutt mail client, MailAvenger SMTP daemon (mailavenger.org) and Postfix MTA■ No modification to SMTP implementation
● LENS email filter uses Community list and vouchers issued by GKs (Gate Keepers)○ Runs as an independent daemon (i.e. spamd for SpamAssasin: spamassasin.apache.org)
● Monitors SMTP transactions and takes different actions based on the results (more on next figure)
● Prototype uses SHA1 (secure hash algorithm 1) for hash and RSA based signatures
LENS Email processing Lens email processing will fall into three categories:
● Message send to Recipient (R) within the community ○ R’s MS (mail server) will verify the sender against the R’s common list
● Message send to a Recipient outside the Sender’s community○ Senders MS will bind a voucher from GK with message ○ At reception, MS verifies the voucher using PK stored in PKList against GKID
● New message is intended for a recipient outside sender’s community with no voucher issued by any GKs ○ Senders MS will hold the message and start a GK selection procedure (stage 3)○ On success, Senders MS will bind a voucher from GK with message and send it out○ At reception, MS verifies the voucher using PK stored in PKList against GKID
Prevention of spam transmission using LENS● If recipient is not in sender (S) community, the sender’
s MS send the RCPT TO: command by appending the voucher and issuing GK’s ID as additional RCPT parameters after establishing TCP connection. ○ e.g . RCPT TO: <[email protected]>
Voucher=1f2a91od85gdd76 GK [email protected]
● Recipient's MS verifies if sender is a community member or has a valid voucher from an authorised GK (GateKeeper)○ Failure of verification will result in the termination of TCP
connection by Recipients MS ■ Transmission of email will not take place
● Thereby preventing spam message being transmitted
LENS backward compatibility● Easy to integrate LENS into current SMTP servers● LENS will run complimentary with with existing spam filters until every user is
familiar with it ● LENS first examines inbound email after sender authentication (figure) ● LENS have 4 different communication scenarios
○ Sender and Receiver both have LENS○ Only Sender has LENS
■ Receivers MS will process email with existing spam filters ○ Only Receiver has LENS
■ LENS will pass the email to existing spam filters after checking for existence of community and voucher
○ Both Sender and Receiver do not have LENS■ Existing mechanisms deployed for email processing at both ends
Security Concerns - I ● False positives and negatives
○ LENS do not encounter any false positives or negatives generated by imprecise content signatures ■ As LENS spam prevention is not based on content filtering
○ Based on false deduction of social context ■ Malicious users could become part of the community and let spam pass through
● Forgery of from: addresses○ Spammers can easily launch a spam attack with forged from: addresses as if they are from
Recipients community ■ Senders address is not authenticated by default in SMTP (Simple Mail Transfer Protocol)
○ LENS utilises iSATS or SPF (Sender Policy Framework) for sender authentication ■ Filters the inbound email at MAIL FROM: command ■ SPF already effective and used in many existing email systems
Security Concerns - II ● Compromised user
○ If a user (GK) is compromised (identity theft), it will only have a local effect within the community ■ Temporary effect as user broadcasts the incident and claims back ownership
○ If victimised user unable to reclaim the ownership, ■ They can request its community to abandon their compromised Id ■ MS will remove all the data associated with it from PKList and VoucherList
● Attacker would not be able to harm the system on a large scale
● Trust farming○ Spammer could launch a trust farming attack on LENS
■ A spammer can vote for other spammers to increase their rank ○ This attack is comparable to link farming on the internet to attack the PageRank.○ Solutions like ‘identifying link farm spam pages’ can be used orthogonally in LENS
■ To identify and protect against trust farming
Security Concerns - III ● Human spamming using GK selection (Stage 3)
○ Stage 3 of GK selection allows new users to send emails to complete strangers ○ New user (spammer) can exploit this to get in as a new GK and spam the Recipient
■ More expensive than normal spamming model ● Repetition of stage 3 will require substantial human involvement
○ The scale of attack will remain low due to rate limit of the emails from new GK
● Voucher misuse and revocation ○ Current version of the LENS do not impose any expiration limit on the vouchers ○ Vouchers become invalid or get removed by MS based on four conditions
■ GK removes any user from their community, MS will also remove the associated voucher■ The user itself gets out of the GK’s community■ UT (UserType) of the users become negative■ If Recipient revokes GK, all associated vouchers will become invalid
Security Concerns - IV ● Malicious GK faking UserType (UT)
○ GK has no control in manipulating the MS to send false UT ○ Two possibilities where an illegitimate GK can send false UT to recipient
■ Malicious GK hosts a private MS certified by CA and configures to send legitimate UT● Never been an option for spammers as malicious GK becomes visible: legal actions
○ The MS is compromised by the malicious GK (GateKeeper)
● Key Theft ○ Protection of keys is the responsibility of the MSs○ Attacks related to key thefts are synonymous to hacking the MS (Mail Server)
● Corresponding defense mechanisms for both are beyond scope of this paper
Evaluation of LENS ● KeyPoints
○ Scalability○ Effectiveness in accepting all inbound emails○ Performance of Gatekeepers Selection○ E mail processing
● Case studies○ ONS (Online Social Networks)
■ Facebook■ Flickr
Evaluation of LENS(Cont’d)- Scalability● Number of GK required
● The number of GK depends on numbers of FoF , therefore depends on the community size.
● Flickr requires <30% of community size (20-400 GKs)
● Increased Reachability via GKs
● LENS is scalable○ With 100s GK, a Recipient can be reached by
millions of users (>40 % of users in F and F)○ but , in RE, we get 0.051% and 0.25%
respectively
Evaluation of LENS(Cont’d)- Effectiveness● Data sets from Enron & Log files from Kiel
Un’ MS.● Social Network Graph created
○ Edge: Between people who exchanged at least 3 emails
● Filters applied○ With Friends only ( 20% of mail were
rejected)○ Comm( <20% rejected)○ LENS (0% rejected)
● But, the number of gatekeepers was reasonable
Evaluation of LENS(Cont’d)- Performance of GK Sel. Prot.
● Latency of GK selection protocol when MS are located in different countries
○ PlanetLab is used with 20 nodes● Each node sends a random GK selection
request to the rest● Average executıon tıme of stage 1 GK sel.Pr.
● An extra step sınce GK are suggested by FoF.
● At stage 3, there is no extra step involved○ Therefore, the execution time reduces
● Success rate is at >94.6%● Few nodes did not achieve 100% (nodes
overloading by too many connections)
Evaluation of LENS(Cont’d)- Performance of Email Proc.
● Objective○ How the size of the message affects the
time required to process it● How
○ 50k messages were sent every second○ Size varying from 1 to 50 kb
● In 4 Scenarios;○ S1: SMTP runs postfix only○ S2: SpamAssassin is used as Content based
F.○ S3: MailAvenger-Postfix-LENS○ S4: similar to S3 with added functionality
of emails based on GKs of the recipient
● Message Size
● SpamAssassin processing delay increases with the message size
Evaluation of LENS(Cont’d)- Performance of Email Proc.
● Objective○ How the size of the message affects the
time required to process it● How
○ 50k messages were sent every second○ Size varying from 1 to 50 kb
● In 4 Scenarios;○ S1: SMTP runs postfix only○ S2: SpamAssassin is used as Content based
F.○ S3: MailAvenger-Postfix-LENS○ S4: similar to S3 with added functionality
of emails based on GKs of the recipient\● To measure throughput, the sender
bombards the MS with 1000 messages of 8kb each as rapid as possible.
● Throughput
● S1, with no filters makes it by accepting 11 messages per second.
Evaluation of LENS(Cont’d)- Performance of Email Proc.
● Objective○ How the size of the message affects the
time required to process it● How
○ 50k messages were sent every second○ Size varying from 1 to 50 kb
● In 4 Scenarios;○ S1: SMTP runs postfix only○ S2: SpamAssassin is used as Content based
F.○ S3: MailAvenger-Postfix-LENS○ S4: similar to S3 with added functionality
of emails based on GKs of the recipient\● CPU & Memory
○ S2 consumes more CPU (many fılters)○ S2 requıres also hıgh memory.
● CPU Usage
● MemoryUsage
CONCLUSION● This paper was explaining the design, implementation of LENS● Evaluation was made on Online Social Networks (OSN) ● The evaluation proved how LENS is scalable when compared to other methods
anti spams● Its effectiveness was proved by accepting all incoming e mails● LENS consumes up to 75% less CPU and 9 % less memory as traditional
solutions like SpamAssassin.
Thank you
Q & A