stylometry and online underground marketsevents.ccc.de/congress/2012/fahrplan/attachments/... ·...
TRANSCRIPT
![Page 1: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/1.jpg)
Stylometry and Online Underground Markets
Sadia Afroz, Aylin Caliskan Islam Co-authors: Ariel Stolerman, Rachel Greenstadt,
Damon McCoy
Friday, December 28, 12
![Page 2: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/2.jpg)
Previously at CCC...
• 26c3:
• Introduced the concept of adversarial stylometry:
• Authorship recognition algorithms can be evaded by changing writing style.
Friday, December 28, 12
![Page 3: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/3.jpg)
Previously at CCC...
• 26c3:
• Introduced the idea of adversarial stylometry
• Authorship recognition algorithms can be evaded by changing writing style.
• 28c3:
• Released two tools:
• JStylo (authorship recognition tool) and
• Anonymouth (authorship anonymization tool)
Friday, December 28, 12
![Page 4: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/4.jpg)
This talk
• How stylometric analysis can be used in real world datasets?
• Identify people based on writing style
• Identify topic of their discussion
• Real world dataset: online underground markets
Friday, December 28, 12
![Page 5: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/5.jpg)
Overview
• Online underground markets
• Analysis
• Limitation and Challenges
• Future work
• Anonymouth
Friday, December 28, 12
![Page 6: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/6.jpg)
Online Underground Markets
• Underground market is a trading place to trade various stolen goods and/or tools such as exploits, malware repackaging kits, phishing kits.
Friday, December 28, 12
![Page 7: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/7.jpg)
Friday, December 28, 12
![Page 8: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/8.jpg)
Friday, December 28, 12
![Page 9: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/9.jpg)
Why is this interesting?
• Cyber-underground ecosystem
• Key information about who controls a given bot
• Who maintains certain tools
• Size and scope of these markets.
Friday, December 28, 12
![Page 10: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/10.jpg)
http://krebsonsecurity.com/2012/12/a-closer-look-at-two-bigtime-botmasters/
Friday, December 28, 12
![Page 11: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/11.jpg)
http://krebsonsecurity.com/2012/12/a-closer-look-at-two-bigtime-botmasters/
Friday, December 28, 12
![Page 12: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/12.jpg)
Online Underground Markets
• Two main markets:
•Internet Relay Chat (IRC)
•Web forums
Friday, December 28, 12
![Page 13: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/13.jpg)
Online Underground Markets
• Two main markets:
•Internet Relay Chat (IRC)
•Web forums
Friday, December 28, 12
![Page 14: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/14.jpg)
Forums
• Antichat - Russian forum (May 2002-Jun 2010)
• BadHacke - English/Hindi (Nov 2003-May 2008)
• BlackHat - English (Oct 2005-Mar 2008)
• Carders - German (Feb 2009- Dec 2010)
• L33tCrew- German (May 2007-Nov 2009)
Friday, December 28, 12
![Page 15: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/15.jpg)
How did we get the data?
• Leaked by anonymous people
• Publicly available
Friday, December 28, 12
![Page 16: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/16.jpg)
0
12,500
25,000
37,500
50,000
Antichat Badhacke Blackhat Carders L33tCrew
9,306
3,0974,2293,026
15,165
9,5285,3284,4895,123
25,871
Members
Active members Lurkers
Friday, December 28, 12
![Page 17: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/17.jpg)
How Transaction Happensi am selling my large targettedemail list of morethan 22.5 million emails
Public
PrivateForum
Register
ICQ/ Email
Friday, December 28, 12
![Page 18: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/18.jpg)
0
562,500
1,125,000
1,687,500
2,250,000
Antichat Badhacke Blackhat Carders L33tCrew
861,459
373,143
65,57237,074
2,160,815Messages
Private messages Public messagesFriday, December 28, 12
![Page 19: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/19.jpg)
Challenges
41,036 Users
2,160,815 Public posts
194,498 Private msgs
Friday, December 28, 12
![Page 20: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/20.jpg)
Challenges
41,036 Users
2,160,815 Public posts
This is just one forum!
194,498 Private msgs
Friday, December 28, 12
![Page 21: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/21.jpg)
Challenges
41,036 Users
2,160,815 Public posts
интересует взлом
194,498 Private msgs
Friday, December 28, 12
![Page 22: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/22.jpg)
Challenges
41,036 Users
2,160,815 Public posts
интересует взломAlles läuft vor eurem PC ab
194,498 Private msgs
Friday, December 28, 12
![Page 23: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/23.jpg)
Challenges
41,036 Users
2,160,815 Public posts
интересует взломAlles läuft vor eurem PC ab I gave 40 bucks and no
program what is up man?
194,498 Private msgs
Friday, December 28, 12
![Page 24: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/24.jpg)
Analysis
• Interaction network analysis
• Member profiling using writing style
• Topic discovery
Friday, December 28, 12
![Page 25: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/25.jpg)
Analysis
• Interaction network analysis
• Member profiling using writing style
• Topic discovery
Friday, December 28, 12
![Page 26: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/26.jpg)
Interaction Network Analysis
Private
Represent a forum with a graph, G=(V, E) whereeach user is a vertex
Friday, December 28, 12
![Page 27: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/27.jpg)
Interaction Network Analysis
Private
Private
Private
Private
Represent a forum with a graph, G=(V, E) whereeach user is a vertexeach private message is an edge
Friday, December 28, 12
![Page 28: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/28.jpg)
Interaction Network Analysis
• Goal:
• Structure of interaction
• Identify central members
Friday, December 28, 12
![Page 29: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/29.jpg)
Interaction Network Analysis• Goal:
• Structure of interaction
• Identify central members:• Eigenvector centrality:
• It is a measure of the influence of a node in a network.
• Higher score == More influential member
Friday, December 28, 12
![Page 30: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/30.jpg)
Interaction Network Analysis: Antichat
With all usersFriday, December 28, 12
![Page 31: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/31.jpg)
Interaction Network Analysis: Antichat
With all users
Bigger nodes are more influential
Darker nodes send/receive more msgs
Friday, December 28, 12
![Page 32: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/32.jpg)
Interaction Network Analysis: Antichat
With all users With top 50 influential users
Friday, December 28, 12
![Page 33: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/33.jpg)
Interaction Network Analysis: Badhacke
With all members With top influential members
Friday, December 28, 12
![Page 34: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/34.jpg)
Analysis
• Interaction network analysis
• Member profiling using writing style
• Topic discovery
Friday, December 28, 12
![Page 35: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/35.jpg)
Profiling using writing style
• Everybody has a *unique* writing style.
• Goal:
• Identify members using their writing style
Friday, December 28, 12
![Page 36: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/36.jpg)
Try JStylo
• https://psal.cs.drexel.edu/index.php/JStylo-Anonymouth
Friday, December 28, 12
![Page 37: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/37.jpg)
0
0.2500
0.5000
0.7500
1.0000
5 10 15 20 25 30 35 40
Accuracy in detecting authorship of regular documents
Number of Authors
9-Feature (NN)Synonym-BasedWriteprints Baseline (SVM)Random
*Thanks to Michael Brennan for the graph
Friday, December 28, 12
![Page 38: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/38.jpg)
Profiling using writing style
Find members with enough texts
Combine all messages of
a member
Extract features Train classifier to model a member
Evaluate
JStylo
Friday, December 28, 12
![Page 39: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/39.jpg)
0
20.0
40.0
60.0
80.0
100.0
2 3 4 5 6 7 8 9 10 11 12 13 14 15
Acc
urac
y
Number of 500-word documentsCarders (private) Carders (public)L33tcrew (private) L33tcrew (public)Antichat Blackhat
How much text is enough?
Friday, December 28, 12
![Page 40: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/40.jpg)
0
20.0
40.0
60.0
80.0
100.0
2 3 4 5 6 7 8 9 10 11 12 13 14 15
Acc
urac
y
Number of 500-word documentsCarders (private) Carders (public)L33tcrew (private) L33tcrew (public)Antichat Blackhat
How much text is enough?:at least 5000 words
Friday, December 28, 12
![Page 41: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/41.jpg)
0
20.0
40.0
60.0
80.0
100.0
2 3 4 5 6 7 8 9 10 11 12 13 14 15
Acc
urac
y
Number of 500-word documentsCarders (private) Carders (public)L33tcrew (private) L33tcrew (public)Antichat Blackhat
How much text is enough?:we used 6500 words
Friday, December 28, 12
![Page 42: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/42.jpg)
Profiling using writing style
Find members with enough texts
Combine all messages of
a member
Extract features Train classifier to model a member
Evaluate
JStylo
Friday, December 28, 12
![Page 43: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/43.jpg)
What are these features?
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Example from Carders
Friday, December 28, 12
![Page 44: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/44.jpg)
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
1337 down? Neh **, the solution! **
Ne nit works, rather guess they have
again DNS problems
*Translation
*Using Google translator
Example from Carders
What are these features?
Friday, December 28, 12
![Page 45: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/45.jpg)
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Freq. of n-grams
What are these features?
Example from Carders
Friday, December 28, 12
![Page 46: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/46.jpg)
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Freq. of n-grams
Freq. of punctuations
What are these features?
Example from Carders
Friday, December 28, 12
![Page 47: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/47.jpg)
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Freq. of n-grams
Freq. of special characters
Freq. of punctuations
What are these features?
Example from Carders
Friday, December 28, 12
![Page 48: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/48.jpg)
Language Independent
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Freq. of n-grams
Freq. of special characters
Freq. of punctuations
What are these features?
Example from Carders
Friday, December 28, 12
![Page 49: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/49.jpg)
Language specific
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Parts of speechFreq. Function wordsFreq. of ngramsFreq. of punctuationsFreq. of special characters
What are these features?
Example from Carders
Friday, December 28, 12
![Page 50: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/50.jpg)
Not all conversationBankname: XX CCNumber: XXXXXXXXCCHolder: XX XXXXCCExpire: X / XXXXCVV2: XXVorname: XXNachame: YYAddresse: XXXXXStadt: XXXXPLZ: XXXXLand: XXTelefon: XXXXX-XXXXXE-mail: [email][email protected][/email]Geburtsdatum: XX / XX / XXXX
Friday, December 28, 12
![Page 51: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/51.jpg)
Bankname: AA CCNumber: XXXXXXXX
What’s wrong with that?
Bankname: AA CCNumber: XXXXXXXX
Member X Member Y
Friday, December 28, 12
![Page 52: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/52.jpg)
Bankname: AA CCNumber: XXXXXXXX
Bankname: AA CCNumber: XXXXXXXX
What’s wrong with that?
Both will look the same to a linguistic classifier
Member X Member Y
Friday, December 28, 12
![Page 53: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/53.jpg)
Profiling using writing style
Find members with enough texts
Combine all messages of
a member
Extract features Train classifier to model a member
Evaluate
JStylo
Identify Product
Friday, December 28, 12
![Page 54: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/54.jpg)
Identify Products
Bankname: XX CCNumber: XXXXXXXXCCHolder: XX XXXXCCExpire: X / XXXXCVV2: XXVorname: XXNachame: YYAddresse: XXXXXStadt: XXXXPLZ: XXXXLand: XXTelefon: XXXXX-XXXXXE-mail: [email][email protected][/email]Geburtsdatum: XX / XX / XXXX
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Product Conversation
1. Product information has repeated patterns2. Conversation usually has verb
Friday, December 28, 12
![Page 55: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/55.jpg)
Bankname: XX CCNumber: XXXXXXXXCCHolder: XX XXXXCCExpire: X / XXXXCVV2: XXVorname: XXNachame: YYAddresse: XXXXXStadt: XXXXPLZ: XXXXLand: XXTelefon: XXXXX-XXXXXE-mail: [email][email protected][/email]Geburtsdatum: XX / XX / XXXX
1. Product information has repeated patterns2. Conversation usually has verb
1337 down? **Neh, die Lösung!**Ne klappt nit, denke mal eher das sie mal wieder DNS probleme haben
Product Conversation
Verbs
Identify Products
Friday, December 28, 12
![Page 56: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/56.jpg)
Identify ProductsBankname: XX CCNumber: XXXXXXXXCCHolder: XX XXXXCCExpire: X / XXXXCVV2: XXVorname: XXNachame: YYAddresse: XXXXXStadt: XXXXPLZ: XXXXLand: XXTelefon: XXXXX-XXXXXE-mail: [email][email protected][/email]Geburtsdatum: XX / XX / XXXX
Product
Noun(NN): CD Noun(NN): CD Noun(NN): CD Noun(NN): CD SYM CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CD-CDNoun(NN): NNNoun(NN): CD SYM CD SYM CD
Tag parts of speech
Friday, December 28, 12
![Page 57: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/57.jpg)
Identify ProductsBankname: XX CCNumber: XXXXXXXXCCHolder: XX XXXXCCExpire: X / XXXXCVV2: XXVorname: XXNachame: YYAddresse: XXXXXStadt: XXXXPLZ: XXXXLand: XXTelefon: XXXXX-XXXXXE-mail: [email][email protected][/email]Geburtsdatum: XX / XX / XXXX
Product
Noun(NN): CD Noun(NN): CD Noun(NN): CD Noun(NN): CD SYM CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CDNoun(NN): CD-CDNoun(NN): NNNoun(NN): CD SYM CD SYM CD
Tag parts of speech
Find repeated patterns
Check for verb
Friday, December 28, 12
![Page 58: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/58.jpg)
Profiling using writing style
Find members with enough texts
Combine all messages of
a member
Extract features Train classifier to model a member
Evaluate
JStylo
Identify Product
Friday, December 28, 12
![Page 59: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/59.jpg)
0
22.5
45.0
67.5
90.0
L33tCrew (private) L33tCrew (public) Carders (private) Carders (public) Antichat (public) Blackhat (public)
Result
Language specific Language independent
Friday, December 28, 12
![Page 60: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/60.jpg)
0
22.5
45.0
67.5
90.0
L33tCrew (private) L33tCrew (public) Carders (private) Carders (public) Antichat (public) Blackhat (public)
Result
Language specific Language independent
Language specific features perform better
Friday, December 28, 12
![Page 61: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/61.jpg)
0
22.5
45.0
67.5
90.0
L33tCrew (private) L33tCrew (public) Carders (private) Carders (public) Antichat (public) Blackhat (public)
Result
Language specific Language independent
Accuracy is better in private messages
Friday, December 28, 12
![Page 62: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/62.jpg)
Relaxed attribution
Who is this author?
This is Alice
Exact attribution:
AliceClassifier
Friday, December 28, 12
![Page 63: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/63.jpg)
Relaxed attribution
Find top n possible authors
They could be Alice
Who is this author?
Alice
Friday, December 28, 12
![Page 64: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/64.jpg)
0
25
50
75
100
1 2 3 4 5 6 7 8 9 10
Antichat (public) Blackhat (public) Carders (private)Carders (public) L33tCrew (private) L33tCrew (public)
Number of possible members
Acc
urac
y
Friday, December 28, 12
![Page 65: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/65.jpg)
0
25
50
75
100
1 2 3 4 5 6 7 8 9 10
Antichat (public) Blackhat (public) Carders (private)Carders (public) L33tCrew (private) L33tCrew (public)
Number of possible members
Acc
urac
y
Friday, December 28, 12
![Page 66: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/66.jpg)
0
25
50
75
100
1 2 3 4 5 6 7 8 9 10
Antichat (public) Blackhat (public) Carders (private)Carders (public) L33tCrew (private) L33tCrew (public)
Number of possible members
Acc
urac
y
Friday, December 28, 12
![Page 67: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/67.jpg)
Tracking members across forums
• Can a member’s posts on one forum identify him in other forums?
Friday, December 28, 12
![Page 68: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/68.jpg)
• Approach:
• Find members using email address
• Train on one forum’s posts and test on another
Tracking members across forums
Friday, December 28, 12
![Page 69: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/69.jpg)
0
100
200
300
400
Carders L33tCrew
8339
Common members
Members with enough textMembers with not enough text
Tracking members across forums
Friday, December 28, 12
![Page 70: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/70.jpg)
Cross forum result
0
18
35
53
70
1 2 3 4 5 6 7 8 9 10
Train on L33tCrew (83 members)
0
18
35
53
70
1 2 3 4 5 6 7 8 9 10
Train on carders (39 members)
Friday, December 28, 12
![Page 71: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/71.jpg)
Who are the possible suspects?
Find top 10 possible identities of Alice
Alice
Friday, December 28, 12
![Page 72: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/72.jpg)
Who are the possible suspects?
Find top 10 possible identities of Alice
Alice
What do they tell us about Alice?
Friday, December 28, 12
![Page 73: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/73.jpg)
Who are the possible suspects?
Find top 10 possible identities of Alice
Alice
What do they tell us about Alice?
Can we find alternate identities of Alice?
Friday, December 28, 12
![Page 74: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/74.jpg)
Who are the possible suspects?
Alice Bob
If Bob is one of the suspects of Alice
Created a graph where each node= a useredge a to b if b is one of the suspects of a
Friday, December 28, 12
![Page 75: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/75.jpg)
• Run this analysis of SpamIt forum associates chat conversation
• Because we had ground truth about duplicate accounts
Who are the possible suspects?
Friday, December 28, 12
![Page 76: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/76.jpg)
Who are the possible suspects?
Alice
Alice2
ICQ:Alice
@lice
Bob
Friday, December 28, 12
![Page 77: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/77.jpg)
Discover Topic
• Topic analysis can identify predominant topic of the forum
• Why is it important?:
• Automatically identify “interesting” subset of the data
• Find relevant people
• Understand trendsFriday, December 28, 12
![Page 78: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/78.jpg)
• We used Latent Dirichlet Allocation (LDA) for identifying topic words.
Discover Topic
Friday, December 28, 12
![Page 79: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/79.jpg)
How LDA works
I wonder what kind of products or services you could advertise using bulk mailing. I'm
really new to bulk mailing, never tried it before. But I know that you'll get in trouble with your hosting company if you promote
stuff that leads to your own server / hosting account. So how does this
work???Example post from Blackhat
Friday, December 28, 12
![Page 80: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/80.jpg)
How LDA works
I wonder what kind of products or services you could advertise using bulk mailing. I'm
really new to bulk mailing, never tried it before. But I know that you'll get in trouble with your hosting company if you promote
stuff that leads to your own server / hosting account. So how does this
work???Example post from Blackhat
Topic 1Topic 2
Topic 3
Topic 2
Friday, December 28, 12
![Page 81: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/81.jpg)
Discover Topic
Find number of topics
Find topics and
topics proportion
Manual analysis
Find top 20 words per topic
Friday, December 28, 12
![Page 82: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/82.jpg)
Discover Topic
Find number of topics
Find topics and
topics proportion
Manual analysis
Find top 20 words per topic
(200-300 on average)
Friday, December 28, 12
![Page 83: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/83.jpg)
Topics discovered
19%14%
5%10%
30%11%
11%
Carders
Anonymity services (web and phone)ExploitsCarding and other accountsCardableBankdropDrugsCurrency: PSC, UKASH, WMZ
12%7%
5%
16%
26%
33%
L33tCrew
Crypting servicesAnonymity servicesCardingOther accountsExploitsAnonymous phone
Friday, December 28, 12
![Page 84: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/84.jpg)
12%7%
5%
16%
26%
33%
L33tCrew
Topics discovered
19%14%
5%10%
30%11%
11%
Carders
Anonymity services (web and phone)ExploitsCarding and other accountsCardableBankdropDrugsCurrency: PSC, UKASH, WMZ
Carding
CurrencyCrypting servicesAnonymity servicesCardingOther accountsExploitsAnonymous phone
Friday, December 28, 12
![Page 85: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/85.jpg)
12%7%
5%
16%
26%
33%
L33tCrew
Anonymityservice
Topics discovered
19%14%
5%10%
30%11%
11%
Carders
Anonymity services (web and phone)ExploitsCarding and other accountsCardableBankdropDrugsCurrency: PSC, UKASH, WMZ
Carding
Currency
Crypting service
Crypting servicesAnonymity servicesCardingOther accountsExploitsAnonymous phone
Friday, December 28, 12
![Page 86: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/86.jpg)
54%
3%3%6%
9%
25%
Antichat
Web moneyExploitsPhone and SMSCaptcha solvingSEO blackhat toolsPassword cracking
Topics discovered
11%7%
18%
16%
49%
Blackhat
SEO blackhatSEO for blogsSEO for youtubeCaptcha solvingBuy followers and friends
Friday, December 28, 12
![Page 87: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/87.jpg)
54%
3%3%6%
9%
25%
Antichat
Web moneyExploitsPhone and SMSCaptcha solvingSEO blackhat toolsPassword cracking
Topics discovered
11%7%
18%
16%
49%
Blackhat
SEO blackhatSEO for blogsSEO for youtubeCaptcha solvingBuy followers and friends
Password cracking
Currency SEO blackhat
Friday, December 28, 12
![Page 88: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/88.jpg)
Next…• Challenges• Limitations• Future work• Conclusions• Tools developed in our lab
Friday, December 28, 12
![Page 89: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/89.jpg)
CHALLENGES• Microtext• Multilingual text• Different types of product information in text• Users with multiple accounts
Friday, December 28, 12
![Page 90: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/90.jpg)
Challenges of microtext• Short writings
“1x Brazzers heut morgen gings noch”
Friday, December 28, 12
![Page 91: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/91.jpg)
Challenges of microtext• Short writings
“1x Brazzers heut morgen gings noch”
• Informal and conversational style “LOL........... nice post”
Friday, December 28, 12
![Page 92: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/92.jpg)
Challenges of multilingual text
• Require multilingual features in machine learning • Language-specific POS tagger• Function words
Friday, December 28, 12
![Page 93: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/93.jpg)
Challenges of multilingual text
• Many authorship tools are designed for English• Translated text gives better results in identifying
members
Friday, December 28, 12
![Page 94: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/94.jpg)
Challenges of multilingual text
• We translated Carders public to see how translation affects the accuracy
Friday, December 28, 12
![Page 95: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/95.jpg)
Challenges of multilingual text
• We translated Carders public to see how translation affects the accuracy
Friday, December 28, 12
![Page 96: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/96.jpg)
Challenges of multilingual text
• Highest accuracy achieved through the translated dataset that used English features
Friday, December 28, 12
![Page 97: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/97.jpg)
Challenges of translating multilingual text
• Large dataset requires automatic language detection for batch translations
• Low quality translations because of microtext properties
Friday, December 28, 12
![Page 98: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/98.jpg)
Challenges of multilingual text
• Automatic language detection is hard
Friday, December 28, 12
![Page 99: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/99.jpg)
Challenges of multilingual text
• Automatic language detection is hard
Friday, December 28, 12
![Page 100: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/100.jpg)
Challenges of multilingual text
• Automatic language detection is hard
Friday, December 28, 12
![Page 101: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/101.jpg)
Challenges of multilingual text
• Low quality translations
Friday, December 28, 12
![Page 102: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/102.jpg)
Challenges of multilingual text
• Low quality translations
Friday, December 28, 12
![Page 103: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/103.jpg)
Challenges of multilingual text
• Low quality translations
Friday, December 28, 12
![Page 104: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/104.jpg)
Challenges of multilingual text
• Low quality translations
• Translation:
Friday, December 28, 12
![Page 105: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/105.jpg)
Challenges of multilingual text
• Low quality translations
• Translation:
Friday, December 28, 12
![Page 106: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/106.jpg)
Challenges of multilingual text
• Low quality translations
• Translation:
Not English
Friday, December 28, 12
![Page 107: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/107.jpg)
Challenges of product information in text
• Problematic for language independent features such as character n-grams • Adds noise to the model for each author
• Messages contain both conversation and product information
Message: халявщики просыпайтесь..этот раздел не умрет ;) (наверно) :D кто забрал,отпишитесь)
1234;xxxx [email protected] 1235;yyyy [email protected] 1236;zzzz [email protected]
Friday, December 28, 12
![Page 108: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/108.jpg)
Challenges of product information in text
• Problematic for language independent features such as character n-grams • Adds noise to the model for each author
• Messages contain both conversation and product information
Message: халявщики просыпайтесь..этот раздел не умрет ;) (наверно) :D кто забрал,отпишитесь)
1234;xxxx [email protected] 1235;yyyy [email protected] 1236;zzzz [email protected]
Conversation
Friday, December 28, 12
![Page 109: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/109.jpg)
Challenges of product information in text
• Problematic for language independent features such as character n-grams • Adds noise to the model for each author
• Messages contain both conversation and product information
Message: халявщики просыпайтесь..этот раздел не умрет ;) (наверно) :D кто забрал,отпишитесь)
1234;xxxx [email protected] 1235;yyyy [email protected] 1236;zzzz [email protected]
Conversation
Product information
Friday, December 28, 12
![Page 110: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/110.jpg)
Challenges of product information in text
• We have this huge dataset
Friday, December 28, 12
![Page 111: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/111.jpg)
Challenges of product information in text
• We have this huge dataset• We need to detect products so that we can
analyze the conversational text
Friday, December 28, 12
![Page 112: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/112.jpg)
Challenges of product information in text
• We have this huge dataset• We need to detect products so that we can
analyze the conversational text• How can you build a method to detect different
types of product information
Friday, December 28, 12
![Page 113: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/113.jpg)
Challenges of product information in text
• We have this huge dataset• We need to detect products so that we can
analyze the conversational text• How can you build a method to detect different
types of product information• We consider a text pattern that doesn’t contain
verbs as product information. This is done with a POS-tagger.
Friday, December 28, 12
![Page 114: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/114.jpg)
Types of products
• Exploits• Copyright infringement• Credit cards• Bank accounts• E-mail accounts• Online accounts• Bankdrops• Shipping/delivery services• Drugs
Friday, December 28, 12
![Page 115: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/115.jpg)
Exploit# Exploit coded by “Alice" and “Bob"##################################################use LWP::UserAgent; $ua = new LWP::UserAgent; $ua->agent("Mosiac 1.0" . $ua->agent);if (!$ARGV[0]) {$ARGV[0] = '';}if (!$ARGV[3]) {$ARGV[3] = '';}my $path = $ARGV[0] . '/index.php?act=Login&CODE=autologin';my $user = $ARGV[1]; # userid to jackmy $iver = $ARGV[2]; # version 1 or 2….if (!$ARGV[2]){print "The type of the file system is NTFS.\n\n";print "WARNING, ALL DATA ON NON-REMOVABLE DISK\n";…
Friday, December 28, 12
![Page 116: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/116.jpg)
Copyright infringementStyle: Format: MP3, 160 kbps Size: 50,7 Mb Country: USA
o1. "Flat Line"o2. "6 6 Sick"o3. "Addiction" (featuring Zakk Wylde)o4. "No Regrets"o5. "My Funeral"o6. "We Are"o7. "Dirty World"o8. "Interlude"o9. "Violence"1o. "Best for Me"11. "Bloodless"12. "Scorn"13. "Rebel Yell" (Billy Idol cover)14. "I Don't Give a..."15. "Die, Boom, Bang, Burn, F*ck"16. "Nothing for Me Here"
Download:http://rapidshare.com/files/123456789/NR-copyrightportal.ru.rarhttp://depositfiles.com/files1a2b3c4d5e
Friday, December 28, 12
![Page 117: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/117.jpg)
Credit CardDE CCV MasterCardI Have got here a Master Card Germany checked, but I can't need it anymore...*~~*CardholderName:Alice Smith*~~*FirstName:Alice*~~*LastName:Smith*~~*Address:Alice’s Street and number*~~*ZIP:99999*~~*City:Alice’s city*~~*Country:DE*~~*Phone:12345678900*~~*Email:[email protected]*~~*1234123412341234*~~*012*~~*0123
Friday, December 28, 12
![Page 118: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/118.jpg)
Bank AccountData: Fri Dec 28, 2012 11:11 pm Login: [email protected]: Alice’s_passwordFirst Name: AliceLast Name: SmithCard Type: Visa Bank Name: Citigroup Smith Barney CC Number: 4321432143214321Month: 01 Year: 2015 CVV2: 215 PIN: 2345
Friday, December 28, 12
![Page 119: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/119.jpg)
More accounts================================================== Software : Windows Live Messenger Protocol : MSN Messenger User : [email protected] Password : Alice’s_password================================================== ================================================== Software : ICQ Lite/2003 Protocol : ICQ User : 123454321 Password : Alice’s_password==================================================
================================================== Name : AliceSmithApplication : Hotmail/MSN Email : [email protected] Server : Type : HTTP User : AliceSmithPassword : Alice’s_passwordProfile : ==================================================
Friday, December 28, 12
![Page 120: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/120.jpg)
Online accountspaypal ...................
254/[email protected]/Alice’s_password253/[email protected]/Alice’s_password
252/[email protected]/Alice’s_password251/[email protected]/Alice’s_password
250/[email protected]/Alice’s_password249/[email protected]/Alice’s_password248/[email protected]/Alice’s_password
247/[email protected]/Alice’s_password246/[email protected]/Alice’s_password
245/[email protected]/Alice’s_password244/[email protected]/Alice’s_password
243/[email protected]/Alice’s_password242/[email protected]/Alice’s_password
241/[email protected]/Alice’s_password240/[email protected]/Alice’s_password
239/[email protected]/Alice’s_password238/[email protected]/Alice’s_password237/[email protected]/Alice’s_password
236/[email protected]/Alice’s_password
Friday, December 28, 12
![Page 121: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/121.jpg)
Bankdrops Bankdrop-tutorial Was wird benötigt? - Socks/VPN etc. - 1 Fake Acc (in 2min. erledigt) - 1 Fake Foto (in 2min. erledigt) - 1 Fake E-Mail (in 2min. erledigt) - Briefkastendrop
Friday, December 28, 12
![Page 122: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/122.jpg)
Shipping/Delivery ServicesOnce we recieve note of order your selected equipment will be shipped to your indicated address no longer than 2-3 working days you will recieve an email with shipping track and trace.
To place order email below.Email:[email protected] support:123*123*123 or 321*321*321SKIMMER PACKAGES AND PRICE LISTSKIMMERS SOLD SEPRATELY
Atm Model:Diebold / Wincor / Ncr (skimmer only)Type:USBPrice:$2000
Atm Model:Diebold / Wincor / Ncr (skimmer only)Type:BluetoothPrice:$2500
Friday, December 28, 12
![Page 123: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/123.jpg)
Skimmer that looks like anti-skimming device
Friday, December 28, 12
![Page 124: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/124.jpg)
DrugsMessage:Verified Vendor of WeedThread: [url]http://www.carders.cc/forum/threads/12345-Alice´s-Weed-Store-Verified-Vendor-[/url])Profil: [url]http://www.carders.cc/forum/members/1234-Alice[/url]
Message:Intresse an Drug Store ?weed pepmdma
Friday, December 28, 12
![Page 125: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/125.jpg)
Challenges caused by users with multiple accounts
• Same user with different IP and e-mail address opens multiple accounts to avoid being banned
Friday, December 28, 12
![Page 126: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/126.jpg)
Challenges caused by users with multiple accounts
• Same user with different IP and e-mail address opens multiple accounts to avoid being banned
• Difficult to identify multiple account holders
Friday, December 28, 12
![Page 127: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/127.jpg)
Challenges caused by users with multiple accounts
• Same user with different IP and e-mail address opens multiple accounts to avoid being banned
• Difficult to identify multiple account holders• In supervised learning, we consider them as
different users
Friday, December 28, 12
![Page 128: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/128.jpg)
Challenges caused by users with multiple accounts
• Same user with different IP and e-mail address opens multiple accounts to avoid being banned
• Difficult to identify multiple account holders• In supervised learning, we consider them as
different users• Authorship attribution classification accuracy and
social connection graphs suffer due to this lack of ground truth
Friday, December 28, 12
![Page 129: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/129.jpg)
Limitations• The required text length is 5000 words
Friday, December 28, 12
![Page 130: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/130.jpg)
Limitations• The required text length is 5000 words• Forums with less well known languages anticipated
Friday, December 28, 12
![Page 131: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/131.jpg)
Limitations• The required text length is 5000 words• Forums with less well known languages anticipated• Separate conversational data from product
information for better stylometric analysis
Friday, December 28, 12
![Page 132: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/132.jpg)
Limitations• The required text length is 5000 words• Forums with less well known languages anticipated• Separate conversational data from product
information for better stylometric analysis• Our method worked well but needs improvements
Friday, December 28, 12
![Page 133: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/133.jpg)
Limitations• The required text length is 5000 words• Forums with less well known languages anticipated• Separate conversational data from product
information for better stylometric analysis• Our method worked well but needs improvements• False-positive product detection:
Friday, December 28, 12
![Page 134: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/134.jpg)
Limitations• The required text length is 5000 words• Forums with less well known languages anticipated• Separate conversational data from product
information for better stylometric analysis• Our method worked well but needs improvements• False-positive product detection:
Msn Piç : Sinir olduğunuz insanlara ister kendi zevkiniz ister programda bulunan küfürleri yollayabilirsiniz.Sizden fazla hızlı küfür edemez :)
Friday, December 28, 12
![Page 135: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/135.jpg)
Limitations• The required text length is 5000 words• Forums with less well known languages anticipated• Separate conversational data from product
information for better stylometric analysis• Our method worked well but needs improvements• False-positive product detection:
Msn Piç : Sinir olduğunuz insanlara ister kendi zevkiniz ister programda bulunan küfürleri yollayabilirsiniz.Sizden fazla hızlı küfür edemez :)
Not a product
Friday, December 28, 12
![Page 136: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/136.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.
Friday, December 28, 12
![Page 137: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/137.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.• This has been very rarely done
Friday, December 28, 12
![Page 138: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/138.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.• This has been very rarely done• Short leetspeak cannot be easily translated.
Friday, December 28, 12
![Page 139: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/139.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.• This has been very rarely done• Short leetspeak cannot be easily translated.• Applying stylometry to big and semi-structured data
is a difficult thing.
Friday, December 28, 12
![Page 140: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/140.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.• This has been very rarely done• Short leetspeak cannot be easily translated.• Applying stylometry to big and semi-structured data
is a difficult thing.• We raised our accuracy to authorship attribution
quality on a dataset that is not clean
Friday, December 28, 12
![Page 141: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/141.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.• This has been very rarely done• Short leetspeak cannot be easily translated.• Applying stylometry to big and semi-structured data
is a difficult thing.• We raised our accuracy to authorship attribution
quality on a dataset that is not clean• Stylometry helps us identify suspects and
predominant topics
Friday, December 28, 12
![Page 142: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/142.jpg)
Conclusions• We applied stylometric analysis to a huge real
world dataset.• This has been very rarely done• Short leetspeak cannot be easily translated.• Applying stylometry to big and semi-structured data
is a difficult thing.• We raised our accuracy to authorship attribution
quality on a dataset that is not clean• Stylometry helps us identify suspects and
predominant topics• We minimize manual analysis time
Friday, December 28, 12
![Page 143: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/143.jpg)
Future work• Use more user-specific features and
temporal information
Friday, December 28, 12
![Page 144: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/144.jpg)
Future work• Use more user-specific features and
temporal information• Add topic information with authorship information
Friday, December 28, 12
![Page 145: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/145.jpg)
Future work• Use more user-specific features and
temporal information• Add topic information with authorship information• Identify multiple account holders
Friday, December 28, 12
![Page 146: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/146.jpg)
Future work• Use more user-specific features and
temporal information• Add topic information with authorship information• Identify multiple account holders• Combine interaction from different media (IRC chat logs with forums)
Friday, December 28, 12
![Page 147: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/147.jpg)
Future work• Use more user-specific features and
temporal information• Add topic information with authorship information• Identify multiple account holders• Combine interaction from different media (IRC chat logs with forums)• Completely automate the process of
identifying users with sufficient text from the datasets and perform topic and authorship analysis
Friday, December 28, 12
![Page 148: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/148.jpg)
Summary• Profiling members of the underground
economy
Friday, December 28, 12
![Page 149: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/149.jpg)
Summary• Profiling members of the underground
economy• Product identification
Friday, December 28, 12
![Page 150: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/150.jpg)
Summary• Profiling members of the underground
economy• Product identification
• Discovering topics being discussed by these members
Friday, December 28, 12
![Page 151: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/151.jpg)
Summary• Profiling members of the underground
economy• Product identification
• Discovering topics being discussed by these members
• Interaction network analysis
Friday, December 28, 12
![Page 152: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/152.jpg)
Summary• Profiling members of the underground
economy• Product identification
• Discovering topics being discussed by these members
• Interaction network analysis• Challenges
Friday, December 28, 12
![Page 153: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/153.jpg)
Summary• Profiling members of the underground
economy• Product identification
• Discovering topics being discussed by these members
• Interaction network analysis• Challenges• Limitations
Friday, December 28, 12
![Page 154: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/154.jpg)
Summary• Profiling members of the underground
economy• Product identification
• Discovering topics being discussed by these members
• Interaction network analysis• Challenges• Limitations• Conclusions
Friday, December 28, 12
![Page 155: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/155.jpg)
Summary• Profiling members of the underground
economy• Product identification
• Discovering topics being discussed by these members
• Interaction network analysis• Challenges• Limitations• Conclusions• Future Work
Friday, December 28, 12
![Page 156: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/156.jpg)
JStylo
Our authorship attribution framework,powered by JGAAP and WEKA
Released in 28C3:http://events.ccc.de/congress/2011/Fahrplan/
events/4781.en.html
Friday, December 28, 12
![Page 157: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/157.jpg)
Anonymouth
Our authorship anonymization framework,powered by JStylo Released in 28C3:
http://events.ccc.de/congress/2011/Fahrplan/events/4781.en.html
Friday, December 28, 12
![Page 158: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/158.jpg)
Anonymouth
• Your writing style can give you away
Friday, December 28, 12
![Page 159: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/159.jpg)
Anonymouth
• Your writing style can give you away• Anonymouth identifies changes required for
document anonymization relative to a corpus
Friday, December 28, 12
![Page 160: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/160.jpg)
Anonymouth
• Your writing style can give you away• Anonymouth identifies changes required for
document anonymization relative to a corpus
• Assists the user making necessary changes accordingly
Friday, December 28, 12
![Page 161: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/161.jpg)
JStylo + Anonymouth = JSANOPEN SOURCE IN GIT
https://psal.cs.drexel.edu/index.php/JStylo-Anonymouth
https://github.com/psal/JStylo-Anonymouth
Friday, December 28, 12
![Page 162: Stylometry and Online Underground Marketsevents.ccc.de/congress/2012/Fahrplan/attachments/... · • BlackHat - English (Oct 2005-Mar 2008) ... Represent a forum with a graph, G=(V,](https://reader034.vdocuments.us/reader034/viewer/2022050216/5f627830ff61b0264746771b/html5/thumbnails/162.jpg)
Questions?• Sadia Afroz
• [email protected] • Aylin Caliskan Islam
• [email protected] • Ariel Stolerman
• [email protected]• Damon McCoy
• [email protected] • Rachel Greenstadt
• [email protected]• Research at PSAL Drexel
• https://psal.cs.drexel.edu/
Friday, December 28, 12