presentation brucon - anubisnetworks and ptcoresec
TRANSCRIPT
![Page 1: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/1.jpg)
1
Real time analysis and visualizationANUBISNETWORKS LABS
PTCORESEC
![Page 2: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/2.jpg)
2Agenda
Who are we?
AnubisNetworks Stream
Stream Information Processing
Adding Valuable Information to Stream Events
![Page 3: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/3.jpg)
3Who are we?
Tiago Martins
AnubisNetworks
@Gank_101
João Gouveia
AnubisNetworks
@jgouv
Tiago Henriques
Centralway
@Balgan
![Page 4: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/4.jpg)
4Anubis StreamForce
Events (lots and lots of events) Events are “volatile” by nature They exist only if someone is listening
Remember?:
“If a tree falls in a forest and no one is around to hear it, does it make a sound?”
![Page 5: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/5.jpg)
5Anubis StreamForce
Enter security Big Data“a brave new world”
Volume
Variety Velocity
We are here
![Page 6: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/6.jpg)
6Anubis StreamForce
Problems (and ambitions) to tackle
The huge amount and variety of data to processMechanisms to share data across multiple systems,
organizations, teams, companies..Common API for dealing with all this (both from a
producer and a consumer perspective)
![Page 7: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/7.jpg)
7Anubis StreamForce
Enter the security events CEP - StreamForce
High performance, scalable, Complex Event Processor (CEP) – 1 node (commodity hw) = 50k evt/second
Uses streaming technologyFollows a publish / subscriber model
![Page 8: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/8.jpg)
8Anubis StreamForce
Data format
Events are published in JSON formatEvents are consumed in JSON format
![Page 9: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/9.jpg)
9Anubis StreamForce
Yes, we love
JSON
![Page 10: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/10.jpg)
10Anubis StreamForce
Sharing Models
![Page 11: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/11.jpg)
MFE
OpenSource / MailSpike community
DashboardDashboard
Complex Event Processing
SinkholesData-theft Trojans
Real Time Feeds
Real Time Feeds
IP Reputation
Passive DNSTraps /Honeypots
![Page 12: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/12.jpg)
MFE
OpenSource / MailSpike community
DashboardDashboard
Complex Event Processing
SinkholesData-theft Trojans
Real Time Feeds
Real Time Feeds
IP Reputation
Passive DNSTraps /Honeypots
![Page 13: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/13.jpg)
13Anubis CyberFeed
Feed galore!Sinkhole data, traps, IP reputation, etc.
Bespoke feeds (create your own view)Measure, group, correlate, de-
duplicate .. High volume (usually ~6,000 events per
second, more data being added frequently
![Page 14: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/14.jpg)
MFE
OpenSource / MailSpike community
DashboardEvent navigation
Complex Event Processing
SinkholesData-theft Trojans
Real Time Feeds
Real Time Feeds
IP Reputation
Passive DNSTraps /Honeypots
![Page 15: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/15.jpg)
15Anubis CyberFeed
Apps (demo time)
![Page 16: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/16.jpg)
Stream Information Processing
Collecting events from the Stream.
Generating reports.
Real time visualization.
16
![Page 17: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/17.jpg)
Challenge
~6k events/s and at peak over 10k events/s.
Let’s focus on trojans feed (banktrojan).
Peaks @ ~4k events/s
{"_origin":"banktrojan","env":{"server_name":"anam0rph.su","remote_addr":"46.247.141.66","path_info":"\/in.php","request_method":"POST","http_user_agent":"Mozilla\/4.0"},"data":"upqchCg4slzHEexq0JyNLlaDqX40GsCoA3Out1Ah3HaVsQj45YCqGKylXf2Pv81M9JX0","seen":1379956636,"trojanfamily":"Zeus","_provider":"lab","hostn":"lab14","_ts":1379956641}
17
![Page 18: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/18.jpg)
18Challenge
![Page 19: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/19.jpg)
19Challenge
![Page 20: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/20.jpg)
Challenge Let’s use the Stream to help
Group by machine and trojan From peak ~4k/s to peak ~1k/s
Filter fields.
Geo location
We end up with{"env":{"remote_addr":"207.215.48.83"},"trojanfamily":"W32Expiro","_geo_env_remote_addr":{"country_code":"US","country_name":"United States","city":"Los Angeles","latitude":34.0067,"longitude":-118.3455,"asn":7132,"asn_name":"AS for SBIS-AS"}}
20
![Page 21: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/21.jpg)
Challenge
How to process and store these events?
21
![Page 22: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/22.jpg)
22Technologies Applications
NodeJS
Server-side Javascript Platform.
V8 Javascript Engine.
http://nodejs.org/
Why? Great for prototyping.
Fast and scalable.
Modules for (almost) everything.
![Page 23: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/23.jpg)
23Technologies Databases
MongoDB
NoSQL Database.
Stores JSON-style documents.
GridFS
http://www.mongodb.org/
Why? JSON from the Stream,
JSON in the database.
Fast and scalable.
Redis
Key-value storage.
In-memory dataset.
http://redis.io/
Why? Faster than MongoDB for
certain operations, like keeping track of number of infected machines.
Very fast and scalable.
![Page 24: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/24.jpg)
Data Collection 24
Storage Aggregate information
MongoDB Redis
Worker
Worker
Worker
Processor
Process real time events
Applications Collector
Worker
Processor
Databases MongoDB
Redis
Collector
Stream
![Page 25: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/25.jpg)
Data Collection 25
Storage Aggregate information
MongoDB Redis
Worker
Worker
Worker
Processor
Process real time events
Events comes from the Stream.
Collector distributes events to Workers.
Workers persist event information.
Processor aggregates information and stores it for statistical and historical analysis.
Collector
Stream
![Page 26: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/26.jpg)
Data Collection 26
Storage Aggregate information
MongoDB Redis
Worker
Worker
Worker
Processor
Process real time events
MongoDB Real time information of infected machines.
Historical aggregated information.
Redis Real time counters of infected machines.
Collector
Stream
![Page 27: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/27.jpg)
Data Collection - Collector 27
Collector
Old data is periodically remove, i.e. machines that don’t produce events for more than 24 hours.
Send events to Workers.Workers
Decrements counters of removed information.
Send warnings
Country / ASN is no longer infected.
Botnet X decreased Y % of its size.
![Page 28: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/28.jpg)
Data Collection - Worker 28
Worker
Create new entries for unseen machines.
Adds information about new trojans / domains.
Update the last time the machine was seen.
Process events and update the Redis counters accordingly.
Needs to check MongoDB to determine if:
New entry – All counters incremented
Existing entry – Increment only the counters related to that Trojan
Send warnings
Botnet X increased Y % in its size.
New infections seen on Country / ASN.
![Page 29: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/29.jpg)
Data Collection - Processor
Processor
29
Processor retrieves real time counters from Redis. Information is processed by:
Botnet;
ASN;
Country;
Botnet/Country;
Botnet/ASN/Country;
Total.
Persisting information to MongoDB creates a historic database of counters that can be queried and analyzed.
![Page 30: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/30.jpg)
Data Collection - MongoDB Collection for active machines in the last 24h
{"city" : "Philippine",
"country" : "PH", "region" : "N/A",
"geo" : { "lat" : 16.4499, "lng" : 120.5499 },
"created" : ISODate("2013-09-21T00:19:12.227Z "), "domains" : [
{ "domain" : "hzmksreiuojy.nl", "trojan" : "zeus", "last" : ISODate("2013-09-21T09:42:56.799Z"), "created" : ISODate("2013-09-21T00:19:12.227Z") }
], "host" : "112.202.37.72.pldt.net", "ip" : "112.202.37.72", "ip_numeric" : 1892296008, "asn" : "Philippine Long Distance Telephone Company", "asn_code" : 9299, "last" : ISODate("2013-09-21T09:42:56.799Z"), "trojan" : [ "zeus” ]}
30
![Page 31: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/31.jpg)
Data Collection - MongoDB Collection for aggregated information (the historic counters database){ "_id" : ObjectId("519c0abac1172e813c004ac3"), "0" : 744, "1" : 745, "3" : 748, "4" : 748, "5" : 746, "6" : 745,
... "10" : 745, "11" : 742, "12" : 746, "13" : 750, "14" : 753,
..."metadata" : { "country" : "CH", "date" : "2013-05-22T00:00:00+0000", "trojan" : "conficker_b", "type" : "daily" }}
31
Preallocated entries for each hour when the document is created.
If we don’t, MongoDB will keep extending the documents by adding thousands of entries every hour and it becomes very slow.
![Page 32: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/32.jpg)
Data Collection - MongoDB Collection for 24 hours
4 MongoDB Shard instances
>3 Million infected machines
~2 Gb of data
~558 bytes per document.
Indexes by ip – helps inserts and updates.
ip_numeric – enables queries by CIDRs.
last – Faster removes for expired machines.
host – Hmm, is there any .gov?
country, family, asn – Speeds MongoDB queries and also allows faster custom queries.
Collection for aggregated information
Data for 119 days (25 May to 11 July)
> 18 Million entries
~6,5 Gb of data
~366 bytes per object
~56 Mb per day
Indexes by metadata.country
metadata.trojan
metadata.date
Metadata.asn
Metadata.type, metadata.country,metadata.date,met....... (all)
32
![Page 33: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/33.jpg)
33Data Collection - Redis Counters by Trojan / Country
"cutwailbt:RO": "1256",
"rbot:LA": "3",
"tdss:NP": "114",
"unknown4adapt:IR": "100",
"unknownaff:EE": "0",
"cutwail:CM": "20",
"unknownhrat3:NZ": "56",
"cutwailbt:PR": "191",
"shylock:NO": "1",
"unknownpws:BO": "3",
"unknowndgaxx:CY": "77",
"fbhijack:GH": "22",
"pushbot:IE": "2",
"carufax:US": "424“
Counters by Trojan
"unknownwindcrat": "18",
"tdss": "79530",
"unknownsu2": "2735",
"unknowndga9": "15",
"unknowndga3": "17",
"ircbot": "19874",
"jshijack": "35570",
"adware": "294341",
"zeus": "1032890",
"jadtre": "40557",
"w32almanahe": "13435",
"festi": "1412",
"qakbot": "19907",
"cutwailbt": "38308“
Counters by Country
“BY": "11158",
"NA": "314",
"BW": "326",
"AS": "35",
"AG": "94",
"GG": "43",
"ID": "142648",
"MQ": "194",
"IQ": "16142",
"TH": "105429",
"MY": "35410",
"MA": "15278",
"BG": "15086",
"PL": "27384”
![Page 34: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/34.jpg)
Data Collection - Redis Redis performance in our machine
SET: 473036.88 requests per second
GET: 456412.59 requests per second
INCR: 461787.12 requests per second
Time to get real time data Getting all the data from Familys/ASN/Counters to the NodeJS application and
ready to be processed in around half a second
> 120 000 entries in… (very fast..)
Our current usage is ~ 3% CPU (of a 2.0 Ghz core)
~ 480 Mb of RAM
34
![Page 35: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/35.jpg)
Data Collection - API But! There is one more application..
How to easily retrieve stored data MongoDB Rest API is a bit limited.
NodeJS HTTP + MongoDB + Redis Redis
http://<host>/counters_countries
...
MongoDB
http://<host>/family_country
...
Custom MongoDB Querys
http://<host>/ips?f.ip_numeric=95.68.149.0/22
http://<host>/ips?f.country=PT
http://<host>/ips?f.host=\bgov\b
35
API
![Page 36: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/36.jpg)
Data Collection - Limitations
Grouping information by machine and trojan doesn’t allow to study the real number of events per machine. Can be useful to get an idea of the botnet operations or how many
machines are behind a single IP (everyone is behind a router).
Slow MongoDB impacts everything Worker application needs to tolerate a slow MongoDB and discard
some information has a last resort.
Beware of slow disks! Data persistence occurs every 60 seconds (default) and can take too much time, having a real impact on performance..
>10s to persist is usually very bad, something is wrong with hard drives..
36
![Page 37: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/37.jpg)
Data Collection - Evolution Warnings
Which warnings to send? When? Thresholds?
Aggregate data by week, month, year.
Aggregate information in shorter intervals.
Data Mining algorithms applied to all the collected information.
Apply same principles to other feeds of the Stream. Spam
Etc..
37
![Page 38: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/38.jpg)
Reports
What’s happening in country X?
What about network 192.168.0.1/24?
Can send me the report of Y everyday at 7 am?
Ohh!! Remember the report I asked last week?
Can I get a report for ASN AnubisNetwork?
38
![Page 39: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/39.jpg)
Reports 39
HTTP API Schedule
Get
Edit
Delete
List schedules
List reports
Check MongoDB for work.
Generate CSV report or store the JSON Document for later querying.
Send email with link to files when report is ready.
Server
Generator
![Page 40: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/40.jpg)
Reports – MongoDB CSVs Scheduled Report{ "__v" : 0, "_id" : ObjectId("51d64e6d5e8fd0d145000008"), "active" : true, "asn_code" : "", "country" : "PT", "desc" : "Portugal Trojans", "emails" : "", "range" : "", "repeat" : true, "reports" : [ ObjectId("51d64e7037571bd24500000d"), ObjectId("51d741e8bcb161366600000c"), ObjectId("51d89367bcb161366600005f"), ObjectId("51d9e4f9bcb16136660000ca"), ObjectId("51db3678c3a15fc577000038"), ObjectId("51dc87e216eea97c20000007"), ObjectId("51ddd964a89164643b000001") ], "run_at" : ISODate("2013-07-11T22:00:00Z"), "scheduled_date" : ISODate("2013-07-05T04:41:17.067Z")}
Report{ "__v" : 0, "_id" : ObjectId("51d89367bcb161366600005f"), "date" : ISODate("2013-07-06T22:00:07.015Z"), "files" : [ ObjectId("51d89368bcb1613666000060") ], "work" : ObjectId("51d64e6d5e8fd0d145000008")}
Files Each report has an array of files that
represents the report.
Each file is stored in GridFS.
40
![Page 41: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/41.jpg)
Reports – MongoDB JSONs Scheduled Report{ "__v" : 0, "_id" : ObjectId("51d64e6d5e8fd0d145000008"), "active" : true, "asn_code" : "", "country" : "PT", "desc" : "Portugal Trojans", "emails" : "", "range" : "", "repeat" : true, “snapshots" : [ ObjectId("521f761c0a45c3b00b000001"), ObjectId("521fb0848275044d420d392f"), ObjectId("52207c2f7c53a8494f010afa"), ObjectId("5221c9df4910ba3874000001"), ObjectId("522275724910ba3874001f66"), ObjectId("5223c6f24910ba3874003b7a"), ObjectId("522518734910ba3874005763")
], "run_at" : ISODate("2013-07-11T22:00:00Z"), "scheduled_date" : ISODate("2013-07-05T04:41:17.067Z")}
Snapshot{
"_id" : ObjectId("51d89367bcb161366600005f"), "date" : ISODate("2013-07-06T22:00:07.015Z"),
"work" : ObjectId("521f761c0a45c3b00b000001"), count: 123}
Results{ "machine" : { "trojan" : [ “conficker_b“ ], "ip" : "2.80.2.53", "host" : "Bl19-1-13.dsl.telepac.pt",
}, … , "metadata" : { "work" : ObjectId("521f837647b8d3ba7d000001"), "snaptshot" : ObjectId("521f837aa669d0b87d000001"), "date" : ISODate("2013-08-29T00:00:00Z") },}
41
![Page 42: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/42.jpg)
Reports – Evolution
Other reports formats.
Charts?
Other type of reports. (Not only botnets). Need to evolve Collector first.
42
![Page 43: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/43.jpg)
43Globe
How to visualize real time events from the stream?
Where are the botnets located?
Who’s the most infected?
How many infections?
![Page 44: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/44.jpg)
44Globe – Stream
origin = banktrojan
Modules
Group
trojanfamily
_geo_env_remote_addr.country_name
grouptime=5000
Geo
Filter fields
trojanfamily
Geolocation _geo_env_remote_addr.l*
KPI trojanfamily
_geo_env_remote_addr.country_name
kpilimit = 10
Stream NodeJS Browser
Request botnets from stream
![Page 45: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/45.jpg)
45Globe – NodeJS
Stream NodeJS Browser
NodeJS HTTP
Get JSON from Stream.
Socket.IO Multiple protocol support (to bypass some proxys and
handle old browsers).
Redis Get real time number of infected machines.
![Page 46: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/46.jpg)
46Globe – Browser
Stream NodeJS Browser
Browser Socket.IO Client
Real time apps.
Websockets and other types of transport.
WebGL ThreeJS
Tween
jQuery
WebWorkers Runs in the background.
Where to place the red dots?
Calculations from geolocation to 3D point goes here.
![Page 47: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/47.jpg)
47Globe – Evolution
Some kind of HUD to get better interaction and notifications.
Request actions by clicking in the globe. Generate report of infected in that area.
Request operations in a specific that area.
Real time warnings New Infections
Other types of warnings...
![Page 48: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/48.jpg)
48Adding Valuable Information to Stream Events
How to distribute workload to other machines?
Adding value to the information we already have.
![Page 49: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/49.jpg)
Minions
Typically the operations that would had value are expensive in terms of resources CPU
Bandwidth
Master-slave approach that distributes work among distributed slaves we called Minions.
49
Master
Minion
Minion
Minion
Minion
![Page 50: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/50.jpg)
Minions 50
Master receives work from Requesters and store the work in MongoDB.
Minions request work.
Requesters receive real time information on the work from the Master or they can ask for work information at a later time.
Process / Storage Minions
Master MongoDB
DNS
Scan
Minion
Minion
Requesters
Minion
![Page 51: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/51.jpg)
51Minions Master has an API that allows custom Requesters to
ask for work and monitor the work.
Minion have a modular architecture Easily create a custom module.
Information received from the Minions can then be processed by the Requesters and Sent to the Stream
Saved on the database
Update existing database
Minion
DNS
Scanning
Data Minin
g
![Page 52: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/52.jpg)
52Extras...
So what else could we possibly do using the Stream? Distributed Portscanning
Distributed DNS Resolutions
Transmit images
Transmit videos
Realtime tools
Data agnostic. Throw stuff at it and it will deal with it.
![Page 53: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/53.jpg)
53Extras...
So what else could we possibly do using the Stream? Distributed Portscanning
Distributed DNS Resolutions
Transmit images
Transmit videos
Realtime tools
Data agnostic. Throw stuff at it and it will deal with it.
FOCUS
FOCUS
![Page 54: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/54.jpg)
54Portscanning Portscanning done right…
Its not only about your portscanner being able to throw 1 billion packets per second.
Location = reliability of scans.
Distributed system for portscanning is much better. But its not just about having it distributed. Its about optimizing what it scans.
![Page 55: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/55.jpg)
55Portscanning
![Page 56: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/56.jpg)
56Portscanning
![Page 57: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/57.jpg)
57Portscanning
![Page 58: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/58.jpg)
58Portscanning
IP Australia(intervolve)
China(ChinaVPShosting)
Russia(NQHost)
USA (Ramnode)
Portugal(Zon PT)
41.63.160.0/19 (Angola)
0 hosts up 0 hosts up 0 hosts up 0 hosts up 3 hosts up (sometimes)
5.1.96.0/21 (China)
10 hosts up 70 hosts up 40 hosts up 10 hosts up 40 hosts up
41.78.72.0/22 (Somalia)
0 hosts up 0 hosts up 0 hosts up 0 hosts up 33 hosts up
92.102.229.0/24(Russia)
20 hosts up 100 hosts up 2 hosts up 2 hosts up 150 hosts up
![Page 59: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/59.jpg)
59Portscanning problems...
Doing portscanning correctly brings along certain problems. If you are not HD Moore or Dan Kaminsky, resource wise you are gonna have a bad time
![Page 60: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/60.jpg)
60Portscanning problems...
Doing portscanning correctly brings along certain problems. If you are not HD Moore or Dan Kaminsky, resource wise you are gonna have a bad time
![Page 61: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/61.jpg)
61Portscanning problems...
Doing portscanning correctly brings along certain problems. If you are not HD Moore or Dan Kaminsky, resource wise you are gonna have a bad
time
You need lots of minions in different parts of the world
Doesn’t actually require an amazing CPU or RAM if you do it correctly.
Storing all that data...
Querying that data...
Is it possible to have a cheap, distributed portscanning system?
![Page 62: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/62.jpg)
62Portscanning problems...
Minion
![Page 63: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/63.jpg)
63Portscanning
![Page 64: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/64.jpg)
64Data….
![Page 65: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/65.jpg)
65Data
![Page 66: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/66.jpg)
66Internet status...
![Page 67: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/67.jpg)
67Internet status...
![Page 68: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/68.jpg)
68If we‘re doing it... Anyone else can.Evil side?
![Page 69: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/69.jpg)
69Anubis StreamForce
Have cool ideas? Contact us
Access for Brucon participants:API Endpoint:
http://brucon.cyberfeed.net:8080/stream?key=brucon2013
Web UI Dashboard maker:http://brucon.cyberfeed.net:8080/webgui
![Page 70: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/70.jpg)
70Lol
Last minute testing
![Page 71: Presentation Brucon - Anubisnetworks and PTCoresec](https://reader033.vdocuments.us/reader033/viewer/2022052900/5558c530d8b42a995d8b46bb/html5/thumbnails/71.jpg)
71Questions?