[elasticstack]what happens when you visualize servers exposed to the world?
TRANSCRIPT
Elastic Stack
Technology Innovation Group
2017.06.20(Tue)Masamitsu Maehara
What happens when you visualize servers exposed to the world?
Self Introductionl Masamitsu Maehara
l Future Architect, Inc.
l Technology Innovation Group
l Messing around with AWS
l yurufuwa Engineer
@micci184
The Purposel Get to know the wonders of Elastic Stack
l Get to know the fun part of visualizing logs
Elastic Stack??
Elastic Stack
Elastic CLoud
LogStash Beats
Elasticsearch
Kibana
+
Security
Alert
Monitor
Graph
l Logstash/Beats:Import Logs
l Elasticsearch:Store/Index/Analyze
l Kibana:User Interface
Expose to the world?
HoneyPot
??
HoneyPotl High Interactive HoneyPot
l Use real OS and applications
l Easy access to information
l High Risk
l Low Interactive Honeypot
l Audit by emulating OS and applications
l Limited function
l Easily noticed by attackers
l Safer than high interactive HoneyPot
Dionaeal Low Interactive HoneyPot
l Gathers malware
l SMB/HTTP/HTTPS/FTP/TFTP/MSSQL/SIP
l Will create front-end view(just to make it look real)
l Low Interactive HoneyPot
l Specialized for SSH
l Better than Kippo
Cowrie
Configuration
l Built on AWS
l Install Beats on HoneyPot
l Gather data into Elastic Stack
Dionaea
Region@Virginia
HaneyPot VPC
Public Subnet
Cowrie ElasticStack
Wordpress
Client・・・
Monitoring
Attack
Logging
Install Dionaea### Ubuntu 14.04$ sudo apt-get update
$ sudo apt-get dist-upgrade
$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:honeynet/nightly
$ sudo apt-get update
$ sudo apt-get install dionaea
### Start Dionaea
$ sudo service dionaea start
Install Cowrie### Ubuntu 16.04$ sudo apt-get install git python-virtualenv libmpfr-dev libssl-dev libmpc-dev libffi-dev build-essential libpython-dev python2.7-minimal authbind
### adduser Cowrie
$ sudo adduser --disabled-password cowrie
$ sudo su - cowrie
### Setup Virtual Enviroment
$ virtualenv cowrie-env
$ source cowrie-env/bin/activate
### Install configuration file
$ export PYTHONPATH=/home/cowrie/cowrie
### Start Cowrie
$ bin/cowrie start
Activating virtualenv “cowrie-env”
Starting cowrie: [twistd -l log/cowrie.log --umask 0077 --pidfile var/run/cowrie.pid cowrie ]...
$ bin/cowrie status
cowrie is running (PID: 5979).
Beatsl Data Shipper
l The Beats FamBam
l Filebeat:Sends log files
l Metricbeat:Sends metric data(CPU/Mem..etc)l Packetbeat:Sends packet capture data
l Winlogbeat:Sends Windows event logs
l Heartbeat:Audits system stats
Data Flowl Store Apache/MySQL for WordPress directly into Elasticsearch
l Store logs from HoneyPot to Elasticsearch via Logstash
Dionaea Filebeat
Ubuntu
+ Log/dionaea.log/binalies/*
Amazon Linux
Logstash+ Input+ Filter+ Output
Cowrie Filebeat
Ubuntu
+ Log/cowrie.log
Elasticsearch
KibanaMetricBeat
Filebeat
Amazon Linux
+ Apache
Apache
WordPress
Packetbeat+ MySQL
Filebeat Modulesl Install Filebeat
$ curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.0.0-alpha2-x86_64.rpm
$ sudo rpm -vi filebeat-6.0.0-alpha2-x86_64.rpm
### Configuring
$ vim /etc/filebeat/filebeat.yml
#------------------------------- Apache2 Module ------------------------------
- module: apache2
# Access logs
access:
enabled: true
var.paths: ["/var/log/httpd/access_log"]
error:
enabled: true
var.paths: ["/var/log/httpd/error_log"]
#-------------------------- Elasticsearch output -------------------------------
output.elasticsearch:
hosts: [“xxx.xxx.xxx.xxx:9200"]
Ingest Pluginsl Install Ingest Geoip & Ingest user agent
l Ingest Geoip:Maps IP addresses to maps
l Ingest user agent:Deals with user agent as it thinks best
l Install Ingenst Plugins on the Elastic Stack server
### Ingest Geoip
$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-geoip
### Ingest user agent
$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-user-agent
!!Attention #01l Be mindful of proxy environments
l Will receive a timeout error when installing Ingest Plugins
l Make sure to define proxy setting on the startup script before installing
$ sudo /usr/share/elasticsearch/bin//elasticsearch-plugin install ingest-user-agent
-> Downloading ingest-user-agent from elastic
Exception in thread "main" java.net.ConnectException: Connection timed out
### Setup Proxy
$ export ES_JAVA_OPTS="-Dhttp.proxyHost=xxx -Dhttp.proxyPort=xxx -Dhttps.proxyHost=xxx -Dhttps.proxyPort=xxx"
### Install ingest-user-agent
$ /usr/share/elasticsearch/bin/elasticsearch-plugin install ingest-user-agent
-> Downloading ingest-user-agent from elastic
[=================================================] 100% ### Ingest Geoip
!!Attention #02l Be mindful of proxy environments
l Handy setting that imports Dashboards when starting Filebeat#Configure dashboard settings on filebeat.yml
l However in environments with proxy settings it will not work! :(
l In that case, install it manually
$ sudo vim /etc/filebeat/filebeat.yml
#============================== Dashboards =====================================
- #setup.dashboards.enabled: false
+ #setup.dashboards.enabled: enable
$ sudo /usr/share/filebeat/scripts/import_dashboards -file /tmp/beats-dashboards-x.x.zip -eshttp://xxx:9200
Visualization♥
Beautiful♥
Until recently…
l Send logs to be visualized from Filebeat to Logstash
l Normalize received logs with Logstash and store it on Elasticsearch
l Create a dashboard on Kibana to make it cool
Sooooo,,,Who needs Logstash?
Wait!
Logstash & Dionaea
Malwarel Are these malwares on Dionaea?
l They’re all over /opt/dionaea/var/dionaea/binaries…
$ ll /opt/dionaea/var/dionaea/binaries
-rw------- 1 dionaea dionaea 53 Jun 6 02:59 d41d8cd98f00b204e9800998ecf8427e.gz
-rw------- 1 dionaea dionaea 162168 Jun 7 22:56 dc8c32d7f26352c8484bc490b6467843.gz
-rw------- 1 dionaea dionaea 153820 Jun 7 02:34 dd0400bed68d272b08d1d0272bc18462.gz
-rw------- 1 dionaea dionaea 129803 Jun 5 01:01 de1e602b2452a95ba57ef53347e50094.gz
-rw------- 1 dionaea dionaea 22778 Jun 6 17:38 e0ddd8bf8e3b97ad25855721dc75daae.gz
-rw------- 1 dionaea dionaea 155154 Jun 7 04:33 e53ed987e82ad7bf076c23d91401cac7.gz
-rw------- 1 dionaea dionaea 1189 Jun 8 15:32 ead49a9b7b0c8ad6894be45674cebf77.gz
-rw------- 1 dionaea dionaea 22777 Jun 6 17:39 eb18a7d302bbc8c0b3ed2cd1612e8d59.gz
…
-rw------- 1 dionaea dionaea 21966 Jun 5 16:52 ee0efafc69a13cd57d714ffdc603d8fc.gz
-rw------- 1 dionaea dionaea 154329 Jun 4 16:48 f09ee5028fd1b1eaaf22df1538de159b.gz
-rw------- 1 dionaea dionaea 156637 Jun 9 08:51 f5f1fd0d093d81a4a769c20aca1d6232.gz
-rw------- 1 dionaea dionaea 29643 Jun 8 15:34 fc9b0b8b711e44ce0d4f91b0cedb1c76.gz
ClamScanl What do you do when you suspect a malware? You scan it.
l Malware FOUND$ clamscan /opt/dionaea/var/dionaea/binaries/
/opt/dionaea/var/dionaea/binaries/f09ee5028fd1b1eaaf22df1538de159b.gz: Win.Worm.Kido-200 FOUND
/opt/dionaea/var/dionaea/binaries/621c0b356c49edc5ce4cf3ee88c30f82.gz: OK
/opt/dionaea/var/dionaea/binaries/90e02a26204ade7771acf7e8521bdf09.gz: Win.Worm.Kido-297 FOUND
/opt/dionaea/var/dionaea/binaries/02830b424d88664cc3576941dd9841f9.gz: Win.Worm.Kido-307 FOUND
/opt/dionaea/var/dionaea/binaries/a7bc14c1bd7271a45391f1e1541afe43.gz: Win.Worm.Downadup-110 FOUND
/opt/dionaea/var/dionaea/binaries/87136c488903474630369e232704fa4d.gz: Win.Worm.Kido-113 FOUND
/opt/dionaea/var/dionaea/binaries/1195dfde6305980ed050a9751b157f42.gz: Win.Worm.Kido-293 FOUND
/opt/dionaea/var/dionaea/binaries/1b4cd56e54d3f9030a153590fb3fa9e5.gz: Win.Worm.Kido-316 FOUND
/opt/dionaea/var/dionaea/binaries/fc9b0b8b711e44ce0d4f91b0cedb1c76.gz: OK
/opt/dionaea/var/dionaea/binaries/cae8a8524eeb0e7de1fb3704bd14b7ba.gz: Win.Trojan.Ramnit-1847 FOUND
/opt/dionaea/var/dionaea/binaries/7bb455ea4a77b24478fba4de145115eb.gz: Win.Worm.Kido-197 FOUND
/opt/dionaea/var/dionaea/binaries/eb18a7d302bbc8c0b3ed2cd1612e8d59.gz: OK
/opt/dionaea/var/dionaea/binaries/smb-az4poq4s.tmp.gz: OK
/opt/dionaea/var/dionaea/binaries/16acf30169d089b8a967f40d9a38d8f7.gz: Win.Trojan.Agent-129152 FOUND
What if we want to surveillance these malwares in realtime?
Data Flowl Regularly runs ClamScan to output logs
l Filebeat sends the logs to the Elastic Stack server
l Logstash normalizes the logs and stores them into Elasticsearch
l Visualize with Kibana
Dionaea Filebeat
Ubuntu
+ Log/binalies/*/log/scan.log
clamscan.sh
Amazon Linux
Logstash+ Input+ Filter+ Output
Elasticsearch
KibanaMetricBeat
By the way, do you use
Logstash?
Logstash vs fluentdl Compared on Google Trend
l By country
l Blue:Logstash
l Red:fluentd
Grok filter
Grok Filter
l Below were the ClamScan results
l We need to somehow normalize it to get certain Key-Value data
l Data we want
l OK/FOUND (Key : check)
l Malware Name (Key : malware)$ clamscan /opt/dionaea/var/dionaea/binaries/
/opt/dionaea/var/dionaea/binaries/f09ee5028fd1b1eaaf22df1538de159b.gz: Win.Worm.Kido-200 FOUND
/opt/dionaea/var/dionaea/binaries/621c0b356c49edc5ce4cf3ee88c30f82.gz: OK
/opt/dionaea/var/dionaea/binaries/90e02a26204ade7771acf7e8521bdf09.gz: Win.Worm.Kido-297 FOUND
/opt/dionaea/var/dionaea/binaries/02830b424d88664cc3576941dd9841f9.gz: Win.Worm.Kido-307 FOUND
Such a pain in the neck…
Grok Constructorhttp://grokconstructor.appspot.com/do/match
Grok Constructor
l Let’s you test on web browsers
l You can also check stdout on Logstash
l Convenient if you don’t want to rewrite Logstash.conf
Paste log here
Grok Filter
Click GO! after pasting
Grok Constructor
l Results look like this
lOK/FOUND is contained in “check”
l But, where is the malware name?
l Work on that Grok Filter again
use contents detected in data
Grok Filter
Click GO! after pasting
Grok Constructor
l Results look like this
l Malware name is matched in malware!
Logstash.conf
l Final product looks like this
input {beats {
port => 5044}
}filter {
grok {match => [ "message", "/[^/]+/[^/]+/[^/]+/[^/]+/(?<field>[^/]+)/%{GREEDYDATA:data}%{WORD:check}"]remove_field => [ "host", "message" ]
}grok {
match => [ "data", "(?:[¥w._/%-]+)%{WORD}(?:[:]*)%{GREEDYDATA:malware}"]remove_field => [ "data" ]
}}output {
elasticsearch {hosts => "http://xxx.xxx.xxx.xxx:9200/"
}}
Looking good
Malware♥
Summaryl Let Elastic Stack do everything from input to output
l Easy visualization with Beats
l Expose your server to expand your log variation
l Are you excited to share your server with the public?
l Don’t let Logstash beat you!
Thanks