Demystifying Data Analytics & Visualization

Make Your Data Dance

2

• This guy?• Definition & Discussion: “Big Data Hype”• What is an analytic?• How do we visualize• Demo: of Data Analytics and Visualization• Questions/Discussion

Today’s Agenda

This Guy?

3

My Wife!

Creepy Kids

My Wife Made

4

• Its everywhere• We all hear it, but what does it mean?• Does it really mean anything or is it just more

marketing hype?• Is bigger really better?

Big Data or Big Hype?

5

• How many logs do we have now?

• Too many to count• Not just on your file

system, but in traffic too!

• Human – Human• Machine – Human• Machine - Machine

Logs Logs Everywhere

• Linux/Unix/Mac(BSD)• Microsoft• Bro Logs

– Or plain Netflow generation

• Snort or other IDS• Switches/Routers

6

What do you do with all this?

7

• How do you decide which logs you want?– Compliance– Policy– Curiosity– Just because

• Normalization– On the fly (streams)– On the remote/local file system (batch)

Get Them In Your Database

8

• Tools for Transport:– Flume, fluentd, rsyslog, syslog-ng, sqoop, logstash

• Tools for Storage:– Note: Relational/Non-relational is important– mySQL, cassandra, Hadoop (HDFS), Elasticsearch

• Degree’s of Wholeness– ELSA, graylog2, Snare

Some Free Tools To Help

9

• All data is not gold• You need a strategy that gets you the right data

at the right time

Data is Big... But So What?

10

• Wikipedia Definition – “the discovery and communication of meaningful patterns in data”

Defining: Analytics

11

• Simple!• What! • A question?!• I can understand that!• These questions can be used to create

– Metrics– Statistics– Network behaviors– These all help the application of Analytics as analytics

help are used to create them.

Simply a Question

12

• I received an IDS alert, is there other similar behavior on my network that I did not receive an alert for?

• I have an IP blacklist, what hosts on my network connected to those IP addresses?

• Better yet, is there other similar behavior on my network to non–black-listed IP addresses?

Ask Questions of Your Data

13

• Unpatched Systems• Misconfigured Devices• File access

– Rates– Personnel

• Visibility– Of your network– Of your hosts

What Other Kinds of Insight

14

• So you normalized and stored the data• You’ve asked good questions of our data with

analytics• Now what?• We visualize• But how?

Visualization.

15

Demo Time!

16

Questions?

Source links in the notes on this slide

jlawler@21ct.com

17