eevi –framework for evaluating the effectiveness …eevi –framework for evaluating the...

EEVi – Framework for Evaluating the Effectiveness of Visualization in Cyber-Security

Aneesha Sethi, Federica Paci, Gary Wills

Aneesha.Sethi@soton.ac.uk

School of Electronics and Computer Science

December 5, 2016International Conference for Internet Technology and Secured Transactions

§ Visualization in Cyber-Security

– Presents information as visual analytics rather than string of text and characters for analysis.

– An effective tool that helps detect, monitor and mitigate sophisticated technical and social attacks in a timely manner1.

– There is an outburst of these tools that focus on different aspects of cyber-security visualization ranging from a high level view of the system to a technical low level view.

Introduction

2University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>1 G. Fink, C. North, A. Endert, and S. Rose. Visualizing cyber security: Usable workspaces. In Visualization for Cyber Security, 2009. VizSec 2009. 6th International Workshop on, pages 45–56, Oct 2009.

§ Visualization in Cyber-Security

Introduction

3University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

4© Created with Storyboard That, Sublime-text Text Editor (https://www.flickr.com/photos/xmodulo/14391734181/) by xmodulo License: Attribution (http://creativecommons.org/licenses/by/2.0/), fon graph (https://www.flickr.com/photos/cromo/185028548/) by Cromo License: Attribution, Non Commercial (http://creativecommons.org/licenses/by-nc/2.0/), 2-core graph of #rstats people (https://www.flickr.com/photos/hjl/4094315135/) by hjl License: Attribution (http://creativecommons.org/licenses/by/2.0/), indica-website-graph (https://www.flickr.com/photos/indi/271647921/) by indi.ca License: Attribution (http://creativecommons.org/licenses/by/2.0/)

University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

What Leads to these Problems?

5

§ The visualizations are rarely evaluated for effectiveness in terms of the task they aid in performing.

§ Most of the visualizations are developed and often evaluated without any user-involvement.

§ The techniques used to evaluate most tools were not standardized.

These factors led to a low adoption rate of tools presenting cyber-security visualisations. Thus, the need for a common framework, evaluating the effectiveness of cyber-security visualisations for the performed task arose.

University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

EEVi - Framework

6University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

Methodology – Thematic Analysis

University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk> 7

5 papers were chosen that provide details about security analyst roles, the types of data used, how analysis were conducted, what the analysts thought about visualization approaches and their experiences, if any, with visualizations. These 5 paper were read and a list of ideas represented was formulated.

The initial list of ideas was further analysed to produce an initial set of codes called a codebook. The codes represent the qualitative aspects of the framework.

The themes were defined on the basis of the results from the codebook and relationships between themes start to appear. A theme captures the significance of the data and represents a patterned response which is reflected by the group of codes it defines.

The themes, codes and relationships or links are identified as a result which were further analysed to design the framework.

Thematic Analysis

1. The first step was to identify the relevant papers and form an initial list of ideas.

2. The next step was to form the codebook of initial set of codes based on the list of ideas formulated.

8University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

Thematic Analysis

1. The first step was to identify the relevant papers and form an initial list of ideas.

2. The next step was to form the codebook of initial set of codes based on the list of ideas formulated.

3. The third step was to form themes by collating the codes:1. Analysis of Data – Task performed by security analysts;

2. Data – Type of data used to perform tasks;

3. Feature of Visualization – Features of visualization required to perform the tasks;

4. Role of Analyst – The security analyst that perform the tasks.

4. Finally, the last step was to record the results along with the relationships formed.

9University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

Thematic Analysis

4. Finally, the last step was to record the results along with the relationships formed.

10University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

EEVi - Framework

11University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

Types of Analysis of Data (Task)

The codes led to the definition of eight type of tasks:

§ Triage Analysis

§ Escalation Analysis

§ Correlation Analysis

§ Threat Analysis

§ Incident Response Analysis

12University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

§ Forensic Analysis

§ Impact Assessment

§ Security Quality Management

Analysis

University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk> 13

Summary

§ Low user-involvement and not having standardised evaluation lead to low adoption rates of cyber-security visualization tools.

§ EEVi presents a common framework, based on user requirements, to standardize evaluation and act as guidelines to develop effective cyber-security visualizations.

14University of Southampton, Aneesha Sethi <Aneesha.Sethi@soton.ac.uk>

Thank you!

Questions?-Aneesha Sethi

University of SouthamptonAneesha.Sethi@soton.ac.uk

15