florian mansmann and svetlana vinnikflorian mansmann and svetlana vinnik abstract—network...

TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, ... 2006 1

Interactive Exploration of Data Traffic withHierarchical Network Maps

Florian Mansmann and Svetlana Vinnik

Abstract— Network communication has become indis-pensable in business, education, and government. Withthe pervasive role of the Internet as a means of sharinginformation across networks, its misuse for destructivepurposes, such as spreading malicious code, compromisingremote hosts or damaging data through unauthorizedaccess, has grown immensely in the recent years. Theclassical way of monitoring the operation of large networksystems is by analyzing the system logs for detectinganomalies. In this work, we introduceHierarchical NetworkMap, an interactive visualization technique for gaining adeeper insight into network flow behavior by means ofuser-driven visual exploration. Our approach is meant asan enhancement to conventional analysis methods basedon statistics or machine learning. We use multidimensionalmodeling combined with position and display awareness toview source and target data of the hosts in a hierarchicalfashion with the ability to interactively change the levelof aggregation or apply filtering. The interdisciplinaryapproach integrating data warehouse technology, infor-mation visualization, and decision support, brings aboutthe benefit of efficiently collecting the input data andaggregating over very large data sets, visualizing theresults, and providing interactivity to facilitate analyticalreasoning.

Index Terms— Data and knowledge visualization, Infor-mation Visualization, Visual Analytics, Network Security

I. I NTRODUCTION

SINCE the Internet has de-facto become theinformation medium of first resort, each host on

the network is forced to face the danger of beingcontinuously exposed to the hostile environment.Therefore, network security has turned into one ofthe central and most challenging issues in networkcommunication for practitioners as well as for re-searchers. Security vulnerabilities in the system areexploited with the intention to infect computers withworms and viruses, to ”hack” company networksand steal confidential information, to run crim-inal activities through the compromised network

F. Mansmann and S. Vinnik are with the University of Konstanz,Germany.

infrastructure, or to paralyze online services throughdenial-of-service attacks. Frequency and intensity ofthe attacks disallow any laxity in monitoring thenetwork behavior of the system.

Various automatic methods, such as firewalls,virus scanners and intrusion detection systems, haveemerged as a response to the need of protectingthe systems from harmful network traffic. However,as there will always exist human and machinefailures, no fully automated method can provideabsolute protection. One may distinguish betweenunintentional defects due to human failures or othermalfunctions, referred to asflaws, and the inten-tional misuses of the system, known asintrusions.

A non-trivial task of detecting different kinds ofsystem vulnerabilities can be successfully solvedby applying the visual analytics approach. When-ever machine learning algorithms become insuffi-cient for recognizing malicious patterns, advancedvisualization and interaction techniques encourageexpert users to visually explore the relevant dataand take advantage of human perception, intuition,and background knowledge. In the process of humaninvolvement acquired knowledge can be further usedfor advancing automatic detection mechanisms.

Intrusion detection is the major preventive mech-anism for timely recognition of malicious use ofthe system endangering its integrity and stability.Our research is concerned primarily withanomaly-basedintrusion detection systems (IDS) which offera higher potential for discovering novel attacks,compared to thesignature-basedIDS. Anomaly-based detection is carried out by defining the normalstate and behavior of the system with alerts sentout whenever that state is violated. It is a rathercomplicated task to define the normal behaviorprecisely enough as to minimize false alerts andespecially not to let any attacks evolve unnoticed.This is where our approach enters.

We offer a visual view on the system behaviorwith the input data read from the system logs of


network gateway routers or other sources loggingnetwork traffic. The user is free to explore theinput along various dimensions and to retrieve thedetails of the network flows. IP addresses and otherattributes are modeled as hierarchical dimensionsin which the instances are grouped into upper-class categories at multiple levels, thus forminga tree structure. The visual interface is optimizedfor displaying hierarchies, preserving geographicalposition, proximity, and the size of network nodes,thus facilitating their comparability.

We call our techniqueHierarchical Network Map,since it organizes the display area as a map witha zoomable rectangle area for each network, us-ing the associated IP addresses for computing thenode’s coordinates. The underlying data model wasinspired by the OLAP (online analytical processing)approach used in building data warehouses for effi-ciently managing huge data volumes and computingaggregates under high performance requirements.

The rest of the paper is organized as follows: inthe next section we discuss work related to our re-search, followed by the introduction of our approachin section 3. The underlying multidimensional datamodel is presented in section 4. Implementationand visualization issues can be found in section5. Section 6 presents the results and findings ofapplying the Hierarchical Network Map, followedby an evaluation of our work. Conclusions andfuture work are given in section 8.

II. RELATED WORK

The goal of visualization for computer security isto display misbehavior and failures within a singlecomputer or a whole network and to ultimatelysupport the administrator in his task to gain insightinto their causes. For network security, this goal canbe achieved either through interpretation of eventsproduced by an IDS or through measurement oftraffic peculiarities. Our work focuses on the latteraspect due to the lack of availability of IDS data.However, this kind of data could be easily importedinto our database and analyzed in our system.

An important subarea is the visualization of portactivity which is an indication to the running net-work applications. Lau [1], for example, presentedthe Spinning Cube of Potential Doom. The visu-alization is based on a rotating cube used as 3Dscatterplot. The variables such as local IP address

space, port number, and global IP address space, areassigned to its axes. Each measured traffic packet ismapped to a point in the 3D scatterplot. The cubeis able to show network scans very easily due toemerging patterns such as horizontal and verticallines or areas. Another example is thePortVis tooldescribed by McPearson et al. in [2]. It implementsscatterplots (e.g., port/time or source/port), port ac-tivity displays and various means of interaction tovisualize and detect port scans as well as suspiciousbehavior on certain ports. Muelder et al. [3] furtherextend the tool by producing wavelets from scat-terplots of network scans. Those wavelets are thenclustered to classify the scans according to theircharacteristics.

In some scenarios, the amounts of alerts producedby an IDS are too large to be scanned manually.Visualization modules such asIDS RainstormbyAbdullah et al. [4] bridge the gap between largedata sets and human perception. They provide ascatterplot-like visualization of local IP addressesversus time enhanced by zooming functionalities.The IDS events are then connected by lines to theglobal IP address space in the detail views.

A. IP Address Space Visualizations

Errors (e.g., routing instabilities) and intrusionsin computer networks were the focus of a studyconducted by Teoh et al. [5]. The authors intro-duced a recursive QuadTree that assigns positionsaccording to the prefixes of the IP addresses. Foreach event, the point representing the IP addressis connected with the two involved autonomoussystems (placed outside of the QuadTree) by lines.Depending on the security events occurring in thenetwork, characteristic patterns appear.

Fink and North [6] introduce theRoot PolarLayoutof the IP address space. Their assumption isthat the classification of the IP address space in trustlevels can reveal useful information. IP addresses ofthe highest trust level are drawn in the inner circle,with IP addresses of lower trust levels arranged onconcentric rings around this circle.

The major advantage of ourHierarchical NetworkMap versus the QuadTree approach and the RootPolar Layout is a more meaningful positioningof the IP addresses on the display. We combinenetwork topology with geographic information foran improved understanding of and orientation in


0 50 100 150 200 250

110

100

1000

0

8 bit IP address prefix

# of

con

nect

ions

from

sou

rce

Fig. 1. Density histogram showing the distribution of sessions overthe IP address space.

the analyzed data sets. However, due to the un-certainty involved in the geographic placement ofautonomous systems, misplacements are to be takeninto account.

B. Commercial IDS Visualization Modules

It is worth mentioning that visualization tech-niques like graphs and Parallel Coordinates havemeanwhile found their way into commercial prod-ucts, such as theRNA Visualization ModuleofSourceFire [7]. In this context, the techniques areused to display the connectivity of network nodesand the correlations in the multi-dimensional char-acteristics of associated network traffic.

III. H IERARCHICAL NETWORK MAP APPROACH

Hierarchical Network Mapis a visual explorationapproach aimed at displaying the distribution ofsource and target data traffic of network nodes(IP addresses) in a more expressive way than asimple density diagram like the one shown in Fig.1. The intention to provide an overview of the entireInternet on the screen has forced us to treat everypixel as a valuable asset and to use a space-fillingtechnique. Compared to node-link diagrams, ourtechnique has three major advantages: 1) no displayspace is wasted, 2) larger data sets can be visualizedmapping network size to the node’s area, and 3)better support for multi-resolution exploration (drill-downs have only local effects on the layout).

The whole network structure can be viewed asa hierarchy with the IP addresses of single hostsas the bottom-level nodes. Each host belongs toa local IP network which, in turn, is membersof an autonomous system. On top of the network

levels mentioned above, the IP address hierarchy isextended by two geographic upper classes, namely,countries and continents. The display rationale is torecursively nest child node rectangles in their parentrectangles, from the continent level down to localnetworks (see Fig. 2), or even single hosts.

In many space-filling hierarchical visualizationssuch as TreeMaps [8], the area is partitioned ac-cording to a variable statistical measure so that thesizes of the resulting rectangles correspond to therespective value. In our scenario, choosing the per-node traffic as a measure would lead to constantlychanging sizes and positions of node rectangleson the display thus aggravating user’s orientation.Since the success of our technique is measured byits applicability for continuous network monitoringand analysis, recognition and familiarity turn intocritical requirements. Therefore, we map the screenarea and its partitioning to a rather non-volatile (atleast in the short-run) measure, namely, the total sizeof the network and its components. Given a constantratio of the display space, this design results in analmost static map layout, as network architectureshardly experience any changes in the short term.

While the containment relationships within rec-tangles show the IP address hierarchy and classmembership, the size of each hierarchical region,from a single IP address all the way up to awhole continent, is proportional to the number of IPaddresses it contains. Furthermore, the geographicalnodes, i.e. those at country and continent level,additionally preserve their relative geographical po-sition (similar to a cartogram). It is this geographicawareness of the node that allowed us to classifyour technique as amap. At the level of autonomoussystems and local networks, the contained rectan-gles appear sorted by IP address in a left-to-rightand top-down fashion, thus accelerating the visuallookup of any sub-node.

In terms of analytical reasoning, the statisticalvaluesX = (xi)i=1,...,n with xi ≥ 0 and xi ∈ R inour scenario are the per-node traffic loads measuredas a total number of bytes, packets, or sessionswithin a specified time frame. These statisticalvalues are represented by the filling color of thenode’s rectangle areas, from dark blue for the lowerbound to dark red for the upper one. We favorthe logarithmic color scale as shown in Fig. 2 dueto its improved discrimination for low values andsmoothing effect on outliers.


(a) World view (b) Continent view (c) Country view

Fig. 2. Multi-resolution approach using theHierarchical Network Mapto investigate the outgoing traffic of a chosen local network withinthe University of Konstanz in terms of number of packets sent from 27th September to 4th of August 2005. a) In the world view, withthe European continent drilled down, one may observe high network traffic targeting Germany. b) A continent view of Europe, with adrill-down on Germany, reveals that the volume at a single autonomous system (AS 553, Belwue) in Germany exceeds the aggregated trafficto Switzerland. c) In the view of Germany, a drill-down on BelWue (the research network of the federal state Baden-Wurttemberg, Germany)confirms the expectation that our university network (134.34.0.0/16) receives most data packets.

Further characteristics of network flows are im-plemented as filters, which are either compulsory oroptional.

Compulsory filters:1) Type of load: Since each IP address functions

both as source (sending packets) and as target(receiving packets), the network load to dis-play can refer either to the packets sent, orthe packets received, or their total.

2) Time window: Each visualization instance dis-plays the traffic within a specified time inter-val.

Optional filters:1) Port or Port Cluster: Single ports or their

groupings can be used as filter to display onlythe portion of traffic occurring at those ports(for instance, showing the outgoing email loadby selecting the source port 25 (SMTP)).

2) Protocol: It might be useful to filter the trafficof a particular protocol, for instance to sepa-rate UDP traffic from that of TCP.

It turned out to be infeasible to process the entire

network data the way it is protocolled by the systemlogs. The generated amount of ”raw” operations issimply too large to store and analyze even on high-performance workstations. For example, to applyour approach for exploring traffic at a gateway of amiddle-sized university would result in storing sev-eral gigabytes of log data per hour. To avoid inputdata overflow, we store the aggregated operations,in which the packets are grouped into sessions andthe sessions of related flows, i.e. those referring tothe same source and target within close timestamps,are summed up. Therefore, the load recorded bysuch aggregated entry is described by the numberof sessions, the number of packets transferred, andthe transferred bytes.

Our proposed visualization can be applied both inonline mode, for actual network traffic monitoring,and in offline mode, for exploring the network’sbehavior in a selected time and space window. Letus consider each operation mode to reveal theirconstraints and implications in terms of requiredinput data and performance.


A. Network Flow Exploration (Offline Mode)

In offline mode, past traffic data is loaded fromthe database. The user specifies the type of loadand the time window to set as compulsory filters.Optional filters can be specified as well or added inthe process of interaction.

Suppose that a network administrator is con-cerned about the degrading performance of UDPtraffic in the network as more and more packetsget dropped. He starts off by choosing the targetload filter to trace where the traffic was sent to, andproceeds by selecting a time period starting at themoment he first discovered the problem. Finally, theprotocol filter is activated to show only UDP traffic.Running theHierarchical Network Maphelps himto formulate a hypothesis about the target distrib-ution of the analyzed data flows. Interactively, hediscovers that most of the traffic is sent to Germanyand selects another date to verify whether such largeamounts of traffic are typical for that target node.To verify whether the spotted traffic pattern is theresult of a continuous growth or whether it appearedall of a sudden, the network administrator runs ananimated view of the daily traffic from the previousmonth. Observation of a continuous upward trendmake it obvious that there is a need for betterconnectivity to the target network.

B. Network Flow Monitoring (Online Mode)

Monitoring network flows in online mode isa challenging task due to high data arrival rateand run-time requirements. Monitoring can be per-formed by refreshing the display either continuouslyor periodically. Continuous re-rendering imposesextreme costs since each incoming event affectsthe visualization. Therefore, it is rather imperativeto define a reasonably short interval in which thevisualization gets updated with newly arrived data.

In many networks, daily operations are moni-tored by measuring the performance of the gatewayrouters. We believe that by extending the mereobservation of traffic statistics to also take intoaccount the source and target distribution in termsof IP addresses involved, may result in improvedadjustment of the routing policies of large networks.In the long term, considerable cost savings can beachieved if the same performance of the network isachieved with less hardware resources due to moreintelligent routing policies.

IV. N ETWORK TRAFFIC ASOLAP CUBES

We have found theOLAP (On-Line AnalyticalProcessing) architecture [9] to be a suitable optionfor processing huge amounts of network data in ananomaly detection scenario. OLAP based solutions,initially adopted by traditional business applications,have proven to be beneficial and extendable to servea much wider range of application domains, suchas health care, biology, government, education, andnetwork security, to name a few. An example foran advanced visual OLAP tool is Polaris [10]. Theunderlying multidimensional data model[11] al-lows mapping of detailed transactional data, denotedfacts, as being arranged in a multidimensional space,or hypercube. The numerical values under analysis,termedmeasures, are characterized by descriptivevalues, drawn from a number ofdimensions. Thevalues of any single dimension can be further or-ganized in a containment-typehierarchy, thus ana-lytical queries can efficiently compute the measureaggregates for various levels of detail.

Transactional facts are periodically retrieved fromnetwork traffic logs and stored in the fact tableLog.Initial log entry describes a single network packettransfer by storing source and target IP addressesand ports, the timestamp, and the packet size. Toreduce the volume of facts to be stored, all packetsand session referring to the same source, destinationand timestamp are grouped into a single fact, withthe number of sessions, the number of packets andtheir total size in bytes as its three measures.

Each ofLog’s dimensions has one or more hier-archies defined upon it, as follows:

• SourceIPandTargetIPare grouped bynetwork→ autonomous system→ country→ continent.Hosting country of an autonomous system isdetermined by looking up the geographicalpositions of the IP addresses of all the networksit contains and choosing the prevailing country.For this, we rely on the GeoIP database ofMaxmind, Ltd. [12] (approx. 99% accuracy).

• Timestampsare aggregated asmillisecond→second→ minute→ hour→ day→ month→year.

• SourcePort and TargetPort may be groupedinto categories(such as well-known, registeredand dynamic ports) on the one hand, and intodomains(e.g., web, email, database, etc.), onthe other hand.


Fig. 3. Modelling network traffic as an OLAP cube.

Overview of the above logical database design isdepicted in Figure 3. Decomposing dimensional ta-bles into subtables for each granularity level resultsin a logical design calledsnowflake schema. Forperformance considerations the dimensionTimewasleft denormalized.

A. Measures

Notice that the originalLog table allows to traceeither the outgoing or the incoming traffic loadat each IP address, but not their total. This totalload is obtained by deriving a new fact tableLoad(Timestamp, IP, Port, NumSessions, NumPackets,NumBytes), whereby the outgoing (SourceIP, Sour-cePort) and the incoming (TargetIP, TargetPort)traffic for each IP address is merged by joining theoriginal Log table or its aggregate with itself.

B. OLAP Operations and Queries

The fact table contains a huge volume of ”raw” orpoorly aggregated data. OLAP operations are usedfor computing a measure’s aggregates (subtotals,averages, bounds) for a chosen combination ofdimensions and their granularity levels:

• Roll-up(aggregating) anddrill-down (disaggre-gating) allow to change the granularity level.

• Slice-and-dicedefines a subcube of interest oreven reduces the dimensionality by ”flattening”single dimensions to one selected value.

• Ranking can be applied as a dynamic filtershowing the specified number of marginal (topor bottom) measure aggregates.

Since our visualization approach restricts itself to arather limited set of OLAP operations, we were ableto relax some of the classical OLAP restrictions inpart of hierarchy properties, such as balancedness,covering and strictness. For instance, one port maybelong to multiple domains or not belong to any.

C. Performance via Pre-aggregation

As elaborated in the previous section, it may beunfeasible or simply impossible to manage the entirelog data ”as it is” in the database. Besides, the olderthe data entries get the less interesting they are forthe analysis, since anomaly detection is basicallyconcerned with recent and especially current trafficflows. Therefore, we suggest to split the Log tableinto multiple subtables with varying level-of-detail:

• ShortTermLogstores the most recent transac-tions (e.g., for the last one hour) “as theyare”, i.e. without any aggregation. This tablewill be used for therun-time network loadvisualization.

• MiddleTermLogstores the network data of thecurrent day with the granularity reduced frommillisecond to minute. This table offers stillrather detailed information for offline explo-ration of the current day’s network behavior.

• LongTermLogaggregates the transactions evenfurther (e.g., by hour or day) to store them ashistorical data for less detailed exploration.

Other granularity levels alongTime dimension canbe defined depending on the application needs.

V. I MPLEMENTATION AND V ISUALIZATION

ISSUES

The starting point of our visualization techniqueis a squarified treemap[13] in which the aspectratio of the resulting rectangles is optimized toproduce nearly square partitions, thus improving thevisibility of the mapped hierarchical structure andthe comparability of rectangle areas. We extend theabove approach by setting rectangle’spositioningwith respect to its neighbors to be another relevant


TABLE I

CONFLICTING OPTIMIZATION CRITERIA

spaceutilization area position aspect ratio

space utilization X X

area X X

position X X X

aspect ratio X X X

optimization criteria. Additionally, we make use ofthe color attribute.

Our work has been inspired by geographicaldistortion methods such as PixelMap [14] and His-toScale [15] that reposition points and polygons.RecMap [16] as a screen partitioning algorithmis capable of displaying leaves of a hierarchicaltree, but does not visualize the hierarchy itself.Hierarchical Network Mapis a further developmentof the spatial distortion methods introduced in His-toMap [17]. The main advancement over HistoMapis an optimization towards the use of hierarchical(network hierarchy) and one-dimensional data (IPaddresses). Furthermore, our algorithm tries to avoidlong and thin rectangles to enhance their visibilityand comparability.

We figured out that the following four crite-ria are to be taken into consideration: 1) Fullspace utilization: an optimal solution for space-filling is achieved by recursively splitting the displayspace into two rectangles. 2) Preservingpropor-tions across rectangles: this criteria is fulfilled bysetting the proportions of the two resulting rectan-gles according to the number of IP addresses theyrepresent. 3)Geographic awareness(position): Theoptimal positions are taken to initialize the layout,but are not considered in the process of furtheroptimization. 4)Aspect ratioof rectangle areas isoptimized by evaluating several alternative layouts.Some of these criteria are conflicting (see Table I),for instance, aspect ratio conflicts with full spaceutilization, since the former cannot avoid renderingrather ”stretched” screen partitions.

A. The Partitioning Algorithm

As an initial setting, we have one spatial pointpi

for each continent, country, autonomous system, ornetwork;P = {p1, . . . , pn} where(pi)i=1,...,n ∈ R2.A weight wi ∈ N is attached to each rectangle andis equal to the number of IP addresses represented

by pi. Functionarea determines the area of eachrectangle on the screen as follows (dx is the widthanddy the height of the display):

area(ri) =wi∑ni=1 wi

× dx × dy (1)

The algorithm searches for a partitioning of the dis-play space into|P| = n rectanglesR = {r1, ..., rn}.Each time a point setP is split into P1 andP2, therepresentative rectangler is split into two rectanglesr1 andr2. Split direction depends on the squarenessof r1 and r2; position of the split line betweenthe rectangles is determined by the sum of weightsassociated with each point set.

procedure partition (P )begin

if |P | > 1 then(P1, P2)← splitRect (P );partition (P1);partition (P2);

elsedrawRect (P );

endprocedure splitRect (P )begin

(P1,P2) = splitHorizontal (P );(P3,P4) = splitVertical (P );if quality (P1,P2) > quality (P3,P4) then

return (P1,P2)

elsereturn (P3,P4)

end

By allowing the algorithm to do onelook-ahead,we considern × 22 solutions; instead of checkingjust two solution options (like above), the functionsplitRect has to check four alternatives. We definea quality function for the aspect ratios of the result-ing rectangles. The optimum is set to squares forvisibility and comparability:

quality(P) =1

n

n∑i=1

wi ×max

(drx

i

dryi

,dry

i

drxi

)(2)

B. Data Preprocessing

The algorithm needs a set of spatial pointsP asinput. Forcontinentandcountrylevels of the hierar-chy, the assignment is a chosen central point withineach continent or country. However,autonomoussystemandnetwork levels are only defined in one-dimensional space. To acquire the missing seconddimension, we setpx

i = pyi = ipmid, whereipmid =


1m

∑mj=1 ipj (m is the number of IP addresses con-

tained in the node). Arising conflicts between twoor more autonomous systems or networks havingthe same spatial point assigned are resolved duringthe recursion by moving the first point to the left.

From the fact thatpxi = py

i , we conclude thatvertical and horizontal partitioning split the initialpoint set into the same resulting point sets. Ex-ploiting this fact, we can calculate the splits andweights a priori and run the optimization only on thewidths and heights of the resulting rectangles. Thisreduces the runtime complexity fromO(n2×2`+1) toO(2`+1+n×log(n)), where` is the number oflook-aheadsat each level. Arranging one-dimensionalitems into a binary tree (each item needs to be aleaf) can be seen as a sorting problem solvable inO(n× log(n)). Note that our algorithm computes agood solution to the problem rather than the optimalone.

C. Descending to the Pixel Level

The technical limit of a visualization is to useevery pixel for displaying distinct values, wherethe latter are mapped to the pixels’ color. Furtherattributes of the data points can be mapped to thepixels’ coordinates. Pixel-based visualization buildsup the finest granularity view of theHierarchicalNetwork Map, namely, the bahavior of single hosts.For instance, it can be employed for determin-ing whether the network traffic just comes froma limited number of IP addresses or is scatteredover the considered network (see Fig. 4). Pixelsare arranged according to therecursive pattern[18]starting at the top left corner going down row byrow with alternating forward-backward direction forbetter cluster preservation. Thereby, the displayed IPaddresses appear sorted in descending order. Usinga large display wall with 8.9 Megapixels, we canshow up to 8.9 million IP addresses mapping eachIP address to one pixel.

D. Interaction

Interactivity is the key to many visual analyticsapplications, and ourHierarchical Network Mapisby no means an exception. The user chooses whichregion of the network traffic data set at hand shouldbe investigated in more detail, with the choice ofthe next step depending on the patterns revealed inthe preceeding exploration steps.

Fig. 4. Recursive pattern pixel visualization can show the activehosts within the network and reveal their distribution behavior. Left:pattern of a network scan, affecting every10th IP address. Right: apattern with only a few target hosts. Moving the cursor over the pixel(or its surrounding region) triggers label appearance.

Explorative tasks are enabled through typicalOLAP database operations such as drill-down (dis-aggregation), roll-up (aggregation) and slice & dice(filtering). All these basic interactions are mappedto the mouse wheel and left mouse button to fostereasy interaction. A popup menu (right mouse but-ton) offers additional support for untrained users aswell as some expert features such as multiple selec-tion or drill-down / roll-up operations on the nodesof the same level. Furthermore, interactive time, hostand port activity diagrams can be accessed throughthis menu.

E. Visual Scalability

The term visual scalability [19] describes thecapability of visualization tools to display large datasets in terms of the number or the dimensionalityof data elements. We designed theHierarchicalNetwork Map to be highly scalable to support alarge tree structure (7 continents, 237 countries,19 731 autonomous systems and 188 248 networks)yet with special effort to facilitate the overview. Towhich extent this explorative potential can actuallybe exploited depends on the available display size.Technological advancements such as large displaywalls enhance scalability while maintaining theoverview (see Fig. 5).

F. Visual Hints

In our technique, the visual variable color is usedfor mapping the specified traffic load measure. Em-ploying a bipolar colormap (blue to red over whiteas the switching point) supports the observation thatnot only the existence of high load values (dark


Fig. 5. Hierarchical Network Mapdisplaying all 19 731 autonomous systems (one can still zoom in twice for details) on a large displaywall (5.20m× 2.15m, 8.9 Megapixels, powered by 8 projectors). The query interface on the top left shows the traffic distribution over timeand specifies the selected data, in this case the traffic entering the gateway of the University of Konstanz onwell-known ports (0-1023)onNov. 29, 2005 using “transferred bytes” as measure with logarithmic color mapping. One recognizes a heavy traffic load from AS 3320 (red)of “Deutsche Telekom” as well as to neighboring autonomous systems in Germany. A port histogram reveals high activity on the web ports80 and 443. For security and privacy reasons, the data was aggregated and sanitized.

continent country autonomous system network

Fig. 6. Borders at each level have distinct colors to preserve thevisibility of various hierarchy levels. Borders are only drawn if arectangle is higher and wider than 2 pixels.

red) can imply interesting findings, but also the non-existence of any traffic (dark blue). Use of squareroot and logarithmic color scales helps to make thevisualization more resistant to outliers, taking intoaccount that comparing nominal values on the colorscale becomes infeasible. For this purpose, linearmapping is also available.

Notice that adjacent rectangles are visually sepa-rated from each other solely by one-pixel thin bor-ders, irrespective of the rectangles’ resolution. Noadditional borders are placed around parent rectan-gles in order to maximize the effective display area.To improve visual perception of the hierarchical

structure represented by the borders and of the depthof rectangle partitions, the borders are highlightedwith a distinct color for each granularity level (seeFig. 6). For better visibility of the labels, these aredisplayed in black or white depending on the bettercontrast to the rectangle in the background.

VI. RESULTS AND FINDINGS

To demonstrate the intrusion detection potentialof the Hierarchical Network Map, let us considersome use scenarios. In case oftrojan horse ac-tivity, malicious code in form of so-called rootkits installs itself on the infected computer andconnects to internet relay chat (IRC) servers (TCPports 194, 529, 994, 6665-6669) from where itreceives further commands. The trojan horseBack-door.Ryknos.B, for example, establishes connec-tions to the public IRC servers at the IP ad-dresses 68.101.14.76, 24.210.44.45, 67.171.67.190,152.7.24.186 or 35.10.203.93. The network admin-istrator can proceed by filtering out the traffic onthe affected ports and checking for possible de-pendencies between chatting activities within thesupervised network and known compromised chatservers.


Another example is theslammer wormthat sendsitself to the UDP port 1434, on which the MicrosoftSQL Server Resolution Service listens. Due to asecurity vulnerability, the worm infected many MSSQL servers and opened countless connections atUDP port 1434 to randomly chosen IP addressesleading to network traffic congestion. We tested ourvisualization by choosing the measure target con-nections on such a scenario. The simulated trafficto random destinations made those networks moreprominent (white to red) in the visualization.

Running two parallel instances of the HierarchicalNetwork Map for source and target IP addresses,respectively, of the same data set is a powerful wayof filtering distinct data flows and exploring the datafrom several perspectives. A specific destinationnetwork can be chosen (such as the network withthe highest load) and matched with the locallyadministrated IP address space. This is especiallyuseful when supervising a large network.

Investigation of the historical trends can be im-proved by pre-normalizing the color map to thehighest occurring value within the whole time span.In the online monitoring mode, however, the max-imum is unknown and, therefore, has to be eitherestimated or adapted dynamically.

With respect to UDP versus TCP traffic behavior,at the time of our measurements we made the fol-lowing observation: TCP traffic prevails at westerndestinations, such as the United States, whereasUDP loads dominate towards Asia. Europe naturallyplays the central role considering the traffic ofboth protocols, apparently due to the geographicallocation of our gateway and the orientation of itsusers towards German web contents and services.

Another usage scenario is shown in Figure 7. Inthis case, the Hierarchical Network Map serves as agrouping function for host activity histograms overthe number of connections and transferred bytes. Ingeneral, few connections and high transfer volumeindicate a download link whereas many connectionsand low transfer suggest “chatty” network services.

VII. E VALUATION

We presented our visualization to the securityexpert of our university. From her perspective, itwas very important to show detailed host activitieswithin a network, autonomous system or country.Usually, she compares the number of connections

TABLE II

PERFORMANCE MEASURES

1 h data set 24 h data set

237 countries 2 sec. 20 sec.

19 731 autonomous systems 9 sec. 36 sec.

188 248 networks 23 sec. 60 sec.

with the transferred bytes of active hosts to findsuspicious patterns. After getting this valuable feed-back, we integrated host activity diagrams (seeFig. 7) as well as port activity diagrams into ourapplication. While using our system, we realizedthat some very small networks with high traffic areonly rendered as a few pixels on the screen and arethus hardly perceivable. Even though a large displaywall would compensate for this shortcoming, weplan to find a solution for improving their visibility.

With respect to the use of geographic informa-tion, we realized two things: a) in some casesneighborhood preservation for countries fails andb) assigning autonomous systems to countries in-volves uncertainty; thus erroneous information canbe communicated. However, until now we havefound only one misplaced autonomous system. Thesystem’s strength lies in the possibility to show net-work traffic at different granularity, so that detailedinformation can be retrieved at continent, country,autonomous system, or network level.

In our performance experiments, we loaded theworld view in three different scenarios, showing all237 countries, all 19 731 autonomous systems andall 188 248 networks (see Table II). Concerningthe performance of our application, updating allcountries in the world view with pre-aggregateddata spanning 1 hour takes about 2 seconds; a dataset spanning 24 hours (hourly aggregated) can beloaded within 20 seconds. The system runs on anApple Power Mac G5 machine with Dual 2.3 GHzclock speed and 2 GB RAM. The input data is man-aged in a PostGreSQL 8.0 database. The databaseperformance could be significantly improved byemploying high-end server hardware and by usinga commercial database product. Note that typicalstarting point of the exploratory analysis process isa highly aggregated representation in which the userperforms a series of successive drill down steps.Thus, the subset of data to be loaded at each step,can normally be done within few seconds.


Fig. 7. Interactive Data Exploration. 1) We start by specifying the data set, the measure (sum of all incoming connections), the filter(discarding heavy mail, DNS, and web traffic) as well as the date. 2) Having launched the Hierarchical Network Map, we select all autonomoussystems within Germany and drill down to the networks. For further analysis, we open a large network (84.128.0.0/10) as we expect moreconnections to different hosts there than in small networks. 3) The host activity diagram is displayed using “transferred bytes” as measure.By applying a logarithmic scale (only outliers are visible on a linear scale), we identify a set of neighboring hosts that sent the same amountof bytes. 4) The second diagram with the measure “number of connections” reveals the same pattern. After an examination of the used ports,we suspected the trojan Netcontroller on several hosts to actively communicate with a computer in our network on port 123. However, thishypothesis was discarded as only UDP traffic was measured, indicating the network time protocol (NTP).

VIII. C ONCLUSIONS

Our work can be seen as a novel application inthe field of computer security visualization. To thebest of our knowledge, such large amounts of IP-related network traffic data have never been shownin a geographically aware hierarchical and space-filling context.

In this work, we presented theHierarchical Net-work Map as a hierarchical space-filling mappingmethod for visualizing traffic of IP hosts. Our algo-rithm is an application of the space-filling variantof the RecMap [16] algorithm which was modifiedto fit our needs with respect to one-dimensional(IP addresses) and hierarchical data (network hierar-chies). The approach aims at finding a compromisebetween four optimization criteria: using displayarea to represent network size, full utilization ofthe screen space, position awareness, and aspectratio. The performance challenges are mastered byemploying OLAP cube operations commonly usedin data warehouses to efficiently aggregate overlarge volumes of detailed data.

We have experimented with traffic data from ouruniversity’s gateway and from our local computersto demonstrate that visual exploration of the actualnetwork behavior as well as simulation of anomaly

scenarios reveal valuable insights in network secu-rity. A nested recursive pattern pixel visualizationfurther extends the exploration paradigm by usingsingle pixels as the lowest physical level of thevisualization. To adapt the visualization for the largehierarchy at hand, we use one-pixel-thin coloredborders between the nodes of each level. Further-more, intuitive mouse interactions are implementedto facilitate the explorative task.

In the future, we want to further optimize thesplit sequence of our algorithm with respect tothe best measured screen layout and to conduct aformal evaluation of this aspect as well as of theperformance aspects of our database and visualiza-tion. Additionally, we plan to introduce some novelvisualization concepts that foster the exploration oftime and port dimensions. Innovative devices tofacilitate the interaction with the large display wallwill be considered as a traditional mouse does notefficiently support the moving analyst.

To master the network traffic flow in real-time,we plan to apply a streaming database for moni-toring purposes in online mode. The challenge tohandle is the update strategy for maintaining thepre-aggregation architecture.

As a further application of the map, we plan


to load IP-related data from Intrusion DetectionSystems to find non-trivial correlations in the IPdimension within cyber attacks. Moreover, we wantto employ our methods to find hacked computerswithin the university network. As long as thosecomputers are not used for network scans or denialof service attacks, their network traffic does notreveal any statistically significant peculiarities. Yet,they are a ticking time-bomb.

ACKNOWLEDGMENT

The authors thank Daniel A. Keim, Mike Sips,Christian Panse, Benjamin Bustos and AlexanderHinneburg for many fruitful discussions and theanonymous reviewers for their valuable input. Fur-thermore, we would like to give special thanks toDuncan Sparrell and his group from AT&T, as wellas to Barbara Lohle and Marcel Waldvogel from thecomputing center of the University of Konstanz forproviding valuable data for designing and evaluatingour approach. This work was partially funded by theGerman Research Foundation (DFG) under grantGK-1042, Explorative Analysis and Visualization ofLarge Information Spaces, University of Konstanz.

REFERENCES

[1] S. Lau, “The spinning cube of potential doom,”Communica-tions of the ACM, vol. 47, no. 6, 2004.

[2] J. McPherson, K.-L. Ma, P. Krystosk, T. Bartoletti, andM. Christensen, “Portvis: a tool for port-based detection ofsecurity events,” inProc. ACM workshop on visualization anddata mining for computer security, 2004.

[3] C. Muelder, K.-L. Ma, and T. Bartoletti, “A visualiza-tion methodology for characterization of network scans,” inProc. IEEE Workshop on Visualization for Computer Security(VizSEC), Minneapolis, U.S.A., October 2005.

[4] K. Abdullah, C. Lee, G. Conti, J. A. Copeland, and J. Stasko,“Ids rainstorm: Visualizing ids alerts,” inProc. IEEE Workshopon Visualization for Computer Security (VizSEC), Minneapolis,U.S.A., October 2005.

[5] S. T. Teoh, T. Jankun-Kelly, K.-L. Ma, and S. F. Wu, “Visualdata analysis for detecting flaws and intruders in computernetwork systems,”IEEE Transactions on Computer Graphicsand Applications, September/Oktober 2004.

[6] G. A. Fink and C. North, “Root polar layout of internet addressdata for security administration,” inProc. IEEE Workshop onVisualization for Computer Security (VizSEC), Minneapolis,U.S.A., October 2005.

[7] Sourcefire. (retrieved on 11/11/2005) Real-time network awareness. [Online]. Available:http://www.sourcefire.com/products/rna.html

[8] B. Johnson and B. Shneiderman, “Tree maps: A space-fillingapproach to the visualization of hierarchical information struc-tures.” in IEEE Visualization, 1991, pp. 284–291.

[9] E. F. Codd, S. B. Codd, and C. T. Salley, “Providing OLAP (on-line analytical processing) to user-analysts: An IT mandate,”Technical report, E.F.Codd & Associates, 1993.

[10] C. Stolte, D. Tang, and P. Hanrahan, “Multiscale visualizationusing data cubes.”IEEE Trans. Vis. Comput. Graph., vol. 9,no. 2, pp. 176–187, 2003.

[11] T. B. Pedersen and C. S. Jensen, “Multidimensional databasetechnology,”IEEE Computer, vol. 34, no. 12, pp. 40–46, 2001.

[12] Maxmind, Ltd. (2005) Geoip database. [Online]. Available:http://www.maxmind.com

[13] D. Bruls, C. Huizing, and J. van Wijk, “Squarified treemaps,”in Proceedings of the joint Eurographics and IEEE TCVGSymposium on Visualization, 2000.

[14] D. A. Keim, S. C. North, C. Panse, and M. Sips, “Pixelmaps: Anew visual data mining approach for analyzing large spatial datasets,” inICDM 2003, The Third IEEE International Conferenceon Data Mining, 19-22 November 2003, Melbourne, Florida,USA. IEEE Computer Society, November 2003.

[15] D. A. Keim, S. C. North, C. Panse, M. Schafer, and M. Sips,“HistoScale: An efficient approach for computing pseudo-cartograms,” inIEEE Visualization 2003, Seattle, Washington,USA, October 2003, pp. 28–29.

[16] R. Heilmann, D. A. Keim, C. Panse, and M. Sips, “RecMap:Rectangular Map Approximations,” inInfoVis 2004, IEEE Sym-posium on Information Visualization, Austin, Texas, October2004, pp. 33–40.

[17] D. A. Keim, F. Mansmann, C. Panse, J. Schneidewind, andM. Sips, “Mail explorer - spatial and temporal exploration ofelectronic mail,” in Proc. Eurographics/IEEE-VGTC Sympo-sium on Visualization (EuroVis 2005), Leeds, United KingdomJune 1st-3rd, 2005.

[18] M. Ankerst, D. A. Keim, and H.-P. Kriegel, “Recursive pattern:A technique for visualizing very large amounts of data,” inProc.Visualization ’95, Atlanta, GA, 1995, pp. 279–286.

[19] S. G. Eick, “Visual scalability,”Journal of Computational &Graphical Statistics, March 2002.

Florian Mansmann finished his Bachelor de-gree in Information Engineering at the Uni-versity of Konstanz (Germany) in 2003. In2004, he graduated from the Solvay Manage-ment School at the Vrije Universiteit Brussel(Belgium) as a Master of Business InformationManagement. Since his graduation he is ascholar of the Graduate College ExplorativeAnalysis and Exploration of Large Informa-

tion Spaces at the University of Konstanz. His research fields areInformation Visualization, Network Monitoring and Data Mining.

Svetlana Vinnik received the Bachelor’s de-gree in International Economic Relations fromthe Belarusian State University, Belarus, in1999, and the Master’s degree in InformationEngineering from the University of Konstanz,Germany, 2003. She is currently working asa research assistant in the Databases and In-formation Systems Group at the Universityof Konstanz, Germany. Her research fields

are data warehousing and business intelligence, decision support,multidimensional data modeling, and e-government.

florian mansmann and svetlana vinnikflorian mansmann and svetlana vinnik abstract—network...

Documents