Monitoring JANET: from photons to flows

Demands - Architectures - Regulation

Steve WilliamsJANET

2

Monitoring JANET

from photons to flows

• Demands:

•From researchers

•From NOCs/institutions/’power’ users

•Layer 1, 2, 3 … metrics

•Routing (BGP/ISIS) – updates/full feed

•Packet capture data (up to 40Gbps)

•Flow data

•Legal requirements

•RIPA, DPA, private data, ethical behaviour

•Indivudual vs institutional data & summary vs detail data

•Research topics in the UK

•Visualization

•Signature analysis / DPI

•Anomaly processing

•Optical networks/switching

•Issues:

•Standards in measurements

•Standard approach across measurement types

•Interchange of data between systems

•Hard to compare measures directly

•What is a ‘good’ link?

Requirements

The SuperJANET55th Objective

-Network Visibility

Steve Williamss.williams@ja.net

3

Monitoring JANET

from photons to flows

JANET and Research

• Aim to support and encourage research and assist where possible by providing bandwidth and services that help researchers

• Assist network research by providing access to the network

• Need to balance:– operational vs research requirements…– legal vs research requirements

Monitoring is research driven

UKLIGHT ‘network’ projects

Real-time Monitoring

Compression (Model Free & Model Dependant)

MASTSAnalysis at All Scales

in Time & Space

Front End Enhancements: Advanced Protocols

Data Analysisand

Traffic Modelling

46PaQProtocol Innovation

Behaviour And PerformanceFor QoS and Control

Tech

nolo

gy

Inn

ovati

on

Tech

nolo

gy

Inn

ovati

on

Netw

ork

Scie

nce

Netw

ork

Scie

nce

Ap

plicati

on

sA

pp

licati

on

s

TowardsNext

Generation

Networks

ESLEAApplications and the Network

ESLEAApplications and the Network

Data Acquisition,Storage & Archiving

SuperJANET and UKLIGHTSuperJANET and UKLIGHT

Research Activities and Projects

OPORONOPORON

OPSnetOPSnet

LUCIFERLUCIFER Control plane and network resource provisioning for Grid and eScience

UPCUPC Networking with ultra-short pulses:-OTDM transmission-OTDM switching-Time-Slot Interchange

40G all-opticalburst Rx

EdgeOPS

Optical Transport Network

OXCLabelSwitching Routers

CoreOPS

SONET/SDH

ephoton/ephoton+ephoton/ephoton+

OBS

PROTAGONPROTAGON

MUFINSMUFINS

TRIUMPHTRIUMPHDevelop switching node for:- Bit-rate adaptation - Multiwavelength regen

Collector ring

HIPNETHIPNETModelling end to end QoS across heterogeneous nets

UFORICUFORICHardware based CBR with optical impairments

Example: Research at ESSEX University

6

Monitoring JANET

from photons to flows

- LHC- JIVE- DEISA

-Layer 0 network

-UKLight STM/GE circuits

7

Monitoring JANET

from photons to flows

YHMAN

Dublin (HEANET)

NIRAN

C&NLMAN

UHIFaTMAN

AbMAN

Clydenet

EaStMAN

NorMAN

NNW

TVN LMN

EastNetMidMAN

LeNSE

WREN

SWERN

Kentish MAN

EMMAN

Bristol

Glasgow

Reading

Warrington Leeds

LondonT-City T-House

Overall Network:- 5,815km New Dark Fibre- 112 optical nodes/sites-746Gbps operationa capacity

-60 x 10G plus 2.5, 1

Core Network:- 8 nodes- 2,290Km fibre- 20 Terminal systems, 23 Amp/OADM- Longest un-regen – 554km- Longest span – 243.6km (51.2db)

Regional Network access:- 3,561km Fibre- 24 optical networks- 73 sites- 36x10G, 26xSTM16,16GE and an STM1

8

Monitoring JANET

from photons to flowsGraphic removed

9

Monitoring JANET

from photons to flows

- IP-SLA- Ping- SNMP- Perl

•Performance•RTT•OWD•Loss SD/DS

•HTTP•DNS•TCP Connect•HTTP transfer•Total time

•DNS•Response time

•VoIP•MOS•Connect

•NTP•Time

RN

RNSuperJANET4/5

V240

Extreme 450

V2403TB V480

Firewall

Cisco 35xx

1000Mbps/TX

Core MP-1

Measurement probes

Inst-meas-1

Inst-meas-1

Inst-meas-1

Private network

Probe/Measure traffic

10

Monitoring JANET

from photons to flows

The JANET Measurement System

11

Monitoring JANET

from photons to flows

12

Monitoring JANET

from photons to flows

- Traffic types- Top talkers- Incident tracking- Anomaly detection- Signature analysis- etc

- Challenges:- >70k flows/sec

at 10:1 sample rate

- T640’s not good at sampling

13

Monitoring JANET

from photons to flows

Multicast

14

Monitoring JANET

from photons to flows

AG conference@5mbps

Multicast

Add soton screen shots

15

Monitoring JANET

from photons to flows

JANET IPJANET

Lightpath

Optical Transmission (DWDM)

Beyond IP

16

Monitoring JANET

from photons to flows

JANET Optical Core

SuperJANET5 – Optical monitoring:

Feed to Optical Switch and Optical monitoring equipment

Packet data to content level:

Incident/Anomaly detection, Signature analysis

DWDMJuniperT640

Flow data:

Traffic typesIncident detection

Packet headers onlySampled

Optical NE data:Alerts - Light on/off

Ciena/VzB - SNMP traps Nortel via SNMP

17

Monitoring JANET

from photons to flows

Optical layer packet capture

1 – 40Gbps

Anomaly detectiionIncident tracking

Packet signature analysisEncrypted signatures

ProductionSJ5 fibres

18

Monitoring JANET

from photons to flows

What was that about sniffing packets

Isn’t that interception?

19

Monitoring JANET

from photons to flows

• JANET is a private network– No public/walk in access– Only access to

staff/students/bona fide visitors– Breach of RIP by network

operations staff is civil case not criminal

– Breach by non-operations staff is still criminal (student hackers etc…)

– http://www.ja.net/development/legislation/laws.html

20

Monitoring JANET

from photons to flows

Research access to data

• Principle of ‘least disclosure’– No access to data not required– Use anonymisation where possible– Access to full data only in cases where

proven case presented– No access to header and payload data– Payload summary data/signatures

• Researchers agree to and sign policy– Non-disclosure of data– Maintain privacy– Scope of research work

• This policy works only because JANET is a private network.

Data Gatherers

TapeBackup

Data ThroughputManager

Measurement Databaseand Central Controller

StandbyDatabase

WebAlertsNRG

...

DeliverySub-systems Gridmon

Flow monitoringCDF

...

ExternalDatabases

Users

MeasurementSystemsJANET

JANET

MeasurementSystem

Measurement systemapplication

NTP Ping

Measurement controller

Data Gatherers

...

Web

ser

vice

AP

I Meas’mentjob configuration

fileData timeslices

Data timeslices Datatimeslices

...

23

Monitoring JANET

from photons to flows

Key issues

• Architecture of monitoring is key• Interchange of data between sytems• Stability and comparability of data

between locations and across time

• The legal framework cannot be ignored– Some researchers try…– Some countries have less stringent

laws

24

Monitoring JANET

from photons to flows