Overview of IETF work on IP traffic flow measurement
and current developments
Dr. Jürgen Quittek
General ManagerNetwork Research Division, NEC Europe Ltd.
Heidelberg, Germany
ITU-T Workshop onIP Traffic Flow Measurement
(Geneva, Switzerland, 24 March 2011)
Geneva, 24 March 2011
… …Flows can be long lasting...
… or have a limited lifetime...
… …
… and packets may belong to more than one flow
Typical reported flow information:•start time•end time•#packets•#bytes
tPeriodically reported for long lasting flows
IP packets and flowsGroups of IP packets sharing common characteristics (e.g IP src/dst address, TOS field, protocol, transport layer ports, etc.)
2
Classification &Flow Recording
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PacketCapturing
Filtering
Samplingpackets
Filtering
Samplingflow records
flow records
packets
packets
flow records flow records
packetreports
both steps may be trivial (1:1 sampling, no filtering)
The general (passive) IP traffic measurement process
Exportingprocess
ObservationPoint
(router, probe, etc.)
Metering process
3
………
…Meter:
Filters packets,timestamps
themand associatesPkts to flow(s)
Flow cache:Creates/Removes/Updates
flow records
• Flow Key• Flow start time• Flow last update time• # Pkts• # Bytes
•….
•….
Collector:Receives export packets, interfaces to applications
info info infoExp HDDatabase
Exporter:Reads Flow cache,
prepares and sends export packets
info info infoExp HD
Router functionalityor dedicated Probe
The flow monitoring process
4
IETF IPFIX(Netflow v9)
Flow monitoring issues
Flows have very different characteristics long-/short-lived, high/low volume, etc.
Creating/updating flow record at high speed links
packet sampling fast memory for flow cache, flow sampling
Timing out flows (TCP FIN/RST vs. timeout)Reporting
flow cache reading effort, reporting frequencyselective report
Reporting formatfixed format: Netflow 5template based: Netflow 9, IPFIX
5
IETF activities on IP traffic measurement
Three working groupsIPPM: IP Performance Metrics
defines metrics for performance measure-ments (delay, roundtrip time, loss, etc.)
IPFIX: IP Flow Information eXportdefines protocol for export of flow data
PSAMP: Packet Sampling (concluded)defines protocol for export of packet databased on IPFIX
6
IPFIX protocolIP Flow Information eXport
Established 2001Main goal: Develop common IP traffic flow reporting protocol to be available on most future routers
meeting requirements of many applicationslow hardware/software costssimple,Scalableextensible
http://datatracker.ietf.org/wg/ipfix/ 7
Distinguishing flows by5-tuple (IP addresses, protocol, port)MPLS label, TOS fieldsinterface & direction
Flexible aggregation of flowsMetering Process
timestampsflow timeouts
Further requirements for IPFIX I
8
Extensible information/data modelflow properties and statistics
many header fields
anonymization
Reliable and secure data transfercongestion awarenesspush model reporting
Configuration
Further requirements for IPFIX II
9
IPFIX architecture
Application
Flow Record
Observation Point
Flow Information Export
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
PAYLOAD HEAD PAYLOAD HEAD
MeteringProcess
ExportingProcess
CollectingProcess
10
OM
E
Probe
OM
E
Simple Router
O OO OM
E
Complex Router
O OO OMO OO O
M
E
Multiple Exporters
O OO OM
E
O OO
OM
E
Protocol Converter
(Meter MIB)
OM
E
OM
E
OM
E
M E
Concen-trator
C E
Proxy
C …
IPFIX devices
11C
EM
OMetering ProcessExporting Process
Collecting Process
Observation Point
IPFIX protocol design
Based on NetFlow version 9Binary-coded flow record arraysTemplates for flow record formats
first send a templatethen send data records with the format defined by the template
Runs over SCTP, TCP, UDP
12
IPFIX information model
A flow record containsheader fields (transport, IP, sub-IP)
"flow keys" used for distinguishing flows
counters for packets, bytes, etc.time stamps further flow properties
min/max values, duration, directionnext hop IP addressBGP source AS, destination AS, next hop AS
may also be used as flow keys
All defined as "Information Elements"
13
IPFIX normative documents
RFC 5101: Specification of the IPFIX Protocol for the Exchange of IP Traffic Flow Information, 2008RFC 5102: Information Model for IPFIX, 2008RFC 5103: Bidirectional Flow Export Using IPFIX, 2008RFC 5473: Reducing Redundancy in IPFIX and PSAMP Reports, 2009RFC 5610: Exporting Type Information for IPFIX Information Elements, 2009RFC 5655: Specification of the IPFIX File Format, 2009RFC 5815: Definitions of Managed Objects for IPFIX, 2010
core protocol specification
14
IPFIX informational documents
RFC 3917: Requirements for IPFIX, 2004
RFC 3955: Evaluation of Candidate Protocols for IPFIX, 2004
RFC 5153: IPFIX Implementation Guidelines, 2008
RFC 5470: Architecture for IPFIX, 2009
RFC 5471: Guidelines for IPFIX Testing, 2009
RFC 5472: IPFIX Applicability, 2009
RFC 5982: IPFIX Mediation: Problem Statement, 2010
15
Current issues in the IPFIX WG
Configurationinterface for configuring IPFIX devicesdefined as YANG module
Mediationparticularly for large networks
driven by NTTaggregationanonymization
Flow selectionStructuring flow records
extending IPFIX capabilities
Using IPFIX for reporting other informationMIB variables, SIP server logs, etc. 16
PSAMP
Established in Summer 2002Focus on sampling and capturing packets and on transferring them to data collectorsTarget applications
traffic profilingmonitoring network behavior
Extends IPFIX export Defines packet sampling with much more detail
packet filtering and sampling information model
17
IPPM"The IPPM WG will produce documents that define specific metrics and procedures for accurately measuring and documenting these metrics:"
connectivityone-way delay and lossround-trip delay and lossdelay variationloss patternspacket reorderingbulk transport capacity (BTC = data_sent / elapsed_time)link bandwidth capacity
Refer to WG official page for list of already published RFCs and ID
http://datatracker.ietf.org/wg/ippm/ 18
Final remarksThe IETF developed IPFIX as standard protocol for reporting IP flow informationTechnology is mature
many implementationsseveral interoperability testing eventsmajor router vendors expected to release IPFIX soon as part of standard installation
IPFIX is extensibleBGP-related flow info can already be reportedadditional information elements can be added
IPFIX can be used to report measurements at peering points appropriate metering hardware required
19