Precise Traffic GeneratorGeoff Salmon, Monia Ghobadi, Yashar Ganjali,

Martin Labrecque, J. Gregory SteffanUniversity of Toronto

What is the Precise Traffic Generator (PTG)?Uses NetThreads on the NetFPGA to

generate packets.Goals

1. Send packets precisely at the requested time.

2. Allow packets to be constantly streamed from host PC.

No separate “load phase”.

Why Are These Goals Interesting?1. Precise transmission times:

Can emulate diverse network conditions and topologies.

2. Stream packets from host PC: Allows generated traffic to be reactive

ie. TCP flows with congestion control

Motivation: Network Testbed ExperimentsTesting systems which are sensitive to packet

arrival times (ie. packet buffering and scheduling in routers) requires realistic test traffic.Need to explore a wide range of traffic

With a small testbed, this is difficult or impossible.

With a large enough testbed, this is difficult and expensive.

Simplified ProblemA few computers can create required traffic

volume.However, resulting traffic may not be

realistic.

How is the traffic different?The PTG focuses on controlling the packet transmission times.

Components of the PTG

3. pktgen

Linux Kernel

NetFPGA

Packet Travel Direction

2. nf2 Linux Driver

NetLink

GigE1.

NetThreads Application

PCI

Host Computer

Userspace Program: pktgenSimple command-line utility used for testing

PTG.Sends descriptions of packets and

transmission times to the driver.Uses inter-transmission times and payload

sizes that are either:fixed, orread from files.

Communicates with driver using a NetLink socket.Easily replaced by other userspace or kernel

code.

nf2 Linux DriverModified version of nf2 driver.Main jobs:

Receive packet descriptions from NetLink socket.

Build command packets containing multiple descriptions.

Send command packets to the NetThreads app over the PCI bus.

NetThreads ApplicationEight threads of execution in NetThreads:

7 threads receive command packets and prepare packets to transmit in output memory.

1 thread sends packets from the output memory at correct times.

Input Memory Output Memory

PCI Bus is a BottleneckCannot copy all packets across PCI bus.Main idea: do not copy packet payloads

Payloads are zeroed when sent from NetFPGA.Experiment often only look at packet headers

anyway.PCI transaction overheads require driver to

gather multiple packet descriptions into single command packet.

Transmission Time PrecisionCompare against Stanford’s Packet

Generator (SPG)PTG/SPG

Sending fixed rate packets

NetFPGA router

measures inter-arrival

times Remember it takes 8ns to send 1 byte on GigE.

Throughput: MTU-Size Packets

Preamble Ethernet Hdr Data CRC

22 bytes 46-1500 bytes 4 bytes

MTU-sized Packet = 1526 bytesTransmits in 1526 * 8 = 12208 nsAdd Inter Frame Gap of 96ns. 12304ns is back to back!

Throughput: Smaller Packets

Throughput: Smaller Packets cont.Errors appear between 6000-7000ns.See the same problem for different packet

sizes.The mean inter-transmission times are

correct,not a PCI bottleneck problem.

Most likely: the sending thread is doing too much work between packet transmissions.In 6400ns each thread has

6400 ns / 8 ns per cycle / 4 threads = 200 clock cycles

Future WorkImplement receiving packets

Currently PTG cannot receive packets and send them to the host PC, so generating reactive traffic requires another NIC.

Augment existing software network emulator (ie. NISTNet) with PTG to transmit packets at the correct times.

ConclusionThe PTG precisely controls the transmission

times of packets streamed from the host PC.It has the potential to

enhance the realism of software network emulators, and

allow researchers to perform experiments which are currently difficult or impossible.

Modified Hardware DesignsNetThreads design:

Removed Output Queues increase accuracy of transmission times.

NetFPGA reference router design used in measurements:Removed Input Arbiter

and Output Port Lookup.

Inaccuracy of NISTNetSoftware network emulators schedule packet

transmissions with fixed-rate timers/interrupts.

Ran experiment to measure inaccuracy using NISTNet.

PTG

NISTNet host

Delays packets by

100ms

NetFPGA router

measuring arrival times

Measure inter-arrivals here

Measure inter-arrivals here

Packets Sent Every 70 μs

Packets Sent Every 640 μs

Packets Sent Every 700 μs