Comparing the Accuracy of Network Simulators for

Packet-Level Analysis using a Network Testbed

Chaudhry Usman AliUNB, Fredericton

TESTING SIMULATOR ACCURACY

• Measure real network traffic.• Compare real network traffic with the

simulator results.

Real Traffic Simulated Traffic

Types of Traffic:CBR (Constant Bit Rate) data traffic. (Simplicity)FTP (File Transfer Protocol) session. (Popularity and Dynamic behaviour)

CBR (Few external factors influence the model)

FTP(Complex)

HTTP(too many degrees of freedom)

METHODS OF SIMULATIONDiscrete Event

Produce predictions in the network at a low level (packet-by-packet).

Accurate.

But SLOW !

Analytical Simulation

Use mathematical models to produce their results.

Sacrifice Accuracy !

Faster speed !

NS-2 and OPNET use a hybrid approach.

SIMULATION TOOLS• OPNET MODELLER

• NS-2

1. Finite State Machine + Analytical Model.2. 400 special purpose modelling

functions.3. Not open source.4. Model parameters can be altered.

1. Numerous models of Internet protocols.2. NAM network animator. 3. Animations and graphs.4. Custom applications and protocols can be created.

C++ : per packet processing.Otcl : Simulation Scenario generation.

NETWORK TESTBED

5 PCs.Client-Server Pair.

G- Traffic Generator.S - Traffic Sink.

Hub

10/100 Mbit links.

* KURT

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

NETWORK REPRESENTATION

OPNET NS-2

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

EXPERIMENTSCBR Traffic FTP Session

1. Characterized by a fixed bandwidth across the network.

2. Typically used by applications such as video and audio.

3. CBR traffic stream can be generated by fixing the packet size and using the same inter-arrival time between packets.

Share, transfer and transmit information between two computers.

Two types of test were performed for the FTP experiments.1- FTP sessions using default settings.2- Finely tuned FTP parameters.

PARAMETER TUNING

Reno to New Reno Fast Recovery capability + recover without retransmission.

Window Scaling Allows advertisement of window sizes bigger than 65 kB.

Timestamp Imitates the echoing capability of the testbed in both directions.

Link Latency Affects RTT.

Max Segment Size Bigger Ethernet packets.

Window Size More range for sliding the window value.G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

TRAFFIC SCENARIOS

IP Traffic monitored using tcpdump.

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

RESULTS

Scenario# Client –Server Load Traffic G. – Traffic S. LoadCBR1 2 Mb/s 0 Mb/s

NS-2 displays a more realistic CBR behaviour.

ROUTER`s PERSPECTIVE

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

Scenario# Client –Server Load Traffic G. – Traffic S. LoadCBR2 2 Mb/s 2 Mb/s

CBR3 5 Mb/s 0 Mb/s

Lower throughput for the network testbed.Scheduling problems?

ROUTER`s PERSPECTIVE

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

Scenario# Client –Server Load Traffic G. – Traffic S. LoadCBR4 5 Mb/s 6 Mb/s

CLIENT`s PERSPECTIVE

Sudden drop

ROUTER`s PERSPECTIVE

SERVER`s PERSPECTIVE

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

Scenario# Client –Server Load Traffic G. – Traffic S. LoadFTP1 10 MB File 0 Mb/s (CBR)

FTP3 10 MB File 0 Mb/s (CBR)

Without tuning. With tuning.

ROUTER`s PERSPECTIVE Server to Client Client to Server

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

Scenario# Client –Server Load Traffic G. – Traffic S. LoadFTP2 10 MB File 6 Mb/s (CBR)

FTP4 10 MB File 6 Mb/s (CBR)

Without tuning. With tuning.

ROUTER`s PERSPECTIVE Server to Client Client to Server

G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh, M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.

Conclusions• Pure CBR type traffic. (NS-2 performed better).• In one scenario OPNET performed “better”.

• FTP Experiments• NS-2 FTP simulation model only indicated general transfer rate

rather than replicating the actual network flow.• OPNET performed closely to the testbed results.• Simulation speed was fast in both simulators.• The learning curve was different and steep at times.• NS-2: “script” generated.• OPNET: User Interface.

Conclusions Contd..• In FTP session, simulators did not model the

dynamic behaviour of FTP in standard form.• FTP adapts its output to prevailing network

conditions.• NS-2 and OPNET did not always mimic this

performance.• However, OPNET performed better.

Future Work

• Establishing a scheme to model HTTP in the simulators and the testbed.

REFERENCE• G. Flores-Lucio, M. Paredes-Farrera, E. Jammeh,

M. Fleury, and M. Reed, OPNET-Modeler and NS-2: "Comparing the Accuracy of Network Simulators for Packet-Level Analysis using a Network Testbed". 3rd WEAS Int. Conf. on Simulation, Modelling and Optimization (ICOSMO 2003), Crete, vol. 2, pp. 700-707, 2003.