internet2 performance tools fall 2011 iips conference - … · 2017-02-20 · 2 10/26/11 ....

Carla Hunt Manager Knowledge and Information Systems, MCNC [email protected]

Tom Throckmorton Sr. Network Management and Information Systems Architect, MCNC [email protected]

Internet2 Performance Tools Fall 2011 IIPS Conference

Overview

  Introduction and Motivation for Performance Tools (Carla)

  On Demand Throughput Measurements (Carla)

  One Way Latency Measurements (Tom)

  Internet2 perfSONAR Performance Toolkit (Tom)

2 10/26/11

Introduction

About Us

  MCNC is the company and NCREN is the network, our !agship product. •  MCNC is an independent, non-pro"t organization that employs

advanced networking technologies and systems to help various sectors of Community Anchor Institutions (CAIs) in North Carolina communicate with their constituents more effectively and meet their speci"c organization's mission, vision and goals.

  Knowledge and Information Systems at MCNC •  The mission of the Knowledge and Information Systems team is to

establish an overall strategy for knowledge capture and dissemination and to develop and provision the MCNC knowledge capture and dissemination infrastructure.

4 10/26/11

About Internet2

  One of two national research networks to which NCREN connects. We send research traffic and subscribe to a commercial peering service.

  Over the last ten years Internet2 has also led the development of a measurement framework and collection of measurement tools

•  perfSONAR-PS •  pS-Performance Toolkit

5 10/26/11

About Internet2 and MCNC

  Relationship extends beyond the network •  Carla, co-chair of Internet2 Performance WG •  Tom, contributor to ps-PS development

  MCNC and Internet2 offered a 1.5 day network performance workshop earlier this month for system and network administrators to learn more about these tools

•  100% of participants surveyed thought workshop a valuable use of time

•  1.5 days into 1hr 15min

6 10/26/11

Motivation

  Are you responsible for a college network?

  Are you or have you experienced network performance problems?

  Do you or have you ever had one or more network users who say the network is slow or broken?

  Would you like to be able to monitor and track short-term and long-term network statistics to identify trends and for capacity planning?

7 10/26/11

About this session

  There is no single tool or approach to diagnose network performance issues

  We’re here to talk about tools, strategies and recommendations that will help

  Patience and diligence will get you to most goals

8 10/26/11

Internet2 End to End Performance Initiative

  Traditional network performance data collection centralized – think Cacti or MRTG

  Approach network performance in the context of multiple administrative domains

9 10/26/11

What do we mean by multiple domains?

10 10/26/11

User typically somewhere on your campus but the resource they are trying to access could be across campus or on a part of the network you don’t manage

Internet2 pS-Performance Toolkit

  What is it? •  A collection of open source tools that can be

easily deployed on your network •  A framework for sharing and discovery of

network performance data •  A resource for users and admins to perform on

demand and scheduled measurement tests

  We will focus on tools that enable throughput and latency measurement but there are other tools in the collection

11 10/26/11

Internet2 pS-Performance Toolkit (cont.)

  Low cost and low effort to deploy

  Measurement endpoints are discoverable

  These tools have been used to solve regional and international network performance issues, as well as locally within NCREN!

  If you have had a network health assessment, we have used some of them on your campus network

12 10/26/11

Myths and Pitfalls

•  “My LAN performance is great, WAN is probably the same” –  TCP recovers from loss/congestion quickly on the LAN (low RTT) –  TCP will cut speed in half for every loss/discard on the WAN – will take a

long time to recover for a large RTT/ –  Small levels of loss on the LAN (ex. 1/1000 packets) will go unnoticed,

will be very noticeable on the WAN.

•  “Ping is not showing loss/latency differences” –  ICMP May be blocked/ignored by some sites –  Routers process ICMP differently than other packets (e.g. may show

phantom delay) –  ICMP may hide some (not all) loss. –  Will not show asymmetric routing delays (e.g. taking a different path on

send vs receive)

•  Our goal is to dispel these and others by educating the proper way to verify a network – we have lots of tools at our disposal but using these in the appropriate order is necessary too

13 10/26/11

  When problems exist, it’s the networks fault! •  Easy to blame a resource, but where else could a problem

be? - Host (Disk, CPU, Kernel, NIC Drivers) - Network Interface Cards -  Routers/Switches, Routing and Con"guration -  Physical Infrastructure -  Protocols

  The network is viewed as a single resource in many cases •  Reality – complex series of components •  Multiple vendors/technologies •  Multiple con"guration options •  Crossing administrative domains

14 – 10/26/11, © 2011 Internet2

Underlying Assumption

Network View (Layman’s Terms)

15 – 10/26/11, © 2011 Internet2

Bob’s Host “The Internets”

Carol’s Host

Network View (Actual)

16 – 10/26/11, © 2011 Internet2

Sw

itch

1

Switch 2 Switch 3

R1 R3

R4

R2 R7

R6 R9

R8 R5

Switch 4

  Finding a solution to network performance problems can be broken into two distinct steps: •  Use of Diagnostic Tools to locate problems

-  Tools that actively measure performance (e.g. Latency, Available Bandwidth) -  Tools that passively observe performance (e.g. error counters)

•  Regular Monitoring to establish performance baselines and alert when expectation drops. -  Using diagnostic tools in a structured manner -  Visualizations and alarms to analyze the collected data

  Incorporation of either of these techniques must be: •  ubiquitous, e.g. the solution works best when it is available

everywhere •  seamless (e.g. federated) in presenting information from different

resources and domains

17 – 10/26/11, © 2011 Internet2

Methodology

  What are the "rst steps to address problems related to network performance? •  Try a Tool

  What tools are out there •  Numerous •  Different metrics (measurements) available •  How to interpret the results?

18 – 10/26/11, © 2011 Internet2

Addressing a Performance Discrepancy

  Ping

  Traceroute

  Iperf

  Tcpdump

  Tcptrace

  BWCTL

  NDT

  OWAMP

  AMP

  Advisor

  Thrulay

  Web100

  MonaLisa

  pathchar

  NPAD

  Pathdiag

  Surveyor

19 – 10/26/11, © 2011 Internet2

Tools, Tools, Tools

•  Ethereal •  CoralReef •  MRTG •  Skitter •  C!owd •  Cricket •  Net100 •  Pathload •  Pathchrip •  MRTG •  Cacti •  Smokeping •  PingER •  FDT •  perfSONAR •  Nagios •  Ganglia •  Thurlay •  Etc. etc. etc.

  Focus on 3 Types of tools (for now) •  Basic Diagnostics

-  Ping, Traceroute •  Advanced User Tools

-  NDT •  Network Admin Focused

-  OWAMP, BWCTL

  What about the others? •  Try them out, learn how they work. •  Most tools are designed to solve a speci"c problem and they may

add value to your organization

  Integration of multiple solutions •  Measurement frameworks integrate use of tools (operation,

collecting results) along with analysis and presentation •  perfSONAR

20 – 10/26/11, © 2011 Internet2

Highlighting some Interesting Tools

  Ping •  Round Trip (e.g. source to destination, and back) •  Con"rms that remote host is ‘up’ •  Some network operators block these packets -  Play w/ command options to see if that will change anything

  Traceroute •  Identi"es the routers along the path •  Same blocking problem as above •  Routers treat ICMP packets with lower priority -  See presentation from prior JTs: -  http://www.internet2.edu/presentations/jt2009jul/20090722-

litvanyi.pdf

21 – 10/26/11, © 2011 Internet2

Basic Diagnostic Tools

 NDT = Network Diagnostics Tool • Perhaps some of you in this room have used an NDT server

 Measure performance to users desktop

 Identify real problems for real users • Network infrastructure could be the problem • Host tuning issues could be the problem

 Simple to use and somewhat simple to understand • Presentation in a method almost all users can access: web browser

 Tool is designed to be useful for users and network administrators • Variables for many aspects of host, protocol, and network performance

22 – 10/26/11, © 2011 Internet2

Advanced User Tool - NDT

  BWCTL – Bandwidth Control •  Allows single person operation over wide area testing environment •  Inject traffic on the network between two measurement nodes and

measure throughput •  Runs NLANR ‘iperf’ program

-  Support for Thrulay, nuttcp

  OWAMP – One way Delay Measurement •  Advanced ‘ping’ command

-  One way vs round trip

•  Inject traffic on the network between two measurement nodes and measure latency or delay

•  Allows single person operation over wide area testing environment

23 – 10/26/11, © 2011 Internet2

Network Administration Tools

  pS Performance Toolkit ISO •  All tools, pre-installed and con!gured •  More info: http://psps.perfsonar.net/toolkit

•  Source Packages (Client and Server) •  http://software.internet2.edu/sources/ •  Typical ‘con"gure/make/make install’

  RPM Installation (CentOS 5.5 Supported): •  Install our RPM package to enable the Internet2 Repository •  See instructions here: http://software.internet2.edu/ •  Support for YUM and APT-RPM

  Others Notes: •  Other RPM based distros (Fedora/RHEL) may work with packaged

RPMs … YMMV •  To install on Debian, consider source. Alien conversions of RPMs

may be problematic

24 – 10/26/11, © 2011 Internet2

Software Availability

On Demand Throughput Measurement Using the Network Diagnostic Tool (NDT)

Advanced User Tool - NDT

  Best on demand throughput test we know about •  http://mitas.csail.mit.edu/papers/

Bauer_Clark_Lehr_Broadband_Speed_Measurements.pdf

  We aren’t the only ones that use it •  FCC – National Broadband Speed Test -

http://www.broadband.gov/qualitytest/ •  Measurement Lab (M-Lab) - http://measurementlab.net/ •  Verizon - http://my.verizon.com/micro/speedtest/broadband/

  Limitations •  Language returned in heuristics may be out of date

26 10/26/11

  M-Lab (Commodity Networking) •  http://www.measurementlab.net/run-ndt

  Internet2 (R&E Networking) •  http://ndt.atla.net.internet2.edu:7123/ •  To not overwhelm the server, also try replacing ‘atla’ with:

-  chic -  hous -  kans -  losa -  newy -  salt -  seat -  wash

  NCREN NDT Service •  https://ndt.ncren.net

27 – 10/26/11, © 2011 Internet2

Hands on Tes6ng of NDT

  Web-based JAVA applet allows testing from any browser •  One Click testing

•  Option to dig deep into available results •  Send report of results to network administrators

  Command-line client allows testing from remote login shell •  Same options available

•  Client software can be build independent of server software

28 – 10/26/11, © 2011 Internet2

NDT User Interface

NDT Results

29 – 10/26/11, © 2011 Internet2

  Measure performance to users desktop •  Lots of tools to measure performance to a nearby server •  Also ‘plugable’ hardware to measure everything up to the network

cable •  Want something to accurately show what the user is seeing

  Develop “single shot” diagnostic tool that doesn’t use historical data

  Combine numerous Web100 variables to analyze connection

  Develop network signatures for ‘typical’ network problems •  Based on heuristics and experience •  Lots of problems have a smoking gun pattern, e.g. duplex mismatch,

bad cable, etc.

30 – 10/26/11, © 2011 Internet2

Motivation for Developing NDT

  Simple bi-directional test to gather end to end data •  Test from client to server, and the reverse

•  Gets the ‘upload’ and ‘download’ directions

  Gather multiple data variables from server •  Via Web100, also some derived metrics (packet inter arrival times)

  Compare measured performance to analytical values •  How fast should a connection be given the observations of the

host and network

  Translate network values into plain text messages

  Geared toward campus area network

31 – 10/26/11, © 2011 Internet2

How It works

  Operates on Any client with a Java enabled Web browser •  No additional client software needs to be installed •  No additional con"guration required

  What it can do: •  State if Sender, Receiver, or Network is operating properly •  Provide accurate application tuning info •  Suggest changes to improve performance

  What it can’t do •  Tell you exactly where in the network the problem is •  Tell you how well or poorly “other” servers perform •  Tell you how well or poorly “other” clients will perform

32 – 10/26/11, © 2011 Internet2

Web Based Performance Tool

  Duplex Mismatch •  This is a serious error and nothing will work right. Reported on

main page, on Statistics page, and mismatch: on More Details page

  Packet Arrival Order •  Inferred value based on TCP operation. Reported on Statistics

page, (with loss statistics) and order: value on More Details page

  Packet Loss Rates •  Calculated value based on TCP operation. Reported on Statistics

page, (with out-of-order statistics) and loss: value on More Details page

  Path Bottleneck Capacity •  Measured value based on TCP operation. Reported on main page

33 – 10/26/11, © 2011 Internet2

Finding Results of Interest

  What is the slowest link in the end-to-end path? •  Monitors packet arrival times using libpcap routine

-  Data and ACK packets -  Is aware of packet sizes – used to calculate speed

•  Use TCP dynamics to create packet pairs •  Quantize results into link type bins

-  Broad classi"cation, e.g. “FastE” -  No fractional or bonded links currently

  Example: •  Consider the following setup

-  1G network card on Host -  1G LAN -  100M (FastE) Wall Jack

•  NDT will report there is a slow link somewhere in the path. It can’t tell you where, but something is limiting the test speed

34 – 10/26/11, © 2011 Internet2

Bottleneck Link Detection

  Duplex Mismatch: •  Operation between a host and an interface are at different

duplex modes (e.g. one half, one full) •  Common in networks where auto negotiation is disabled, or

faulty •  Classic example of a “soft failure”, connectivity is present and

speeds are poor

35 – 10/26/11, © 2011 Internet2

Duplex Mismatch Detection

  Detect non-congestive loss due to •  Faulty NIC/switch interface

•  Bad Cat-5 cable •  Dirty optical connector

36 – 10/26/11, © 2011 Internet2

Faulty Hardware or Link

  Shared network infrastructures will cause periodic congestion episodes •  Detect/report when TCP throughput is limited by cross traffic •  Detect/report when TCP throughput is limited by own traffic

37 – 10/26/11, © 2011 Internet2

Congestion Detection

  Provide basic tuning information

  Features: •  Basic con"guration "le •  FIFO scheduling of tests, support for testing with simultaneous

clients •  Simple server discovery protocol and ability to federate (e.g. load

balance) servers •  Logging of all test results on the server side

  Command line client support

  Other Clients can be developed against open Javascript API: •  http://www.internet2.edu/performance/ndt/api.html

  Posted on Google Code: •  http://code.google.com/p/ndt/

38 – 10/26/11, © 2011 Internet2

Additional Functions and Features

Architecture

39 – 10/26/11, © 2011 Internet2

Client Web

Browser

Java

Applet

NDT - Server

Web

Server

Testing

Engine

Child

Test Engine

Spawn child

Well Known NDT Server

Web Page Request

Web page response

Test Request

Control Channel

Specific test channels

  There are no settings or options for the Web based java applet. •  It allows the user to run a "xed set of tests for a limited time

period

  Test engine settings •  Turn on admin view (-a option) •  If multiple network interfaces exist use –i option to specify

correct interface to monitor (ethx)

  Simple Web server (fakewww) •  Use –l fn option to create log "le

•  Could also use a ‘real’ web server like Apache

40 – 10/26/11, © 2011 Internet2

Recommended Settings

  Non-standard kernel required •  Web100 patching may be difficult to apply to new kernels

•  Hard to keep up with vendor patching •  GUI tools can be used to monitor other ports •  Consider using pS Performance Toolkit enhancements if this

scares you…

  Public servers generate trouble reports from remote users •  Respond or ignore emails

  Test streams can trigger IDS alarms •  Con"gure IDS to ignore NDT server

41 – 10/26/11, © 2011 Internet2

Potential Risks

One Way Latency Measurement Using OWAMP

  OWAMP is: •  Command line client application •  Policy and scheduling daemon •  Used to determine one way latencies between hosts.

  Implementation of the OWAMP protocol as de"ned by http://www.rfc-editor.org/rfc/rfc4656.txt •  Command Protocol to speak between client and server, server and

server •  Test protocol

  Different attempts to do this in the past: •  Surveyor •  RIPE

43 – 10/26/11, © 2011 Internet2

OWAMP: What is it

  Passive Measurements alone not enough (e.g. SNMP) •  Higher polling interval may mask queue depths

•  Active probing gives a better picture of real traffic

  Round Trip Measurements: •  Pros

-  Can be done with a single ‘beacon’ (e.g. using ICMP responses) -  e.g. latency measurements in the NCREN Community Portal are round

trip

•  Cons -  Hard to isolate the direction of a problem -  Congestion and queuing can be masked in the "nal measurement -  ICMP packets are given a low priority, dropped "rst

44 – 10/26/11, © 2011 Internet2

Why One Way Latency?

  One Way Measurements: •  Direction of a problem is implicit

•  Detects asymmetric behavior •  See congestion or queuing in one direction "rst (normal

behavior) •  Requires ‘2 Ends’ to measure properly

  OWAMP Graphs: •  Feature two lines for each direction.

•  Very sensitive, even to NTP differences •  Loss/duplication shows up as an event !ag •  Congestive loss as well loss caused by other reasons (equipment

failures)

45 – 10/26/11, © 2011 Internet2

One Way Latency != Round Trip Latency

Different Latencies with pockets of Loss

46 – 10/26/11, © 2011 Internet2

  Supports authentication and authorization of the users that will test

  Used to con"gure the parameters of a test •  Endpoint controlled port numbers •  Extremely con"gurable send schedule •  Con"gurable packet sizes

  Used to start/stop tests

  Used to retrieve results •  Provisions for dealing with partial session results in the event of a

failure

47 – 10/26/11, © 2011 Internet2

OWAMP Control Protocol

  “Lightweight” compared to the control protocol

  Uses UDP as the transport protocol, since the protocol needs to be able to measure individual packet delivery times

  Supports varying packet sizes

  Data needed to calculate experimental errors on the "nal result is in every packet

  Packets can be “open”, “authenticated”, or “encrypted”

48 – 10/26/11, © 2011 Internet2

OWAMP Test Protocol

  Applications •  Daemon (owampd) •  Clients (owping, powstream)

  Open Source License & Development •  Modi"ed BSD (http://www.internet2.edu/membership/ip.html) •  Mailing lists for developer communication – come join us!

  Protocol Abstraction Library •  Will support development of new clients •  Add custom ‘hooks’ into the policy (e.g. add authentication

via OpenID or similar)

49 – 10/26/11, © 2011 Internet2

Sample Implementa6on

  Meant to operate like traditional “ping”

  owping client requests OWD tests from an OWAMP server (owampd)

  Client can be ‘sender’ or ‘receiver’ •  Both directions are tested unless otherwise speci"ed

  Communication can be “open”, “authenticated”, or “encrypted”

  Supports the setup of many tests concurrently

  Supports the storage of results on the server for later retrieval

50 – 10/26/11, © 2011 Internet2

Func6onality (owping client)

  Accepts requests for OWD tests

  Responds with accepted/denied

  Tests are formally started with a StartSessions message from the client.

  Runs tests

  Sessions with packets received at the server are buffered for later retrieval

51 – 10/26/11, © 2011 Internet2

Func6onality (owampd server)

OWPING Example

52 – 10/26/11, © 2011 Internet2

OWAMP GUIs – Delay/Loss Plot

53 – 10/26/11, © 2011 Internet2

Similar Latencies, Some Loss

54 – 10/26/11, © 2011 Internet2

OWAMP GUIs -‐ Mesh

55 – 10/26/11, © 2011 Internet2

OWAMP GUIs -‐ JiYer

56 – 10/26/11, © 2011 Internet2

  Clock requirement is the strongest •  Doesn’t work well in virtualized environments

•  Doesn’t work well when machine is doing heavier testing (e.g. BWCTL), results may be suspect

  NTP (ntpd) synchronized clock on the local system •  Speci"c con"guration requirements as speci"ed in

NTP talk… •  Strictly speaking, owamp will work without ntp.

However, your results will be meaningless in many cases

57 – 10/26/11, © 2011 Internet2

OWAMP Requirements

  NTP (ntpd) synchronized clock on the local system •  Con"gure NTP properly (don’t rely on system defaults!)

•  Strictly speaking, owamp will work without NTP. However, your results will be meaningless in many cases

•  More information about NTP con"guration here: http://www.internet2.edu/performance/owamp/details.html#NTP

58 – 10/26/11, © 2011 Internet2

General Requirements – Time Source

Plot – Note the “wavy” activity = NTP

59 – 10/26/11, © 2011 Internet2

NCREN NTP Servers

  Each rPoP is equipped with a Stratum II time server that allow end users to synchronize the time on individual workstations and other end user devices.

  These devices peer with each other and synchronize with our Stratum I servers, to ensure users get the most accurate time.

  More information - https://www.mcnc.org/news/network-time-server-upgrade.html

60 10/26/11

  “Bare Metal” – virtualization is tricky

  Stable System Clock •  Temperature controlled environment •  No power management of CPU

•  Reduction of “background” services – may institute noise

  No strict requirements for CPU, Memory, Bus speed •  More tasking schedules will require more capable

hardware

61 – 10/26/11, © 2011 Internet2

General Requirements – Hardware

  Time: •  NTP issues predominate the problems •  Determining an accurate timestamp “error” is in many ways

more difficult than getting a “very good” timestamp •  Working as an “open” server requires UTC time source (For

prede"ned test peers, other options available)

  Firewalls: •  Port "lter trade-off - Administrators like pre-de"ned port numbers -  Vendor manufactures would probably like to “prioritize” test

traffic - Owampd allows a range of ports to be speci"ed for the

receiver

62 – 10/26/11, © 2011 Internet2

Opera6onal Concerns

  Third-Party DoS source •  Compromised server may send packets to other locations.

  DoS target •  Excessive traffic will harm measurement results •  Someone might attempt to affect statistics web pages to see how

much impact they can have

  Resource consumption •  Time slots

•  Memory (primary and secondary) •  Network bandwidth

63 – 10/26/11, © 2011 Internet2

Policy/Security Considera6ons

  Restrict overall bandwidth to something relatively small •  Most OWAMP sessions do not require much

  Limit “open” tests to ensure they do not interfere with precision of other tests

64 – 10/26/11, © 2011 Internet2

Policy Recommenda6ons

Internet2 pS-Performance Toolkit

Care and Feeding

  The pSPT nodes are designed to help two constituencies •  Engineers – Regular monitoring to catch problems earlier •  Users – Distributed test points to validate their own performance

  Work as a system, the following will describe: •  The tools •  The operations methodology •  Maintenance steps

  Can integrate with others •  Deployed on Backbones •  Other regional networks •  Other connected members of NCREN

66 – 10/26/11, © 2011 Internet2

Your pSPT Deployment and You

  pS Performance Toolkit ISO •  All tools, pre-installed and con"gured •  Two !avors; LiveCD or NetInstall •  LiveCD demo after the presentation

  pS Performance Toolkit source •  All components are open source and available •  packages for RPM-based systems

  Consider installing multiple toolkit nodes

  Scheduled bandwidth and latency probably won’t go well on the same node

  If you install a publicly connected toolkit node and would like to participate in testing with other members of NCREN, we'd suggest you con"gure your node(s) to join the 'NCREN' community. As other test points on NCREN are added, you'll be able to identify them more easily by looking for nodes in that community.

67 – 10/26/11, © 2011 Internet2

Deployment Strategies

Home Page

68 – 10/26/11, © 2011 Internet2

NDT Tests

69 – 10/26/11, © 2011 Internet2

  Perfect for the end user to run from their laptop •  Convince your campus users to test to this, instead of

‘speedtest.net, or FCC’. Install one on *every* campus •  Runs client to server and reverse •  As mentioned, Detects common problems

  The LiveCD is the quickest way to get NDT deployed on your network

  Caveats •  Test results on higher bandwidth links can be skewed by poor

client OS tuning; many still shipping with untuned stacks •  NDT won’t tell you precisely what’s wrong, but will provide

several clues

70 – 10/26/11, © 2011 Internet2

NDT - quick review

Other tools on the toolkit…

  BWCTL •  scheduled throughput testing •  uses iperf and can perform TCP/UDP tests

  Traceroute •  Reverse traceroute; regular traceroute testing

  NPAD •  Detailed path diagnostics

  PingER •  Scheduled ICMP RTT testing

  Cacti •  Scheduled SNMP collection

73 10/26/11

Summary

Contact Info

Questions? Need help?

  [email protected]

  Join the NCREN Network Visualization and Management Discussion Group mailing list: https://list.mcnc.org/mailman/listinfo/net-mgmt-wg

  Post to the Technology Forums, https://www.mcnc.org/forums/ncren/performance-monitoring-tools

  Internet2 pS-Performance Toolkit •  http://psps.perfsonar.net/toolkit

75 10/26/11

76 10/26/11

extra slides (this page intentionally left blank)

  Experimental use of OWAMP data

  Plots ‘activity’ more than anything. When you see data on the graph its an indication that there is “something” going on •  Network congestion •  NTP Skew (common) •  Host activity (other processes/measurements)

  The points do correspond to some amount of “Jitter”, e.g. packet delay variation. •  +’s are noise, the larger values may mean something •  Red is a large chunk, green are smaller chunks. •  Add ‘’alpha_max=0.99” to your string to see a change (e.g. 99th percentile of

Jitter only).

  Graphs are still a work in progress, will be ingreated into the toolkit GUIs in the next release

81 – 10/26/11, © 2011 Internet2

Jitter?

  TCP Testing •  Recommend to do at least a 20 second test every 4 hours. (less

duration is ok for short RTT, can test more or less often) •  TCP is elastic, numbers should be close to capacity unless link is

heavily used. If they drop it may be congestion related or real problem related

  UDP Testing •  Be careful, not elastic. If you are going to do it, pick your peers.

Set a bandwidth limit, try to mimic a UDP application (e.g. video)

  Testing from others: •  Encourage connectors to test to a measurement node that is

close. •  Test to Internet2/MCNC etc.

85 – 10/26/11, © 2011 Internet2

Regular testing

  Use in conjunction with regular traceroute: •  Workstation -> Server via your terminal

•  Server -> Workstation via the web

  Detect asymmetric routes

  Cautionary notes: •  Don’t rely on traceroute for latency times (e.g. icmp and routers/

switches) •  Asymmetric routing by itself is not harmful, it’s a way of life. If

one of the routes has a performance problem then it is harmful.

89 – 10/26/11, © 2011 Internet2

Reverse Traceroute Use Case

  Similar to NDT, but spits out a lot more •  Tries to “simulate” a network use based on current conditions.

•  E.g. expectations are a latency and a speed, will tell you if this is or is not possible

  Also replies on Web100, same caveats apply

  “Mom Feature” … you can retrieve the HTML results from past tests. Users can mail these in trouble ticket reports.

92 – 10/26/11, © 2011 Internet2

NPAD = Network Path Diagnostics

  Currently only monitoring the localhost •  Memory/cpu things

•  Plans to add NTP monitoring as well as some more advanced localhost things (process breakdown, etc.)

  Can be used for SNMP monitoring if desired •  Note the poll is defaulted to about 5 minutes

98 – 10/26/11, © 2011 Internet2

Using Cacti

  The term “throughput” is vague •  Capacity: link speed

-  Narrow Link: link with the lowest capacity along a path -  Capacity of the end-‐to-‐end path = capacity of the narrow link

•  U6lized bandwidth: current traffic load

•  Available bandwidth: capacity – u6lized bandwidth -  Tight Link: link with the least available bandwidth in a path

•  Achievable bandwidth: includes protocol and host issues

Throughput? Bandwidth?

100 – 10/26/11, © 2011 Internet2

(Shaded portion shows background traffic)

45 Mbps 10 Mbps 100 Mbps 45 Mbps

Narrow Link Tight Link

source sink

internet2 performance tools fall 2011 iips conference - … · 2017-02-20 · 2 10/26/11 ....

Documents