making the grade: ensuring application performance in an education network presented by: sean...

Making the Grade: Ensuring Application Performance in an Education Network

Presented By:

Sean Applegate

Mid-Atlantic Systems Engineer

© 2001 Packeteer, Inc.

The Bottleneck

WAN Access LinkWAN Access LinkBottleneckBottleneck

High-SpeedHigh-SpeedLANLAN

High-Speed High-Speed BackboneBackbone

OC-3, OC-12, OC-48, OC-192OC-3, OC-12, OC-48, OC-192 10/100/100010/100/1000

T-1 – OC3T-1 – OC3

Problem: Traffic on the high-speed LAN hits the lower-speed WAN access link and congestion (queuing/dropped packets) occur.


Introductory Questions

How many people here own a PacketShaper?How many people here are evaluating a PacketShaper?How many people have NEVER heard of Packeteer or the PacketShaper?How many people don’t know what your top 10 applications are and the percent of bandwidth they are using?How many people are considering increasing WAN bandwidth speeds?How many people are using other bandwidth mgnt or policing technologies to control traffic?


Applications Drive Today’s Educational Institutions

Mission-critical applications are critical to educationAll traffic is not created equal

eMaileMail

File TransfersFile Transfers

Peer-to-PeerPeer-to-Peer

Web SurfingWeb Surfing

CitrixCitrixClient/Svr AppsClient/Svr Apps

Streaming Streaming VideoVideo

Streaming Music, Quake, etc.

TCP / IPTCP / IPApplication-Application-

NeutralNeutral

TCP / IPTCP / IPApplication-Application-

NeutralNeutral

++

----

Mission-Mission-CriticalCritical

Time-SensitiveTime-Sensitive ++


What am I spending my $ on?

Are you spending 60-85% of your WAN budget on P2P applications?


Agenda

Who is Packeteer?

What is PacketShaper?

Who is using PacketShaper?

Implementing Packeteer's Four Step Process

- Classify

- Analyze

- Control

- Report

Summary

Questions


Packeteer Fact Sheet

Founded in 1996, Pioneer of Proactive Bandwidth ManagementHeadquarters in Cupertino, CA

US Offices: New Jersey, Chicago, Atlanta, Dallas, Seattle, Washington D.C., San DiegoOffices Abroad: Netherlands, Hong Kong, Japan, and AustraliaRegional Resellers

Employees: 197Customer proven

Shipping since February 1997Thousands of PacketShapers shipped worldwide5th generation of software


Packeteer Product Family

PacketShaper™

Internet BandwidthManagement Solutions

AppVantage™

Application SubscriberManagement Solutions

Central ConfigurationManagement

PolicyCenter™

AppCelera™

Internet Application Acceleration Solutions


What is PacketShaper?

Industry Leading QoS Solution

CMP Network Telecom: Network Infrastructure Product of the Year - 2001

Firmware, Real-time OS (PSOS)Classifies 350+ Apps at OSI Layers 2-7Uses Patented TCP Rate Control to proactively control application traffic and prevent queuing and reduce latency.

Over 55 measurement variables for detailed analysisManaged through an onboard web interface and CLI, no external hardware/software requiredBecomes a piece of wire if it fails


PacketShaper Product Line

PacketShaper 1500 Series

Up to 2 Mbps WAN capacity








Typical k-12 School District Topology

Internet

PacketShaper to manage the Internet link

PacketShapers at each shool to manage school-to-district office and school-to-school traffic


380+ Higher Education Customers in US

Stanford UnivCalifornia TechYale UnivVanderbilt UnivUniv of MiamiTexas A&MClemson UnivUniv of Notre DameAll Universities of CaliforniaCase Western Reserve UnivOhio Northern UnivDartmouth CollegeHoward Univ

Univ of DaytonMiami UnivCase Western Reserver UnivCleveland State UnivXavier UnivYoungstown StateDenison UnivBowling Green UnivCapital Univ…Ohio has more higher ed users than any other state in the US.…A Sites are catching up fast!

Higher Ed Customers in the US: Ed Customers in Ohio:


A Complete Solution

Our Example Customer: Our Example Customer: Randolph Macon CollegeRandolph Macon College

Problems:Problems:•Congested WAN LinkCongested WAN Link•Poor Visibility at App LayerPoor Visibility at App Layer•Poor Response TimesPoor Response Times•Needed to get controlNeeded to get control•Needed better WAN ROINeeded better WAN ROI


Step 1: Classify - What’s Running on My Network?

Physical

Network

Data Link

Transport

Session

Presentation

Application

Pac

ketS

hap

er

Mo

st R

ou

ters

Sw

itch

es

1

7

6

5

4

3

2

Automatically Classify 350+ Apps at Automatically Classify 350+ Apps at OSI Layers 2-7OSI Layers 2-7

Peer-to-Peer Apps:• Aimster• AudioGalaxy• CuteMX• DirectConnect• Gnutella• Hotline• iMesh• KaZaA/Morpheus• Napster• ScourExchange• Tripnosis….

Some Other Apps:• H.323• RTP-I/RTCP-I • PASV FTP• HTTP• Real• WinMedia• Shoutcast• MPEG• Quicktime• RTSP• Chatting Apps• Games

If you can’t classify it you can’t shape it!If you can’t classify it you can’t shape it!


Step 1: Classify – Traffic Class Criteria

Inbound/Outbound (travel direction)Protocol familyService (very diverse, see online list)Inside/Outside (location of relevant server)Port(s)Service ProxyIP Address, MAC Address, host name, or host listSubnet MaskURL (including wildcards)Further details (criterion) for Citrix-ICA, Oracle-netv2, HTTP 1.1, RTP-I Diffserv, IP Precedence, COS/TOS


Step 1: Classify – Traffic’s INs and OUTs

PacketShaperIn ternet

Client

Server Client

Server

inbound/HTTP/outside (data)

outbound/HTTP/outside (get)

inbound/HTTP/inside (get)

outbound/HTTP/inside (data)

Outbound

Inbound

OutsideInside


Step 1: Classify – Manual Class Creation

3. Define traffic’s criteria (details on the next slide).

1. From the MANAGE screen, select the parent class from the traffic tree

2. Click on CLASS, then ADD


Classes are made up of matching rules

Classes can have many matching rulesmultiple matching rules are OR’d together

1 rule for each of 3 servers.

single matching rules are AND’d together1 rule that catches traffic from a specific server to a specific client.

Step 1: Classify – Matching Rules


Step 1: Classify – More on Matching Rules

The definition of the traffic in a class is a matching rule

It’s a collection of values for the criteria we listed

Traffic Discovery defines matching rules for the classes it creates

You define matching rules for the classes you create


Step 1: Classify – Adv. Matching Rules

In addition to the basic criteria, such as IP address and port numbers, the following advanced options are available:

Host Lists

Details for Citrix, Oracle, HTTP 1.1 and RTP

Diffserv and IP COS/TOS


Step 1: Classify – Host Lists

Instead of a single IP address or a range of IP addresses, specify a list of hosts.

Lets you take advantage ofLDAP directory services.


Citrix-ICA, Oracle-netv2, HTTP 1.1, and RTP-I can be further classified using the Matching Rule Criterion field:

Citrix-ICA: by published application, client name or priority level*

Oracle-netv2: by database name

HTTP 1.1 by DNS name or IP address

RTP-I (real-time protocol for media streaming) by Encoding Name, Media Type ("a" for audio, "v" for video), or Clock Rate (8000, 16000, 44100, 90000)

Step 1: Classify – Application Criteria


Step 1: Classify – Diffserv, COS/TOS …

Diffserv Code Point (DSCP) (6-bit field)Value of 0-63

COS - Class of Service (3-bit field)IP precedence value 0-7

TOS - Type of Service (4-bit field)

802.1q/ISL VLANs

MPLS


Step 1: Classify - Other Settings

Type, Traffic Discovery(within class), Top Talkers/Top Listeners, RTM, Comments


Step 1: Classify – Traversing the Class Tree

PacketShaper examines all passing traffic. - Every flow must be assigned to a class.

It traverses the tree to find the traffic’s correct class. Traversal starts at the topIf you have a special-case class you want searched first, make it an Exception class.Example: All PCs in a subnet to be treated the same except one. E.g.: DifferentPC

SubnetASubnetB


Step 1: Classify – RMC After 24 Hrs

55 Applications AutoDiscovered6 Peer-to-Peer (circled)7 Streaming3 Chat5 GamesAnd the usual Internet and network service protocols

This traffic tree was automatically built by turning on Traffic Discovery. Only shows applications on the network.


• Click Report in the PolicyConsole navigation bar.

• You’ll see 3 graphs for the Inbound link and 3 graphs for the Outbound link:- Link Utilization- Network Efficiency- Top 10 Classes

• Shows what’s competing for the bandwidth.

Step 2: Analyze


Step 2: Analyze - Top Ten Tab


Step 2: Analyze - Context-Specific Reports


Step 2: Analyze - Monitor Tab

Class Hits/Rates tell you how busy a service is

RTM Summary

Top Talkers/Top Listenerstell you which IP addressesare using the most bandwidth


Step 2: Analyze - Top Talkers / Listeners

Enable up to 12 top talkers/listeners (total).

Create classes for top users.


Step 2: Analyze - Response-Time Summary

View delay statistics for all measured classes:


Step 2: Analyze - Transaction Delay

Is my network causing problems? Or is it one of my servers?


Approx. 90% of transactions at the

serverexperience 0 sec delay

Approx. 75% of transactions

experience .1 sec delay on

the network—Here’s our culprit!!

Who is the most common culprit? The Server or The Network?

Step 2: Analyze – Delay Distribution


Why measure response time?

Quantify performance.

Identify performance problems.

Develop strategies for bandwidth management, server balancing, and topology upgrades.

Assess results after you’ve made configuration changes.

Step 2: Analyze – Response Times


Step 2: Analyze - Measuring Delay

Server Delay - # of ms the server uses to process a client’s request after all data received.

Total Delay - # of ms from client’s request to receipt of response.

Network Delay = Total Delay - Server Delay

Round-Trip Time (RTT) is the # of ms for client-server exchange of precisely one packet.


- Link is fully congested, observe how close the avg & peak bps are. - What are my top 10 types of traffic?- Am I using this for recreational or business use?

30% of all TCP data is retransmitted

Approx 1/3 of the WAN budget is wasted (~$700/month).

Inbound Link avg & peak bps

% of TCP Retransmitted Bytes

Step 2: Analyze - RMC Link Performance


Step 2: Analyze – RMC – Top Applications

GnutellaiMesh

HTTP

KaZaA

FTP

Over 72% P2P

GnutellaHTTP

iMeshKaZaA


Step 2: Analyze – RMC – HTTP/SSL Response Times

Users are waiting 2+ secs for each HTTP connection to complete

Users are waiting 3 to 8+ secs for each

SLL connection to complete


Step 3: Control – How Do I Control Performance?

Set policies to control performancePer-flow minimum/maximum bandwidth policiesPer-user minimum/maximum bandwidth policiesPriority-based policiesAdmissions ControlPartitions for control of aggregate flows

PacketShaper implements TCP Rate ControlControl when and how much data end-systems transmitUsing industry-standard TCP/IPManage traffic flows and aggregate classes with bits-per-second accuracy


Step 3: Control – Applying Policies


Step 3: Control – Priority Policies

A Priority policy has only one parameter

Low High

0 1 2 3 4 5 6 7


Step 3: Control – Priority Policy Guidelines

Use a priority policy:When rate is not your primary objectiveIf traffic does not burst (surge)If traffic is latency-sensitiveIf high-priority flows are small, orif low-priority flows are large but not bursty

Priority policies are appropriate for interactive traffic like TN3270 or Telnet (latency-sensitive, don’t burst, small)


Step 3: Control – Rate Policy Page


Step 3: Control – Rate Policy Guidelines

Guarantee each flow a minimum bits-per-second rateGive each flow prioritized access to excess bandwidthKeep a lid on surging, bandwidth-hungry flowsGuard mission-critical flowsGive delay-sensitive flows a chanceMake sure behind-the-scenes TCP Rate Control is active

Remember not to over-commit guaranteed rates!


Step 3: Control – Never-Admit Policies

Use a Never-Admit policy:

For TCP or Web traffic, to block a session and inform the user


Step 3: Control – Discard Policies

• When you simply want to toss all packets for a traffic class.

– Block a service– Provide security

• Recommended for blocking non-TCP classes because they’re not session-oriented


Step 3: Control – Ignore Policies

Ignore policies:

Treat traffic as pass-through

Exempt a traffic class from bandwidth management

PacketShaper does not count the statistics


Step 3: Control – How flows Compete for Excess

Rate policies are satisfied first!

Then, at each priority level, rate policies aregiven their burstable chunks and priority polices get what they want.


Flow A Rate (5) 10k

Flow BPriority 4

Flow CRate (3) 5k

Guaranteed Rate:

Priority 7:

Priority 6:

Priority 5:

Priority 4:

Priority 3:

Priority 2:

Priority 1:

Priority 0:

10(10) 0 5(5)

0(10) 0 0(5)

00(10) 0(5)

010(20) 0(5)

65(65)10(30) 0(5)

0(30) 0(5)0(65)

0(5)0(30) 0(65)

0(5)0(30) 0(65)

0(5)0(30) 0(65)

Step 3: Control –How Flows Compete For Demand


Step 3: Control - TCP Rate Control

Steps:1. Measure end-to-end latency2. Forecast when packets will be

needed to meet the policy3. Tell the Client/Server how much data

to send (set TCP Window Size)4. Tell the Client/Server when to send

the data (schedule ACKs)

PROACTIVE CONTROL!!

• Speed up latency-sensitive flows• Throttle back big file transfers• Smooth traffic throughput• Improve multiplexing, reduce jitter

Sender Receiver

Natural TCP

Tim

e

Bursty Traffic Flow

Receiver

TCP Rate Control

Tim

e

Sender

Smooth Traffic Flow


“Gravel”

“Sand”

Unmanaged Traffic

Managed Traffic

Step 3: Control – Multiplexing Gains


Step 3: Control – Queuing versus Rate Control

Queuing TCP Rate Control

Efficiency • Tosses packets (RED, WRED)• Induces packet loss (tail-end drops)• Generates retransmissions (timeouts)

• no queues form• More efficient data transfer• Reduces packet loss & retransmissions (better ROI)

Precision • Limited traffic classification• No bits-per-second control• No detailed flow-by-flow QoS

• Rich traffic classification• Explicit bits-per-second control• Rate-based QoS for individual flows

Full-Duplex

No inbound control •Inbound & outbound control

Proactive • Reactive• Congestion has already occurred if queues are forming

• Proactive• Prevents congestion BEFORE it occurs


Deadline scheduling mechanism:Provides rate control for UDP

Not as good as TCP rate control

Uses a delay bound to Set the maximum delay

Limit buffer utilization per flow

Allows setting the delay bound from 200 to 10,000 milliseconds

Step 3: Control – UDP Delay Bound


A partition:

Creates a virtual pipe within a link for an aggregate traffic class

Provides a minimum and maximum bandwidth guarantee

Ranges from 0 Kbps to 45 Mbps

Enables efficient bandwidth use

Step 3: Control – Partitions


Partitions can:

Limit — restrain a traffic class to keep it from becoming predatory

Protect — shelter a traffic class’ bandwidth from predators

Step 3: Control –Partitions’ Two Purposes


Step 3: Control – Partitions Can Burst

You can:

Create a static partition

Create a partition that can grow (burst) if extra bandwidth is available

Partitions can burst to use:

The entire link

A predetermined maximum amount of bandwidth


Step 3: Control – Dynamic Partitions

Automatically setup and tear down partitions based on active users.Limit each user to a maximum amount of b/w at all times.Set a cap on number of active users assigned a partition.Create an overflow partition for everyone else Dynamic Partition usage graph


Step 3: Control – Creating a Partition


Select details to specifysizing and traffic flow

Step 3: Control – Dynamic Sub-Partitions


Step 3: Control – Dynamic Sub-Partition Details


Step 3: Control – Time of Day Scheduling

CLI onlySyntax: schedule <time rge> <cmd> | <-f cmd file>Use “schedule show” to see scheduled items.Use “schedule delete <#>” to remove scheduled items.Schedule commands are stored in RAM so they do not span resets.To span resets create a file named startup.cmd in 9.256/Put schedule commands in startup.cmd to change shaping by time of day.When PS boots up it reads startup.cmd and schedules commands.To immediately apply a new schedule command delete old scheduled times and enter “run startup.cmd” to initialize the new commands.


Step 3: Control – Organizing the Traffic Tree

Logically organized the classes

Used low priority rate policies and partitions to throttle back aggressive non-latency-sensitive file downloads such as P2P traffic, FTP and SMTP

Used high priority rate policies to improve performance of longer lived time-sensitive traffic, such as HTTP, SSL, Citrix, RTP-I, etc.

Used priority policies for short lived flows, such as Telnet, RTCP, H.323, tn3270, rsh, rlogin, etc.

Final Config Used:10 Partitions 35 Policies


Step 3: Controlling VoIP and Video Traffic

Classify and control H.323's at OSI Layers 5-7Q.931 (call setup)H.245 (call control)Gatekeeper DiscoveryGatekeeper Control (Registration, Admission, and Status)RTCP-IRTP-IRTCP-BRTP-B

Classify and control RTP-I traffic by at OSI Layers 5-7 by:Audio or VideoCodec – provide exact amount of BW required per-flow with a Rate policyEncoding type (GSM or JPEG)

Prevent other traffic, such as casual web browsing, P2P and large file transfers from impacting VoIP performance by proactively throttling back inbound and outbound bandwidth.


Step 4: Report - How Do I Measure Performance and Plan for the Future?

PacketShaper lets you make more intelligent decisionsEvaluate the effectiveness of shapingSee what traffic you are spending your WAN Budget onPlan for the future of your network through capacity planning, trend analysis, etcTrack application service level agreements based on total delay, server delay and network delaySet and meet user expectations

Import data into other reporting systemsCSV, SNMP, XML

Complex plugins for HP Openview, Concord eHealth, InfoVista, NetCool and other NMS…

Can notify via email or SNMP trap when performance is poor or when there is a possible DoS attack


Step 4: Report – Establish Acceptable Performance

Set a threshold to define “good service.”


Step 4: Report – What’s Good, What’s Bad?

Thresholds let you easily quantify good/bad service.

Definition of “good” responses Definition of SLA


Step 4: Report – Monitoring SLAs

SLA Problems

SLA Problems are gone!


Step 4: Report – RTM: Transaction Delay

User-set threshold


Step 4: Report – Worst Clients/Servers

Tells you which clients/servers have the most delay


Step 4: Report – Statistics Data Dump

Extract lists of variable values for any class. Two months of data stored.

Specify:One or more variables (definite variety)Time periodSort orderIndividual statistics or sum totals


Step 4: Report – How to Get the Data


Step 4: Report – PacketShaper Events

PacketShaper Events notify you when thresholds are exceeded.

Currently command line only

Viewable via the Events Monitor

Several steps to set it up.


Step 4: Report – Event set up.

4 steps to Event Notification

Identify the mail server PacketShaper will use to send messages.

Identify the recipients of the email

Identify the SNMP Server PacketShaper will send traps to.

Register the event.


Step 4: Report – Setting up email notification

<setup email>


Step 4: Report – Setting the Recipients List

<event email>


Step 4: Report – Setting SNMP Server


Step 4: Report – Defining Events

Events come in 2 flavors:

User-Defined: Any measurement engine variable

Pre-Defined: 17 PacketShaper Pre-defined Events


Step 4: Report – Defining Events

<event new>


Step 4: Report – Registering Events

<event register>


Step 4: Report – Event Summaries

<event show>


Step 4: Report – Event Monitor


Standard MIBS

MIB II

10 Basic Groups (system, interfaces, at, ip, icmp,tcp,udp,egp,transmission,snmp)

Private MIBS

Packeteer MIB

Packeteer RTM MIB

Step 4: Report –SNMP MIBs

Download from support.packeteer.com


Step 4: Report - “Roll Your Own” Reports

Useful API’sPolicyConsole – HTTP/Javascript

XML

PacketWise Server-side Tags

CGI API


Report- Custom Reports via SNMP Authentication

Step 1: Determine report type

Step 2: Get an example URL from the WUI

Step 3: Replace the respective variables with your new variables

Step 4: Turn on snmp look authentication:CLI: sys set dataRetrievalUseSMMPPassword 1

Append &SNMPPASSWORD=<community> to end of URL

Step 5: Put new URL in a web page and the graph will be created


Step 4: Report – RMC Link Performance

Inbound Throughput Inbound Efficiency

Outbound Throughput Outbound Efficiency

Improved Efficiency, better WAN ROI

No Shaping Shaping No Shaping Shaping

No Shaping Shaping No Shaping Shaping

Restricted P2P to 300Kbps


Step 4: Report – RMC Top 10

Inbound Before Shaping

71% P2P

Inbound After

Shaping

34% P2P

HTTP

HTTP100%+ Increase


Step 4: Report – RMC Main Apps

Rate shaped P2P back and capped at 1.5Mbps with a partition

Rate shaped HTTP/SLL so they would perform faster

Rate shaped P2P back and capped with at 300kbps

Rate shaped HTTP/SSL

so they would perform better


Step 4: Report – RMC HTTP Response Times

Outside Web Server Normalized Network Response Times

No Shaping Shaping

No Shaping Shaping

Inside Web Server Normalized Network Response Times


Packeteer’s PacketShaper

Provides the application infrastructure that enables you to:

Know what’s on your networkGet visibility into and control over bandwidth usageControl recreational traffic Reserve bandwidth for teaching, learning, and researchMake intelligent decisions about capacity planningAnd much more…


Tools

http://support.packeteer.com PolicyConsole API (ask support for it)Boilerplate Reporting PortalStanford PacketShaper email list

Send email to: [email protected] body (no subj): subscribe packeteer-eduArchive: http://www.stanford.edu/group/networking/netlists

Initial Shaping ScriptTons of Perl ScriptsOnline White PapersPacketGuide (v5.2+)FREE Online Training every FridayRegional Training Classes


Questions & Contact Info

Questions?

Sean Applegate, Packeteer Mid-Atlantic SE(540) [email protected]

ResellersStratacache – 937-224-0485Vector – 513-786-6618DPS – 513-489-4200DDS – 216-676-1760

making the grade: ensuring application performance in an education network presented by: sean...

Documents