service performance manager user guide...

SPECTRUM ®

Service Performance Manager User Guide (5093)r9.0.1

This documentation and any related computer software help programs (hereinafter referred to as the “Documentation”) is for the end user’s informational purposes only and is subject to change or withdrawal by CA at any time.

This Documentation may not be copied, transferred, reproduced, disclosed, modified or duplicated, in whole or in part, without the prior written consent of CA. This Documentation is confidential and proprietary information of CA and protected by the copyright laws of the United States and international treaties.

Notwithstanding the foregoing, licensed users may print a reasonable number of copies of the Documentation for their own internal use, and may make one copy of the related software as reasonably required for back-up and disaster recovery purposes, provided that all CA copyright notices and legends are affixed to each reproduced copy. Only authorized employees, consultants, or agents of the user who are bound by the provisions of the license for the product are permitted to have access to such copies.

The right to print copies of the Documentation and to make a copy of the related software is limited to the period during which the applicable license for the product remains in full force and effect. Should the license terminate for any reason, it shall be the user’s responsibility to certify in writing to CA that all copies and partial copies of the Documentation have been returned to CA or destroyed.

EXCEPT AS OTHERWISE STATED IN THE APPLICABLE LICENSE AGREEMENT, TO THE EXTENT PERMITTED BY APPLICABLE LAW, CA PROVIDES THIS DOCUMENTATION “AS IS” WITHOUT WARRANTY OF ANY KIND, INCLUDING WITHOUT LIMITATION, ANY IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NONINFRINGEMENT. IN NO EVENT WILL CA BE LIABLE TO THE END USER OR ANY THIRD PARTY FOR ANY LOSS OR DAMAGE, DIRECT OR INDIRECT, FROM THE USE OF THIS DOCUMENTATION, INCLUDING WITHOUT LIMITATION, LOST PROFITS, BUSINESS INTERRUPTION, GOODWILL, OR LOST DATA, EVEN IF CA IS EXPRESSLY ADVISED OF SUCH LOSS OR DAMAGE.

The use of any product referenced in the Documentation is governed by the end user’s applicable license agreement.

The manufacturer of this Documentation is CA.

Provided with “Restricted Rights.” Use, duplication or disclosure by the United States Government is subject to the restrictions set forth in FAR Sections 12.212, 52.227-14, and 52.227-19(c)(1) - (2) and DFARS Section 252.227-7014(b)(3), as applicable, or their successors.

All trademarks, trade names, service marks, and logos referenced herein belong to their respective companies.

Copyright © 2009 CA. All rights reserved.

Contents

PrefaceText Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viiiDocumentation Location and Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ixContact Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix

Chapter 1: IntroductionSPM Concepts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

Test Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Test Types Supported by SPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3Test Templates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4SPM Alarms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4Performance Agents and MIBs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

SPM Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Multiple Ways to Create Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Test Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Automated Test Creation with Test Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Single-Point Test Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Report Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7SPM Tests and Service Level Agreement Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

User Roles and SPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8SPM for Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8SPM for Administrators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Getting Started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Access SPM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Basic Tasks Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10About Test Host and Test Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Secure an SPM Test Host . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Secure an SPM Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11Overwrite a Security String . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Chapter 2: Finding SPM Components with SPM SearchesPerform an SPM Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Search for Specific SPM Components. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

All Test Component Searches. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13Criteria-Based Test Host Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14Criteria-Based Test Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Contents iii

Chapter 3: Working with Test TemplatesCreate Test Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Create a New Test Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Create a Different Version of an Existing Test Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

Apply a Test Template to Test Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21Selectively Apply a Test Template to One or More Test Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . 21Apply a Test Template to a Global Collection Container . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

Example Implementation Scenario. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24Manage Test Templates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Edit a Test Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27Remove the Association Between a Template and a Global Collection. . . . . . . . . . . . . . . . . . 27

Delete a Test Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Chapter 4: Working with Performance TestsCreate Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Create a New Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Create a Different Version of an Existing Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30Information About Specific Test Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

DHCP Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31DNS Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31FTP Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31HTTP Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32HTTPS Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32ICMP (Ping) Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32Jitter Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33Traceroute Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33POP3 Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34SMTP Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34TCP Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34UDP Echo Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

Discover Tests in the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35About Discovery Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Read-Only Discovery Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35Read/Write Discovery Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

Run Discovery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36Configure SPM Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

Configure General Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39Standard General Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40General Parameters for Specific Test Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40About the Test Host Location Parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Configure Advanced Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44Schedule a Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45Specify Alarm Thresholds for a Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

Establish Baseline Data to Determine Valid Thresholds . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50Run Tests on an On-Demand Basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Activate and Deactivate Test Hosts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

iv Service Performance Manager User Guide

Manage Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52Edit a Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53Delete a Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

Chapter 5: Viewing SPM InformationTest Host Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Test Host Information in the Component Detail View . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57Test Host Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57Test List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

Test Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58Test Information in the Component Detail Panel. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59Test Performance Information in the Component Detail Panel . . . . . . . . . . . . . . . . . . . . . . . . . 60

Specify the Interval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60Specify the Test Result Type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61View Test Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Adjust an Axis Dimension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61About the Effect of a Test’s Filter Timeout Data Setting on a Performance Graph . . . . . . . . . . 61

SPM Alarms and Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

Chapter 6: Generating Reports on Test DataSPM Reports from SRM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

Report Management Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65How to Generate Performance Reports with SRM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

SPM Reports from SRG . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69Data Logging Event Codes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70SPM Result Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70SPM Data Export Sample . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71SPM Test Modification Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Chapter 7: Creating and Managing RTM Tests with Command Line Interface (CLI)Create Tests in CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74Run Tests in CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Edit Tests in CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Get Test Results in CLI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75Test Parameters Used in CLI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

Required Parameters for Creating Tests. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76Required Parameters for Scheduling Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77Optional Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78Threshold Type Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

Latency Threshold Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80Packet Loss Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81HTTP DNS Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82HTTP TCP Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82HTTP Download Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83Jitter Source to Destination Packet Loss Threshold. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84Jitter Destination to Source Packet Loss Threshold. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85Jitter MIA Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

Contents v

Jitter Late Arrival Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87Jitter Busies Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88Jitter Positive Source to Destination Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 89Jitter Positive Destination to Source Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 89Jitter Negative Source to Destination Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . 90Jitter Negative Destination to Source Threshold Parameters . . . . . . . . . . . . . . . . . . . . . . . . 91

Test Action Codes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92Test Status and Test Results Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

SPM LatestStatus (0x4560004) Attribute Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92Test Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

Appendix A: TroubleshootingFirmware Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

Cisco IOS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Cisco IOS 12.0(9) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95Cisco IOS 12.0(7)T2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Cisco IOS below 12.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Cisco IOS 12.2(2)T . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Cisco IOS 12.2(11)T. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Cisco IOS 12.3(4) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Cisco IOS 12.3(5) and below. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96Cisco IOS 12.2(18)SXF3 and 12.2(18)SXF4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Juniper (all JUNOS devices). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97Riverstone RS-8000 FW 9.0.0.4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

Timeout Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97ICMP Ping Tests and Extreme Summit Test Host Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Traceroute Tests and iAgent Test Hosts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

Test Scheduling Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98Scheduling Tests in Geographically Distributed Environments . . . . . . . . . . . . . . . . . . . . . . . . . 98Schedule Time Interval and CA SystemEDGE Agent Test Hosts . . . . . . . . . . . . . . . . . . . . . . . . 99Large Number of Tests on a Router Running Off Schedule. . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

Delayed SpectroSERVER Activation Resulting from External Reads . . . . . . . . . . . . . . . . . . . . . . . 100

Appendix B: SPM Event CodesEvent Code Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101About SPM Timeout Event . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Index

vi Service Performance Manager User Guide

Preface

This guide is intended for network administrators who use Service Performance Manager (SPM) to configure and run Response Time Tests (RTTs). This guide assumes that you have a working knowledge of SPECTRUM, OneClick, and the various RTT agents included with your network assets.

This guide is organized as follows:

Chapter 1: Introduction provides an overview of SPM concepts and features, describes user roles and privileges, describes how to access SPM components from OneClick Console, provides SPM implementation guidelines, and outlines the tasks you can perform with SPM.

Chapter 2: Finding SPM Components with SPM Searches describes how to use OneClick Console search capabilities to locate SPM components.

Chapter 4: Working with Performance Tests describes how to create tests, discover tests configured on tests hosts, configure tests, schedule tests, set test thresholds, run tests, and manage tests.

Chapter 3: Working with Test Templates describes how to create test templates, apply test templates to Global Collection containers, and manage test templates.

Chapter 4: Working with Performance Tests describes SPM information views in OneClick Console.

Chapter 6: Generating Reports on Test Data describes SPECTRUM resources for generating SPM reports.

Chapter 7: Creating and Managing RTM Tests with Command Line Interface (CLI) provides example response time test scripts and describes command line parameters and action codes.

Appendix A: Troubleshooting identifies error messages that may be generated during SPM operations and describes corrective actions. It also addresses other maintenance and optional configuration issues.

Appendix B: SPM Event Codes lists SPM events by event code, event name, the model types the event can be asserted on, the alarm generated or cleared (if any), and alarm severity.

vii

Text Conventions

Text ConventionsThe following text conventions are used in this document:

Element Convention Used Example

Variables

(The user supplies a value for the variable.)

Italic in angle brackets (<>)

Type the following:

DISPLAY=<workstation name>:0.0 export display

The directory where you installed SPECTRUM

(The user supplies a value for the variable.)

<$SPECROOT> Navigate to:

<$SPECROOT>/app-defaults

Linux, Solaris, and Windows directory paths

Unless otherwise noted, directory paths are common to both operating systems, with the exception that slashes (/) should be used in Linux and Solaris paths, and backslashes (\) should be used in Windows paths.

<$SPECROOT>/app-defaults on Linux and Solaris is equivalent to <$SPECROOT>\app-defaults on Windows.

On-screen text Courier The following line displays:path=”/audit”

User-typed text Courier Type the following path name:C:\ABC\lib\db

viii Service Performance Manager User Guide

Documentation Location and Feedback

Documentation Location and FeedbackCheck the following site for the latest updates and additions to SPECTRUM documents:

http://ca.com/support

To send feedback regarding SPECTRUM documentation, access the following web address:

http://supportconnectw.ca.com/public/ca_common_docs/docserver_email.asp

Thank you for helping us improve our documentation.

Contact Technical SupportFor online technical assistance and a complete list of locations, primary service hours, and telephone numbers, contact Technical Support at the following web address:

http://ca.com/support

ix

Contact Technical Support

x Service Performance Manager User Guide

Chapter 1: Introduction

Service Performance Manager (SPM) lets you create, run, and manage performance tests supported by various vendor products managed by SPECTRUM. Performance testing addresses multiple network management requirements. A few representative examples are listed as follows:

Establishing IT services delivery standards—Lets you establish service (HTTP transactions, login validations, file transfers) delivery benchmarks against which you can measure how well services are delivered to service consumers. Understanding what constitutes optimal, acceptable, and unacceptable levels of service delivery is a prerequisite to developing realistic service level agreement goals and obligations.

Capacity planning—Lets you determine whether service demands by consumers are underutilizing or exceeding current IT infrastructure capacity.

Proactive fault management—Lets you determine where service delivery degradation trends are developing before they impact service customers.

SPM ConceptsThis section describes the SPM components you work with to configure and run performance tests.

Test Hosts

An SPM test host is a model of a device or software agent that supports one or more types of performance tests for IT services. SPECTRUM creates a test host model of type RTM_TestHost during model discovery for each SPM capable device.

Introduction 1

SPM Concepts

Some examples of test host devices and agents are a Cisco router, a Sun Workstation running iAgent, and a SpectroSERVER.

Important! Devices and agents that support SPM must be modeled with read/write community strings in order to run SPM tests. Attempts to run tests on test hosts (with the exception of SpectroSERVERs) that are not modeled with a read/write community string will cause an alarm on the test host model.

If it is not desirable to create the device model with the read/write community string, you can configure the community string of the application model that is used by SPM.

Note: See Performance Agents and MIBs on page 4, which lists tests supported by the performance agents as well as the associated SPECTRUM Application model type.

To configure the community string of the application model used by SPM

1. In OneClick, open the Locater tab.

2. Open the Application Models folder.

3. Double-click the By Device Address field and enter the IP address of the device.

4. Filter for the appropriate application model (for example, for Cisco IPSLA supporting hosts, enter ‘CiscoRTTMonApp’) and select it.

5. Open the Attributes tab.

6. Filter for the Community_Name.

7. Select Community_Name/0x10024 and select the right arrow.

It now appears in the right panel.

8. Double-click to select and change the value.

Tests

A performance, or response time, test is an IT service operation of a particular type (retrieve a Web page, download a file, establish a TCP connection) that returns a result you can use to assess the service’s performance. Where a single test returns a result that can help you determine whether there is a current problem with a service’s viability, a group of tests return aggregate results that can help you determine a service’s viability over a particular timeline or to determine baseline performance standards you can factor into any number of decision making processes regarding your infrastructure.

SPM enables you to create tests (model type RTM_Test) for your test hosts and discover tests previously configured on managed devices. You can schedule tests, run tests on an on-demand basis, specify test thresholds, and incorporate tests into service management and service level agreement systems.

2 Service Performance Manager User Guide

SPM Concepts

Test Types Supported by SPM

Performance tests supported by SPM fall into three different test categories:

Network response time tests: These are the most basic network response time tests.

Network service response time tests: These measure the response time of essential network services.

Network application response time tests: These measure the response time of essential network applications.

The following test types are supported by SPM:

Network Response Time TestsICMP Ping

Tests ICMP Echo Request messages from the test host to the destination address. If the test host and source addresses are not the same, then another ICMP Echo Request is issued from the test host to the source address. The resulting ICMP Echo Reply messages are used to determine the round-trip time between the source address and the destination address. A second metric, packet loss, results from these tests. When a series of ICMP Echo requests are made, which is typical of these tests, a coarse measure of packet loss is possible.

Jitter

Tests latency and losses between two endpoints. This test is designed to measure the quality of the network for applications that cannot tolerate loss or latency, such as VoIP.

Traceroute

Tests round-trip ICMP Echo from the host address to each layer three hop in the discovered path.

Network Service Response Time TestsDHCP

Dynamic Host Configuration Protocol IP address assignment.

DNS

Domain Name Service domain name to IP address translation.

Introduction 3

SPM Concepts

Network Application Response Time TestsFTP

File Transfer Protocol transaction time.

HTTP/HTTPS

Hyper Text Transfer Protocol transaction time.

POP3

Post Office Protocol transaction time.

SMTP

Simple Mail Transfer Protocol transaction time.

TCP

Transmission Control Protocol connection time.

UDP Echo

User Datagram Protocol transmission echo round-trip delay.

Test Templates

Test templates contain test parameters for a particular test type that you can selectively apply to multiple test host models or automatically apply to test host models added to Global Collection models. SPECTRUM monitors Global Collection content and automatically creates the test specified by the template on test hosts added to the Global Collection. Regardless of how a test template is applied to test hosts, the result is the same—the test defined by the template is created on the test hosts.

See Working with Test Templates on page 19 for more information.

SPM Alarms

SPM generates various events representing significant occurrences related to response time testing. Some of these events produce alarms designed to notify you that a user-actionable event has occurred. See SPM Event Codes on page 101 for information about these events and alarms.

Also, you can configure SPM to generate an alarm whenever the duration of a response time test exceeds a predetermined threshold on a specific link or path. See Specify Alarm Thresholds for a Test on page 47 and SPM Alarms and Events on page 63 for more information.

Performance Agents and MIBs

SPM supports the following performance agents and MIBs:

RFC2925


SPM Concepts

Cisco IOS IP SLAs Agent

CA eHealth SystemEDGE Service Availability agent

Network Harmoni SLAplus Agent

iAgent

JUNOS Real Time Performance Monitor

Cisco Ping MIB

Wellfleet Ping MIB

Note: Only current Cisco routers support the RTTMON MIB. Older, pre 11.2 deployments may support only the Cisco Ping MIB.

The following table lists tests supported by the performance agents and MIBs.

Agent/MIB SPECTRUM Application Model Type

Tests

RFC2925 RFC2925App ICMP Ping, Traceroute

Note: You must verify with the vendor whether or not a particular device supports the RFC2925 MIB.

Cisco IOS IP SLAs Agent (Cisco IOS IP SLAs Agent is supported on Cisco routers running IOS 12.0 or greater.)

Note: For full-mesh measurements between hubs, Cisco recommends that shadow routers dedicated for IOS IP SLAs use be employed. For more information see Cisco IOS IP SLAs documentation.

CiscoRTTMonApp DHCP, DNS, FTP, HTTP, ICMP Ping, Jitter, TCP, UDP

You can discover tests configured on Cisco agents with the SPM discovery feature. See Discover Tests in the Network on page 35 for details.

CA eHealth SystemEDGE Service Availability agent

Emp_SvRsp_App DNS, FTP, HTTP, HTTPS, ICMP Ping, POP3, TCP, SMTP

You can discover tests configured on SystemEDGE agents with the SPM discovery feature. See Discover Tests in the Network on page 35 for details.

Note: SystemEdge agents require the eHealth Service Availability module to run performance tests.

Introduction 5

SPM Features

Note: SPM does not support the Nortel Contivity Ping MIB (CONTIVITY-INFO-V1-MIB).

SPM FeaturesSPM supports multiple vendor and agent performance tests solutions and multiple implementation features.

Multiple Ways to Create Tests

SPM enables you to create tests in the following ways:

Create a new test from scratch.

Create a new version from a copy of another test.

Use SPM Discovery to locate tests configured on test hosts.

Apply test templates to Global Collection containers. When a test host is added to the container, a test with the configuration settings provided by the template is automatically created on the test host.

Network Harmoni SLAplus Agent

OEM Vendors:Agilent Technologies, InfoVista, Peregrine Systems, Ericsson, Opticom, RedPoint, HP, Micromuse, Response Networks

HrmniSvcRspApp DNS, HTTP, ICMP Ping

iAgent SRISvcMonApp DNS, FTP, HTTP, ICMP Ping, POP3

JUNOS Real Time Performance Monitor

JnprRFC2925ExtApp ICMP Ping, Jitter, Traceroute

Other Support

Cisco Ping MIB CiscoPingApp ICMP Ping

Wellfleet Ping MIB WFPingApp ICMP Ping

Agent/MIB SPECTRUM Application Model Type

Tests


SPM Features

Test Scheduling

You can use SPM scheduling capabilities to automate your performance testing. You can use preconfigured schedules provided by OneClick with SPM or you can create your own custom schedules. Scheduling tests enables you to implement performance testing during peak and off-peak hours so you can compare the results from each period and determine realistic performance standards.

For example, you may want to schedule a test to run at a specific interval continually, 24 hours a day, seven days a week. Or you may want to schedule tests to run during or after intervals when the infrastructure is in high demand. Whatever the case, you can create test schedules that meet your requirements.

Automated Test Creation with Test Templates

SPM Test templates enable you to set up test configurations and apply them to multiple test hosts. Test templates leverage Global Collection container capabilities. By applying a template to a Global Collection container, you automate the test creation process. SPM creates the test specified by the template for any test host added to the container that supports the test. Modifications to test parameters are equally simple to perform. When you change template parameters, those changes extend to the parameters for all tests created from the template.

Single-Point Test Management

SPM provides you with complete access and control over your test components on multiple landscapes from a single landscape. This means you can create, configure, locate, and manage tests from a single OneClick Console.

Report Options

You can generate summary and detailed SPM performance test result information in a variety of text and graphical formats using the SPECTRUM Report Manager (SRM) application. You can also generate graphical timeline reports using SPECTRUM’s SSLogger and Report Gateway options.

SPM Tests and Service Level Agreement Management

Performance testing is an indispensable component for establishing IT service-obligation benchmarks and monitoring service performance. SPM response time test results provide a finer measure of service performance and viability than a service monitoring system that merely determines whether a service is up or down. Service consumers may consider service delivery speed the determining factor in whether a service meets or does not meet their requirements.

Introduction 7

User Roles and SPM

For example, a service consumer may consider a 20-second wait for a Web page as unacceptable when it takes only five seconds on average to retrieve and load a page. In this case, service consumers may consider the sluggish Web service as essentially unavailable, even though the service is accessible.

SPM enables you to specify response time thresholds to measure service performance in terms of latency. This means a service level agreement can stipulate that a service provider guarantee a particular level of response time that meets the service consumer’s requirements. See the Service Manager User Guide (5155) for information about creating and managing services and service level agreements in SPECTRUM.

User Roles and SPMA user’s access to SPM functionality depends on the rights granted to the user. A SPECTRUM administrator can configure user roles and rights in OneClick. For more information about configuring user rights, see the OneClick Administration Guide (5166).

SPM for Operators

When logged into SPM as an operator, you can view information for tests and test hosts and run existing response time tests. As an operator, you cannot create, delete, edit, or copy any tests. Operators cannot change the state of the test on any test hosts.

SPM for Administrators

When logged into SPM as an administrator, you have full access to all SPM functionality, including creating, editing, deleting, and discovering tests.

Getting StartedThis section describes how to access SPM components from OneClick Console, outlines tasks you perform with SPM and where to find information about those tasks, and provides important information about how to establish security for tests and test hosts.

Access SPM

SPM components are integrated with and accessed from the OneClick application interface.


Getting Started

To access SPM components

1. Click the Locater tab in the OneClick Navigation panel.

2. Click the SPM Searches folder.

The following image shows SPM components in the SPM Searches folder.

Introduction 9

Getting Started

Basic Tasks Overview

Once you have accessed the SPM components list in the OneClick Console Locater, you can begin to work with SPM components:

To locate a specific test component or groups of components (tests, test hosts, test templates), run a search of SPECTRUM. See Finding SPM Components with SPM Searches on page 13 for information about performing criteria-based searches for SPM components.

To create tests, configure test parameters, schedule tests, specify test thresholds, manually run tests, discover tests on test hosts, and edit and delete tests, see Working with Performance Tests on page 29 for instructions.

To create test templates, selectively apply them to test hosts or to Global Collection containers that include test hosts, and edit and delete test templates, see Working with Test Templates on page 19 for instructions.

To create, run, and edit tests from the Command Line Interface, see Creating and Managing RTM Tests with Command Line Interface (CLI) on page 73 for instructions.

To generate reports on tests, see Generating Reports on Test Data on page 65 for information about generating reports with SPECTRUM Report Manager and information about time-series reports you can generate with SPECTRUM Report Gateway.

About Test Host and Test Security

You can use the security string setting for test and test host models to restrict access to them only to authorized personnel.

Note: This section assumes you have experience invoking detailed views of SPECTRUM models in OneClick Console and privileges to modify security string settings for models. See the OneClick Administration Guide (5166) for more information about setting up SPECTRUM model security.

For instructions on how to set or modify a security string for a test host or test, see one of the following procedure descriptions:

Secure an SPM Test Host on page 11

Secure an SPM Test on page 11

Overwrite a Security String on page 11


Getting Started

Secure an SPM Test Host

To secure a test host

1. In the Component Details panel, expand Test Host Details for the test host you want to secure.

2. Click set for the Security String parameter, as shown in the following image:

3. Specify the security string. Permission to access the test host is granted only to privileged users defined by the security string entered.

Note: All SPM tests associated with a “secured test host” automatically inherit the host security string. To overwrite or remove a security string at a test level (or test host level), see Overwrite a Security String on page 11.

Secure an SPM Test

To secure a test

1. In the Component Details panel, expand Test Details, General for the test you want to secure.

2. Click set for the Security String parameter.

3. Specify the security string. Permission to access the test is granted only to privileged users defined by the security string entered.

Overwrite a Security String

To overwrite a security string for a test host or a test

1. In the Component Details panel, Click set for the Security String parameter.

2. Edit or remove the security string.

Introduction 11

Getting Started


Chapter 2: Finding SPM Components with SPM Searches

SPM Searches provide multiple search criteria options you can use to find existing SPM tests, test hosts, and test templates in SPECTRUM. For more information about OneClick searches and configuration options, see the OneClick Administration Guide (5166).

Perform an SPM SearchYou perform search operations from the Locater view in the OneClick Navigation panel.

To perform an SPM search, select the type of search you want to run from the SPM Searches folder and click the Search button. Some searches require you to enter values (typically IP addresses) in a Search dialog before you execute the search.

Search results appear under the Results tab in the OneClick Contents panel. You can select an item returned by a search, view information about it, and perform whatever operation it supports.

Search for Specific SPM ComponentsThis section describes the different types of searches supported by SPM.

All Test Component Searches

This search category enables you to search for all test hosts, tests, and test templates modeled in SPECTRUM.

Finding SPM Components with SPM Searches 13

Search for Specific SPM Components

Criteria-Based Test Host Searches

You can search for test hosts modeled in SPECTRUM that meet criteria you specify.

IP Address

This search finds the test host with the Network Address or tests hosts associated with the address you specify in the Search dialog.

Note: The Search dialog does not support searching on partial IPs (for example, 10.253).

State

This search finds test hosts in the following states:

Active

Test hosts that have been activated.

Contact Lost

Test hosts that have stopped responding to polls.

Inactive

Test hosts that have not been activated or have been deactivated.

Supported Test Type

This search finds test hosts that support the following test types:

DHCP

DNS

FTP

HTTP

ICMP

Jitter

POP3

SMTP

TCP



TRACEROUTE

UDP

Test Discovery Supported

This search finds test hosts that support test discovery.

Test Discovery Unsupported

This search finds test host that do not support test discovery.

Criteria-Based Test Searches

You can search for tests modeled in SPECTRUM that meet criteria you specify.

Destination Address And Port Number

This search finds tests configured with the Destination Address and Destination Port you specify in the Search dialog. You can enter a partial address string in the dialog (for example, 138.42). Searches with the destination port set to 0 return tests for which destination port is not applicable.

Discovery State

This search finds tests that meet the following criteria related to SPM Discovery:

Discovery Read-Only

SPM tests discovered in Read-Only mode via SPM Discovery.

Discovery Read/Write

SPM tests discovered in Read/Write mode via SPM Discovery.

RTM Domain

SPM Tests created via the SPM user interface (and not through SPM Discovery).



Stale Different Type

SPM-discovered tests that have a corresponding test on the device of a different test type.

Stale Entry Not Present

SPM-discovered tests that no longer have a corresponding test on the device.

Note: See Discover Tests in the Network on page 35 for information about the SPM Discovery feature.

Name

This search enables you to find the test with the Test Name you specify in the Search dialog.

Scheduled

This search finds all scheduled tests.

Source Address

This search finds tests configured with the Source Address you specify in the Search dialog. You can enter a partial address string (i.e., 135.48.200) in the dialog.



Source Address, Destination Address, and Port Number

Enter the Source Address, Destination Address, and Destination Port for the test and click OK. You can enter partial strings to search for a test using this search. Note that searches with port number set to 0 will return tests for which destination port is not applicable (N/A).

Tests By - Status

This search provides the capability to search for a test by one of the following test statuses:

Bad Community String

Bad Configuration

Device Disabled

Ready To Run

Running

Scheduled

Stopped

Timeout

User Disabled

Tests By - Threshold Exceeded

This search provides a list of all of the tests exceeding their thresholds.



Tests by - Type

Use this search to find all tests of one of the following test types:

DHCP

DNS_LOOKUP

FTP

HTTP

Jitter

POP3

SMTP

TCP_CONNECT

TRACEROUTE

UDP


Chapter 3: Working with Test Templates

A test template is a test configuration that you can apply to multiple test hosts in a single step to create tests on the test hosts. You configure and edit test templates the same way you do with tests. Where they are different is in their scope of application. When you create a test, you create it on a single test host. When you create a test template, however, you can apply it to multiple tests hosts in the following ways:

Selectively apply a template to multiple test hosts that support the test specified by the template.

Apply a template to any number of test hosts that support the tests specified by the templates and that are added to a Global Collection container.

The primary advantages of using a test template are listed as follows:

You can implement automated bulk performance testing.

All modifications to test template parameters are automatically applied to the tests on each test host to which the template is applied. For example, changing a threshold value in a template changes that threshold value for all tests derived from it.

A test template allows you to specify that a test is either a test host or a destination type. A test host type specifies a variable test host but a constant destination. A destination type specifies a constant test host but variable destinations.

Create Test TemplatesSPM provides two ways to create performance test templates:

Create a new test template from scratch

Save a different version of an existing test template

Whichever method you use to create a test template, you configure it the same way as you would a test, as described in Configure SPM Tests on page 39. You also configure the following template-specific configuration parameters:

Working with Test Templates 19

Create Test Templates

Template Type

Specifies Test Host or Destination. (The Destination configuration is not available for DHCP and HTTP test templates.)

Global Collections

(Optional) Specifies one or more Global Collection containers. OneClick automatically applies the test specified by the template to all test hosts added to it that support the test.

Create a New Test Template

This section describes how to create a new test template from scratch. You must use this method to create a test template if no others have been created.

To create a test template

1. Use SPM Searches to find test templates as described in Finding SPM Components with SPM Searches on page 13.

Test templates may or may not appear in the Contents panel, depending on whether any have been created.

2. Click the New icon and select a test type.

The New Test Template dialog opens. It enables you to configure test template parameters and also specify one or more Global Collection containers to which you want to apply the template.

3. Configure test template settings and click OK.

The new template is saved and information about the new test template appears in the Information tab in the Component Detail panel.

Create a Different Version of an Existing Test Template

You can create a new test template from a copy of an another template.

Note: You cannot copy test templates between domains running different versions of SPECTRUM.

To create a new test template from a copy of another template


The test templates appear in the Contents panel. If no templates appear, create a new template as described in Create a New Test Template on page 20.


Apply a Test Template to Test Hosts

2. Select the test template you want to copy and use one of the following methods to open the Copy Test dialog:

Click the Copy icon, as shown in the following image.

Right-click the test template you want to copy and click Copy Test Template.

The Copy Test Template dialog opens. It includes configuration settings you can modify for your new template. Copied test templates are disabled by default, and “_COPY” is appended to the test name. You may want to rename the template, but it is not required.

3. Modify test template configuration settings and click OK.

The new template is saved and information about the new test template appears under the Information tab in the Component Detail panel.

Apply a Test Template to Test HostsYou can selectively apply a test template to a group of test hosts that support the template test type, or you can automate the process by placing test hosts in a Global Collection container and applying a test template to the container. When a test template is applied to a Global Collection container, tests are automatically created for all test hosts that support the template test type added to the container.

See the Modeling Your IT Infrastructure Administrator Guide (5167) for information about creating and configuring Global Collection containers.

Selectively Apply a Test Template to One or More Test Hosts

Manually applying a test template to a selection of test hosts enables you to precisely specify only those test hosts you want to create tests for.

To apply a test template to a test host


The test templates appear in the Contents panel.



2. Select the test template you want to apply and use one of the following methods to apply the template:

Click the Apply Test Template icon, as shown in the following image.

Right-click the template and click Apply Test Template.

The Select Test Host dialog opens, listing test hosts that support the template test type, as shown in the following image:

3. Select one or more test hosts to which you want to apply the test template and click OK.

OneClick creates the test defined by the test template for the selected test hosts. The Contents panel lists tests created from a test host template type. There are two naming formats for tests created from test templates, tests with names that indicate the test target IP address and tests that indicate the test target model name. The target may be the test host or a particular device, depending on the whether the template is of a test host or destination type.

The default format:

Template Name_IP Address

where



Template Name is the name of the template you applied to a test host.

IP Address is the IP address for the test target.

You can change the naming convention using the SPM Template Naming option available from the OneClick Administration Pages to this format:

Template Name_Model Name

where

Template Name is the name of the template you applied to a test host.

Model Name is the model name for the test target.

Changing the SPM Template Naming option setting does not change the naming format for tests that have already been created from a template.

Apply a Test Template to a Global Collection Container

Applying test templates to Global Collection containers enables you to automate the process of creating tests of different types for multiple test hosts. For example, assume a Global Collection container includes a group of test hosts that only supports ICMP Ping tests, a group that only supports HTTP tests, and a group that supports both. Assume also that numerous tests hosts belonging to each group will be added to the container from time to time.

In this case, two different test templates could be applied to the container to create the tests for the test hosts included in it. One template would specify an ICMP Ping test and the other an HTTP test. ICMP Ping tests would be created for the test hosts that support ICMP Ping tests, and HTTP tests would be created for the test hosts that support HTTP tests.

This example, of course, understates the potential complexity of a “real-life” performance testing implementation, but it does illustrate the ease with which you can use test templates to set up tests on multiple test hosts as an alternative to setting them up on a test-by-test basis. See Example Implementation Scenario on page 24 for a more detailed example.



To apply a test template to a Global Collection procedure

1. In the Create Test Template, Copy Test Template, or Edit Test Template dialog, click the Browse button next to the Global Collection parameter.

The Select Collections dialog opens, as shown in the following image:

2. Select the Global Collections to which you want to apply the template and click OK. If there are no Global Collections available or you would prefer to create a new Global Collection for your test hosts, click Create, configure the new Global Collection, and proceed with the selection process.

Example Implementation Scenario

This section illustrates how test templates can be applied to a Global Collection to implement automated bulk performance testing between a central office location and branch office locations.

ScenarioAn organization’s central location provides domain name lookup and Web services to numerous remote branch locations that connect to the central location on an intermittent basis.

Infrastructure administrators want to ensure that remote connections to the central location are maintained and that remote branch locations have continuous access to DNS and HTTP services provided by the central location.

To determine whether service delivery from the main location to the remote locations remains viable, administrators decide to run Ping, DNS, and HTTP tests from remote location edge routers (that support the tests) on a regularly scheduled basis whenever they are brought online (connect to the central location edge router).

Administrators create a Global Collection configured to include all remote location edge router models as they are brought online.



Three test templates are applied to a Global Container:

– An ICMP Ping test template configured to create a Ping test on remote-location edge routers that tests connectivity to the central-location router.

– A DNS test template configured to create a DNS lookup test on remote-location edge routers that tests DNS lookup time from the DNS server from the central location.

– An HTTP test template configured to create an HTTP download test on remote-location edge routers that tests the round-trip time to download a Web page from the HTTP server from the central location.



Test Template Scenario

The following illustrates the example scenario.

DNS Server

HTTP Server

RTR

Central Office

Branch Offices

RTR

RTRRTR RTR

Global Collection

Destination

Source Source Source Source

DNS Test Template

HTTP Test Template

Ping Test Template

NY LA Chi Bos

DNS test from each routerHTTP test from each router

Ping test from each router

Applied to collection


Manage Test Templates

Manage Test TemplatesWhen you edit a template, all the tests created from the template are updated with the modified settings. When you remove the association between a template and a Global Collection or delete a template, all tests created from the template are deleted.

Important! Editing and deleting test templates should be performed with caution by qualified personnel, especially in cases when SPM performance test results are being monitored by SLAs (service level agreements) modeled in SPECTRUM or when tests are being run for infrastructure performance analysis purposes.

Edit a Test Template

You can edit all test template parameter settings as required. Changes apply to all tests created by the template. You can also remove the association between a template and a Global Collection when you edit a template.

To edit test template settings



2. Select the test template you want to edit and do one of the following:

Click the Edit icon.

Right-click the template and click Edit Test Template.

The Edit Test Template dialog opens.

3. Modify the template’s settings and click OK.

The changes are applied to the template and all tests created from it.

Remove the Association Between a Template and a Global Collection

You can remove the association between a template and a Global Collection by deleting the value in the Global Collections parameter in the Edit Test Template dialog. When you remove the association, all tests on the test hosts in the collection are deleted.

Delete a Test Template

You can delete all test templates as required. When you delete a template, you also delete all tests created from it.


Manage Test Templates

To delete a template

1. Use SPM Searches to find test templates as described in Chapter 2: Finding SPM Components with SPM Searches.


2. Select the test template you want to delete and do one of the following:

Click the Delete icon.

Right-click the template and click Delete Test Template.

A confirmation dialog opens, as shown in the following image:

3. Click Yes.

The test template is deleted.


Chapter 4: Working with Performance Tests

Note: For information about creating and managing response time tests using the SPECTRUM Command Line Interface, see Creating and Managing RTM Tests with Command Line Interface (CLI) on page 73.

Create TestsYou can create a performance test on a test host using either of the following methods:

Create a new test from scratch

Save a different version of an existing test

You can create tests on test hosts that have not been activated. To run the tests, however, the test hosts must first be activated. See Activate and Deactivate Test Hosts on page 51 for instructions.

You can also discover preconfigured tests on test hosts using SPM Discovery. It models tests that have been created on test hosts using a non-SPM method (i.e., command line). See Discover Tests in the Network on page 35 for more information.

Working with Performance Tests 29

Create Tests

Create a New Test

You can create a new test of any type supported by the test host you select.

To create a new test

1. Use SPM Searches to find test hosts as described in Finding SPM Components with SPM Searches on page 13.

The test hosts appear in the Contents panel.

2. Select the test host for which you want to create the test and do one of the following:

Click the New icon and select a test type.

Right-click a test host entry, select New Test, and select a test type.

The New Test dialog opens. It enables you to configure test parameters. See Configure SPM Tests on page 39 for more information about test parameters.

3. Configure test settings and click OK.

The new test is saved and information about the new test you created on the test host appears under the Information tab in the Component Detail panel.

Create a Different Version of an Existing Test

You can create a test by saving a unique version of an existing test. The test can include the same parameter settings, but it must have a different name. Also, you can save the new test to the base test’s test host or a different test host.

Note: You cannot copy tests between domains running different versions of SPECTRUM.

To create a new test from a copy of another test

1. Use SPM Searches to find tests as described in Finding SPM Components with SPM Searches on page 13.

The tests appear in the Contents panel.


Create Tests

2. Select the test you want to copy and do one of the following:

Click the Copy icon.

Right-click the test you want to copy and click Copy Test.

The Copy Test dialog opens. It includes configuration settings you can modify for your new test. See Configure SPM Tests on page 39 for more information about test parameters.

Note: Copied tests are disabled by default, and “_COPY” is appended to the test name. You may want to rename the test, but it is not required.

3. Modify test configuration settings and click OK.

The new test is saved and information about the new test you created on the test host appears under the Information tab in the Component Detail panel.

Information About Specific Test Types

This section includes information about specific test types you should review before you run the tests.

DHCP Tests

DHCP tests measure the round-trip time (latency) required to get an IP address. The DHCP server must be on the same subnet as the test host performing the DHCP test. To configure the test host to work with your DHCP server’s IP address, see the documentation for the device. For DHCP tests to work on a router, one of the neighboring routers must be a DHCP agent or relay. See the documentation for your device for more information.

Note: DHCP test latency result values that are higher than the value set for timeout for tests run on Cisco router test hosts are the result of a known issue with Cisco IOS 12.2(2)T. See Firmware Issues on page 95.

DNS Tests

DNS tests measure DNS lookup time. DNS-based host name-to-address translation must be enabled on the test host device performing the DNS test. To check to see if DNS lookup is enabled and/or to enable it, see the documentation for the device.

DNS test results include the following metrics:

Latency

Packet Loss

FTP Tests

FTP tests measure the round-trip time to transfer a file.


Create Tests

FTP test results include the following metrics:

Latency

Packet Loss

Note: FTP tests run on Cisco test hosts using the RTTMON MIB require that you enter the user name, password, and file name. FTP tests run on CA eHealth SystemEDGE Service Availability test hosts require that you enter a user name and password, but do not need a file name.

HTTP Tests

HTTP tests measure the round-trip time to get a Web page.

HTTP test results include the following metrics:

Latency

HTTP DNS Resolution Time

HTTP TCP Connection Time

HTTP Download Time

ConsiderationsHTTP tests performed from Harmoni and iAgent test hosts produce latency results only.

A HTTP test may fail on some Cisco systems using HTTP 1.1. For more information see Firmware Issues on page 95.

The HTTP version configuration setting is only available for HTTP tests run from Cisco test hosts. The Proxy URL setting is only available for HTTP tests run from Cisco and CA eHealth SystemEDGE Service Availability test hosts. See Configure Advanced Parameters on page 44 for more information.

SPM HTTP tests that require authentication are not supported on Cisco test hosts.

HTTPS Tests

HTTPS tests measure the round-trip time to get a Web page over an encrypted Secure Sockets Layer (SSL) or Transport Layer Security (TLS) connection. An HTTPS test measures the same metrics as an HTTP test.

ICMP (Ping) Tests

ICMP Ping tests measure round-trip time from a source to a destination address.

ICMP Echo (Ping) test results include the following metrics:

Latency


Create Tests

Packet Loss

Note: Cisco and Juniper test hosts support Virtual Routing and Forwarding (VRF) Ping tests.

ConsiderationsWhen using the Cisco IOS IP SLAs Agent, you can configure the ICMP Echo operation payload size by setting the request size. The router adds 36 bytes to the size specified. For example if the request size is 28 bytes, the actual ICMP Packet size is 64 bytes (of IP packet). See Cisco’s IOS IP SLAs Agent documentation for more information.

The Harmoni agent does not support the configuration of sample count or packet size for ICMP Ping tests.

Jitter Tests

Jitter tests both latency and losses between a test host and a voice-enabled endpoint, but adds a finer measure of the statistical behavior of a sequence of requests. Additionally, Mean Opinion Scoring (MOS) is available when using Cisco test hosts. MOS provides a numerical measure of the quality of human speech at the destination end of the circuit. Jitter test results include the following metrics:

Latency

Destination to Source Packet Loss

Late Arrival Packet

Jitter Positive Source to Destination

Jitter Negative Source to Destination

Mean Opinion Scoring

Source to Destination Packet Loss

Missing in Action Packet

Jitter Busies

Jitter Positive Destination to Source

Jitter Negative Destination to Source

Traceroute Tests

Traceroute provides both the discovery of the layer three hops between the source and destination addresses, as well as the round-trip ICMP Echo measurement from the host address to each hop in the discovered path.

Traceroute test results include the following metrics:

Latency

Packet Loss


Create Tests

For each hop, IP address and round trip time

Note: Cisco and Juniper test hosts support Virtual Routing and Forwarding (VRF) traceroute tests.

POP3 Tests

POP3 tests measure POP3 response (transaction) time.

POP3 test results include the following metrics:

Latency

Packet Loss

SMTP Tests

This measures SMTP mail server response (transaction) time.

SMTP test results include the following metrics:

Latency

Packet Loss

TCP Tests

TCP tests measure the time required to create a TCP connection.

TCP Connection test results include the following metrics:

Latency

Packet Loss

UDP Echo Tests

UDP Echo measures round-trip delay.

Results from UDP Echo tests are:

Latency

Packet Loss

UDP tests must be configured to target a destination port known to be listening for UDP traffic. For example, many Cisco devices running IOS IP SLAs use port 1967 and UNIX systems use port 7. Failure to do this can result in timeouts for UDP tests. The Destination Port parameter is set in the General option for UDP tests. (See Configure General Parameters on page 39 for more information.)


Discover Tests in the Network

Discover Tests in the NetworkSPM Test Discovery enables you to discover and model performance tests that have been configured on test hosts but not with SPM.

ConsiderationsYou must have administrator role privileges to use SPM Test Discovery.

You must use a device that is supported by SPM Test Discovery.

About Discovery Modes

When you run a discovery, you are prompted to choose whether you want the tests to be created in SPM in one of the following discovery modes:

Read-Only

Read/Write

Read-Only Discovery Mode

When you discover tests in read-only mode, you will not be able to edit test configurations after the tests are modeled in SPECTRUM. That is, you will not have SNMP set privileges to the tests. Test Discovery takes schedule information from the tests configured on the device and reads test results at the appropriate interval.

ConsiderationsAdministrator access to read-only tests is similar to that of the Operator user role, with the exception of full access to Threshold parameters. For more information see Specify Alarm Thresholds for a Test on page 47.

Read-only tests cannot be run manually; they must be scheduled.

Sample Count is always 1.

You can copy read-only tests to create SPM-based tests.

To ensure that data from read-only tests are available for response time reports you can generate with SPECTRUM Report Manager, make sure that the test is in the Active state at the time of the read-only discovery. You should also verify that the Events service indication in OneClick is showing the “Up” status.



Read/Write Discovery Mode

When you discover tests in read/write mode, you will be able to edit test configurations after the tests are modeled in SPECTRUM. That is, you will have SNMP set privileges to the tests. Tests on these test hosts can be treated exactly like tests created manually in SPM. That is, they can be run on demand, stopped, and edited. See Configure SPM Tests on page 39 for more information.

ConsiderationsIf the test was activated on the device, the Schedule State field for the Schedule parameter field is set to Enabled.

Sample Count is set to 1.

Run Discovery

Running a test discovery involves first locating test hosts that support test discovery and then running the discovery on the test hosts.

To locate test hosts that support test discovery and run test discovery

1. Run the SPM Searches option, Test Hosts By, Test Discovery Supported, to locate discovery-compatible tests hosts.

The Contents panel lists test hosts that support test discovery.

2. Select one or more test hosts for which you want to discover tests and click the Discover Tests icon, as shown in the following image:

Note: The Discover Tests button is disabled if a selected test host is not active or if contact with the host is not established. For more information about activating and deactivating a test host, see Activate and Deactivate Test Hosts on page 51.



If the selected test host supports both Read/Write and Read-Only discovery modes, the Discover Tests: Select Discovery Mode dialog opens, as shown in the following image:

3. Click the button for the test discovery mode you want to run.

The Discover Tests Complete dialog opens when discovery finishes searching, as shown in the following image. It indicates how many tests were created and or updated.

The Events tab in the Component Detail panel provides additional information about the Discovery for the test host. The following table lists SPM Test Discovery event codes and descriptions.

Event Code Event Description

0x04560059 SPM Test No Longer On Device Event

If SPM Test Discovery fails to match an existing SPM Read-Only test to a table entry on the device, this event and a corresponding yellow alarm will be generated on the test model.

0x0456005a SPM Test No Longer Running On Device Event

If SPM reads the test results and detects that the numberOfPktsSent object on the device has not increased, and the operational state on the device is read and found to be InActive, this event will be generated and the following actions occur:

No data is processed.

Schedule state on the SPM test is set to Disabled, and no more data will be read.



0x0456005b SPM Duplicate Result Event If SPM reads the test results and detects that the numberOfPktsSent object on the device has not increased, and the operational state on the device is read and found to be Active, this event will be generated and the following actions occur:

No data is processed.

Schedule state on the SPM test remains Enabled, therefore data will be processed from the next scheduled test.

0x0456005c SPM Test Discovery Completion Event

This event, generated after an SPM Test Discovery has run, indicates the mode (Read-Only or Read/Write) in which the discovery was run. May contain any of the following error output:

No Errors

No SPM Tests Were Discovered

Test type is invalid

Test name is null

Test timeout is 0

Test frequency is 0

Test packet size is 0

Test sample count is 0

Test port number is invalid

Test IP Address is invalid

Test URL is invalid

Test host name is invalid

Test user name is invalid

Test password is invalid

Test file name is invalid

0x0456005d SPM Test Type Mismatch Event If SPM Test Discovery matches an existing SPM Read-Only test to a table entry on the device, but it’s of the wrong test type, generate this event and corresponding yellow alarm on the test model.

0x0456005e SPM Stale Test Clear Event Clears 0x4560059 or 0x456005d if subsequent SPM Test Discovery clears condition.

Event Code Event Description


Configure SPM Tests

Configure SPM TestsWhenever you issue a create, copy, or edit test command, SPM displays a test configuration dialog where you can specify test parameters, set up a test schedule, and specify test thresholds. The following image shows example test configuration categories.

Depending on the type of test you are configuring, the test configuration dialog contains different options under each configuration category, or may not offer all of the options described in this section.

To configure a test

1. Enter values for the test in the parameter categories for your particular test. For example, test scheduling is disabled by default. If you are not interested in scheduling test runs, you do not have to configure scheduling parameters. The same applies to the test thresholds.

2. Click OK in each parameter category you work with to save your settings. The OK button is disabled if you do not enter required values.

Configure General Parameters

The General tab lets you configure required parameters for a test. The following image shows an example configuration dialog for an ICMP Ping test.


Configure SPM Tests

Standard General Parameters

The following general parameters are available for all performance tests supported by SPM.

Name

Specifies the test name.

Default: New SPECTRUM Response Time Test

Test Host

Indicates the IP address of the test host for the test.

Latency Timeout

Specifies the number of milliseconds allowed for the response. If no response is received before this timeout occurs, then SPECTRUM generates a timeout event. Any response that arrives after this timeout is ignored. Set the timeout to be higher than the threshold setting.

Default: 5000 milliseconds

Note: The Harmoni agent does not support latency timeout configuration.

State

Enables and disables the test.

Default: Enabled

Description

Specifies test annotations.

Default: SPECTRUM Response Time Test

General Parameters for Specific Test Types

These parameters may or may not be included under the General tab, depending on the type of test you are working with.

Alternate Source Address

Specifies the IPv4 address or host name of the test source location when it is not the test host, as in mid path location and extended path location ICMP Ping test scenarios.

Note: IPv6 addresses are not supported.

Browse

Lets you choose an Alternate Source Address if the test host on which you are configuring the test is not the source (ICMP) or if you want to choose another test host if you are copying test configurations.

Codec Type

Specifies the VOIP codec (audio compression/decompression) type to test: G.711 U-law, G.711 A-law and G.729A.


Configure SPM Tests

Note: You can set a voice quality threshold value (100 - 500) for the selected codec with the Mean Opinion Score parameter under the Threshold tab.

Destination Address

Specifies the destination address for the test except those of type DHCP, DNS, HTTP. You can enter an IPv4 address or a host name.

Note: IPv6 addresses are not supported.

Destination DNS Server

Specifies the destination address of the DNS server for DNS tests. You can enter an IP address or a host name.

Destination Port

Specifies the port number used by the Jitter, TCP, UDP test. For Mean Opinion Score (MOS) support in Jitter tests, the destination port should be an even numbered port in the range 16384 to 32766 or 49152 to 65534.

Destination URL

Specifies the URL used in HTTP tests.

File Name

Specifies the file path used for the FTP test.

Lookup Name

Specifies the IP address, host name, or fully qualified domain name (FQDN) of the host in DNS tests.

Note: Some agents, such as iAgent, require the use of a FQDN rather than a host name.

Password

Specifies the password for FTP, POP3, or HTTP test authentication.

Test Host Location

Specifies the location of the test host on the path between the source and destination for ICMP Ping tests.

Default: Source

Note: See About the Test Host Location Parameter on page 42 for more information.

User Name

Specifies the user name for FTP, POP3, or HTTP test authentication.

Voice Test

Specifies whether to test voice quality for Jitter tests.


Configure SPM Tests

About the Test Host Location Parameter

Test host location refers to the path between the source and destination of a response time test. The location is critical to response time measurement.

Test host locations:

The test host is the source of the test. See Source Location on page 42.

The test host is located between the source and destination points. See Mid Path Location on page 43.

Test host is not located between the source and destination points. See Extended Path Location on page 43.

Note: Only ICMP Ping tests support mid path or extended path tests.

Mid path and Extended path tests are useful only when either the source or destination address is not a test host. In both of these test types, you must configure test host location.

Source Location

In the most common response time test scenario, the source, or starting point, of the test is also a test host, as shown in the following diagram:

In this case, the test host generates a transaction directly with the destination and measures the RTT.

When the source of a test is a test host, response time measurements are the most direct and accurate. For this reason, you should set up tests with a test host as the source whenever possible.

Source&

Test HostDestination

RTT


Configure SPM Tests

Mid Path Location

In a mid path test configuration, a test host lies on the path between the source and destination points for which RTT is being measured. The source of the test is not capable of being a test host, so it cannot initiate or perform any response time measurements. A mid path test measures response time as the sum of the following:

(response time (RTT1) from test host to the source) + (response time (RTT2) from test host to the destination)

This is a relative value which approximates actual RTT. The following image is an example of a mid path test configuration:

Extended Path Location

In an extended path test configuration, the source of the test is not capable of being a test host. Additionally, there is no test host located between the source and destination as in the Mid Path Location on page 43 scenario. However, the source lies directly in the path between a test host and the destination. In this case, response time is measured as the difference between the following:

(response time (RTT2) from the test host to the destination) — (response time (RTT1) from the test host to the source)

This is a relative value which approximates actual RTT. The following image is an example of an extended path test configuration:

Source Destination

RTT

Test Host RTT2RTT1

Test Host Destination

RTT

Source RTT2

RTT1


Configure SPM Tests

Configure Advanced Parameters

The Advanced tab lets you configure additional parameters for a given test type. Each test type has its own specific parameters. The following image shows an example configuration dialog for a ICMP Ping test.

The following section describes Advanced parameters for all test types.

Sample Count

Specifies the number of times a test is performed during a test run.

Note: RTTMON may occasionally perform more than the specified sample count repetitions for an SPM test.

Default: 5

Note: The Harmoni agent does not support the configuration of sample count.

Packet Size

Specifies the value (in octets) that limits the size of the packets used in the test.

Note: The Harmoni agent does not support packet size configuration.

Type of Service

Specifies the Type of Service (TOS) parameter for test packets sent by the test host. A typical implementation example would be to create a response time test with the TOS parameter that VOIP uses and test the performance of the network in routing those packets.

The Type of Service octet in an IP datagram header enables packets with different TOS values to be routed differently.

Proxy URL

Specifies the proxy URL for HTTP tests. See HTTP Tests on page 32 for more information.


Configure SPM Tests

Alternate Packet Address

Specifies an alternate source IP address instead of using the default network address of the device discovered in SPECTRUM.

Note: The Alternate Packet Address field is available for all Cisco test hosts that support IOS IP SLAs Agent.

Alternate Packet Port

Specifies an alternate packet port instead of using the default packet port of the device discovered in SPECTRUM.

Note: The Alternate Packet Port field is available for all Cisco test hosts that support IOS IP SLAs Agent.

Filter Timeout Data

Specifies whether the Performance tab the OneClick Component Detail panel displays all SPM data or a subset of data minus timeouts.

Default: True (filter timeout data)

HTTP Version

Specifies the HTTP version (1.0 and 1.1) for HTTP tests.

Default: 1.1

VRF Name

The VPN routing instance for the test (Ping and Traceroute for Cisco and Juniper test hosts).

Schedule a Test

The Schedule tab lets you schedule a test. The following image shows an example configuration dialog for a DHCP test.


Configure SPM Tests

The following section describes Schedule parameters.

Schedule State

Enables and disables a test schedule.

Default: Disabled

Schedule time interval

Specifies the interval between scheduled test runs.

Default: 900 seconds

Schedule

Specifies one of the predefined schedules available from the drop-down list. See Create a Schedule on page 46 for information about creating additional schedules.

Default: 24/7

Note: Time zones for Schedule are local to the SpectroSERVER where the test host running the scheduled test is modeled.

Threshold Violation Interval

Specifies an alternate test interval that can be used during a threshold violation period. Decreasing the interval during a period of high latency can provide more precise data.

Default: 300 seconds

Note: When the Threshold Violation is exceeded, the test will not resume again until the Threshold Violation Interval is reached whether it is a lower or higher value than the Scheduled Time Interval.

Important! Setting this value too low can create enough additional traffic and load on the router during a period of high latency to exacerbate the situation.

Create a Schedule

You can create schedules if the SPM-provided schedules do not meet your particular requirements. Schedules you create are not test-specific; they are available for all tests.

To create a test schedule

1. Click Create in the Schedule tab.

The Create Schedule dialog opens.

2. Configure schedule settings and click OK.

The new schedule appears in the Schedule drop-down list.

Note: If you create a schedule with Recurrence set to None, the test will run once on the start date, and then the schedule will revert to the default schedule and the schedule will be disabled.


Configure SPM Tests

The following image shows an example of the Create Schedule dialog:

Specify Alarm Thresholds for a Test

The Threshold tab lets you configure alarm thresholds to monitor response time measurements on a specific link or path. Threshold parameters vary from test to test. When a threshold value is exceeded, SPECTRUM generates an event and/or an alarm based on the settings you specify.


Configure SPM Tests

The following image shows an example threshold configuration dialog for a Jitter test.

The following section describes Threshold option parameters. Threshold parameters vary depending on the type of test you are creating.

Active Thresholds Schedule

Specifies one of the predefined schedules during which thresholds are in effect available from the drop-down list. You may want to use this feature if you want to measure test against thresholds only during certain times of the day. See Create a Schedule on page 46 for information about creating additional schedules.

Default: 24/7

Note: Time zones for Schedule are local to the SpectroSERVER where the test host running the scheduled test is modeled.

Threshold Events Asserted On

Specifies the model from the drop-down list on which events are asserted.

Threshold Types

Identifies threshold settings for the threshold types related to a test.

Status

Enables and disables each threshold type for a test.

Event

Specifies a response time threshold value (greater than 0) that if exceeded results in SPECTRUM generating an event.

Depending on the threshold type, a threshold value can be specified as one the following:


Configure SPM Tests

A time interval in milliseconds (ms)

A percentage (%) of packet errors encountered

Minor Alarm

Specifies a response time threshold value (greater than 0) that if exceeded results in SPECTRUM generating a minor alarm.




Major Alarm

Specifies a response time threshold value (greater than 0) that if exceeded results in SPECTRUM generating a major alarm.




Critical Alarm

Specifies a response time threshold value (greater than 0) that if exceeded results in SPECTRUM generating a critical alarm.




Threshold Cycles

Specifies a value (greater than 0) that indicates the number of consecutive test cycles this test must run in violation of the threshold before SPECTRUM generates an event or alarm.

Clear Threshold

Specifies a response time threshold value (greater than 0) that if not exceeded results in SPECTRUM clearing an event or alarm.




Clear Cycles

Specifies a value (greater than 0) that indicates the number of consecutive test cycles this test must run in compliance with a threshold before SPECTRUM clears an event or alarm.


Run Tests on an On-Demand Basis

Establish Baseline Data to Determine Valid Thresholds

If you are unsure of the threshold values to use for a test (too high or too low), establish baseline response times that you can use to determine appropriate thresholds.

To determine valid thresholds for a test

1. Schedule multiple test runs over the interval to which you want to apply thresholds to tests.

2. Analyze test result data to determine realistic performance thresholds. See Viewing SPM Information on page 55 for more information about test performance data.

3. Configure test thresholds based on the results of your analysis.

Run Tests on an On-Demand BasisYou can run tests manually on an on-demand basis if the following criteria are met:

The test is not scheduled or is scheduled and the schedule is disabled.

The test is enabled.

The test host for the test is activated. See Activate and Deactivate Test Hosts on page 51 for more information.

You would typically manually run tests for diagnostic purposes. For example, you would run an HTTP test to determine whether a Web server is responsive to requests, or you would run a Jitter test to determine the quality of VoIP transmission.


Run Tests on an On-Demand Basis

To manually run a test

1. Use SPM Searches to find the tests you want to run as described in Finding SPM Components with SPM Searches on page 13.


2. Select the tests you want to run and do one of the following, as shown in the following diagram:

Click the Run Tests icon.

Right-click the test and click Run Test.

Information about the test appears in the Component Detail panel.

Activate and Deactivate Test Hosts

All test hosts are modeled in SPECTRUM in the active state. Tests can be configured on a test host in either the activated or deactivated stated. However, tests can only be run on activated test hosts.

Note: If the test host's associated device model is in maintenance mode, an error will be displayed and the test host will not be activated. When the device model is taken out of maintenance mode the test host will be activated if it was activated when the device model went into maintenance mode.

To activate a test host

1. Use SPM Searches to find test hosts as described in Chapter 2: Finding SPM Components with SPM Searches.

The test hosts appear in the Contents panel. The State field indicates whether test hosts are Active or Inactive.

2. Select the inactive test host you want to activate and do one the following:

Click the Activate Test Host icon.

Right-click the test host and click Activate Test Host.

To deactivate a test host

1. Use SPM Searches to find test hosts as described in Finding SPM Components with SPM Searches on page 13.

The test hosts appear in the Contents panel. The State field indicates whether test hosts are Active or Inactive.

2. Select the active test host you want to deactivate and do one of the following:

Click the Deactivate Test Host icon.

Right-click the test host and click Deactivate Test Host.

Note: Any tests that are scheduled to run will stop running until the test host is re-activated.


Manage Tests

Manage TestsYou can edit parameter settings for any test and delete any test, provided you have the administrative-user and read/write privileges to the test.

Important! Editing and deleting tests should be performed with caution by qualified personnel, especially in cases, for example, when SPM performance test results are being monitored by SLAs (service level agreements) or when tests are being run for infrastructure performance analysis purposes.


Manage Tests

Edit a Test

You can edit all test parameter settings as required.

Note: When you edit a test, SPM must stop and restart the test after all changes have been completed. This can disrupt a test schedule. For example, if a test has a 60-minute interval between scheduled runs and it is edited 58 minutes after the last scheduled run, it will run 1 hour and 58 minutes after it has been restarted.

To edit a test

1. Use SPM Searches to find the tests you want to edit as described in Finding SPM Components with SPM Searches on page 13.


2. Use one of the following methods to open the Edit Test dialog:

Click the Edit Test icon.

Right-click the test, and then select Edit Test.

3. Modify test parameters as required. See Configure SPM Tests on page 39 for details on configuration options.

Delete a Test

When you delete a test, you remove it permanently from SPECTRUM.

To delete a test

1. Use SPM Searches to find the tests you want to delete as described in Finding SPM Components with SPM Searches on page 13.


2. Use one of the following methods to delete the test:

Click the Delete icon.

Right-click the test, and then select Delete.

The Confirm Delete dialog opens.

3. Click Yes in the Confirm Delete dialog.


Manage Tests


Chapter 5: Viewing SPM Information

OneClick Console provides views of test hosts, test templates, tests, and test results derived from SPM searches as described in Chapter 2: Finding SPM Components with SPM Searches. It also provides detailed information about events and alarms for SPM components.

Tip: Click the Refresh button in every OneClick Console view where it is available to ensure you are viewing up-to-date information.

Note: See the OneClick Console User Guide (5130) for more information about working with OneClick Console views (refreshing views and docking and undocking panels for example) and customizing views (setting table column preferences) to display only the information types you want to view.

Viewing SPM Information 55

Test Host Information

Test Host InformationOneClick Console displays summary and detailed information about test hosts modeled in SPECTRUM and enables you to perform operations on tests for test hosts. The following image shows an example view of a test host located by a “Test Hosts By IP Address” search.

The Contents panel table displays information about the selected test host model, the landscape where it is modeled, its state (activated or deactivated), and the test types it supports among the various test types supported by SPM. The Contents panel also provides command icons for creating tests for the test host, for discovering tests configured on the test host, and for activating and deactivating the test host.


Test Host Information

Test Host Information in the Component Detail View

The Component Detail panel provides detailed information about a selected test host. The Information tab of the Component Detail panel includes two categories of information about the test host:

Test Host Details

Test List

See SPM Alarms and Events on page 63 for information about accessing information about alarms and events for SPM components.

Test Host Details

The Test Host Details view provides all the information about the test host, which may or may not be displayed in the customizable Contents panel table, and enables you to set the test host model security string.

Test List

Like the Contents panel table, the Test List table displays information about tests for the selected test host. It also provides test command icons for running tests, stopping tests, managing tests, and invoking a Component Detail views for tests.


Test Information

Test InformationOneClick Console displays summary and detailed information about tests modeled in SPECTRUM and enables you to perform operations on tests. The following image shows an example view of tests located by an “All Tests” search.

The Contents panel table displays information about the selected test model, the landscape where it is modeled, the IP addresses of the test source, test destination, and test host for the test. It also indicates whether the test is scheduled, running, or stopped, the last time the test was run and the resulting status of the test (did or did not violate a threshold), and the test discovery state. The Contents panel also provides command icons for running, stopping, and managing tests.


Test Information

Test Information in the Component Detail Panel

The Information tab in the Component Detail panel provides detailed information about a selected test:

Details

Provides information about test configuration settings, including scheduling and test threshold settings.

Last Run Results

Provides information about the most recent test run, including latency and packet loss values.

Threshold Results

Provides information about a threshold violations (event, minor, major, critical) resulting from the test.

Note: Regarding the sample count and the % of Threshold values, RTTMON may occasionally perform more than the sample count repetitions specified for an SPM Test. SPM takes this into account, and properly calculates the average, maximum, and minimum results for latency and packet Loss. Because of this, the value for percentage of packet loss can be other than a multiple of (1 / sample count) * 100. For example: If you set sample count to 5, normally the percentage of packet loss will be a value of 0%, 20%, 40%,... 100%. If the agent performs more than 5 repetitions, you may see 0%, 16.66%, 33.33%... 100%.


Test Information

Test Performance Information in the Component Detail Panel

The Performance tab in the Component Detail panel shows graphical representations of test results. The following image shows an example test performance graph.

Note: The Archive Manager for the test’s landscape must be running or SPM to display performance test results.

Specify the Interval

You can specify the following time intervals for which you want to display performance results:

1 Hr

3 Hr

6 Hr

12 Hr

1 Day

1 Week

4 Weeks

All Results

Note: The All Results option retrieves all result data stored in the SPECTRUM Distributed Data Manager (DDM) database, up to a maximum of 365 days. By default, the DDM stores up to 45 days worth of data. See the Database Management Guide (2601) for more information.


Test Information

Important! Retrieving large numbers of result events may negatively impact performance.

To specify the interval, select an interval from the drop-down list.

Specify the Test Result Type

You can specify a graph that displays performance latency or packet loss statistics for the selected test.

To specify the test result type, select either Latency of Packet Loss from the drop-down list.

View Test Data

You can view data from a particular data point in a graph.

To view test data, position your mouse pointer over the end of a data point line in the graph to display a descriptive label (including date, time, and value) for that data point.

You can also view additional test data (averages, high and low values) in tabular form under the graph.

Adjust an Axis Dimension

You can adjust the dimensions of the X and Y axes as necessary to meet your viewing requirements. For example, you can adjust the X axis to display latency outliers not visible from the default view.

To adjust an axis dimension, select a zoom percentage from the drop-down list for the X or Y axis, or enter a value.

Note: 100% is the minimum allowable zoom percentage.

About the Effect of a Test’s Filter Timeout Data Setting on a Performance Graph

When you configure a test, you can specify that data generated as a result of a test timeout is filtered out of test results. The setting’s default value is True. It will cause gaps in the graph where timeouts or, possibly, test failures have occurred, and you will be unable to discern them. Examples of test failures include lost contact to device, device in maintenance mode, or device failure.

A setting of False, however, will cause the following effects:

Skewed performance graph auto-scaling: Timeout values may be so far greater than the response time values that exact response time values on the performance graph may be difficult if not impossible to discern.


Test Information

Skewed average calculation: Timeout values may be so far greater than response time values that they cause inaccurate average calculations.

Skewed data while using data export: Timeout values will be prevalent in the exported data, so depending on your post-processing mechanisms, this could produce undesirable results.

The following image illustrates some of these effects.

The timeout value is of sucha far greater scale . . .

. . . that these valuescannot be discernedon the graph.

Gap may represent timeoutsor test failures


SPM Alarms and Events

SPM Alarms and EventsYou can view alarms for SPM components from multiple points in OneClick Console. For comprehensive information about viewing alarms, configuring alarm filters, and performing other activities with alarms in OneClick Console, see OneClick Console User Guide (5130).

To view events and alarms for SPM components from the Locater

1. List the components for which you want to view event and alarm information. See Finding SPM Components with SPM Searches on page 13 for more information.

2. In the Contents panel, select a component.

3. In the Component Detail panel, click the Alarms or Events tab to view information.

To view events and alarms for SPM components from the Explorer

1. Click the Alarms tab in the Contents panel.

2. Enter filtering text (RTM_Test for example) to display the component or group of components.

3. In the Component Detail panel, click the Alarm Details or Events tab to view information.


SPM Alarms and Events

The following image shows an example alarm details view.


Chapter 6: Generating Reports on Test Data

SPECTRUM provides two resources you can use to generate reports on performance testing results:

SPECTRUM Report Manager (SRM)

Lets you generate summarized and detailed performance test, or round trip time (RTT) test, results in a variety of text and graphical formats on an on-demand or scheduled basis.

SPECTRUM Report Gateway (SRG)

Lets you generate graphical reports on time-series performance test data.

SPM Reports from SRMSee the Report Manager User Guide (5158) for detailed information about SRM features referenced but not fully described in this section.

SPM report options are included under the Response Time report pack in SRM. SRM provides numerous report content, format, and report organization options. This means you can generate reports with the appropriate type and scope of information for different audiences in your organization interested in performance testing results.

Report Management Options

SRM provides you with multiple options for generating and managing your RTT reports:

Generate reports on demand. Use this option to generate the most recent RTT test results obtainable.

Schedule RTT test report generation on a one-time or periodic basis.

Specify how long you want SRM to retain scheduled RTT test reports or how many reports to retain.

Specify email recipients for scheduled RTT test reports.

Schedule RTT test reports for other SRM users.

Generating Reports on Test Data 65

SPM Reports from SRM

Save RTT test report configurations as favorites that you can run repeatedly without having to reconfigure the report.

Publish RTT test reports in PDF, text, and spreadsheet formats. Use the documents for briefings and presentations on network performance issues.

How to Generate Performance Reports with SRM

You can generate performance reports on demand or schedule reports. This section provides an overview of how you perform these tasks.

To generate RTT reports

1. Select the type of test you want to generate.

The following image shows the SPM report options:

2. Configure the report.



The following image shows an example portion of a configuration page for an ICMP Ping report:

3. Generate (view) the report, save the report configuration as a favorite, or schedule the report.



The following image shows report command buttons on the configuration page

4. View and manage report results. At this point you can also schedule the report and save the report as a favorite as you could before you generated the report, you can page through the report and search for text, and you can also publish the report in multiple industry-standard formats and print the report.


SPM Reports from SRG

The following image shows a portion of the report results:

SPM Reports from SRGYou can export SPM result data compiled over an extended period of time using the SPM result logger. SPM result logger output files include a model handle, time stamp, and a list of test-specific result statistics on each line. Once SPM logging is enabled, SPM produces text files containing SPM result data in SSLogger format that can then be used by the SRG reporting utility. All result files are created and saved in an output directory you create. New log files are created after an interval you specify.

See the OneClick Administration Guide (5166) for information about configuring SPM data export parameters.

See the Report Gateway User Guide (5141) for information about graphical reports.



Data Logging Event Codes

The following table lists SPM data logging event codes:

SPM Result Events

For SPM result events, the data above is followed by statistical data particular to the RTM_Test. The event code dictates which of the statistics are relevant.

Note: Irrelevant statistics for a given test type (for example, Jitter statistics for a ping test) are reported as 0.

The remaining 12 reported statistics in a logged event correspond to (in order):

Latency

Packet loss

HTTP response time

DNS resolution time

TCP connect time

HTTP download time

Jitter response time

Jitter source to destination time

Jitter destination to source time

Jitter MIA

Jitter late arrival

Jitter busies

Event Code Event Contains Statistics For...

0x04560000 SPM Result Event Latency and packet loss

(Except for DHCP tests which do not contain data for packet loss.)

0x0456002e SPM Result Event (HTTP) HTTP response time, DNS resolution time,TCP connect time, and HTTP download time

0x04560010 SPM Result Event (Jitter) Jitter response time, Jitter source to destination time, Jitter destination to source time, Jitter MIA, Jitter late arrival, and Jitter busies

0x0456003e SPM Result Event (Traceroute) Latency



SPM Data Export Sample

The sample below illustrates the structure of logged SPM test result data:

0x830201d,6PM-7AM_6.6.0.2,1054176719,0x4560000,1,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x8302010,Jitter_6.6.0.2,1054176724,0x4560010,0,0.0,0,0,0,0,5,0.0,0.0,0.0,0.0,0.0

0x8302012,FTP_6.6.0.2,1054176730,0x4560000,843,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x8302017,TCP_6.6.0.2,1054176748,0x4560000,1,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x830201b,24x7_6.6.0.2,1054176767,0x4560000,1,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6a00f8a,TCP_6.6.0.0,1054176941,0x4560000,6,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009de,Traceroute_6.6.0.1,1054176872,0x456003e,40,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009d6,FTP_6.6.0.1,1054176872,0x4560000,858,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009e0,TCP_6.6.0.1,1054176872,0x4560000,12,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x8302011,HTTP_6.6.0.2,1054176837,0x456002e,0,0.0,753,5,32,716,0,0.0,0.0,0.0,0.0,0.0

0x830201f,6-11M-F_6.6.0.2,1054176870,0x4560000,1,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009e6,24x7_6.6.0.1,1054176927,0x4560000,5,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009e8,6PM-7AM_6.6.0.1,1054176937,0x4560000,5,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009d5,HTTP_6.6.0.1,1054176939,0x456002e,0,0.0,19,0,7,12,0,0.0,0.0,0.0,0.0,0.0

0x6a00f8b,ICMP_6.6.0.0,1054177035,0x4560000,1,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009df,UDP_6.6.0.1,1054176943,0x4560000,3,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x8302019,UDP_6.6.0.2,1054176906,0x4560000,7,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009eb,6-11M-F_6.6.0.1,1054176960,0x4560000,8,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

0x6c009dd,ICMP_6.6.0.1,1054176978,0x4560000,4,0.0,0,0,0,0,0,0.0,0.0,0.0,0.0,0.0

SPM Test Modification Event

SPM test modification events (event code 0x0456000a) contain the following statistical data (in addition to the common data described under Data Logging Event Codes on page 70):

Test name

Destination address

Packet size

Source address

Test interval

Sample count


Chapter 7: Creating and Managing RTM Tests with Command Line Interface (CLI)

Response time tests are represented in SPECTRUM as models of model type RTM_Test. You can use SPECTRUM’s CLI to create, run, and edit response time tests and get test results as an alternative method to using the SPM user interface in OneClick (see Working with Performance Tests on page 29). You can issue test management commands from the CLI command line, or you can embed CLI commands in scripts, which you can run on an ad hoc or scheduled basis.

Important! Before you attempt to create and manage tests with CLI, you should understand Service Performance Manager concepts, CLI commands and command syntax, and SPECTRUM modeling concepts.

See the Command Line Interface User Guide (0664) for more information about using the following CLI commands to create and manage tests:

ack alarm

connect

create

current

destroy

disconnect

jump

seek

setjump

show

stopShd

update

Note: CLI command references and examples in this chapter use the typographical conventions described in the Command Line Interface User Guide (0664).

Creating and Managing RTM Tests with Command Line Interface (CLI) 73

Create Tests in CLI

Create Tests in CLIYou can create any of the response time test models supported by SPECTRUM using the CLI create command. See Required Parameters for Creating Tests on page 76 for complete information about required test attributes for all tests and specific tests.

The following example script shows how to create a regularly scheduled Ping test. It enables a latency threshold, a latency value that if exceeded triggers a minor alarm, and a latency value that if not exceeded triggers an alarm clear.

#!/bin/ksh

cd $SPECROOT/vnmsh

connect

./create model mth=0x4560000 \

attr=0x1006e,val="Ping_Test_1" \ # Test Name

attr=0x4560005,val=0 \ # Test Type is ICMP_Ping

attr=0x4560006,val=10.253.9.8 \ # TestHost address

attr=0x456001f,val=10.253.9.12 \ # Destination IP

attr=0x4560022,val=1 \ # Schedule is enabled

attr=0x4560014,val=600 \ # Test interval is 10 minutes

attr=0x4560035,val=1 \ # Latency Threshold is enabled

attr=0x456009b,val=100 \ # Latency Minor Threshold Set value

attr=0x4560017,val=100 \ # Latency Threshold Clear value

attr=0x4560027,val=2 # Clear Cycles

The following example script shows how to create an HTTP test. Note the formatting required for URL values.

#!/bin/ksh

cd $SPECROOT/vnmsh

connect

./create model mth=0x4560000 \

attr=0x1006e,val="Http_Test_1" \ # Test Name

attr=0x4560005,val=5 \ # Test Type is HTTP

attr=0x4560006,val=10.253.9.8 \ # TestHost address

attr=0x456000f,val="http:\/\/www.ca.com\/about.htm" # Destination URL

attr=0x456008d,val="http:\/\/proxyServer" # Proxy URL

attr=0x4560022,val=1 \ # Schedule is enabled


Run Tests in CLI

Run Tests in CLIYou can run a response time test using the CLI update command by specifying the SPM_RUN_TEST action code (0x4560009) and the test model handle. See Test Action Codes on page 92 for more information. The following example script shows how to run a single test. You can run multiple tests by specifying multiple tests in the script.

#!/bin/ksh

# this will run a test with the given model handle.

cd $SPECROOT/vnsmh

connect

./update action=0x4560009 mh=<testMH>

Edit Tests in CLIYou can edit a response time test using the CLI update command to change test parameter values by specifying new parameter values and issuing an update test action code (0x4560008). The following example script shows how to modify a tests’s schedule interval and test timeout values.

#!/bin/ksh

# change the schedule interval from 15 minutes to 30 minutes (1800 seconds)

# change the test timeout from 5000 ms to 1000 ms

# have the test not filter its timeout data

cd $SPECROOT/vnmsh

connect

./update mh=<testMH> attr=0x4560014,val=1800

./update mh=<testMH> attr=0x4560025,val=1000

./update mh=<testMH> attr=0x45600d6,val=FALSE

./update action=0x4560008 mh=$i

Get Test Results in CLIYou can get various test status indications and test results using the CLI show command along with the parameters that return the values you want.


Test Parameters Used in CLI

with ded

ged. akes

The following example script shows how to get test status and test results. See Test Status and Test Results Parameters on page 92.

#!/bin/ksh

# obtain the latestResult status and latency/packet loss values for a given test

cd $SPECROOT/vnmsh

connect

./show attributes -e attr=0x4560004 attr=0x4560015 attr=0x456007d mh=<testMH>

Test Parameters Used in CLIThis section lists and describes parameters for creating and scheduling tests and specifying test thresholds. (Configure SPM Tests on page 39 describes how to configure RTM_Test model parameters using the Service Performance Manager user interface in OneClick.)

Required Parameters for Creating Tests

These parameters are required for creating response time test models.

Attribute Attribute ID Default Value Description/Usage

Model_Name 0x1006e None Name of the test. Only one test per host can existthe same name. Duplicate test names are appenwith “_COPY”

Test_Type 0x4560005 None Specifies the type of test to create.

0=ICMP_Ping

1=UDP_ECHO

2=TRACEROUTE

3=TCP_CONNECT

4=DNS_LOOKUP

5=HTTP

6=POP3

7=DHCP

8=FTP

9=SMTP

10=Jitter

Once a test is created, this value cannot be chanIf it is changed, it will be reset when the update tplace.



he .

f it is lace.

tests.

lose cape eate

tests he

P3

.

3

Required Parameters for Scheduling Tests

Use these parameters to specify test timeout and test schedule management values. (Schedule a Test on page 45 describes how to configure test schedule parameters using the Service Performance Manager user interface in OneClick.)

Test_Host_Address 0x4560006 None The IP Address of a test host. This must match tnetwork address of the associated device model

Once a test is created, this cannot be changed. Ichanged, it will be reset when the update takes p

Dest_IP_Address 0x456001f None The destination IP address. Used for ICMP_Ping,TRACEROUTE, UDP, TCP, DNS, SMTP, and Jitter

Dest_URL 0x456000f None The destination URL required for HTTP tests. Encthe URL between double quotes (“ ”), and use esslashes ( \ ) before forward slashes ( / ). See CrTests in CLI on page 74 for a format example.

Dest_Host_Name 0x456000a None The destination test host name required for DNS as the lookup name, and for FTP/POP3 tests as taddress on which to perform the transaction.

Dest_Port_Number 0x4560011 None Must be non-zero for TCP/UDP/Jitter tests.

Dest_User_Name 0x456000b None The destination user name. Required for FTP/POtests. Optional for HTTP tests.

Dest_File_Name 0x456000d None The destination file name. Required for FTP tests

Dest_Password 0x456000e None The destination password. Required for FTP/POPtests. Optional for HTTP tests.


Attribute Attribute ID Default Value

Description/Usage

Test_Timeout 0x4560025 5000 The number of milliseconds before a test connection to an unresponsive test host times out.

Sched_Frequency 0x4560014 900 Interval in seconds between scheduled test runs.

Schdule_State 0x4560000 0 Specifies whether or not the test schedule is in effect.

0 = DISABLED

1 = ENABLED

Thresh_Frequency 0x456001a 300 Interval in seconds after which a test runs while in a threshold condition.



Optional Parameters

Use these parameters to specify various, optional test parameters. (Configure Advanced Parameters on page 44 describes how to specify these parameters using the Service Performance Manager user interface in OneClick.)

TestSchedule_Type 0x456008b 5 Preconfigured test schedules provided with the SPM application.

0 = 7x24

1 = 7A-6P

2 = 6P-7A

3 = MF 8A-8P

4 = MF 6A-11P

Use as an alternative to ACTIVE_SCHED_MH.

Note: Not specifying a value results in a 7x24 schedule

ACTIVE_SCHED_MH 0x456008f None Model handle of a test schedule model.

Use as an alternative to TestSchedule_Type.

Note: Not specifying a value results in a 7x24 schedule.


Description/Usage


Description/Usage

FILTER_TIMEOUT_DATA 0x45600d6 TRUE Specifies whether the RTM_Test will generate result events for timeouts.

Sample_Count 0x4560068 5 Specifies the number of times during a test run that the test is performed.

Packet_Size 0x4560067 64 Specifies the value (expressed in octets) that limits the size of the packets used in the test.

State 0x4560003 1 Specifies whether or not the test is enabled.

1 = ENABLED

2 = DISABLED

Thresh_Model 0x4560024 0 Specifies the test entity on which threshold events are asserted.

0 = TEST

1 = SOURCE

2 = DESTINATION

3 = HOST



ThreshSchedule_Type 0x4560090 5 Preconfigured threshold schedules provided with the SPM application. Specifies periods during which you want to see SPM threshold alarms.

0 = 7x24

1 = 7A-6P

2 = 6P-7A

3 = MF 8A-8P

4 = MF 6A-11P


THRESH_SCHED_MH 0x4560090 None Model Handle of a Schedule model. Used with the ThreshSchedule_Type parameter.


Source_IP_Address 0x4560008 None For use by Mid_Path/Extended_Path Ping tests only.

Test_Host_Position 0x4560030 0 Specifies test host location.

0 = END_POINT

1 = MID_PATH

2 = EXTENDED_PATH

Proxy_URL 0x456008d None Used by HTTP tests only. Specifies the URL for the test. Enclose the URL between double quotes (“ ”), and use escape slashes ( \ ) before forward slashes ( / ). See Create Tests in CLI on page 74 for a format example.

TypeOfService 0x4560099 None Cisco hosts only. Specifies the Type of Service (TOS) octet in an IP datagram header that enables packets with different TOS values to be routed differently. Must be less than 256.

EchoAdminSourceAddress 0x45600b0 None Cisco hosts only. Specifies the IP address or host name of the test when it is not the test host. See About the Test Host Location Parameter on page 42 for more information.

EchoAdminSourcePort 0x45600b1 None Cisco hosts only. Specifies the port number used by the tested application (Jitter, TCP, UDP only). It must be less than 65536.


Description/Usage



Threshold Type Parameters

You can specify threshold management parameters for all test types. Thresholds are expressed either in terms of response, or transaction time, or packet error or loss, depending on the type of test for which the threshold is specified.

Latency Threshold Parameters

CodecType 0x45600e7 0 Specifies the type of codec used by the router to perform audio compression and decompression. This is important for correct calculation of the Mean Option Score (MOS).0 = NONE

1 = G.711 U-law

2 = G.711 A-law

3 = G.729A


Description/Usage


Description/Usage

Latency_Thresh_State 0x4560035 0 Specifies whether the latency threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Latency_MinorSetValue 0x456009b 500 Specifies the latency threshold period in milliseconds that must be exceeded before a minor alarm is generated.

Latency_MajorSetValue 0x456009c None Specifies the latency threshold period in milliseconds that must be exceeded before a major alarm is generated.

Latency_CriticalSetValue 0x456009d None Specifies the latency threshold period in milliseconds that must be exceeded before a critical alarm is generated.

Thresh_Set_Value 0x4560016 None Specifies the latency threshold period in milliseconds that must be exceeded before an event is generated.

Thresh_Clear_Value 0x4560017 500 Specifies the latency threshold period in milliseconds the test must not exceed before an event is cleared.

Thresh_Set_Delay 0x4560026 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.



Packet Loss Threshold Parameters

Thresh_Clear_Delay 0x4560027 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.


Description/Usage


Description/Usage

PL_Thresh_State 0x4560034 0 Specifies whether the packet loss threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

PacketLoss_MinorSetValue 0x456009e 20% Specifies the packets lost percentage that must be exceeded before a minor alarm is generated.

PacketLoss_MajorSetValue 0x456009f None Specifies the packets lost percentage that must be exceeded before a major alarm is generated.

PacketLoss_CriticalSetValue 0x45600a0 None Specifies the packets lost percentage that must be exceeded before a critical alarm is generated.

PL_Set_Value 0x456002c None Specifies the packets lost percentage that must be exceeded before an event is generated.

PL_Clear_Value 0x456002e 20% Specifies the packets lost percentage that must not be exceeded before a minor alarm is generated.

PL_Set_Delay 0x456002d 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

PL_Clear_Delay 0x456002f 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.



HTTP DNS Threshold Parameters

HTTP TCP Threshold Parameters


Description/Usage

Statistic_1_Thresh_State 0x4560036 0 Specifies whether the HTTP DNS threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_1_MinorSetValue 0x45600a1 500 Specifies the resolution time threshold period in milliseconds that must be exceeded before a minor alarm is generated.

Statistic_1_MajorSetValue 0x45600a2 None Specifies the resolution time threshold period in milliseconds that must be exceeded before a major alarm is generated.

Statistic_1_CriticalSetValue 0x45600a3 None Specifies the resolution time threshold period in milliseconds that must be exceeded before a critical alarm is generated.

Statistic_1_Set_Value 0x4560039 None Specifies the resolution time threshold period in milliseconds that must be exceeded before an event is generated.

Statistic_1_Clear_Value 0x456003a 500 Specifies the resolution time threshold the test must not exceed before an event is cleared.

Statistic_1_Set_Delay 0x456003b 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

Statistic_1_Clear_Delay 0x456003c 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.


Statistic_2_Thresh_State 0x456003e 0 Specifies whether the HTTP TCP threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_2_MinorSetValue 0x45600a4 500 Specifies the connection time threshold period in milliseconds that must be exceeded before a minor alarm is generated.



HTTP Download Threshold Parameters

Statistic_2_MajorSetValue 0x45600a5 None Specifies the connection time threshold period in milliseconds that must be exceeded before a major alarm is generated.

Statistic_2_CriticalSetValue 0x45600a6 None Specifies the connection time threshold period in milliseconds that must be exceeded before a critical alarm is generated.

Statistic_2_Set_Value 0x4560041 None Specifies the connection time threshold period in milliseconds that must be exceeded before an event is generated.

Statistic_2_Clear_Value 0x4560042 500 Specifies the connection time threshold that must not exceed before an event is cleared.

Statistic_2_Set_Delay 0x4560043 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

Statistic_2_Clear_Delay 0x4560044 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.



Description/Usage

Statistic_3_Thresh_State 0x4560046 None Specifies whether the HTTP download threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_3_MinorSetValue 0x45600a7 500 Specifies the download time threshold period in milliseconds that must be exceeded before a minor alarm is generated.

Statistic_3_MajorSetValue 0x45600a8 None Specifies the download time threshold period in milliseconds that must be exceeded before a major alarm is generated.



Jitter Source to Destination Packet Loss Threshold

Statistic_3_CriticalSetValue 0x45600a9 None Specifies the download time threshold period in milliseconds that must be exceeded before a critical alarm is generated.

Statistic_3_Set_Value 0x4560049 None Specifies the download time threshold period in milliseconds that must be exceeded before an event is generated.

Statistic_3_Clear_Value 0x456004a 500 Specifies the download time latency threshold the test must not exceed before an event is cleared.




Description/Usage


Description/Usage

Statistic_1_Thresh_State 0x4560036 0 Specifies whether the Jitter source to destination packet loss threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_1_MinorSetValue 0x45600a1 20% Specifies the percentage of packets that must be lost before a minor alarm is generated.

Statistic_1_MajorSetValue 0x45600a2 None Specifies the percentage of packets that must be lost before a major alarm is generated.

Statistic_1_CriticalSetValue 0x45600a3 None Specifies the percentage of packets that must be lost before a critical alarm is generated.



Jitter Destination to Source Packet Loss Threshold

Statistic_1_Set_Value 0x4560039 None Specifies the percentage of packets that must be lost before an event is generated.

Statistic_1_Clear_Value 0x456003a 20% Specifies the percentage of packets lost that must not be exceeded before an event or alarm is cleared.

Statistic_1_Set_Delay 0x456003b 1 Specifies the number of consecutive cycles the test must run in violation of a threshold before an event or alarm is generated.



Description/Usage


Description/Usage

Statistic_2_Thresh_State 0x456003e 0 Specifies whether the Jitter destination to source packet loss threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_2_MinorSetValue 0x45600a4 20% Specifies the packets lost percentage that must be exceeded before a minor alarm is generated.

Statistic_2_MajorSetValue 0x45600a5 None Specifies the packets lost percentage that must be exceeded before a major alarm is generated.

Statistic_2_CriticalSetValue 0x45600a6 None Specifies the packets lost percentage that must be exceeded before a critical alarm is generated.



Jitter MIA Threshold Parameters

Statistic_2_Set_Value 0x4560041 None Specifies the packets lost percentage that must be exceeded before an event is generated.

Statistic_2_Clear_Value 0x4560042 20% Specifies the packets lost percentage that must not be exceeded before a minor alarm is generated.

Statistic_2_Set_Delay 0x4560043 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.



Description/Usage


Description/Usage

Statistic_3_Thresh_State 0x4560046 0 Specifies whether the Jitter MIA threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_3_MinorSetValue 0x45600a7 20% Specifies the percentage of missing in action packets that must be exceeded before a minor alarm is generated.

Statistic_3_MajorSetValue 0x45600a8 None Specifies the percentage of missing in action packets that must be exceeded before a major alarm is generated.

Statistic_3_CriticalSetValue 0x45600a9 None Specifies the percentage of missing in action packets that must be exceeded before a critical alarm is generated.



Jitter Late Arrival Threshold Parameters

Statistic_3_Set_Value 0x4560049 None Specifies the percentage of missing in action packets that must be exceeded before an event is generated.

Statistic_3_Clear_Value 0x456004a 20% Specifies the percentage of missing in action packets that must not be exceeded before an event or alarm is cleared.

Statistic_3_Set_Delay 0x456004b 1 Specifies the number of consecutive cycles the test must exceed the threshold before an event or alarm is generated.

Statistic_3_Clear_Delay 0x456004c 1 Specifies the number of consecutive cycles the test must not exceed the threshold before an event or alarm is generated.


Description/Usage


Description/Usage

Statistic_4_Thresh_State 0x456004e 0 Specifies whether the Jitter late arrival threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_4_MinorSetValue 0x45600aa None Specifies the percentage of late packets that must be exceeded before a minor alarm is generated.

Statistic_4_MajorSetValue 0x45600ab 20% Specifies the percentage of late packets that must be exceeded before a major alarm is generated.

Statistic_4_CriticalSetValue 0x45600ac None Specifies the percentage of late packets that must be exceeded before a critical alarm is generated.

Statistic_4_Set_Value 0x4560051 None Specifies the percentage of late packets that must be exceeded before an event is generated.

Statistic_4_Clear_Value 0x4560052 None Specifies the percentage of late packets that must not be exceeded before an event or alarm is cleared.



Jitter Busies Threshold Parameters

Statistic_4_Set_Delay 0x4560053 20% Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.



Description/Usage


Description/Usage

Statistic_5_Thresh_State 0x4560056 0 Specifies whether the Jitter busies threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_5_MinorSetValue 0x45600ad 20 Specifies the percentage of busy failures that must be exceeded before a minor alarm is generated.

Statistic_5_MajorSetValue 0x45600ae None Specifies the percentage of busy failures that must be exceeded before a major alarm is generated.

Statistic_5_CriticalSetValue 0x45600af None Specifies the percentage of busy failures that must be exceeded before a critical alarm is generated.

Statistic_5_Set_Value 0x4560059 None Specifies the percentage of busy failures that must be exceeded before an event is generated.

Statistic_5_Clear_Value 0x456005a 20 Specifies the percentage of busy failures that must not be exceeded before an event is generated.





Jitter Positive Source to Destination Threshold Parameters

Jitter Positive Destination to Source Threshold Parameters


Description/Usage

Statistic_6_Thresh_State 0x45600ba 0 Specifies whether the Jitter positive source to destination threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_6_MinorSetValue 0x45600b4 25 Specifies the percentage of positive Jitter that must be exceeded before a minor alarm is generated.

Statistic_6_MajorSetValue 0x45600b5 None Specifies the percentage of positive Jitter that must be exceeded before a major alarm is generated.

Statistic_6_CriticalSetValue 0x45600b6 None Specifies the percentage of positive Jitter that must be exceeded before a critical alarm is generated.

Statistic_6_Set_Value 0x45600b3 None Specifies the percentage of positive Jitter that must be exceeded before an event is generated.

Statistic_6_Clear_Value 0x45600b7 25 Specifies the percentage of positive Jitter that must not be exceeded before an event is generated.

Statistic_6_Set_Delay 0x45600b8 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

Statistic_6_Clear_Delay 0x45600b9 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.


Description/Usage

Statistic_7_Thresh_State 0x45600ce 0 Specifies whether the Jitter positive destination to source threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED



Jitter Negative Source to Destination Threshold Parameters

Statistic_7_MinorSetValue 0x45600bd 25 Specifies the percentage of positive Jitter that must be exceeded before a minor alarm is generated.

Statistic_7_MajorSetValue 0x45600be None Specifies the percentage of positive Jitter that must be exceeded before a major alarm is generated.

Statistic_7_CriticalSetValue 0x45600bf None Specifies the percentage of positive Jitter that must be exceeded before a critical alarm is generated.

Statistic_7_Set_Value 0x45600bc None Specifies the percentage of positive Jitter that must be exceeded before an event is generated.

Statistic_7_Clear_Value 0x45600c0 25 Specifies the percentage of positive Jitter that must not be exceeded before an event is generated.

Statistic_7_Set_Delay 0x45600c1 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

Statistic_7_Clear_Delay 0x45600c2 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.


Description/Usage


Description/Usage

Statistic_8_Thresh_State 0x45600cc 0 Specifies whether the Jitter negative source to destination threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_8_MinorSetValue 0x45600c5 25 Specifies the percentage of negative Jitter that must be exceeded before a minor alarm is generated.

Statistic_8_MajorSetValue 0x45600c6 None Specifies the percentage of negative Jitter that must be exceeded before a major alarm is generated.



Jitter Negative Destination to Source Threshold Parameters

Statistic_8_CriticalSetValue 0x45600c7 None Specifies the percentage of negative Jitter that must be exceeded before a critical alarm is generated.

Statistic_8_Set_Value 0x45600c5 None Specifies the percentage of negative Jitter that must be exceeded before an event is generated.

Statistic_8_Clear_Value 0x45600c9 25 Specifies the percentage of negative Jitter that must not be exceeded before an event is generated.

Statistic_8_Set_Delay 0x45600ca 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

Statistic_8_Clear_Delay 0x45600cb 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.


Description/Usage


Description/Usage

Statistic_9_Thresh_State 0x45600d5 None Specifies whether the Jitter negative destination to source threshold is enabled or disabled for a test.

0 = DISABLED

1 = ENABLED

Statistic_9_MinorSetValue 0x45600cf 25 Specifies the percentage of negative Jitter that must be exceeded before a minor alarm is generated.

Statistic_9_MajorSetValue 0x45600d0 None Specifies the percentage of negative Jitter that must be exceeded before a major alarm is generated.

Statistic_9_CriticalSetValue 0x45600d1 None Specifies the percentage of negative Jitter that must be exceeded before a critical alarm is generated.

Statistic_9_Set_Value 0x45600ce None Specifies the percentage of negative Jitter that must be exceeded before an event is generated.


Test Action Codes

Test Action CodesUse action codes with the CLI update command to save changes to tests and run tests.

Test Status and Test Results Parameters

SPM LatestStatus (0x4560004) Attribute Values

You can use the latest status attribute with the CLI show command to get status information for the most recently run test.

Statistic_9_Clear_Value 0x45600d2 25 Specifies the percentage of negative Jitter that must not be exceeded before an event is generated.

Statistic_9_Set_Delay 0x45600d3 1 Specifies the number of consecutive cycles the test must run in violation of the threshold before an event or alarm is generated.

Statistic_9_Clear_Delay 0x45600d4 1 Specifies the number of consecutive cycles the test must run without exceeding the threshold before an event or alarm is cleared.


Description/Usage

Action Code Description/Usage

Update test 0x4560008 Update the test after making attribute changes.

Run test 0x4560009 Run the test.

Update and Run test 0x456000a Combination of Update and Run actions.

Attribute Description

1=Ok Test ran successfully.

2=Threshold Test result has exceeded its “Set Value.”

3=Timeout Test timed out.



Test Results

Use the CLI show command to get results for a particular test.

4=Failed Test failed to create a response time test table entry due to configuration issue.

5=Initial Test has never run.

6=Bad_comm Test failed to create a response time test table entry because of an invalid community string.

7=Running Test is running.

8=Stopped Test has stopped running.

9=Threshold_minor Test result has exceeded its “Minor Set Value.”

10=Threshold_major Test result has exceeded its “Major Value.”

11=Threshold_critical Test result has exceeded its “Critical Value.”

Attribute Description

Attribute Attribute ID Description

Result_Timestamp 0x456005e The time test last completed.

Latest_Result 0x4560015 Average Response Time (scalar).

TRACEROUTE_Result 0x4560075 TraceRoute Result. List of IP Address Latency Result pairs.

PL_Result 0x456007d % Packet Loss.

DNS_Latest_Result 0x4560037 Average HTTP DNS Lookup Time (scalar).

TCP_Latest_Result 0x456003f Average HTTP TCP Connection Time (scalar).

DL_Latest_Result 0x4560047 Average HTTP Page Download Time (scalar).

JPLSD_Latest_Result 0x456007e % Jitter Packets Loss between Source and Destination.

JPLDS_Latest_Result 0x456007f % Jitter Packets Loss between Destination and Source.

JBUS_Latest_Result 0x4560080 % Jitter Packets Busy.

JMIA_Latest_Result 0x4560081 % Jitter Packets Missing in Action.

JLATE_Latest_Result 0x4560082 % Jitter Packets Arriving Late.

PosJitterSD_LatestResult 0x45600b2 Average Positive Jitter between Source and Destination.



PosJitterDS_LatestResult 0x45600bb Average Positive Jitter between Destination and Source.

NegJitterSD_LatestResult 0x45600c4 Average Negative Jitter between Source and Destination.

NegJitterDS_LatestResult 0x45600cd Average Negative Jitter between Destination and Source.

MOS_Latest_Result 0x45600e5 Mean Opinion Score Value (0 - 500). It provides a numerical measure of the quality of human speech at the destination end of the circuit.

Attribute Attribute ID Description


Appendix A: Troubleshooting

This chapter identifies error messages that may be generated during SPM operations and describes corrective action where feasible. Other maintenance and optional configuration issues are also addressed.

Firmware IssuesCertain router firmware revisions, such as the one described in Cisco IOS 12.0(9) on page 95, can exhibit instability. CA follows published interfaces to the SNMP agents, and rely on the device vendors to fully support these interfaces. Before deploying SPM, it is advisable that users review device and firmware documentation from the vendor and apply any updates as appropriate.

Cisco IOS

When managing Jitter tests on Cisco IP SLA hosts, note that certain Cisco IOS versions do not allow codec type changes once the test has been run.

Cisco IOS 12.0(9)

Cisco IOS 12.0(9) has an issue that causes the router to reload upon the first SNMP SET performed to validate that the MIB is writable. See the following URL for more information:

http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120cavs/120mcavs.htm

Cisco IOS 12.0(9) has an issue that causes the router to reload upon an SNMP Get of the supported test types table, which occurs during model activation.

Troubleshooting 95

Firmware Issues

Cisco IOS 12.0(7)T2

Cisco IOS 12.0(7)T2 has an issue which causes the test described in Traceroute Tests on page 33 to fail. The problem is that the device does not properly report hop data in the result tables of the CISCO-RTTMON-MIB, which causes SPM to put erroneous data in the result event for average response time. To address this, upgrade your firmware to Cisco IOS 12.1(17).

Cisco IOS below 12.2

Cisco IOS below 12.2 has an issue which causes HTTP version 1.1 tests to fail with a “Request Timed Out” error message. Upgrading to Cisco IOS 12.2 or later versions will fix this issue.

Workaround: Changing the HTTP version from 1.1 to 1.0 may correct the timeout error message. See Configure Advanced Parameters on page 44 for more information.

Cisco IOS 12.2(2)T

Cisco IOS 12.2(2)T has an issue that causes the router to intermittently report incorrect operation error codes such that DHCP response time tests that have timed out are reported as “OK.” When this happens during a run of a DHCP test, SPM can report a latency value for that DHCP test that is greater than the timeout value set for the test. To address this, Cisco recommends upgrading your firmware to Cisco IOS 12.2(15)T2.

Cisco IOS 12.2(11)T

Cisco devices running IOS firmware 12.2(11)T and higher will now correctly function as test hosts in SPM. Previously, configured tests would not operate correctly due to changes in the RTTMon MIB.

Cisco IOS 12.3(4)

When running SPM tests, a router running Cisco IOS version 12.3(10a) that is configured for SAA/RTR may crash because of a memory leak in the SAA/RTR process. This issue has been resolved in IOS version 12.3 (11) TO4.

Cisco IOS 12.3(5) and below

In Cisco IOS versions 12.3(5) and below changing the packet size on a Jitter Tests on page 33 test will cause the IOS to crash and reboot the router. This issue has been addressed in Cisco IOS 12.3(5.013).


Timeout Errors

Cisco IOS 12.2(18)SXF3 and 12.2(18)SXF4

Cisco Routers running IOS 12.2(18)SXF3 and Version 12.2(18)SXF4 will crash when SPM tests are run because of Cisco bug CSCin62031. Routers running these IOS versions should not be modeled as test hosts capable of running Cisco IPSLA. To prevent these routers from being modeled as test hosts capable of running Cisco IPSLA, add these commands to the router configuration prior to modeling:

snmp-server view NoRTTMON internet included

snmp-server view NoRTTMON ciscoRttMonMIB excluded

snmp-server community TEST view NoRTTMON RO

If the above configuration commands are added after modeling, then any attempt to run SPM tests will fail gracefully and not cause the router to crash.

Juniper (all JUNOS devices)

On a Juniper host device, running a response time test with a name longer than 32 characters will return an error. If this is the case, recreate the test with a shorter name. When test templates are used, be aware that the model name or IP address is appended onto the template name. For this reason, take special care to leave enough characters when using templates so that the final, full test name is at or under the 32 character limit.

Riverstone RS-8000 FW 9.0.0.4

Response time tests run from Riverstone RS-8000 (firmware 9.0.0.4) test host devices may return “Bad Configuration” errors. If this occurs, ensure that no two tests configured for the Riverstone test host have exactly the same character length names and rename any such tests as necessary. See Configure General Parameters on page 39 for more information.

Timeout ErrorsIn most cases, lack of access is the cause of timeout errors during SPM tests. For example, when Network Address Translation (NAT) is enabled, it can deny access to networks that are not in the list of networks to translate. Thus response time tests for HTTP or ICMP echo from an unlisted range of IPs would result in timeouts.

The solution in this case would be to add the test network to the NAT list and re-run the tests. Other tests, such as DNS, DHCP, and UDP tests require that the test host device be properly configured for the service being tested.

Troubleshooting 97

Test Scheduling Caveats

Timeout errors can also be caused by setting the latency timeout parameter to an invalid value. Valid values for test timeouts can vary from test type to test type, and even from device to device. For example, the CISCO-RTTMON-MIB provides the following guideline in describing its timeout value:

To prevent unwanted closure of connections, be sure to set this value to a realistic connection timeout.

Common solutions for timeout errors are:

Ensure that you have SNMP read/write access on the test host device.

Ensure that you have access from the test host device to the service being tested (ICMP, HTTP, etc.).

Ensure that you have set reasonable values for test timeout.

Note: HTTP tests using certain Cisco Routers that run HTTP version 1.1 may fail with a “Request Timed Out” error. See Firmware Issues on page 95.

ICMP Ping Tests and Extreme Summit Test Host Devices

Ping tests run from Extreme Summit test host devices can result in timeouts. When you execute a ping from certain Extreme Summit devices, the RFC2925 branch of their MIB agents incorrectly reports a response time of 0. SPM interprets this as a timeout, when in fact the device has replied to the ping. The current workaround is to use a different device as a test host.

Traceroute Tests and iAgent Test Hosts

SPM always reports a timeout when traceroute tests are run on an iAgent test host. This issue has been resolved by iAgent version 16.2.

Test Scheduling CaveatsThis section contains the following information about scheduling tests:

Scheduling Tests in Geographically Distributed Environments on page 98

Schedule Time Interval and CA SystemEDGE Agent Test Hosts on page 99

Large Number of Tests on a Router Running Off Schedule on page 99

Scheduling Tests in Geographically Distributed Environments

In a geographically distributed environment, the following network components can potentially be in different time zones:

SpectroSERVERs

OneClick Consoles


Test Scheduling Caveats

Web servers

Test host devices

Test scheduling in SPM is based on the time zone of the SpectroSERVER where the test host is modeled. Test results in SPM are based on the time zone local to the OneClick Console. To address the time discrepancy this confusion, choose the 24/7 option when scheduling.

Schedule Time Interval and CA SystemEDGE Agent Test Hosts

In general, to help ensure that scheduled tests run according to the configured schedule time interval the following algorithm is used to calculate when the scheduled test will run. This is applied after the previous test results are returned.

(Schedule Time Interval) -

(elapsed time since test last initiated) = (test time)

where

(elapsed time since test last initiated) =

(sample count) * (timeout).

SPM performs this calculation differently for tests run from CA SystemEDGE agent test hosts. Concord systemEDGE agents require the period over which response time statistics are calculated to be at least 60 seconds.

Tests scheduled to run on a CA SystemEDGE test host calculate:


(sample count) * (timeout) + (60 seconds).

For example, using the default values for sample count and timeout of 5 and 5000 ms, respectively, yields the following calculation:


(5 * 5000) + 60 = 85 seconds

Thus, the schedule time interval in this example must be set to greater than 85 seconds for scheduling to work properly.

Large Number of Tests on a Router Running Off Schedule

Tests may run off schedule in cases where there are large numbers of tests on a single router. This is caused by the large number of snmp-gets performed to enforce the limit of a number of probes of the router. If tests begin to run off schedule, you can disable enforcement off by setting the EnforceMaxProbes/0x45600dc attribute to FALSE for one or more RTM_Test models with the OneClick Attribute Editor.

Troubleshooting 99

Delayed SpectroSERVER Activation Resulting from External Reads

Delayed SpectroSERVER Activation Resulting from External Reads

For agents that support performance test types via MIB objects, SPECTRUM must perform external reads of these MIB objects at SpectroSERVER activation time to obtain the list of supported test types. This can delay the SpectroSERVER activation time. To remedy delayed activation, you can configure the spm_wait_activate parameter in the .vnmrc file to delay the external reads until the SpectroSERVER has been activated.

Set the default value of spm_wait_activate from No to Yes to prevent SPM from performing external reads until the SpectroSERVER is activated. Note that the agent test host will remain inactive until the SpectroSERVER is activated.


Appendix B: SPM Event Codes

Event Code DescriptionsThe following table lists SPM events by event code, event name, the model types the event can be asserted on, the alarm generated or cleared (if any), and alarm severity.

Event Code Event Name Model Type Asserted On

Alarm Generated or Cleared

Alarm Severity

0x04560000 SPM Result Event RTM_Test None Not Applicable

0x04560001 SPM Timeout Event (See About SPM Timeout Event on page 111 for more information.)

RTM_Test 0x04560001 Yellow

0x04560002 SPM Test Host Configuration Failed Event


0x04560003 SPM Threshold Exceeded Event

RTM_Test, RTM_TestHost, Source, or Destination

None Not Applicable



0x04560003 Yellow



0x04560003 Orange



0x04560003 Red

0x04560007 SPM Threshold Cleared Event


Clears 0x04560003

Not Applicable

SPM Event Codes 101

Event Code Descriptions

0x04560008 SPM Test Creation Event RTM_Test None Not Applicable

0x0456000a SPM Test Modification Event RTM_Test None Not Applicable

0x0456000b SPM Packet Loss Threshold Exceeded Event


None Not Applicable

0x0456000c SPM Packet Loss Exceeded Event


0x0456000b Yellow

0x0456000d SPM Packet Loss Exceeded Event


0x0456000b Orange

0x0456000e SPM Packet Loss Exceeded Event


0x0456000b Red

0x0456000f SPM Packet Loss Cleared Event


Clears 0x0456000b

Not Applicable

0x04560010 SPM Result Event (Jitter) RTM_Test None Not Applicable

0x04560011 SPM Jitter Packet Loss Source to Destination Exceeded Event


None Not Applicable



0x04560011 Yellow



0x04560011 Orange



0x04560011 Red

0x04560015 SPM Jitter Packet Loss Source to Destination Cleared Event


Clears 0x04560011

Not Applicable

0x04560016 SPM Jitter Packet Loss Destination to Source Threshold Exceeded Event


None Not Applicable

0x04560017 SPM Jitter Packet Loss Destination to Source Exceeded Event


0x04560016 Yellow



Alarm Severity





0x04560016 Orange



0x04560016 Red

0x0456001a SPM Jitter Packet Loss Destination to Source Threshold Cleared Event


Clears 0x04560016

Not Applicable

0x0456001b SPM Jitter Packet MIA Threshold Exceeded Event


None Not Applicable

0x0456001c SPM Jitter Packet MIA Threshold Exceeded Event


0x0456001b Yellow

0x0456001d SPM Jitter Packet MIA Threshold Exceeded Event


0x0456001b Orange

0x0456001e SPM Jitter Packet MIA Threshold Exceeded Event


0x0456001b Red

0x0456001f SPM Jitter Packet MIA Threshold Cleared Event


Clears 0x0456001b

Not Applicable

0x04560020 SPM Jitter Packet Late Arrival Threshold Exceeded Event


None Not Applicable



0x04560020 Yellow



0x04560020 Orange



0x04560020 Red

0x04560024 SPM Jitter Packet Late Arrival Threshold Cleared Event


Clears 0x04560020

Not Applicable



Alarm Severity

SPM Event Codes 103


0x04560025 SPM Jitter Busies Threshold Exceeded Event


None Not Applicable



0x04560025 Yellow



0x04560025 Orange



0x04560025 Red

0x04560029 SPM Jitter Busies Threshold Cleared Event


Clears 0x04560025

Not Applicable

0x0456002a SPM Test Pause (Test Host Down) Event

RTM_Test None Not Applicable

0x0456002b SPM Test Restart (Test Host Recontacted) Event


0x0456002c SPM Test Entering Management Mode Event


0x0456002d SPM Test Exiting Management Mode Event


0x0456002e SPM Result Event (HTTP) RTM_Test None Not Applicable

0x0456002f SPM HTTP DNS Resolution Threshold Exceeded Event


None Not Applicable

0x04560030 SPM HTTP DNS Resolution Threshold Exceeded Event


0x0456002f Yellow



0x0456002f Orange



0x0456002f Red

0x04560033 SPM HTTP DNS Resolution Threshold Cleared Event


Clears 0x0456002f

Not Applicable



Alarm Severity



0x04560034 SPM HTTP TCP Connect Threshold Exceeded Event


None Not Applicable



0x04560034 Yellow



0x04560034 Orange



0x04560034 Red

0x04560038 SPM HTTP TCP Connect Threshold Cleared Event


Clears 0x04560034

Not Applicable

0x04560039 SPM HTTP Page Download Threshold Exceeded Event


None Not Applicable

0x0456003a SPM HTTP Page Download Threshold Exceeded Event


0x0456003c Yellow

0x0456003b SPM HTTP Page Download Threshold Exceeded Event


0x0456003c Orange

0x0456003c SPM HTTP Page Download Threshold Exceeded Event


0x0456003c Red

0x0456003d SPM HTTP Page Download Threshold Cleared Event


Clears 0x0456003c

Not Applicable

0x0456003e SPM Result Event (Traceroute)


0x0456003f SPM Traceroute Threshold Exceeded Event


None Not Applicable

0x04560040 SPM Traceroute Threshold Exceeded Event


0x0456003f Yellow



Alarm Severity

SPM Event Codes 105




0x0456003f Orange



0x0456003f Red

0x04560043 SPM Traceroute Threshold Cleared Event


Clears

0x0456003f

Not Applicable

0x04560044 SPM Test SNMP Set Failure Event

RTM_TestHost 0x04560044 Yellow

0x04560045 SPM Test SNMP Set Failure Cleared Event

RTM_TestHost Clears 0x04560044

Not Applicable

0x04560046 SPM Test Timeout Cleared Event

RTM_Test Clears 0x04560001

Not Applicable

0x04560047 SPM Test Host Configuration Failed Cleared Event


Not Applicable

0x04560048 SPM Too Many Probes On Test Host Event


0x04560049 SPM Too Many Probes on Test Host Cleared Event

RTM_TestHost Clears

0x04560048

Not Applicable

0x0456004a SPM HTTP Result Event RTM_Test None Not Applicable

0x04560054 Bad Ping Result RTM_Test None Not Applicable

0x04560055 Bad Jitter Result RTM_Test None Not Applicable

0x04560056 Bad HTTP Result RTM_Test None Not Applicable

0x04560057 Bad Traceroute Result RTM_Test None Not Applicable

0x04560058 Bad HTTP Result RTM_Test None Not Applicable

0x04560059 SPM Test No Longer On Device Event


0x0456005a SPM Test No Longer Running On Device Event


0x0456005b SPM Duplicate Result Event RTM_Test None Not Applicable

0x0456005c SPM Test Discovery Completion Event

RTM_TestHost None Not Applicable



Alarm Severity



0x0456005d SPM Test Type Mismatch Event

RTM_Test 0x0456005d Yellow

0x0456005e SPM Stale Test Cleared Event

RTM_Test Clears 0x04560059 0x0456005d

Not Applicable

0x0456005f SPM Positive Jitter Source to Destination Exceeded Event


None Not Applicable

0x04560060 SPM Positive Jitter Source to Destination Exceeded Event


0x0456005f Yellow



0x0456005f Orange



0x0456005f Red

0x04560063 SPM Positive Jitter Source to Destination Cleared Event


Clears 0x0456005f

Not Applicable

0x04560064 SPM Positive Jitter Destination to Source Exceeded Event


None Not Applicable



0x04560064 Yellow



0x04560064 Orange



0x04560064 Red

0x04560068 SPM Positive Jitter Destination to Source Cleared Event


Clears 0x04560064

Not Applicable

0x04560069 SPM Negative Jitter Source to Destination Exceeded Event


None Not Applicable



Alarm Severity

SPM Event Codes 107


0x0456006a SPM Negative Jitter Source to Destination Exceeded Event


0x04560069 Yellow

0x0456006b SPM Negative Jitter Source to Destination Exceeded Event


0x04560069 Orange

0x0456006c SPM Negative Jitter Source to Destination Exceeded Event


0x04560069 Red

0x0456006d SPM Negative Jitter Source to Destination Cleared Event


Clears 0x04560069

Not Applicable

0x0456006e SPM Negative Jitter Destination to Source Exceeded Event


None Not Applicable

0x0456006f SPM Negative Jitter Destination to Source Exceeded Event


0x0456006e Yellow

0x04560070 SPM Negative Jitter Destination to Source Exceeded Event


0x0456006e Orange

0x04560071 SPM Negative Jitter Destination to Source Exceeded Event


0x0456006e Red

0x04560072 SPM Negative Jitter Destination to Source Cleared Event


Clears 0x0456006e

Not Applicable

0x04560073 SPM Too Many Probes Event RTM_Test None Not Applicable

0x04560074 SPM Bad Community String Event


0x04560075 SPM Invalid Destination Address Event


0x04560076 SPM Invalid Destination Address Cleared Event


Not Applicable

0x04560077 SPM Invalid Test Host Event RTM_Test 0x04560077 Yellow

0x04560078 SPM Invalid Test Type Event RTM_Test 0x04560078 Yellow

0x04560079 SPM RTM_TestHost No Device Model Event




Alarm Severity



0x0456007a SPM RTM_TestHost No Device Model Cleared Event

RTM_TestHost Clears 0x04560079

Not Applicable

0x0456007b SPM Result Failure Event RTM_Test None Not Applicable

0x0456007c SPM Mean Opinion Score (MOS) event


None Not Applicable

0x0456007d SPM Mean Opinion Score (MOS) event


0x456007c Yellow

0x0456007e SPM Mean Opinion Score (MOS) event


0x456007c Orange

0x0456007f SPM Mean Opinion Score (MOS) event


0x456007c Red

0x04560080 SPM Mean Opinion Score (MOS) event


Clears 0x456007c

Not Applicable

0x04560081 Juniper Jitter Result Event RTM_Test, RTM_TestHost, Source, or Destination

None Not Applicable

0x04560082 Bad Juniper Jitter Result RTM_Test, RTM_TestHost, Source, or Destination

None Not Applicable

0x04560083 Juniper RTT Jitter Exceeded Event


None Not Applicable



0x04560083 Yellow



0x04560083 Orange



0x04560083 Red

0x04560087 Juniper RTT Jitter Cleared Event


0x04560083 Not Applicable



Alarm Severity

SPM Event Codes 109


0x04560088 Juniper Egress Jitter Exceeded Event


None Not Applicable



0x04560088 Yellow

0x0456008a Juniper Egress Jitter Exceeded Event


0x04560088 Orange

0x0456008b Juniper Egress Jitter Exceeded Event


0x04560088 Red

0x0456008c Juniper Egress Jitter Cleared Event


Clears

0x04560088

Not Applicable

0x0456008d Juniper Egress Jitter Exceeded Event


None Not Applicable

0x0456008e Juniper Egress Jitter Exceeded Event


0x0456008d Yellow

0x0456008f Juniper Egress Jitter Exceeded Event


0x0456008d Orange



0x0456008d Red

0x04560091 Juniper Egress Jitter Cleared Event


Clears

0x0456008d

Not Applicable



Alarm Severity


About SPM Timeout Event

About SPM Timeout EventSPM determines when it is appropriate to generate SPM Timeout Event and its associated alarm:

SPM Timeout Event and its related alarm is suppressed if the destination IP of the SPM test represents a SPECTRUM device model that does not have ICMP contact. The alarm appears as a symptom of the “DEVICE HAS STOPPED RESPONDING TO POLLS” alarm of the SPECTRUM device model.

SPM Timeout Event and its related alarm is not generated if the SPM Test Host does not have SNMP contact at the time that the RTM results are read for a given SPM test.

SPM Event Codes 111

About SPM Timeout Event


Index

A

Accessing SPM • 8

Action codesused with CLI update command • 92

Activate Test Host icon • 51

Activating test hosts • 51

Administrator user role • 8

Agents and MIBsCA eHealth SystemEdge Service Availability agent • 5Cisco IOS IP SLAs Agent • 5iAgent • 6JUNOS Real Time Performance Monitor • 6Network Harmoni SLAplus Agent • 6, 44RFC2925 • 5

Alarms • 4critical for test threshold violations • 49for test threshold violations • 47major for test threshold violations • 49minor for test threshold violations • 49viewing • 63

Alternate source address for test source • 40

Apply Test Template icon • 22

C

CA eHealth SystemEdge Service Availability agent • 5

Caveats, test scheduling • 98

Cisco IOS IP SLAs Agent • 5

Cisco Ping MIB • 5

CISCO-RTTMON-MIB • 98

Codec typeJitter tests • 40mean opinion scoring • 40

Command Line Interface (CLI) • 73creating tests with • 73editing tests with • 73optional test parameters • 78required test parameters • 76

retrieving test results • 93retrieving test status results • 92running tests with • 73schedule test parameters • 77test action codes • 92test parameters • 76test schedule parameters • 77threshold parameters • 80

Community strings for test hosts • 2

Configuring testschedule parameters • 45

Configuring tests • 39advanced parameters • 44general parameters • 39threshold parameters • 47

Copy (test template) icon • 21

Copy (test) icon • 31

Copyingtests • 20, 30

Create Schedule dialog • 46

Creatingtest schedules • 46test templates • 19tests • 29tests automatically with test templates • 23tests by copying • 20, 30tests with CLI • 73

D

Deactivate Test Host icon • 51

Deactivating test hosts • 51

Delete (test template) icon • 28

Delete (test) icon • 53

Deletingtest templates • 27tests • 53

Destination address for a test • 41

Destination DNS server • 41

Destination port for mean opinion score • 41

Destination URL for HTTP tests • 41

Index 113

DHCP tests • 3, 31

Disable a test • 40

Disabling test thresholds • 48

Discovering tests • 35

Discovery modechanging for test discovery • 37

DNS tests • 3, 31destination server • 41lookup name • 41

E

Edit (test template) icon • 27

Edit Test icon • 53

Editingtest templates • 27tests • 53tests with CLI • 73

eHealth Service Availability module • 5

Enable a test • 40

Enabling test thresholds • 48

Event codes • 101

External test reads, effect on SpectroSERVER activation • 100

Extreme Summit host devices, ping time-outs • 98

F

Filter timeout data • 45

Filter timeout data setting • 61

Firmware issuesCisco IOS 12.0(7)T2 • 96Cisco IOS 12.0(9) • 95Cisco IOS 12.2 • 96Cisco IOS 12.2(11)T • 96Cisco IOS 12.2(18)SXF3 • 97Cisco IOS 12.2(18)SXF4 • 97Cisco IOS 12.2(2)T • 96Cisco IOS 12.3(4) • 96Cisco IOS 12.3(5) • 96, 97Riverstone RS-8000 FW 9.0.0.4 • 97

FTP tests • 4, 32file name • 41password • 41

user name • 41

G

Global Collection • 4

Global Collections • 20

Graphsof test results • 60

H

HTTP tests • 4, 32destination URL • 41HTTP version • 45password • 41proxy URL • 44user name • 41

HTTPS test, Secure HTTP test • 32

I

iAgent • 6

iAgent test hosts, traceroute timeout • 98

ICMP • 3

ICMP Ping tests • 32

J

Jitter tests • 3, 33codec type • 40destination port • 41voice quality testing • 41

JUNOS Real Time Performance Monitor • 6

L

Latency timeout • 40

Logging, test data • 69

Lookup name for DNS tests • 41

M

Maintenance mode for test hosts • 51

Mean opinion score • 94destination port range • 41

MOS, see Mean opinion score


N

Naming convention for tests created by a test template • 23

Network application response time testsFTP • 4HTTP • 4POP3 • 4SMTP • 4TCP • 4UDP • 4

Network Harmoni SLAplus Agent • 6, 44

Network response time testsICMP Ping • 3Jitter • 3traceroute • 3

Network service response time testsDHCP • 3DNS • 3

New (test) icon • 30

O

Operator user role • 8

P

Packet Loss • 59

Packet loss • 3

Packet size used in tests • 44

Performance graph for tests • 61

Performance Tabgraphs of test results • 60viewing test results in • 60

Ping tests • 3alternative source location • 40

POP3 tests • 4, 34password • 41user name • 41

R

Read-only test discovery mode • 35

Read/Write test discovery mode • 36

Reports, test resultsReport Gateway • 69

Report Manager • 65

RFC2925 • 5

RTM_Test model parametersrequired for tests • 76

RTTMONsample count • 44, 59

RTTMON MIB • 5, 98

Running tests on demand • 50

Running tests with CLI • 73

S

Sample Count • 59

Sample count for tests • 44

Schedule test parameters • 77

Scheduling a test • 45

Scheduling testscaveats • 98

Searchesfor SPM components • 13test hosts • 13test templates • 13tests • 13

Secure Sockets Layer (SSL), secure HTTP test • 32

Securing SPM tests • 11

Securing tests hosts • 11

Security strings, overwriting • 11

SMTP tests • 4, 34

Specifying the test host location • 41

SPM searches • 13

SPM searches, see Searches

SPM Template Naming option • 23

T

TCP tests • 4, 34

Template type • 20

Test discovery • 35

Test host location • 41

Test hostsactivating and deactivating • 51component detail view • 57

Index 115

extended path location • 43information view • 56maintenance mode • 51mid-path location • 43read/write community strings • 2searches • 13securing • 11source location • 42specify the location between a source and destination

Ping testslocation of test host • 41

Test parametersactive thresholds schedule • 48alternate packet address • 45alternate packet port • 45alternate source address • 40codec type • 40description • 40destination address • 41destination DNS server • 41destination URL • 41event • 48file name • 41filter timeout data • 45HTTP version • 45latency timeout • 40lookup name • 41optional • 78packet size • 44password • 41proxy URL • 44required • 76sample count • 44schedule • 77state • 40status • 48test host location • 41test name • 40threshold events asserted on • 48threshold types • 48thresholds • 80type of service • 44user name • 41voice test • 41

Test parameters, thresholdsclear cycles • 49clear threshold • 49critical alarm • 49

major alarm • 49minor alarm • 49threshold cycles • 49

Test result attributesused with CLI show command • 93

Test Resultsgraphical representation of • 60retrieving with CLI • 73

Test status result attributesused with CLI show command • 92

Test template parametersglobal collections • 20template type • 20

Test templates • 4applying to select test hosts • 21applying to test hosts in global collection • 23creating • 19deleting • 27editing • 27removing association with a Global Collection • 27searches • 13specify naming convention for tests created from • 23

Test types • 31network application response time • 3network response time • 3network service response time • 3

Testscomponent detail view • 59configuration • 39copying • 20, 30creating • 29default name • 40deleting • 53discovering • 35editing • 53enabling and disabling • 40event codes • 101information view • 58managing with CLI • 73name for • 40performance view • 60running • 50scheduling • 45scheduling caveats • 98searches • 13securing • 11


service level agreements • 7specifying thresholds • 47timeout errors • 97

Thresholds for tests • 47enabling and disabling • 48establishing baseline values • 50number of violations before an event/alarm is generated • 49scheduling when in effect • 48types • 48when event is generated • 48when events/alarms are cleared • 49where violation events are asserted • 48

Timeout errors • 97

Traceroute tests • 3, 33

Transport Layer Security (TLS), secure HTTP test • 32

Troubleshooting • 95

Type of Service (TOS) • 44

U

UDP tests • 4, 34

User roles and rights • 8

V

Violations of test thresholds • 47

Virtual Routing and Forwarding (VRF) • 33, 34

VRF name • 45

Index 117

service performance manager user guide...

Documents