automated testing of network service interactions contributions to automated testing of network...
Post on 08-Jan-2018
217 Views
Preview:
DESCRIPTION
TRANSCRIPT
Automated Testing of Network Service Interactions
Contributions to Automated Testing of Network Service
Interactions
Rudy Deca1 ,Omar Mahrez1, Omar Cherkaoui1, Yvon Savaria2 and
Doug Slone3
Notere 2005, Ottawa/Gatineau, Canada
1University of Québec at Montréal, 2Ecole Polytechnique de Montréal, 3Cisco Systems, Inc.
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
2
Contents
The problem context The Meta-CLI Model The ScriptMaker tool Conclusion
3
The Problem context
Connectionless statelessnetworks
Connection-oriented
circuit-switching stateful network services
Hybrid, contradictory
network service characteristics
Network convergence
NewSolutionsrequiredMore services
more features more parameters
More combinationsmore interactions
more configurationsmore testsrequired
More environment
states
Increase in the number of states
Causes: Effects:
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
4
Contents
The problem context The Meta-CLI Model
General presentation Structure• Operations
Abstracting the environment The ScriptMaker tool Conclusion
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
5
The Meta-CLI Model solution Abstracts:
the network service and device configuration states
the environment states the dynamic changes of the connectionless,
stateless IP protocols Benefits:
Automated adaptation to environment changes Efficient generation and modification of
configurations, test scripts and tests for service interactions.
6
Modelling a service configuration as a conceptual tree structure
Arcs: Hierarchical
relationships Inclusion relationships
Nodes: Configuration command modes Configuration commands Configuration parameters
Meta-CLI modellingCreated
Modified
device_1
service_1
service_1
Command mode
M
Command P
Command Q
Parameter R
Parameter S
Command mode
NConfigurationcommands and
parameters
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
7
Contents
The problem context The Meta-CLI Model
General presentation • Structure Operations
Configuration Validation
Abstracting the environment The ScriptMaker tool Conclusion
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
8
Meta-CLI Model Operations Operation types:
Configuration Validation
Distribution: Single device Multiple devices of a network
Operation attributes: Atomicity Order (causal, sequential) Granularity (varying from parameters to
services)
Automated Testing of Network Service Interactions
9
Configuring services using the Meta-CLI Model The main idea of the Meta-CLI configuration: compose thedevice and network service configuration models.
Deviceconfiguration
record
Networkservice
Conversion
Con
vers
ion
Meta-CLI Model
Capsule
Construct
Com
posi
tion
Configuration Construct + RetroversionDevice
configurationrecord +
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
10
Generic service configuration operations Instantiate service configurations on devices. Instantiate service configuration parameter values. Instantiate service configuration constraints. Adapt service configuration to device and interface
environment. Adapt service configuration to device and interface
role. Determine environment support for various feature
combinations Determine compatibility of services and features. Determine environment support for service and
feature combinations.
11
Examples of validation dependencies
Validation dependencies between: configuration commands: T depends on Q configuration parameters: S and U depend on R
Dep.U-R
device_1
Command P
Value1
Parameter R
Value3
Parameter U
Value4
Command Q
Value2
device_2
Command P
Value5
Parameter S
Value7
Parameter V
Value8
Command T
Value6
Dependency S-R
Dependency T-Q
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
12
Network service and feature interactions
Mutual exclusion, due to: Competition for resources Accessing the same parameters/variables Incompatible goals
Influence, due to : Scale factor Performance factor
Service combination constraints: Common supporting environment Existence of mandatory parameters
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
13
Contents
The problem context The Meta-CLI Model
• General presentation Abstracting the environment
The ScriptMaker tool Conclusion
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
14
Environment heterogeneity solutionThe configuration environment: network-level component
network topology, technology, Protocol, Role.
equipment-level component. hardware components
device type, interface type, medium type, etc.
software components operating system, software version, etc.
15
Adapt CLI commands to different environments
Command P is instantiated on device_1with one parameter (R) and on device_2 with two parameters (R,S).
Command P
Parameter R
Parameter S
service_1
env1, env2
Command P
env1, env2
Paramet. R
env1, env2
Paramet. S
env2
device_1 + service_1 device_2 + service_1
device_1
env1
Command P
Parameter R
device_2
env2
16
Adapt CLI commands to different device and interface roles
Command P is instantiated: on device_1 with parameter R and on device_2 with parameter S, based on the different roles assumed
by the two devices.
Command P
Parameter S
service_1
role1, role2
Command P
role1, role2
Paramet. R
role1
Paramet. S
role2
device_1 + service_1
device_1
role1
Command P
Parameter R
device_2
role2
device_2 + service_1
17
Addition of a network service to a device configuration
The generic network service model is accessed from the repository.
A service instance model is generated and adapted to the given environment
and then composed with the device configuration model.
Modification of a network service in a device configuration
The network service is already configured on an equipment when an environment change occurs.
The network service model instance in the old environment is extracted from the device configuration model.
The generic network service model is accessed from the repository.
A new instance model is generated for the new environment and composed with the device configuration model.
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
19
Determining environment support for various feature combinations
The information is obtained by querying the tree model.
Service features
service_1 P P P P
Feature R NP P NP P
Feature S NP NP P P
Environment instances
env1 S S NS NS
env2 S S S S
service_1
env1, env2
Command P
env1, env2
Feature R
env1, env2
Feature S
env2 P = is presentNP = is not present
S = supportsNS = does not support
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
20
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
Functionality• Architecture• Generation scenario• Conversion scenario
Conclusion
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
21
Functionality of the ScriptMaker tool
Aimed to regression testing. Provides an effective, automated and easy way to
generate/modify testing scripts with test cases featuring various combinations of network services and features.
Solves the problem of the existence of a large number of : combinations among existing network services; features and parameters of the services; environment states in which the service parameters are
configured differently. Uses a modular approach for components:
Service configurations Traffic Routing, topology and connection among router interfaces Configurations validation
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
22
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
• Functionality Architecture• Generation scenario• Conversion scenario
Conclusion
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
23
Architecture of the ScriptMaker tool
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
24
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
• Functionality• Architecture Generation scenario• Conversion scenario
Conclusion
Automated Testing of Network Service Interactions
25
The script generation scenario
Specific CLI servicecode
Genericservice
capsule stubs
Envi-Ron-ment
Selected Specific TC, TS,
Templates
Selected Generic TC, TS,
Templates
Signaling
OK
Environment (descriptor)
Feature name(s)
Script
Environment (descr.)Feature name(s)
OK
Script
Script Manager Block
Test SelectionBlock
Test Filling Block Test Binding Block
Meta-CLI Model Block
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
26
The test selection block
Selected TC, TS,
Template names
Featurename(s)
Env.desc.
Envi-ron-
ment
Feature name(s) Constr
Constraints
Selected TC, TS, Templates
TC, TS, Template
names
Feature name(s)
Test Selector
Test Organizer
Spec Archive
Test Bench
Environment Constraint Database
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
27
The test filling block
Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology
commands
Test Filler
Converter
InstantiatorCode Generator
Selected Gen CLI auxiliary, validation commands and TCL
Traffic, Topology commands
Selected generic stubs Kx, Kv, Trf, Top
Selected Gen stubs Kx, Kv, Trf, Top
Selected Gen CLI auxiliary, validation commands and TCL
Traffic, Topology commands
Environment
Selected Spec CLI auxiliary, validation commands and TCL Traffic, Topology commands
Ks = Service capsuleKv = Validation capsuleKx = Auxiliary capsule
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
28
The Meta-CLI Model block
Meta-CLI Model manager
Service Instantiator
Service Retriever
Selected genericservice capsule
Selected generic service stubs Ks
Selected generic service capsule Environment descriptor
Selected specific
CLI service
Capsules (models)
Selected generic service capsule
Selected generic
service stubs
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
29
Contents
The problem context The Meta-CLI Model The ScriptMaker tool
• Functionality• Architecture• Generation scenario Conversion scenario
Conclusion
Automated Testing of Network Service Interactions
30
Envi-Ron-ment
The script conversion scenario
selected servicesauxiliariesvalidationstopologies
traffics
Generic TC, TS, Template
servicesauxiliariesvalidationstopologies
traffics
service
Filled Selected Specific TC, TS,
Template
Selected Generic TC, TS,
Template
Signaling
Old script
Environment
Feature name(s)
Script
Environment
Feature name(s)
Script
New script
Script Manager Block
Test Selection
Block
Test Filling Block
Test Binding Block
Meta-CLI Model Block
ParsingBlock
Specific CLI servicecode
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
31
Conclusion Problem:
New Internet services, hybrid, with numerous: parameters features interactions heterogeneous environments.
FSMs not always useful. Solutions:
The Meta-CLI Model: Generic service configuration Environment-aware
The ScriptMaker tool: Automated generation/modification of testing scripts; Modular design; Uses the Meta-CLI for abstracting and reusing service
configurations.
Automated Testing of Network Service Interactions
R. Deca , O. Mahrez, O. Cherkaoui, Y. Savaria and D. Slone
32
Questions
top related