opnfv/functest: lessons learned and best practices for vnf test automation
TRANSCRIPT
June 20–23, 2016 | Berlin, Germany
Functest: lessons learned and best practices for VNF test automation
V.Boucher, Orange, M.Richomme Orange
Agenda
• Back to vIMS test case in Brahmaputra
• Lessons learned
• The future of VNF testing in Functest
Back to vIMSTest-case
Back to vIMS test-case
• vIMS Functest test-case in Brahmaputra => Get a first complex VNF test case
• vIMS test-case includes orchestration/VNF/Test
• vIMS test-case fully open source including both VNF and orchestrator
• vIMS fully integrated in CI pipeline
Back to vIMS test-case
vIMS Clearwater
architecture
Back to vIMS test-case
VNF Descriptor (Cloudify Blueprint) :
homestead:type: clearwater.nodes.homesteadproperties:private_domain: clearwater.local
relationships:- type: cloudify.relationships.contained_in
target: homestead_host- type: app_connected_to_bind
target: bind
homestead_host:type: clearwater.nodes.MonitoredServerinstances:deploy: 1
relationships:- target: base_security_group
type: cloudify.openstack.server_connected_to_security_group
- target: homestead_security_grouptype: cloudify.openstack.server_connected_to_security_group
See more : https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/
Back to vIMS test-case
vIMS Testcase step by step
Create user/tenantUpload OS ImagesChoose flavorsGet external network
1 Environment preparation
Back to vIMS test-case
vIMS Testcase step by step
Install orchestrator components
2.2 Orchestrator deployment
Create router/networkCreate security groupsCreate VM
2.1 Orchestrator deployment
Back to vIMS test-case
Create security groupsCreate VMCreate floating ip
3.1 VNF deployment
Install and configure VNF software
3.2 VNF deployment
Back to vIMS test-case
Create numbers
4.1 Test VNFSignaling testing
4.2 Test VNF
Back to vIMS test-case
Push results of deployment duration and test results into result api
5 Push results
DB
RESULT API
Back to vIMS test-case
Reporting status example
Lessons learned&
Best practices
Lessons learned
• Valuable test case to give a high level of confidence:• Creation of user/tenant/networks
• Access to Internet, public APIs
• Use of an orchestrator (Cloudify) to deploy a complex VNF based on Tosca descriptors
• Spawning of more than 5 VMs cross connected
• Association orchestrator/VNF makes sense for functional testing
• Realistic test suite for signaling
Brahmaputra lessons learned
• Success criteria• Orchestration + vIMS deployment
• Tests not part of the criteria due to some issues
• Not possible to run it on all the installers/scenarios• vIMS launched systematically
• Timeout adaptations needed to reduce test duration
• When previous tests failed (e.g. vPing) no need to try vIMS => improvement in Colorado
• Issues detected and reported (e.g. access to public API)
• Multi-tenant/Multi-sites would be also valuable
Future of VNF testing
Future of VNF testing
• Functest test slicing => 1 category dedicated to VNFs
• VNF part of the weekly loop i.e. not used anymore for scenario validation
• Not scenario specific – shall work on all the scenarios….
• Potentially long duration test
Future of VNF testing
• VNFs validation = 1 of the plugfest activity
• Reference/consider Open Source VNFs to enrich VNF category
• Goal develop test suite/ref open source VNFs to • Define ref baselines / harmonize test strategy
• Create a “VNF library”
• Initiate MANO stack testing
Future of VNF testing
Proprietary VNF
• Cannot be integrated to CI (license)
• Can be tested in Pharos Community labs
• Can be tested during Plugfests
• Open source test suite even towards proprietary VNFs should be encouraged
Thank you