Introduction to Quality Assurance for

Service Provider Network Functions Virtualization

Carsten Rossenhoevel, Managing Director European Advanced Networking Test Center

2015-09-08

Company Introduction

Vendor-neutral test lab based in Berlin Manufacturer Testing and Certification: Performance,

Scalability, Availability, Interoperability Proof of Concept testing and Audits for service providers,

large enterprises and government organizations Focusing on innovative technologies on layers 2 to 7

Unique, state of the art testing expertise for MPLS, SDN, NFV, 4G/LTE/VoLTE

Emulating fully realistic scenarios representative for today’s production networks

EANTC is 100% independent and vendor-neutral

20-year testing experience matches highest quality assurance standards

Industry Differentiation

EANTC is the most innovative independent test lab publicly active in NFV Testing

We are experienced in testing NFV use case applications and use case-agnostic NFV infrastructure

The EANTC team masters latest technologies – on par with vendor experts

We create independent reports and technical white papers with unique, truly validated content

We organize turn-key live showcases, including all the logistics

Future NFV Testing Directions

Time

Read

ines

s for

SP

Depl

oym

ent

2013 2014 2018 2017 2015

Prototype

Field Trial

Deployment

Legacy

ETSI NFV PoCs

Open Source Baseline Testing

2019 2020

EANTC’s Role in the NFV Ecosystem

• Active member • Defining testing standards for performance and

interoperability ETSI NFV ISG

• Following OPNFV, OpenDaylight, OpenStack • Will contribute to OPNFV testing projects

Open Source Projects

• Created New IP Agency Test Program with Light Reading • Intel Network Builders Partner for Testing

Industry Initiatives

• Service Provider Proof of Concept Tests • Verification of NFV functions, performance, reliability • Integration tests/audits for trials and early deployments

Private testing

First Public NFV World Showcase October 2013

EANTC staged the showcase at Layer123’s SDN & OpenFlow World Congress

Focus on instantiation & provisioning, portability and elasticity: Virtualized CPE Environment Virtualized BNG Virtualized Middleware (CDNs,

Cache Servers, Load Balancers, Application accelerators, firewalls)

Virtualized Carrier-Grade Network Address Translation (CG-NAT)

EANTC is a participating member of ETSI's NFV Industry Specification Group (ISG) which defined the use cases chosen for the demonstration

Participants in 2013:

Procera Virtual PacketLogic Software Performance Test Published October 2014

Performance Results Highlights: NFV implementation demonstrates bare metal performance Layer 7 full DPI throughput of 40Gbit/s Average TCP Response Time under 1ms

Verifying ETSI NFV Proof of Concept (PoC) #21 October 2014

PoC members showed the benefits of hardware acceleration in NFV environments for functions that are compute or network intensive

Participants provided various solutions to the gateway function

Tested at EANTC‘s lab in Berlin Results were published and presented live at the SDN &

OpenFlow World Congress 2014

PoC Participants:

NFV Industry Schedule (as disclosed)

Q4/2015 Q4/2016 Q1 Q2 Q3

ETSI NFV ISG Phase 2 Define functional blocks, ref points; focus on interop

OPNFV Brahmaputra

OPNFV C Release

Openstack Liberty

OpenDaylight Beryllium

Standards and open source progress swiftly and in parallel

Many upstream projects are getting aligned

Some vendors in turn base development on OPNFV

ETSI NFV Release 2015

NFV Projects And Ecosystems Intersect

ETSI NFV ISG

OPNFV

OpenDaylight

OpenStack There are more projects. And I am certainly wrong about the exact overlap.

NFV Commercial Solution Environment

ETSI NFV ISG

OPNFV

OpenDaylight

OpenStack

Commercial NFV Infrastructure

Based on OPNFV Commercial NFVI

Based on Proprietary Solutions

Commercial OSS

Solutions

Commercial Orchestration

Solutions

Landscape of NFV Testing

Each open source project does code/system testing OPNFV has more testing projects beyond ETSI NFV ISG

creates testing guidelines

IETF Benchmarking WG creates or coordinates performance testing standards

https://wiki.opnfv.org/testing, as of Sept 8, 2015

Most Important OPNFV Testing Projects

Functest: Basic OPNFV release testing for functionality

Yardstick: Infrastructure verification from the point of view of the VNF

Qtip: Performance benchmark suite for OPNFV platform, bottom-up approach (computing / networking / storage)

VSPERF: Virtual switch testing framework and test cases

Pharos: Distributed lab infrastructure

ETSI NFV Testing Projects

NFV001: Pre-deployment Testing Methodology NFV002: Interoperability Testing Methodology NFV003: Open source components for NFV

(report and awareness creation/human interface) Other ETSI NFV working groups tap into Vswitch performance testing Acceleration hardware implications Public information: Work program – https://portal.etsi.org/tb.aspx?tbid=789&SubTB=789,832,831,801,798,799,802,828 Public drafts - http://docbox.etsi.org/isg/nfv/open/Drafts/

Testing in the ETSI NFV Reference Model

[1] Based on GS NFV 002Document

(Multiple Vendors) Server Hardware optionally with acceleration such as DPDK

VIM: vanilla or enhanced

OPNFV, or proprietary

VNFM(s) generic or

aligned with VNFs

Orchestrator of SP’s Choice

vCPE vFW vDPI vAAA vEPC

OPNFV or other NFVI

Vswitch & network access

Storage

Potentially with individual EMs

ETSI NFV TST001

ETSI NFV TST002

We are here Future

Interoperability Reference Architectures ETSI NFV-TST002

Interoperability Features ETSI NFV-TST002

Identify topics („features“) of interoperability

Define which functional blocks are involved

Summarize all relevant interfaces

Next step: Test suite diving into these topics

Test Methodologies and Coverage

Functionality / Compliance

Interoper-ability

Performance

VNF Lifecycle Management VNF Package / Software Image Management VNF Fault and Performance Management Network Service Lifecycle Management Network Service Fault and Performance Management Virtualized Resource Management NFVI Networking Management (vSwitch) () Infrastructure Resource Management NFV Resilience () NFV Policy Administration NFV Acceleration Management ()

NFV Use Cases

Switching/ Routing

Elements

Broadband Network Gateway

Carrier-Grade NAT

Provider Edge, CPE Routers

Home Routers, Set-top Boxes

Mobile Network Nodes

Packet and Serving

Gateways

HLR/HSS

IMS

Security/ Filtering

IPsec Gateways

Deep Packet Inspection

Firewalls IPv4/ IPv6

IPS/ IDS

Virus Scanners/ Spam Filters

Policy Management

Video, Voice QoE

Session Border Controllers

AAA Servers

Mobile Policy Mgmt (PCRF)

Content Management

Cache Servers

Application-layer Load Balancers

Don’t Forget About NFV Application Testing

NFV Service Integration Validate multi-layer

independence or interoperability or performance

Verify ETSI defined requirements being met

vEPC, vIMS vGI-LAN vCPE vRouter,

vBNG

vFirewall, vContent

Filter

NFV

Inte

grat

ion

Use Case Integration

Orchestration

Management

VNF

Hypervisor

Host Hardware

NFV Use Case Validation Test multiple virtualized

solutions on the same platform(s) across all layers

Verify ETSI defined requirements being met

New IP Agency Interoperability Testing Initiative

Ensuring virtualized network function solutions work together industry-wide Infrastructure (NFVI) VNF Lifecycle Forwarding Graphs Orchestration OSS First Phase starting in Q4/2015

Phase 1 Testing

• VNF to NFVI Interoperability (vertical interop) • Service chains interoperability (horizontal interop) • VNFM to VNF interoperability (if exist) • VIM to NFVI interoperability (if exist)

VNF …N VNF 2

VIM…N VIM B

NFVI …N NFVI B

NFVI A

VNF 1

VIM A

VNFM…N VNFM 2

VNFM 1

Phase 2 Testing Orchestrator focus

Orchestrator to VNFM interoperability

VNF …N VNF 2

VIM…N VIM B

NFVI …N NFVI B

NFVI A

VNF 1

VIM A

VNFM…N VNFM 2

VNFM 1

Orch…n Orch b

Orch a

Thank you for your interest!

For further information, please contact us: EANTC AG Salzufer 14 D-10587 Berlin Germany Phone: +49.30.318 05 95-0 Fax: +49.30.318 05 95-10 E-mail: info@eantc.de www.eantc.de Follow us