NFV-SDN Opensource

MP Odini/HPE – ETSI NFV Vice Chair

Dec, 2015

F

Many Opensource projects on NFV-SDN …

2

SDN Controller

VIM

DPDK

NFVI

NFVO

Non exhaustive list …

Federator

used as part of testing

MANO OpenSource InitiativesOpenSource Link Status

NEW !!

ETSI OSG ‘OSM’ : OpenSource

MANO

http://osm.etsi.org new

NEW – OPNFV expanding scope www.opnfv.org Expending Scope to MANO stack

Tacker (OpenStack) – Apache2 https://wiki.openstack.org/wiki/Tacker Renaming of an opensource project called

ServiceVM (spin-off from Neutron)

Openmano (Telefonica) - Apache2 https://github.com/nfvlabs/openmano Launched beginning 2015 – issued from

Telefonica test team

OpenBaton (Fraunhofer) http://openbaton.github.io/ Oct 2015 – issued from OpenSDN FK

project

Cloudify http://getcloudify.org/ support from Orange and Metaswitch

Open-O (China Mobile) (not available yet) Presented at OPNFV Summit 2015

Gohan (NTT Data) – Apache2 https://github.com/morika-t/gohan Presented at OpenStack Summit Tokyo

TCS Telco Cloud (Tata) https://github.com/TCS-

TelcoCloud/OpenVNFManager

Backup

MANO OpenSource

4

Overview

MANO opensource has emerged this past 1-2 years from 3 main sources :

• Cloud Opensource projects : ie Tacker in OpenStack, coming from neutron-ServiceVM project

• Operator labs => who opensourced their orchestration/test tool : Telefonica openmano, NTT Gohan, China Mobile Open-O

• New small projects => generally led by vendors, small community, or University : ie Cloudify, OpenBaton,

ETSI OSG ‘OSM’ : Open Source MANO

6

Jan’16: New proposal led by Telefonica , currently being reviewed by ETSI Board, and submitted to ETSI Director General Approval

Scope: Develop an opensource reference implementation of ETSI MANO stack

Quote: “This ISG defines a framework to facilitate the

software development of a reference implementation of

ETSI MANO. ISG OSM complements the work of ISG

NFV. The code (reference implementation) developed by

ISG OSM shall be non-normative. Any potential

standardization work identified by ISG OSM will be

submitted to ISG NFV for further consideration and

eventual inclusion in the ISG NFV Group Specifications.”

BT PLC

Intel Corporation (UK) Ltd

Telefonica S.A.

Telekom Austria AG

Telenor ASA

RIFT.io Inc

Canonical Group Ltd (applicant

member)

Mirantis (applicant member)

Founding members

OPNFVChairman: Prodip Sen, HPE

7

The scope of projects chosen by the OPNFV TSC can focus on any of the ETSI NFV architecture functional blocks and reference points, along with supporting tools and infrastructure.

These functional blocks include:* Virtualised Infrastructure Manager (VIM)* Network Functions Virtualisation Infrastructure (NFVI)* Virtual Network Function Managers* Orchestrator* Virtual Network Functions and associated Element

Management Systems* Service, VNF, and Infrastructure DescriptionOperations Support System / Business Support System

The associated reference points are (ETSI NFV):

* Vi-Ha: The reference point between hardware resourcesand the Virtualisation Layer

* Vn-Nf: The reference point between NFVI and a Virtualised Network Function (VNF)

* Or-VNFM: The reference point between the Orchestrator and VNF Manager

* Vi-Vnfm: The reference point between a VNF Manager (VNFM) and the VIM

* Or-Vi: The reference point between the Orchestrator and the VIM

* Nf-Vi: The reference point between the VIM and the NFVI* Os-Ma: The reference point between the OSS/BSS

and Management and Orchestration* Ve-Vnfm: The reference point between the VNF/EMS

and VNF Manager* Se-MA: The reference point between the Serice, VNF,

and Infrastructure Description and the NFV Management and Orchestration

NEW Jan’16: OPNFV expanding its scope to all ETSI NFV Functional Blocks, incl MANO

Tacker

8

VNFM

1.VNF Catalog

2.Basic life-cycle of VNF (define/start/stop/undefine)

3.Performance and Health monitoring of deployed VNFs

4.Auto Healing VNFs based on Policy

5.Facilitate initial configuration of VNF

NFVO

1.Templatized end-to-end Network Service deployment

using decomposed VNFs

2.VNF placement policy – ensure efficient placement of

VNFs

3.VNFs connected using a SFC - described in a VNF

Forwarding Graph Descriptor

4.VIM Resource Checks and Resource Allocation

5.Ability to orchestrate VNFs across Multiple VIMs

Tacker uses TOSCA for VNF meda-data definition

https://wiki.openstack.org/wiki/Tacker

http://git.openstack.org/cgit/openstack/tacker/

Data Model initial draft from June 2014 (??old)https://docs.google.com/document/d/1xs8TvEVMszzND5uoWTHtd1tJnu7105Ekgq9hxiyXABQ/edit?pref=2&pli=1

9

This project introduces a new service to

organize and manage devices

capable of hosting cloud services. Such

devices may take a virtual

form factor (like virtual machines or

containers) or physical form`

factors like hardware devices. The

responsibility of the new services

is to:

1.act as a data store for such devices

2.act as a data store for physical topology

information

3.offer life cycle management for

compatible device types

4.offer capacity allocation control in such

devices

class DeviceTemplate(model_base.BASE, models_v1.HasId, models_v1.HasTenant):

name = sa.Column(sa.String(255))

description = sa.Column(sa.String(255))

infra_driver = sa.Column(sa.String(255))

attributes = orm.relationship('DeviceTemplateAttribute',

backref='template')

class DeviceTemplateAttribute(model_base.BASE, models_v1.HasId):

template_id = sa.Column(sa.String(36),

sa.ForeignKey('devicetemplates.id'),nullable=False)

key = sa.Column(sa.String(255), nullable=False)

value = sa.Column(sa.String(255), nullable=True)

class Device(model_base.BASE, models_v1.HasId, models_v1.HasTenant):

template_id = sa.Column(sa.String(36), sa.ForeignKey('devicetemplates.id'))

template = orm.relationship('DeviceTemplate')

instance_id = sa.Column(sa.String(255), nullable=True)

mgmt_url = sa.Column(sa.String(255), nullable=True)

status = sa.Column(sa.String(255), nullable=False)

flavor = sa.Column(sa.String(255))

class DeviceArg(model_base.BASE, models_v1.HasId):

device_id = sa.Column(sa.String(36), sa.ForeignKey('devices.id'), nullable=False)

device = orm.relationship('Device', backref='kwargs')

key = sa.Column(sa.String(255), nullable=False)

value = sa.Column(sa.String(4096), nullable=True)

TOSCA – latest specs in progress

10

Current latest (as of Oct 2015) is: https://www.oasis-open.org/committees/download.php/56577/tosca-nfv-v1.0-wd02-rev03.doc

Tacker sample VNFD template (YAML)

11

Tacker presentations

12

https://www.openstack.org/summit/vancouver-

2015/summit-videos/presentation/tacker-virtual-network-

function-life-cycle-management-for-openstack

https://www.openstack.org/summit/tokyo-

2015/videos/presentation/brocade-nfv-orchestration-

demo-with-openstack-tacker

https://github.com/nfvlabs/openmano

OpenMANO follows an NFVO-centric approach, with a simplified VNF instance

lifecycle management at the NFVO (VNF instantiation and termination)

Openmano – released by Telefonica lab

• FRIENDLY FOR NETWORK ENGINEERS

• NETWORK SCENARIOS & SNAPSHOTS

• Provides NFVO & VIM (+ GUI and CLI)

• FULL SUPPORT OF NFV PER001 GS

• EPA-awareness for I/O-intensive

workloads

• REST-BASED APIs, OpenStack-friendly

• OpenStack support also coming!

• MULTI-VENDOR by design

• No formal integration needed

• Assures optimal VNF deployment and IXC

• >30 VNFs tested

• OpenMANO code @ GitHub:o Python-based

o 40k code lines

o Released under Apache 2 license

o 30 forks

OpenMANO‘s main features

OpenMANO automates network creation process

SCENARIOS & SNAPSHOTS MAKE NETWORK DESIGN SIMPLE

AVOIDS CLASH WITH LEGACY OSS (PURE RESOURCE

ORCHESTRATION)

EPA SUPPORT (NFVO & VIM)

PROPER VARIETY OF L2 CONNECTIVITIES

VNF ON-BOARDING DOES NOT REQUIRE FORMAL INTEGRATION

• Hide complexity to network engineer

• Scenario snapshots allow

Saving

Cloning

Re-deployment

• Allows to deploy for high performance

• Passthrough, SR-IOV, Virtio

• E-Line & E-LAN

• Descriptor-based (openly available)

VNF VNF

VNF VNF

VNF VNF

E-Line E-LAN

VNF

VNFVNF

VNFVNF

VM

VM VM

VM

VM

VMVM

VM VMVNF

VNF

VNF

VM

VM VM

VM

VM

VMVM

VM VM

NS

(NETWORK

SCENARIO)

VNF

(SW-BASED NODE)

+

-

Ab

stra

ctio

n

VM

(DEPLOYMENT UNIT)

OpenMANO works with abstractions of VNFsand Network Scenarios via descriptors...

x100

Line rate with 192

bytes frame size

x100

With the right exposure of HW resources to the VNFs (as described at

NFV PER001 GS), carrier-grade performance can be achieved

Enhanced Platform Awareness (EPA) natively to the VNF when required…

Support of L2 networks with passthrough, SR-

IOV or virtio interfaces:

• E-Line

• E-LAN

Traditional E-LAN based on virtual

bridges/switches is still supported

VNF VNF

VNF VNF

VNF VNF

comprehensive set of connectivities available

hiding complexity to network engineers

1 2New VNF: virtual Router New scenario

3 Deployment 4 Test with traffic

vRouter C

VNFD- Deployment

requirementsNSD

VM Image(s)

OpenMANO demo https://youtu.be/5Szc-VGDhi4

Openbaton

21

•the Network Function Virtualisation

Orchestrator (NFVO), always required

for creating a composition of different

Virtual Network Function Descriptors

generic-VNFM, needed only when the

VNFP approach is used. Using the

generic-VNFM implies also the usage

of the EMS which is automatically

installed on the VDUs where VNF are

to be installed.

openbaton

– OpenBaton is easily extensible. It integrates with OpenStack, and provides a plugin mechanism for supporting additional VIM types. It supports Network Service management either using a generic VNFM or interoperating with VNF-specific VNFM. It uses different mechanisms (REST or PUB/SUB) for interoperating with the VNFMs. Its initial focus was to provide the main functionalities for provisioning and managing Network Services, however in its future releases new additional features will provide mechanisms for increasing the automation in NS management. Those new features will include autoscaling, fault management, TOSCA, etc.

22

Openbaton code

23

files language blank comment code

159 Javascript 5958 5059 26409

348 Java 5732 8782 22917

71 HTML 5670 460 14484

23 CSS 627 318 11789

27 JSON 12 0 1791

18 LESS 102 58 1640

15 SASS 36 26 1399

66 Groovy 276 665 1055

6 Bourne Shell 85 28 394

2 Bourne Again Shell 49 35 248

1 XML 50 0 108

1 DOS Batch 24 2 64

2 PHP 7 3 45

3 YAML 1 0 31

1 Mustache 0 0 4

1 SQL 2 4 2

Total 18631 15440 82380

The LOC is about

83K, the core Java

code is about 23K

Cloudify

24

Cloudify is an open source TOSCA based cloud orchestration software platform (like Amazon’s OpsWorks).

It automates the process of installation, deployment and also post-deployment such as monitoring,

remediation, and auto-scaling of your application stack.

Cloudify

25

Application Orchestration

Your application in its entirety (Infrastructure, Middleware,

Application Code, Scripts, Tool Configuration, Metrics and Logs)

can be desribed in what we call a blueprint.

Ex: OpenStack blueprint => https://github.com/cloudify-

cosmo/cloudify-nodecellar-example/blob/master/openstack-

blueprint.yaml

Written in a human readable YAML format, a blueprint allows for

potentially high granularity of configuration of your application.

By defining the complete lifecycle of each part of your

application in a blueprint, and by utilizing the different IaaS APIs

and the plugin abstractions of different tools, Cloudify can

deploy and manage your application automatically.

Cloudify will launch the compute instances, and configure

network and security in order to manifest your infrastructure.

Then, it will execute scripts (remotely via SSH or locally on the

machines) or configuration management tools to configure your

servers and deploy your middleware and code.

Application Maintenance

Cloudify’s ability to run custom workflows will grant

you the ability to manually or automatically change

your application’s structure, deploy code to your

servers, heal or scale your system.

Cloudify will use metrics collectors (and soon, log

collectors) to stream application (and Cloudify

specific) data to Cloudify so that you’re able to

monitor and analyze your system.

Data aggregation and visualization within Cloudify

will allow you to execute the different workflows so

that either you or Cloudify itself can make smart,

actionable decisions based on business/application

KPIs.

Those decisions can either manually or

automatically trigger workflows (such as scaling or

healing) on the tactical front; or as application

behavior analysis on the strategic front.

Cloudify

26

Orange Cloudify project in opensource with Clearwater vIMS

27

https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/

Clearwater vIMS

28

TOSCA like YAML description

29

Gohan – NTT opensource SDN/NFV Orchestration

30

https://www.openstack.org/summit/tokyo-2015/videos/presentation/gohan-an-open-source-service-development-engine-for-sdnnfv-orchestration

Gohan architecture

31

Gohan Policy

32

Policy

You can configure API access policy using this resource.

Policy has following properties.

•id : Identitfy of the policy

•principal : Keystone Role

•action: one of create, read, update, delete for CRUD

operations on resource or any custom actions defined by

schema performed on a resource or * for all actions

•effect : Allow api access or not

•resource : target resource you can specify target

resource using “path” and “properties”

•condition : addtional condition (see below)

•tenant_id : regexp matching the tenant, defaults to

TATA TCS Open VNF Manager

33

Tata TCS Open VNF Manager - github

34

https://github.com/TCS-TelcoCloud/OpenVNFManager

Basic VNF managerOn top of OpenStackUsing Heat

8 commitsLast Aug 2014

YangForge

YangForge provides runtime JavaScript execution based on YANG schema modeling language – IETF (RFC 6020).

Basically, the framework enables YANG schema language to becomea programming language.

It utilizes YAML with custom tags to construct a portable module with embedded code. It is written primarily using CoffeeScript and runs on Node.js and the web browser (yes, it's isomorphic).

35

What is it?

• A Model-driven Architecture (MDA) software framework for building schema defined software modules

Why was it created?

• To serve as a rapid prototyping framework for model driven software architecture

• To support OPNFV Promise implementation for abstracting ResourceElement(s) of the virtualized infrastructure for state management of capacity and reservations for future usage

How does it work?

• It uses Metaprogramming techniques to process YANG schema (RFC 6020) files as a software language

• It uses Creational Patterns to dynamically construct hierarchy of runtime class objects based on YANG data models

What’s the current status?

• It’s early stage with working compilers and generators

Stormforge

stormforge framework

37

Schema

• provides definitions for generating the data models

Model/View/Controller

• decouples logic layers

Data Store

• handles record instances of data models

Events/Messages

• enable reactive control logic integration

Interface Layer

• auto-generates data model derived interfaces

Import/Export

• facilitates exchange of data models

stormforge core components

38

meta-class

• The core base class that houses the meta-data and enables Class definitions to be mutable

yang-compiler

• Parses YANG schema text and produces runtime JS class object hierarchy (composed of meta-class)

• Implemented based on YANG version 1.0 specifications (RFC 6020)

• Provides compile, generate, import, and export facilities (can be used as a server-side API service)

storm-compiler

• Extensions beyond YANG version 1.0 to introduce complex-types (RFC 6095)

• Enhanced resolvers to infuse with data-storm objects providing type validators and model relationships

yang-storm

• Defines a collection of common data models

• Serves as a DataStore for housing schemas, modules, models, controllers, views, and instantiated records

stormforge compiler features

39

Parser parse YANG schema files and generate runtime JavaScript

meta-class semantic tree hierarchy

Map/Reduce traversal of the parser output to dynamically resolve YANG

statement extensions and transform nodes in the tree as well

as collapsing them into a final output module

Import/Export capabilities to load modules using customizable importers

based on regular expressions and custom import routines.

Ability to serialize module meta data into JSON format that is

portable across systems. Also exports serialized JS functions

as part of export meta data.

Runtime

Generation

allows compiler to directly create a live JS class object

definition so that it can be instantiated via new keyword and

used immediately

Dynamic

Extensions

enable compiler to be configured with alternative resolver

functions to change the behavior of produced output

stormforge usage examples

40

Prototyping a new module

1. Write a new YANG schema definition file, such as hello-world.yang

2. stormforge run -p 5000 hello-world

3. curl http://localhost:5000

Validating an external module

• stormforge validate http://intercloud.net/hello-world.yang

• stormforge validate http://intercloud.net/hello-world.json

• stormforge run -p 5000 http://intercloud.net/hello-world.json

Building/Signing/Publishing a new module

1. stormforge build hello-world

2. stormforge sign -k <keyfile> hello-world

3. stormforge publish --registry=<url> hello-world

The steps above can apply for external sources as well

TOSCA vs Yang

41

Defining TOSCA

TOSCA is Topology and Orchestration Specification for

Cloud Applications. TOSCA’s mission is to facilitate the

creation cloud applications and services. TOSCA

provides mechanisms to control workflow, describe

relationships and reflect dependencies between

resources.

Defining YANG

YANG is a data modeling language used to describe

configuration and state information. YANG has been used

to model networking devices and services – i.e., an object

and its attributes. YANG defines the data models that are

then manipulated through the NETCONF protocol.

YANG is a modeling "language." YANG provides a way for describing "what a

service is." However, YANG is not an orchestration language. YANG does NOT

provide a mechanism for describing "what to do." In other words, YANG is not

effective for describing how a service should be implemented, the topology of

underlying resources, or how resources may be related or dependent upon

each other.

XML, Yang, HOT, TOSCA

XML, YAML and Yang = languages

XML: Extensible Markup Language (XML) is a simple, very flexible text format

http://www.w3.org/XML/Core/#Publications

YAML: “YAML Ain't Markup Language” – inspired by XML but simplified, more readable - data-oriented, rather than document markup.

http://www.yaml.org

Ex; Sample VNFD Yaml (Tacker example): https://github.com/openstack/tacker/blob/master/devstack/samples/sample-vnfd.yaml

YANG: data modeling language used to model configuration and state data manipulated by the Network Configuration Protocol(IETF)

https://tools.ietf.org/html/rfc6020

• XML

Templates: HOT and TOSCA

HOT: Heat Orchestration Template

• Declarative

• YAML

TOSCA : Topology & Orchestration Standard for Cloud Application (OASIS)

TOSCA NFV is refining TOSCA for NFV:

https://www.oasis-open.org/committees/download.php/56577/tosca-nfv-v1.0-wd02-rev03.doc

• Declarative & Imperative

• XML and now YAML

Ex: Sample TOSCA template in YAML (Cloudify example)

https://github.com/Orange-OpenSource/opnfv-cloudify-clearwater/blob/master/openstack-blueprint.yaml

42http://fr.slideshare.net/openstackil/heat-tosca

TOSCA principles

43

TOSCA is made of following metadata: - TOSCA Templates: Node, relationship (between node) and Topology- Node Types with Capabilities and requirements- Relationship Types with interfaces and properties- Policies

A TOSCA Definitions document MUST define at least one of the elements :ServiceTemplate, NodeType, NodeTypeImplementation, RelationshipType, RelationshipTypeImplementation, RequirementType, CapabilityType, ArtifactType, ArtifactTemplate, PolicyType, or PolicyTemplate

Simple example of a Node ‘Compute’ Template

TOSCA data model for a VNF

44

TOSCA for NFV is a new initiative in OASISThat is refining the TOSCA model to map to ETSI NFV model

Baseline:TOSCA-Simple-Profile-YAMLTOSCA Simple Profile in YAML Version 1.0

ETSI NFV VNF model: (source MANO)

TOSCA VNFD Template

45

Extract/simplified view of the Template:

tosca_definitions _version: tosca_simple_profile_for_nfv_1_0_0topology_template: inputs: subsititution_mappings:

node_type: tosca.nodes.nfv.VNF.VNF2requirements:

virtualLink1: [CP21, virtualLink]capabilities:

forwarder1: [CP21, Forwarder]

VNF2 above has : 3 VDU (VDU1,2,3) = VNFCs4 CP : 3 (CP22,23,24) for internal VL (inter-VNFV communication)and 1-CP21 to external VL2

CP21: #endpoints of VNF2

type: tosca.nodes.nfv.CP

properties:

type:

requirements:

virtualbinding: VDU1

capabilities:

Forwarder

node_templates:

VDU1:

type: tosca.nodes.nfv.VDU

properties:

requirements:

- host:

node_filter:

capabilities:

# for selecting “host” (Container Capability)

- host

properties:

- num_cpus: { in_range: [ 1, 4 ] }

- mem_size: { greater_or_equal: 2 GB }

# for selecting “os” (OperatingSystem Capability)

- os:

properties:

- architecture: { equal: x86_64 }

- type: linux

- distribution: ubuntu Interfaces:

# omitted here for brevity

artifacts:

VM_image:vdu1.image #the VM image of VDU1

Interface:

Standard:

create:vdu1_install.sh

configure:

implementation: vdu1_configure.sh

internal_VL

type: tosca.nodes.nfv.VL.ELAN

Same for VDU1,2,3

CP22 type: tosca.nodes.nfv.CPproperties:

type: requirements:

virtualbinding: VDU1virtualLink: internal_VL

Same for CP23,24 But with VDU2

Parsers & Yang

46

Parser URL Comment

Yang to TOSCA (? exists somewhere)

TOSCA to Yang Cloudify plug in at least , I think it is also in OpenStack (tbc)

TOSCA to HOT (Heat)

(OPNFV Parser)

https://github.com/op

enstack/heat-

translator

Contributed by HPE to OpenStack/OPNFV Parser - Heat-Translator

project takes a non-Heat template (e.g. TOSCA flat YAML template or template embedded in

TOSCA Cloud Service Archive (CSAR) format) as an input, calls an appropriate Parser (e.g.

TOSCA Parser) per the type of input template to parse it and create an in-memory graph, maps it

to Heat resources and then produces a Heat Orchestration Template (HOT) as an output.

YangForge (OPNFV

Promise)

https://github.com/op

nfv/yangforge

YangForge provides runtime JavaScript execution based on

YANG schema modeling language – IETF (RFC 6020).

StormForge http://github.com/stor

mstack/stormforge

Contributed by ClearPath – used in promise to model VIM NBI in

Yang

Yang plug-in for HPE

HPSA (HP only)

https://gitlab.ch.hpec

orp.net/hpsa/pluginN

etconfYang

Developed by HPE Portugal

Some high level conclusions

Fragmented …many opensource pop up with lots of overlap and no dominant

Few committers in each, less than 10, less than 5 really active in general but this is not unusual for a small project, still new topic

All reference ETSI NFV and MANO (which was still high level though)

Most if not all use TOSCA, (check if any use Yang?)

Most if not all use Apache 2 licence

Most if not all on github

Most if not all use Python

Most if not all use YAML

All support OpenStack

47

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank you