cloud-based nokia sbc performance testing and function ... · interface (webui) and group of sbcs...

Cloud-based Nokia SBC

Performance Testing and Function Validation

September 2017

DR170831E

Miercom.com

www.miercom.com

Cloud-based Nokia SBC Performance Verified 2 DR170831E

Copyright ©2017 Miercom 25 September 2017

Contents

1.0 Executive Summary ............................................................................................................................... 3

Test Summary ............................................................................................................................................. 4

2.0 Product Overview ................................................................................................................................... 5

3.0 How We Did It ......................................................................................................................................... 9

4.0 Performance Testing ........................................................................................................................... 15

4.1 Signaling performance ................................................................................................................... 15

4.2 Media performance ......................................................................................................................... 16

4.3 Performance Under Attack ........................................................................................................... 17

5.0 Security Testing .................................................................................................................................... 20

5.1 Nessus Vulnerability Scan ............................................................................................................. 20

5.2 Codenomicon .................................................................................................................................... 21

6.0 Functional Testing................................................................................................................................ 23

6.1 SBC Resilience & High Availability............................................................................................. 23

6.2 Support of Ultra-HD Voice EVS codec ..................................................................................... 25

6.3 SIP Header Manipulation .............................................................................................................. 25

6.4 WebRTC ............................................................................................................................................... 26

7.0 Management and Orchestration Testing .................................................................................... 29

7.1 SBC VNF Cloud Orchestration ..................................................................................................... 29

7.2 SBC Operations, Administration and Management ............................................................ 31

7.3 SBCs Cluster Management with NetAct .................................................................................. 38

About Miercom ............................................................................................................................................ 41

Customer Use and Evaluation ................................................................................................................ 41

Use of This Report ...................................................................................................................................... 41

Appendix ...................................................................................................................................................... A-1



1.0 Executive Summary

Nokia commissioned Miercom to perform an independent testing of the Cloud-based Nokia

SBC and validate the product’s performance, security, functionalities and management.

The Nokia SBC is a field-proven virtualized session border controller software designed to

control and secure the signaling and media stream of Communication Service Provider’s (CSP)

or Enterprises’ IP-communications networks. Nokia SBC operates as a Virtual Network Function

(VNF) in Nokia’s end-to-end cloud-native solution or in competitive standalone deployment.

The document provides a complete description of the test plan and results, using real-world

network parameters. It also covers Nokia SBC VNF integration with OpenStack-based

Management and Operation (MANO) framework.

Key findings:

• Validated signaling plane performance: Processed 126,000 concurrent signaling sessions

over TLS and 48,000 concurrent VoLTE signaling sessions over IPSec on a single VM.

• Validated media plane performance and high-density transcoding: Processed 77,500

concurrent media sessions on a single media plane VNF and a 4.23 MOS for AMR-WB codec.

Processed 1,400 concurrent G.711µ-G.729ab transcoded calls on a single VM.

• Powerful encryption/de-encryption for service security: Supported 10,600 concurrent

sRTP-RTP media sessions on a single VM.

• Effective denial of service/distributed denial of service (DoS/DDoS) protection:

Prevented every TCP, UDP and SIP flood attacks. No calls were dropped and CPU usage

remained constant for high-volume call loads.

• High resiliency and availability: Showed high resiliency during failover test with 16,200

concurrent calls. All established calls were successfully transferred to other VMs.

• Supports multiple services and new features: Supported simultaneous calls from access

and peering, Enhanced Voice Services (EVS) codec and a Web real-time communications

(WebRTC) gateway and APIs for third party to develop in-browser communications services.

• Unified and easy SBC operations: Demonstrated management flexibility on three levels:

SBC VNF with CloudBand Application Manager (CBAM), individual SBC with SBC web user

interface (WebUI) and group of SBCs with Nokia NetAct.

Based on results of our testing, the Nokia SBC offers a safe way to evolve

SBCs to support key cloud architectural principles such as fully virtualized

network functions and MANO. Cloud-based Nokia SBC proved

impressive high performances, security and scalability potential, earning

the Miercom Performance Verified certification.

Robert Smithers

CEO

Miercom

https://networks.nokia.com/products/session-border-controller

https://networks.nokia.com/products/session-border-controller



Test Summary

Performance

Signaling

Section 4.1

SIP calls over TLS (30 sec)

• 600 rps max registration rate

(MD5 authentication)

• 800 cps max. call rate

• 126,000 max concurrent sessions

VoLTE calls over IPSec (30 sec)

• 600 rps max registration rate (IMS-

AKA authentication)

• 580 cps max call rate

• 48,000 max concurrent sessions

Media

Section 4.2

Max RTP media session capacity:

• 24,000 on a single VM, 4.23 MOS

• 77,500 on a single media plane

VNF, 4.23 MOS

• 124,000 on a dual media plane

VNF, 4.23 MOS

Max sRTP to RTP

media session

capacity

• 10,600 on a

single VM,

4.23 MOS

Max RTP media

session capacity

(G.729 -G.711

transcoding)

• 1,400 on a single

VM, 4.3 MOS

Attacks

Section 4.3

No impact on call load for TCP SYN, UDP, SIP INVITE and SIP Registration DoS

and DDoS floods. Maximum of 5% CPU increase.

Security

Nessus Vulnerability scan

Section 5.1

No vulnerabilities found during local OA&M interface and remote signaling

interface scans. Found 7 medium-risk flaws during remote OA&M interface scan.

Codenomicon

Section 5.2 Security flaw tester passed. Ran 275,488 tests. No security flaws found.

Functional

Resilience & High Availability

Section 6.1

Failover and rebooting of VMs; Call rate of 120 cps, 300 ms to 2 sec failover time;

all failovers passed. Call rates sustained. No active calls dropped.

EVS Codec

Section 6.2

Verified EVS codec support, transcoding to AMR-WB and backwards

compatibility. Call rate of 1 cps with 4.22 MOS.

SIP Header Manipulation

Section 6.3

Verified SIP filtering effects on quality; 4.16 MOS was unaffected. Minimal CPU

increase for calls with manipulated SIP messages impacted call performance.

WebRTC

Sections 6.4

Successful OPUS-G.711 connection, chat and media sharing. Custom Slack web

application by third-party developer integrated with SBC WebRTC APIs.

Management and Orchestration

VNF Cloud Orchestration

Section 7.1

Verified Nokia CloudBand integration – Successful, intuitive use of CBAM GUI for

install; deployment; extraction; healing; updating; scaling of VNF components.

OA&M

Section 7.2

Verified VMs control on WebUI - Full visibility and control verified for charts;

measurement; alarms; call tracing; backup; component/media plane status and

SIP screening.

Element Management System

Section 7.3

Verified NetAct integration - The centralized NetAct monitoring system provided

real-time visibility into every individual SBC or group of SBCs.



2.0 Product Overview

Cloud-based Nokia SBC

The Nokia SBC is a carrier grade, service-aware gateway which controls IP media and signaling

streams across both the access and peering network boundaries, for both CSPs and enterprises.

Hundreds of millions of subscribers have relied on its service and security in 11 of the top 25

mobile networks. This performance is now available in the Cloud-based Nokia SBC.

The Nokia SBC operates as a VNF, providing a cloud-based software-only SBC for deployment

on any OpenStack or VMware cloud infrastructure. It helps organizations adapt to new network

and technology shifts, take advantage of the speed, flexibility and efficiency of the cloud while

also enabling 3GPP-standard functions migration to the cloud, including:

• Proxy-Call Session Control Function (P-CSCF) and Access Border Gateway (IMS-AGW) for

end-user signaling and media connectivity to core network,

• Access Transfer Control Function and Gateway (ATCF/ATGW) for seamless VoLTE call

handover on a circuit-switched mobile network,

• Interconnect Border Control and Gateway Function (I-BCF/I-BGF) for signaling and media

connectivity to or from peering networks,

• Enhanced P-CSCF function for Web Real-Time Communications i.e. audio, video and

data transfer on any device with a browser and embedded IP into the context of any apps

and website.

Network functions can be deployed as separate VNF instances or in combination, providing

maximum configuration flexibility and faster time-to-market for new services. Each VNF instance

is decomposed into VNF Components (VNFCs) that can scale independently to meet the

growing control plane demands of VoLTE, VoWi-Fi, the future IoT/MTC, and ultimately the

transition to 5G. As shown in the picture below, these components process all the traffic coming

from the untrusted access and the peering sides.

Cloud-based Nokia SBC components Source: Nokia



The Nokia SBC communicates with the core network, which contains a registrar of allowable

callers into the secured network. If registered, the call can be processed and connected inside

the core. The core is the secured side of the network, an IP multimedia Subsystem (IMS) or an

enterprise IP network, depending the use case (SBC for CSPs or enterprise SBC).

All functions and services are protected using an internal traffic distributor and firewall at the

network edge to secure the media and signaling plane. The Nokia SBC media plane security uses

packet filtering on network (L3) and transport (L4) layers of the network. Attacks on the

application layer (L7) are prevented by using SIP and HTTPS filtering.

Protected services include:

• Fixed VoIP for consumer and business (cVoIP/bVoIP),

• Mobile VoLTE,

• Video over LTE (ViLTE),

• Voice over Wi-Fi (VoWi-Fi),

• Rich Communication Services (RCS),

• Enterprise SIP-trunking / Cloud-PBX,

• Enterprise Unified Communication and Collaboration (UC&C),

• In-browser communications services.

The SBC web user interface (WebUI) provides web-based Operation, Administration and

Management (OA&M) interface on individual SBC. Operations include: configuration, fault

management (system status, alarms etc.), performance indicators and management, backup and

troubleshooting (call tracing). Through the WebUI dashboard, control profiles can be created

and security logs can be read or written.



Nokia SBC VNF deployment on Cloud NFV

Nokia SBC supports standard ETSI Network Functions Virtualization (NFV) architecture and

interfaces to help organizations cost-effectively bolster their SBC deployments on any customer

chosen OpenStack or VMware cloud infrastructure. The figure below shows the software

components of the Nokia SBC on OpenStack cloud NFV solution.

Nokia SBC VNF deployment on cloud NFV

The Nokia SBC integrates with Nokia CloudBand to reduce the overall integration efforts,

provide performance, resiliency, scalability and operational efficiency while also enabling a

seamless transition to NFV and software-defined networking (SDN):

• CloudBand Infrastructure Software (CBIS) virtualizes and manages compute, storage, and

network resources. It enables the Nokia SBC VNFs to run and ensures that they meet

strict robustness, performance, and security requirements.

• CloudBand Application Manager (CBAM) automates lifecycle management actions on

the Nokia SBC VNFs by managing resources and applying associated workflows.

The Nokia SBC also integrates with Nokia NetAct Element Management System (EMS). The

Nokia NetAct is virtualized for minimal downtime and resilience and gives one consolidated

view and full visibility and control over the SBC network. NetAct also offers a uniform set of tools

for radio, core and transport network management. This means reduced administration,

maintenance and system integration cost.

Source: Nokia

https://networks.nokia.com/products/cloudband

https://networks.nokia.com/solutions/netact



Nokia SBC VNF Components

The Nokia SBC VNF consists of several VNFCs that run on multiple VMs on the HPE c7000 server

blades. The table below provides a sample configuration for a 2 million subscriber footprint.

VNFC Abbrev. Description No.

VM

No.

CPU

Memory

Size

Operation,

Administration and

Maintenance

OA&M Oversees all element activity;

controls all VM blades. 2 4 16 GB

Charging iCCF Provides optional charging and local

CDR storage 2 4 16 GB

Firewall FW

Layer 3-4 packet filtering and layer 7

SIP filtering and untrusted network

front end distributor

2 16 32 GB

Session Controller SC Handles the access and peering

signaling processing 8 32 192 GB

Core Front End

Distributor CFED

Distributes the trusted signal-

sessions to core. 2 16 64 GB

Diameter Front End

Distributor DFED

Distributes the trusted diameter

traffic 2 8 32 GB

Border Gateway

Controller BGC

H.248 media gateway control

(server side) 2 4 16 GB

System Control

Module SCM H.248 control (client side) 4 16 32 GB

Packet Interface

Module PIM Processes media traffic 16 128 256 GB

Media Conversion

Module MCM

Provides software-based

transcoding of media sessions

12

+2 168 224 GB

TOTAL 54 396 880 GB



3.0 How We Did It

Cloud Infrastructure Setup

The test plan covered Nokia SBC combined access and peering for cloud deployment on

OpenStack CloudBand Infrastructure (CBIS) and CloudBand Application Manager (CBAM).

The Virtualized Infrastructure Manager (VIM) CBIS runs on two C7000 chassis and 32 Gen9 hosts:

one host is needed for under cloud VM, and three hosts for Controller with High Availability,

leaving 28 hosts for compute nodes. The CBIS memory design accessed local memory quickly

to handle scalable workloads. On compute nodes, CBIS used 6 vCPUs and 2G memory dedicated

for hypervisor (4 vCPUs from NUMA0 and 2 vCPUS from NUMA1), that means for a Gen9 blade

with 48 vCPUs, 42 vCPUs were available for SBC VMs. All computes were configured to enable

single root I/O virtualization (SR-IOV) for higher virtualized media plane performance.

Compute Host Configuration Table

Server HPE BL460c Gen9

CPU 2 x Intel Xeon(R) CPU E5-2680 v3 @ 2.50GHz (12 cores) - total 24 cores – total 48

HT – total 42 HT for guest vCPU

RAM DDR4, 2133 MHz, 128GB

Network 2x Dual port NICs: Flexible LOM HP Ethernet 10GB 2-port 560FLM adapter and

Mezzanine card: HP Ethernet 10GB 2-port 560M adapter

Storage Integrated HDD: HP 1.2TB 12Gb/s SAS 10Krpm, RAID0

HP Smart Array P244br

The VNF Manager (VNFM) CBAM managed the life-cycle of the Nokia SBC VNF. The test plan

covered several LCM events (deploy, heal, scale etc.) implemented using VNF templates, VNF

descriptors, Heat Orchestration Templates (HOT), Mistral workflows and Ansible playbooks.



Onboarding Nokia SBC VNF

Test Tools

Source: Miercom

Source: Nokia



• The Ixia XGS12-SD generated signaling and media loads with Transport Layer Security

(TLS), VoLTE with IP Multimedia Subsystem (IMS) Authentication and Key Agreement

(AKA) authentication, Real-time Transport Protocol (RTP), Secure RTP (SRTP)

and transcoding.

• DDoS and flood attacks traffic was generated by the Nokia TCP/IP RightTrack

DoS toolkit.

• The SIPp tool simulated signaling-only loads.

• WebRTC call was initiated using the Nokia WebRTC API.

Lab Topology

Source: Miercom

Ixia XGS-12SD

with IxLoad

IMS core, SIPp, Nessus,

Codenomicon, RightTrack on CBIS

(Openstack Liberty cloud on HP Gen9

server blades)



Test Bed Diagram

Traffic Profile

Different traffic profiles were used to represent a real-world network environment:

Traffic Type Description

SIP secured by TLS

• Registration using MD-5 authentication

• SIP over Transport Layer Security (TLS) v1.2

• Call flow consisting of 7 messages: INVITE, 100

Trying, 180 Ringing, 200-OK, ACK, BYE, 200-OK

VoLTE with AKA

• Registration using IMS-AKA authentication

• SIP over IPSec

• Call flow consisting of 12 messages: INVITE, 100

Trying, 183 Session Progress, PRACK, 200-OK, 180

Ringing, PRACK, 200-OK, 200-OK (INVITE), ACK,

BYE, 200-OK

Source: Miercom

Signal and Media Calls with TLS, VoLTE+AKA, RTP sRTP, DoS/DDoS Generator

Unsecure Network User Equipment Callee (UEC)

Secure Network User Equipment Server (UES)

8-1

0G

Inte

rfac

es

6-10G Interfaces

4-10G Interfaces

Ixia, SIPp, Nessus, CN, Right Track

Layer 3

HP Switch

Nokia SBC

Nokia IMS

Signal Calls, RTP



Call Rate

Standard IMS call

As shown in the picture above, in a standard IMS call (assume the calling and called parties

belong to the same CSP), 1 call per session is managed on the originating SBC to connect the

calling party to the network and 1 call per session is managed on the terminating SBC to

connect the network to the called party.

Lab test call

During the tests, Nokia SBC managed both the calling and called party sides emulated by the

load generator. That means when a call was placed, 2 calls were being handled by the SBC: 1

call per session to connect the calling party to the IMS network and 1 call per session to connect

the IMS network to the called party.

Source: Nokia

Source: Nokia



The next sections define the call rate as calls per second (cps) as follows:

• 100 cps call load from the Ixia XGS12-SD generator means the SBC handles 100 cps on

the originating side and 100 cps on the terminating side, 200 cps in total.

• 200 cps call rate on SBC means the load generator initiated a 100 cps call load.

Test Plan

The test plan covered four main areas:

• Performance

Performance metrics for the signaling and media planes and performance under attack.

• Security

Find vulnerabilities that may expose the secured network to peering or attacks from

untrusted network sources. Testing was performed using the Nessus Vulnerability

Scanner and Codenomicon test tools.

• Functionality

Observe and validate four key features: SBC Resilience, SBC support of EVS VoLTE codec,

SIP Header Manipulation, WebRTC Gateway and APIs.

• Management and orchestration

Validate the different levels of management of SBC VNF using CBAM, SBC WebUI and

Nokia NetAct.

The results of the tests are presented in the following four sections.



4.0 Performance Testing

4.1 Signaling performance

The Session Controller (SC) is one of the Nokia SBC’s virtual machines (VMs) responsible

for processing calls that have passed through the firewall VM. There are 8 SC VMs per SBC

VNF consisting of 4 pairs of active/stand-by VMs. The following tests have been passed against

1 SC VM pair.

This VM pair is rated to run up to 75 percent of the HPE server CPU and use 80 percent of

memory during high performance handling. The CPU and memory are monitored at 400

millisecond (ms) intervals, and when thresholds are reached, alarms and overload controls are

activated. Calls and SIP messages may be throttled. Additional alarms are generated as critical

limits of resources are met.

4.1.1 Subscriber registration performance

Description Result

Generate registrations using MD5 authentication over

TLS. Verify the registration rate at which the SC VM can

successfully signal registrations.

• Max. registration rate = 600

registrations per second (rps)

• SC CPU = 27%

Generate VoLTE registrations using IMS-AKA

authentication over IPSec. Verify the maximum

registration rate at which the SC VM can successfully

signal registrations.

• Max. registration rate = 600 rps

• SC CPU = 17%

4.1.2 Call signaling performance over TLS

Description Result

Generate SIP calls over TLS with 30 sec hold time.

Determine the maximum call rate at Session Controller

VM resource capacity (75% CPU, 80% memory).

• Call duration = 30 sec

• Max. call rate = 800 cps

Generate SIP calls over TLS with 30 sec hold time.

Measure the SIP session capacity, i.e. maximum number

of concurrent sessions the SC VM can manage


• Max. concurrent sessions = 126,000



4.1.3 VoLTE call signaling performance over IPSec

Description Result

Generate VoLTE calls using AKA authentication over

IPSec with 30 sec hold time. Determine the maximum

call rate at Session Controller VM resource capacity

(75% CPU, 80% memory).


• Max Call rate = 580 cps

Generate VoLTE calls using AKA authentication over

IPSec with 30 sec hold time. Measure the SIP session

capacity, i.e. maximum number of concurrent sessions

the SC VM can manage.


• Max. concurrent sessions = 48,000

4.2 Media performance

Encrypted VoLTE signaling was generated using SIPp and Ixia to determine maximum sessions.

4.2.1 Media Calls performance with RTP

Description Result

Generate media calls using AMR-WB codec and

RTP protocol on the access and core side.

• Measure the media session capacity, i.e.

maximum number of concurrent media

sessions supported.

• Determine the max. call rate

• Verify quality with MOS score.

Perform these tests with different capacities,

from smaller to larger and verify the MOS score.

Case 1: one pair of (active/stand-by) PIM VM

(8 vCPUs)



• Max media session capacity = 24,000

• CPU = 90%

• MOS = 4.23

Case 2: Single Media-Plane VNF (4 pairs of

(active/stand-by) PIM VM)


• Max call rate = 72 cps

• Max media session capacity = 77,500

• CPU = 85%

• MOS = 4.23

Case 3: Dual Media-Plane VNF (8 pairs of

(active/stand-by) PIM VM)



• Max. media session capacity = 124,000

• CPU = 85%

• MOS = 4.23



4.2.2 Media performance with sRTP

Description Result

Generate media calls using G.729 codec and sRTP

protocol on the access side and G.729 codec and

RTP protocol on the core side. Use 130 sec call

hold time.



sessions supported on one PIM.

• Determine the maximum call rate

• Verify quality with MOS score

One pair of (active/stand-by) PIM VM (8 vCPUs)

with G.729 sRTP to RTP interworking:




• CPU = 83%

• MOS = 4.16

4.2.3 Media transcoding performance

Description Result

Generate media calls using G.729 codec and RTP

protocol on the access side and G.711 codec and

RTP protocol on the core side. Use 155 sec call

hold time.



sessions supported on one MCM

• Determine the maximum call rate

• Verify quality with MOS score.

Single MCM (12 vCPUs) with G.729-G.711

transcoding:




• Call peak = 712 calls

• CPU = 84%

• MOS = 4.3

4.3 Performance Under Attack

Denial-of-Service (DoS) and Distributed (DDoS) attacks overwhelm a target with traffic, creating

vulnerability. DDoS attacks have legitimate looking connections and no single attack source. To

test the SBC for DoS/DDoS protection, we used a combination of packet flooding and targets.

Floods of TCP, UDP and SIP packets were sent to signaling interfaces with public IP addresses.

The following results were expected: no expected noise; no impact on call load; alarms are

generated and cleared; no critical issues (SegV, memory leak, core dump, switchover/failover,

unexpected ASSERT, ALARM, HIGH log). A 46 percent CPU baseline was recorded with 1.1

million busy-hour call attempts and 150 second call hold time.



4.3.1 DoS - TCP SYN Flood

Description Result

Generate SYN flood attack, the continuous opening of

new connections with SYN messages to the server,

without the follow-up ACK message to close them.

Increasing amounts of half-open connections will drain

system resources until DoS results; malfunction or

failure may occur. Measure resource use, system

impact and prevention.

Background load = 2 million subscribers

Max calls = 95,700 IPsec sessions

Registration rate = 267 rps

Call rate = 311 cps

Attack flood rate = 600,000 pps SYN attacks

• Initial CPU = 46%

• CPU during attack = 51.3%

• No impact on call load and system

• Alarms issued

• Firewalls reported dropped packets

and associated measurements in the

WebUI

4.3.2 DDoS – Distributed TCP SYN Flood

Description Result

Generate DDoS SYN flood attack, but incorporate

connection rate limiting. Measure resource use, system


Background load = 2 million subscribers

Max calls = 95,700 IPsec sessions


Call rate = 311 cps

Attack flood rate = 560,000 pps

• CPU during attack = 51%


• Alarms issued



WebUI

4.3.3 DDoS - UDP Flood

Description Result

Generate a DDoS UDP flood on a specific port

to cause the system failure, restart or memory loss.

Measure resource use, system impact

and prevention.

Background load = 100,000 subscribers

Max sessions = 82,000 IPSec sessions

Attack flood rate = 1.2 to 1.3 million pps with 1

GB small packets for 20 min



• Alarms issued, indicating dropped

packets



WebUI



4.3.4 SIP INVITE Flood

Description Result

Generate a SIP INVITE flood using a SIP INVITE

message. Measure resource use, system impact

and prevention.

Background load = 100,000 subscribers



Call rate = 311 cps

Attack flood rate = 15,000 INVITE pps



• Alarms issued, indicating dropped packets

• Firewalls reported dropped packets and

associated measurements in the WebUI

4.3.5 SIP Registration Flood

Description Result

Generate a SIP REGISTER flood using a SIP

REGISTER message. Measure resource use, system


Registered users = 2 million



Call rate = 311 cps

Attack flood rate = 15,000 REGISTER pps



• Alarms issued, indicating dropped packets

• Firewalls reported dropped packets and

associated measurements in the WebUI



5.0 Security Testing

5.1 Nessus Vulnerability Scan

Vulnerability scanning was performed for all IP addresses of different network interfaces. These

vulnerabilities are categorized as low, medium, high or critical.

5.1.1 Nessus Local Scan - Guest OA&M Interface

Description Result

Verify the following:

• Missing operating system (OS)

security patches

• Documentation of missing OS patches

• List of open ports for each host

• Post-scan LCP logs for analysis

• No core dump or tenant services

A Nessus scan revealed no vulnerabilities.

5.1.2 Nessus Remote Scan - Guest OA&M Interface

Description Result



security patches





A Nessus scan revealed:

• 1 High-grade vulnerability; this was reported

as a false positive, as it is a policy

compliance summary.

• 14 Medium-grade vulnerabilities; this was

reduced to 7 vulnerabilities. Half were false

positives, resolvable and resulting from

policy compliance.

5.1.3 Nessus Remote Scan – Access Signaling Interface

Description Result



security patches








5.1.4 Nessus Remote Scan – Peering Signaling Interface

Description Result



security patches






5.2 Codenomicon

Codenomicon tools identify security flaws by sending loads of malformed protocol messages

and observing system response. The system is expected to handle all stress-test conditions.

5.2.1 Codenomicon IPv4 Protocol

Description Result


• No unexpected noise

• No impact to call load

• CPI alarm generated and cleared

• No critical issues (SegV, memory leak,

core dump, switchover/failover,

unexpected ASSERT, ALARM, HIGH log)

Pass; a total of 50 test cases were executed and

no system flaws were found.

5.2.2 Codenomicon TCP for IPv4 Server Protocol

Description Result








Pass; a total of 1,026 test cases were executed

and no system flaws were found.



5.2.3 Codenomicon SIPUAS Protocol

Description Result










5.2.4 Codenomicon Diameter Client Protocol

Description Result












6.0 Functional Testing

6.1 SBC Resilience & High Availability

Resilience and high availability functionality is unique to each SBC. Traditional failover is

accomplished using two separate SBC devices, an active SBC and a standalone SBC to pick up

when failure occurs in the primary controller. Failover can occur when the primary SBC loses

power, reboots, loses physical connectivity or encounters a computing crash.

In the case of the Nokia SBC, which utilizes virtual components to handle call-processing, each

VM was rebooted to cause failover. High availability is expected to minimize or eliminate other

VM failure, call failure and impact on stable calls.

6.1.1 Failover OA&M VM with VoLTE Load

Description Result

Using a VoLTE call load, create an OA&M

failover by VM reboot and verify the following:

• No impact to transient or stable calls.

• Call rate = 120 cps

• Call hold time = 135 sec

• No transient or stable calls lost

6.1.2 Failover FW with VoLTE Load

Description Result

Using a VoLTE call load, create a FW failover by

VM reboot and verify the following:

• Some transient call failure, where the

number of failed calls determines duration

of outage

• No impact on stable calls



• Pass; 150-300 ms failover time

• No stable calls fail

6.1.3 Failover SC with VoLTE Load

Description Result

Using a VoLTE call load, create a SC failover by

VM reboot and verify the following:


number of failed calls determines

duration of outage




• Pass; 2 sec failover time




6.1.4 Failover CFED with VoLTE Load

Description Result

Using a VoLTE call load, create a CFED failover

by VM reboot and verify the following:



of outage






6.1.5 Failover BGC with VoLTE Load

Description Result

Using a VoLTE call load, create a BGC failover




of outage




• Pass; 500 ms failover time


6.1.6 Failover PIM with VoLTE Load

Description Result

Using a VoLTE call load, create a PIM failover




of outage






• Slight dip in MOS, but quickly recovers

6.1.7 Failover SCM with VoLTE Load

Description Result

Using a VoLTE call load, create a SCM failover




of outage




• Pass; 2-3ms failover time




6.2 Support of Ultra-HD Voice EVS codec

EVS codec supports sampling rates for narrow, wide, super-wide and full bands. It minimizes

jitter and packet loss; concealing packet loss when it occurs. It is backwards compatible with

Adaptive Multi-Rate Wideband (AMR-WB), a codec that supports wideband rates.

Using a VoLTE call being transcoded from EVS to AMR-WB, we observed a maintained high-

quality call on the Nokia SBC. See Appendix Section 6.2 for screen captures of the transcoding

process in the WebUI.

6.2.1 EVS Codec

Description Result

Show a VoLTE call using EVS codec transcoded

to AMR-WB.

Call load: VoLTE call

Call rate: 1 cps

Transcoding: EVS/AMR-WB

MOS: 4.22

6.3 SIP Header Manipulation

The message screening feature utilizes filters to manipulate both the SIP message and body

information for interoperability and security. Filtering and manipulating SIP headers resolve

protocol differences from multiple vendors and enhance security by hiding SIP topology, like IP

addresses vulnerable to attacks.

Using the SC VM, we initiated calls at 40 cps to determine if the call quality could be maintained

despite header filtering and manipulation. Other effects were also observed, such as the load

placed on the CPU. Screen captures can be viewed in Appendix Section 6.3.

The SIP filter tested was a complex one and commonly used in VoLTE environments to strip

preconditions from both the message and body. It was run against one SC VM pair to measure

the MOS score impact.

6.3.1 SIP Filter to Manipulate SIP Message Header and Body

Description Result

Create a SIP filter to manipulate the SIP header

and body.

• Call rate: 40 cps

• Call duration: 130 sec

• MOS: 4.16



6.3.2 SIP Filter Verified with IMS Call Trace

Description Result

Create a SIP filter to manipulate the SIP message

header and body. Verify its effect using the IMS

call trace tool.



• MOS: 4.16

• No effect

6.3.3 SIP Filter Verified with Call Processing Signaling Performance

Description Result

Create a SIP filter to manipulate the SIP message

header and body. Verify its effect by observing

any impact to call processing signaling

performance.



• MOS: 4.16

• 0.25% CPU increase on 100% calls

6.4 WebRTC

WebRTC technology enables HTML-5 based web browsers with Real-Time Communications

capabilities via simple JavaScript APIs and without the need for plugins. WebRTC enhances the

customer experience, moving away from the traditional communications experience to deliver

communications contextually in web-based applications and websites. WebRTC also offers new

monetization opportunities providing a suite of WebRTC APIs that developers can use to create

new web applications with universal access from legacy or IP networks.

Our testing showed Nokia SBC is fully supportive of WebRTC, providing a secured transport and

interworking functions with IMS. Additionally, we experienced the Nokia WebRTC in-browser

communication API, in-browser collaboration API and device-switching API. Using these easy-

to-use APIs, web developers can augment the multi-device service to multi-device

communications via web applications that deliver voice, video, chat, and share features.

https://networks.nokia.com/solutions/webrtc



WebRTC-enabled in-browser communications

Registrations and calls can be setup and broken down using mobile or web applications on

desktop devices, involved parties can chat and collaborate during the call and calls can be

pushed or pulled from one device to the other.

This flexibility was demonstrated using three test cases:

6.4.1 WebRTC Call with Audio-Transcoding

Description Result

A video call with audio was placed between

two WebRTC clients over DTLS/SRTP using

Nokia WebRTC ORCA client on Chrome

browser. One client uses OPUS codec and the

other uses G.711. Verify successful

demonstration.

• Client A: OPUS audio and VP8 video codec

• Client B: G.711 audio and VP8 video codec

• Connection made and media shared

Source: Nokia



6.4.2 WebRTC with Chat and File Transfer

Description Result

A video call with audio was placed between two

WebRTC clients over DTLS/SRTP using Nokia

WebRTC ORCA client on Chrome browser. One

client uses OPUS codec and the other uses

G.711.

• Chat session was placed

• File was sent and opened

• Chat session was closed

• Verify use with different browser

• Client A: OPUS audio and, VP8 video codec

• Client B: G.711 audio and, VP8 video codec

• Connection made and media shared

• Chat session started and a 78K image file was

sent from Client A to Client B.

• Chat session was closed and video chat

resumed.

• Repeated using Mozilla Firefox and observed

the same results.

6.4.3 WebRTC with Slack WebUI

Description Result

Nokia’s developer partner Quobis leveraged

the Nokia’s WebRTC APIS to enable WebRTC

on Slack, a web application for team’s

communication and collaboration. With such

integration Slack becomes a SIM-less

secondary mobile device.

• A call was made from the legacy network

to a mobile number

• The call terminated on Slack and mobile

devices

• The call was answered on Slack

• Call was pushed to mobile

• Call was answered on mobile

• Call pulled back on the browser

• At user login on Slack, the WebRTC-enabled

Slack client is registered on IMS network

• Incoming call was synchronized on Slack and

mobile device (simultaneous ringing)

• Call was answered on Slack WebRTC client

• Call was then successfully pushed from Slack

UI to the mobile device.

• After call was answered by the mobile device,

the call was successfully pulled from Slack UI

and moved in the browser.

• Flexibility of communication was shown to

extend to any device with a browser.



7.0 Management and Orchestration Testing

7.1 SBC VNF Cloud Orchestration

In this series of tests we observed the Nokia SBC VNFs orchestration and life cycle management

using CBAM.

CloudBand Application Manager

The CBAM GUI allows the user to install and deploy, or extract the following SBC VNF

components: OA&M, SC, BGC, FW, CFED, DFED, iCCF, SCM, PIM and MCM. For additional

screenshots, see Appendix Section 7.1

7.1.1 Deploy

Description Result

Verify CBAM operation to install, deploy or

extract SBC VNF components.

The following VNF components were

successfully installed, deployed or extracted:

OA&M, SC, DFED, BGC, FW, CFED, iCCF, SCM,

PIM and MCM.



CBAM standard lifecycle operations

The CBAM standard lifecycle operations provide healing function, software upgrade and scale

functions. For additional screenshots, see Appendix Section 7.1.

7.1.2 Heal

Description Result

Verify CBAM Heal operation on SBC media

plane (PIM VM).

PIM VM was successfully healed and recovered

to “in-service” using CBAM heal operation. See

Appendix Section 7.1.2 for more screenshots.

7.1.3 Software Upgrade

Description Result

Verify CBAM Software upgrade operation on

both SBC signaling and media VMs.

Verify CBAM software roll back operation on

both SBC signaling and media VMs.

A software update was performed for both

signaling and media planes. The signaling plane

was able to evolve a database from an

older version to a newer version using the

software upgrade.

The roll back of this software was also verified.



7.2 SBC Operations, Administration and Management

The OA&M oversees all element activity. It controls all VM blades. The dashboard provides many

display options, analysis tools and controls.

7.2.1 Performance Charts in OA&M UI Dashboard

Description Result

Display the following Performance Management

(PM) charts:

• PM data display with area zoom

• PM data display with zoom out

• Restore

• Line chart

• Histogram

• Data review

• Save as image

• Charts with large PM data

Charts were displayed and verified. For various

screenshots of the OA&M UI, see Appendix

Section 7.2.1.

All charts contained correct data, and the

interface was easy to navigate.

7.1.4 Scale

Description Result

Verify CBAM procedure for scaling SBC media

plane. Add one additional pair of MCM VM

during transcoding work load.

MCM was successfully scaled-out using CBAM

scale operation.

• Transcoding background call load



• Scale-out: from 3 active (+ 1 stand-by) to

4 active VM (+ 1 stand-by)

See Appendix Section 7.1.4 for more

screenshots.



7.2.2 Traffic Measurement in OA&M UI Dashboard

Description Result


• Login to WebUI

• Reports Scheduling

• Add Traffic Job

• Modify Traffic Job

• Successfully logged into WebUI

• Used PM for measuring traffic, filterable by

many variables

• PM reports were verified as correct and

filterable

• Traffic schedules could be created based on

measurement type and modified if disabled

Data was correct and easily navigated. For more

screenshots, see Appendix Section 7.2.2.

The PM report allowed for measurement of all traffic, filterable by type, time, object ID, granularity,

value and flag. Traffic job schedules measured groups of traffic. Groups could be based on VM, resources,

policies or other components. These jobs could be created, enabled or disabled. Only disabled jobs could be

modified.



7.2.3 Fault Management Alarm Capabilities

Description Result


• Alarm view

• Alarm filter

• Alarm meaning

• Alarm clearing

Viewed, filtered and cleared alarms. For

additional screenshots, see Appendix 7.2.3.

Alarms were viewed and filterable by severity, time, user, event, cause and problem. Details for the

alarm were available to determine the meaning for the alert. Alarms were viewable in a chart, to display

type and frequency.

7.2.4 Fault Management Alarm Filter Configuration

Description Result

Log into OA&M, click an alarm from the table. In

the Filter tab, configure alarm setting

parameters in alarm creation page and save.


• Filter page is shown

• All filters are shown on the left and

Create Filter page is shown on the right

• The new alarm is shown in the filter list

• After clicking Choose Filter selector in

the alarms page, the new filter is shown

All alarm functionality and visibility was

observed and verified. For additional

screenshots, see Appendix Section 7.2.4.



The Alarm table showed all current alarms. These were filterable by label, type, cause and problem. They

could be sorted by severity or time.

7.2.5 OA&M Call Trace Tool

Description Result

Create two call trace records with trace types SIP

URI and tel num. Make a call using the same

subscriber, then stop the call trace using SIP URI

manually. Let another call trace using tel num

stop automatically, when the duration time

expires. Verify the following:

• Two call traces can be stopped

successfully and displayed in WebUI

• Download one call trace

• Content can be captured in call trace for

the basic call

Call trace functionality was verified. See

Appendix Section 7.2.5 for screenshots.

7.2.6 OA&M Backup

Description Result

Confirm that multiple backup output packages

can be displayed in WebUI. Create multiple

backup jobs or create one job to generate

multiple backup packages. Verify that all output

can be displayed in WebUI.

All backup creation and details were observed in

the WebUI, verified.



Backup jobs were created under OA&M Tool Backup Management > Scheduling.

Backup jobs were displayed in the Scheduling list. These jobs could be enabled or disabled at any time.

Details of each backup were available.



The status of each backup was listed under Backup

Management > Status.

7.2.7 OA&M Signaling Components and Inventory

Description Result

Verify the status of signaling components:

• Host services

• Service members

• Components

• Diameter links

• SIP links

• Network links

Check system inventory.

All components were verified and found useful

for visibility of signaling components.

For additional screenshots, see Appendix

Section 7.2.7.

Host services were displayed under the State tab and indicated its status, actions and other information.



Information was shown for each service network

interface. The CNFG interface is shown above.

7.2.8 OA&M Media Plane Status

Description Result

Open the WebUI State Management and verify

registered status of the MGC media plane.

Using the Node Status > Virtual Media Gateway,

we verified the registered status of the MGC

media plane.

Media plane status was displayed and verified as registered.

7.2.9 OA&M SIP Screening

Description Result

Use the WebUI for SIP Screening related tables.

• Check that modifications are synced to

signaling plane

• Add filter to the filter set

• Assign filter set ID to P-CSCF profile

The WebUI was used to view modified signaling

configuration and filter.



Using Signaling > SIP Screening > SIP Filter, the signaling plane was viewed and filtered.

7.3 SBCs Cluster Management with NetAct

The Nokia NetAct is an optional web-based tool that enhances the network and SBC experience

as a real-time management system, dashboard and report generator of network performance.

The centralized monitoring system provides visibility into every individual SBC or group of SBCs.

Data is collected from the manage networks to provide the following building blocks:

NetAct building blocks

The customizable NetAct monitor detects incidents with alarm and filtering technology. Alarms

can be monitored for individual SBC or group of SBCs by grouping multiple SBCs into SBC

Cluster as shown below.

Source: Nokia



SBCs cluster alarm monitoring

The NetAct Performance Manager tracks network performance and identifies bottlenecks with

reporting tools, such as reporting suites, report creator, KPI builder, threshold and profiler and

NetAct Self Monitor reporting.

Dashboards display performance data in real-time for flexible management and adjustment.

Time Overlay uses timestamps to make multiple report comparisons, and any report can be

grouped or filtered based on specific parameters. Also PM report can be displayed for individual

SBC or group of SBCs by grouping multiple SBCs into SBC Cluster.

Source: Nokia



SBCs cluster performance monitoring

Source: Nokia



About Miercom

Miercom has published hundreds of network product analyses in leading trade periodicals and

other publications. Miercom’s reputation as the leading, independent product test center is

undisputed. Private test services available from Miercom include competitive product analyses,

as well as individual product evaluations. Miercom features comprehensive certification and test

programs including: Certified Interoperable, Certified Reliable, Certified Secure and Certified

Green. Products may also be evaluated under the Performance Verified program, the industry’s

Customer Use and Evaluation

We encourage customers to do their own product trials, as tests are based on the

average environment and do not reflect every possible deployment scenario. We offer

consulting services and engineering assistance for any customer who wishes to perform an

on-site evaluation.

Use of This Report

Every effort was made to ensure the accuracy of the data contained in this report but errors

and/or oversights can occur. The information documented in this report may also rely on various

test tools, the accuracy of which is beyond our control. Furthermore, the document relies on

certain representations by the vendors that were reasonably verified by Miercom but beyond

our control to verify to 100 percent certainty.

This document is provided “as is,” by Miercom and gives no warranty, representation or

undertaking, whether express or implied, and accepts no legal responsibility, whether direct or

indirect, for the accuracy, completeness, usefulness or suitability of any information contained in

this report.

All trademarks used in the document are owned by their respective owners. You agree not

to use any trademark in or as the whole or part of your own trademarks in connection with

any activities, products or services which are not ours, or in a manner which may be

confusing, misleading or deceptive or in a manner that disparages us or our information,

projects or developments.

© 2017 Miercom. All Rights reserved. No part of this publication may be reproduced, photocopied, stored on a retrieval system, or

transmitted without the express written consent of the authors. Please email [email protected] for additional information.

Cloud-based Nokia SBC Performance Verified A-1 DR170831E


Appendix

6.2 Support of Enhanced Voice Services (EVS) Codec

Figure 1: Session Description

Figure 2: Call Trace List



Figure 3: EVS to AMR-WB Transcoding

Figure 4: Transcoding Measurement Information



6.3 SIP Header Manipulation

Section 6.3.1 SIP Filter to Manipulate SIP Message Header and Body

Figure 5: Precondition SIP Filter

Figure 6: SIP Filtering



Section 6.3.2 SIP Filter Verified with IMS Call Trace

Figure 7: SIP Filtering with IMS Call Trace

Figure 8: Before SIP Filtering Verified with IMS Trace Tool



Figure 9: After SIP Filtering Verified with IMS Trace Tool

Section 6.3.3 SIP Filter Verified with Call Processing Signaling Performance

Figure 10: Performance Measurement Report (without filter)



Figure 11: Performance Measurement Report (with filter)

7.1 VNF Cloud Orchestration

Figure 12: CBAM GUI Operation Status



Figure 13: CBAM GUI Operation View

Figure 14: CBAM GUI Trigger - Signal and Media Plane Application Level Back Up

Figure 15: CBAM GUI - SBC Restore for Signal and Media Planes



Section 7.1.2 Heal VM Function

Figure 16: CLI for SBC Heal

Figure 17: Media PIM VM Malfunction

Figure 18: Heal Process for Media PIM VM



Figure 19: Node Status after Heal

Figure 20: Healed Media PIM VM



Section 7.1.4 Scale

Figure 21: CLI for Information on Media Planes

Figure 22: Planes to Scale



Figure 23: Scaling in Progress

Figure 24: Scaling Complete

Figure 25: Verified Scaling Finished

Figure 26: Confirmed Scale



7.2 OAM Web User Interface

Section 7.2.1 Performance Charts in OAM UI Dashboard

Figure 27: OAM Web User Interface

Figure 28: PM Zoom



Figure 29: PM Traffic Filtering

Figure 30: Signaling Chart Configuration

Figure 31: Media Chart Configuration



Section 7.2.2 Traffic Measurement in OAM UI Dashboard

Figure 32: PM Report Selection and Filter

Figure 33: Traffic Job Creation

Figure 34: Traffic Job Enable/Disable



Section 7.2.3 Fault Management Alarm Capabilities

Figure 35: Alarm Frequency Chart

Figure 36: Recovery Procedures to Clear Alarms

Section 7.2.4 Fault Management Alarm Filter Configuration

Figure 37: Alarm Filters (Left) and Parameters (Right)



Section 7.2.5 OAM Call Trace Tool

Figure 38: Call Trace List

Figure 39: Call Trace Create



Figure 40: Call Trace Details



Section 7.2.7 OAM Signaling Components and Inventory

Figure 41: OAM Signaling Status

Figure 42: OAM Service Member Status



Figure 43: OAM Component Status

Figure 44: OAM Diameter Status



Figure 45: OAM SIP Link Status

Figure 46: OAM Network Interface

cloud-based nokia sbc performance testing and function ... · interface (webui) and group of sbcs...

Documents