icirrus contract no. 644526 1 jan 2015 – 31 dec 2017 · enb enhanced node b (radio base station...

iCIRRUS Contract No. 644526 1 Jan 2015 – 31 Dec 2017

This project has received funding from the European Union’s Horizon 2020

research and innovation programme under grant agreement No 644526

intelligent Converged network consolidating Radio and optical access aRound USer equipment

DELIVERABLE: D5.4

Validation test results and analysis evaluation

Contract number: 644526

Project acronym: iCIRRUS

Project title: Intelligent converged network consolidating radio and optical access around user equipment

Project duration: 1 January 2015 – 31 December 2017

Coordinator: Nathan Gomes, University of Kent, Canterbury, UK

Deliverable Number: D5.4

Type: Report

Dissemination level Public

Date submitted: 20 February 2018

Editors: Howard Thomas (VIAVI)

Authors / contributors (contributing partners)

Kai Habel (HHI), Jim Zou (ADVA), Philippos Assimakopoulos, Nathan Gomes, Huiling Zhu, Yuan Kai (UniKent), Howard Thomas (VIAVI), Xuan Du (UEssex), Philippe Chanclou (Orange), Patrik Ritoša (TS), Guadalupe Flores (WT), Gregor Linne (IAF)

Internal reviewers Mike Parker (UEssex), Michael Georgiades (PrimeTel), Nathan Gomes (UniKent)


of 75

This project has received funding from the European

Union’s Horizon 2020 research and innovation

programme under grant agreement No 644526

Documenthistory

0.0 Document creation 17/10/2017

0.1 Updated TOC to reflect D5.3 included Section on aggregation of results and analysis

30/11/2017

0.2 Included inputs to Validation test from ADVA, HHI. Elaboration of KPI Section by VIAVI

21/12/2017

0.25 Demo schematic update and added LTE femto‐cell case description 22/12/2017

0.3 Wellness version

0.4 Howard comments 3/01/2018

0.45 Minor updates and accept additions 4/01/2018

0.5 Initial inputs about mobile cloud from UEssex 07/01/2018

0.55 Combine V5, V7 & Orange 15/01/2018

0.6 Integrate Philippos content 15/01/2018

0.65 Accept Xuan content 17/01/2018

0.7 Changes accepted, acronym list, Summary text from Jim 17/01/2018

0.72 Additions from Gregor on CPRI 22/01/2018

0.75 Accept changes, remove fixed comments, minor edits 24/01/2018

0.8 1st attempt at completing intro and the evaluation Sections 24/01/2018

0.85 Re‐emphasis, re‐write in context of 5G KPIs 09/02/2018

0.90 Accept changes to format, typos, include text from Kai and my editspost review. This is a baseline to edit the Demo as a KPI in a broader scheme

12/02/2018

0.91 Restructure and emphasise KPI validation 13/02/2018

0.92 revision of text 3.9.1 13/02/2018

0.93 Re‐structure to separate testbed KPI from demo KPI 14/02/2018

0.94 Added evaluation of the Showcase section, incorporated inputs on D2D and mobile cloud

17/02/2018

1.00 Added Conclusions and general tidy‐up 19/02/2018


of 75




Abstract

The deliverable presents the results obtained throughout the iCIRRUS project and compares them

with the 5G requirements including business, societal and performance KPIs. One KPI that is relevant

to both the business and societal KPIs is the number of proof of concept demonstrators as this is

recognised as a measure of “European availability of a competitive industrial offer for 5G services

and technologies.” Consequently, the showcase demonstrator hosted by Telekom Slovenije in

December 2017 is a focus of this report as is the separate demonstrator for evolved‐fronthaul higher

layer split hosted by Orange. Three families of use‐case are explored: enhanced front haul, mobile

cloud, and network controlled device‐to‐device communication, aiming to demonstrate that the

planned 5G networking requirements are met. The architecture and component parts of the

demonstrator are described and the performances of the use‐cases are presented. The results are

encouraging, in that all the iCIRRUS quantitative performance targets are met and there are

significant contributions towards the 5G performance KPIs. Additionally, significant contributions

towards achievement of the 5G business and societal are demonstrated by the three use‐case

classes.


of 75




ExecutiveSummary

The report presents the results obtained throughout the iCIRRUS project and, to validate them

against 5G Key Performance Indicators (KPIs), compares the results against 5G requirements

including business, societal and performance KPIs as summarised in [1].

Two elements of elements of the KPI validation relate to technology demonstrators. One, hosted by

Orange, addressed evolved‐fronthaul with a higher layer split. The second, hosted by Telekom

Slovenije in December 2017, was the iCIRRUS showcase demonstrator that simultaneously

demonstrated a variety of evolved‐fronthaul lower layer splits and cloud‐assisted Unified

Communication applications.

The demonstrators are KPI validations in themselves as implementation of 5G proof of concept

demonstrators is recognised as a measure of the KPI “European availability of a competitive

industrial offer for 5G services and technologies,” which links to both B3 and S3 business and

societal 5G KPIs respectively [1].

The use cases explored in the demonstrators and the KPIs used to validate their performance are set

out and are related to the overall 5G‐compliant architecture that has been developed by the project.

In this deliverable, the architecture of the showcase demonstrator is illustrated and its component

parts are described, which includes the physical extent of the fibre network that supports the

fronthaul/backhaul.

The performance of the component fronthaul use‐cases and the results obtained for transport over

a 100G aggregated Ethernet link are presented. Respectively, these use‐cases aim to demonstrate: a

low layer split fronthaul for a 60GHz link; CPRI over Ethernet; and femto‐cell backhaul. The

performance achieved by the aggregated link itself is then presented, including synchronisation and

latency.

The results of a separate demonstrator to evaluate the performance of a higher layer split LTE link

over point‐to‐point fibre and PON is presented.

Moreover, the performance of the mobile cloud (Unified Communications) use‐cases, and the

results obtained when these are supported by the LTE link operating over the 100G aggregated

fronthaul are presented.

Finally, the results achieved in all the experiments and their relation to KPIs are summarised, and it is

concluded that they support the assertion that the iCIRRUS concept is valid and has potential for

deployment in 5G networks.


of 75




Indexofterms

3GPP 3rd Generation Partnership Project

5G 5th Generation Mobile Radio Technology

BBU Baseband Unit

BER Bit Error Rate

BS Base Station

BTS Base Station

BW Bandwidth

CAPEX Capital Expense

CFO Carrier Frequency Offset

CO Central Office

CoMP Coordinated MultiPoint

CoS Class of Service

CPRI Common Public Radio Interface

CPRIoEth CPRI over Ethernet

CPU Central Processing Unit

C‐RAN Cloud Radio Access Network

C‐SON Centralised SON

CU Central Unit (abstraction of BBU pool for alternative functional splits)

D2D Device‐to‐Device

D2I Device‐to‐Infrastructure

DAC Digital to Analogue Converter

DBA Dynamic Bandwidth Allocation

DL Downlink

DSCP Differentiated Services Code Point

D‐SON Distributed SON

DSP Digital Signal Processing

DSPLL Digital Signal Phase Locked Loop

DU Digital Unit

eMBB enhanced Mobile Broadband (5G use‐case)

eNB Enhanced Node B (radio base station unit for LTE)

EPC Evolved Packet Core

F1 3GPP NR interface between CU and DU

FAX Facsimile

FDV Frame Delay Variation

FEC Forward Error Correction

FH Fronthaul

FIFO First In, First‐Out

FLR Frame Loss Rate


of 75




FMC Fixed Mobile Convergence

FPGA Field Programmable Gate Array

FPU Floating Point Unit

FTTH Fibre to the home

FTTx Fibre to the x

GMC Grand Master Clock

GNSS Global Navigation Satellite System

G‐PON Gigabit Passive Optical Network

GPS Global Positioning System

GST Guaranteed Service Traffic

HARQ Hybrid Automatic Repeat Request

HetNet Heterogeneous Network

HLS Higher Layer Split

H‐SON Hybrid SON

ICT Information and Communication Technology

ID Identifier

IEEE 1588 Ethernet timing protocol

IETF Internet Engineering Task Force

IF Intermediate Frequency

IFDV Inter Frame Delay Variation

IP Internet Protocol

IPDV Inter Packet Delay Variation

iRRH Intelligent Remote Radio Head

IT Information Technology

KPI Key Performance Indicator

LLS Lower Layer Split

LTE Long Term Evolution

MAC Media Access Control

MB Mobile Cloud

MCS Modulation and Coding Scheme

MEC Mobile Edge Computing / Multiple access Edge Computing

MIMO Multiple‐Input Multiple‐Output

MME Mobility Management Entity

MMW Millimetre Wave

NGFI Next‐Generation Fronthaul Interface

NIC Network Interface Card

NID Network Interface Device

NOC Network Operations Centre

NR 5G New Radio

ns Nano second 10‐9


of 75




OAI Open Air Interface

OAM Operations, Administration and Management

OC Ordinary Clock (may be master or slave)

ODN Optical Distribution Network

OLT Optical Line Termination

OMC Operations and Maintenance Centre

OPEX Operating Expense

OS Operating System

OSS Operation Support System

OWD One Way Delay

PDCP Packet Data Convergence Protocol

PDG Packet Delivery Gateway

PDV Packet Delay Variation

PGW Packet Gateway

PHY Physical (Layer)

PLL Phase‐Locked Loop

PM Performance Management

PNF Physical Network Function

PoC Proof of Concept

PON Passive Optical Network

PRC Primary Clock

PRE Packet Routing Engine

PtMP Point‐to‐Multi‐Point

PtMP Point‐to‐Multipoint

PtP Point‐To‐Point

PTP Precision Time Protocol

QAM Quadrature Amplitude Modulation

QoE Quality of Experience

QoS Quality of Service

RACH Random Access Channel

RAN Radio Access Network

RAR RACH Response

RAT Radio Access Technology

RAU Radio Aggregation Unit

RCC Radio Cloud Centre

RE Radio Equipment

RF Radio Frequency

RFC Request for Comment (IETF)

RLC Radio Link Control

RN Radio Network


of 75




RoE Radio over Ethernet

RRC Radio Resource Control

RRH Remote Radio Head

RRS Radio Remote System

RRU Remote Radio Unit

RTD Round Trip Delay

RTMP Real Time Messaging Protocol

RTT Round Trip Time

RU Radio Unit (abstraction of RRH for alternative functional splits)

RU Remote Unit

RX Receiver

SDN Software Defined Network

SFP Small Form‐factor Pluggable

SI System Information

SLA Service Level Agreement

SM Statistically Multiplexed (traffic)

SMS Short Message Service

SON Self‐Optimising Network

SP Strict Priority

STD Standard Deviation

SyncE Synchronous Ethernet

TCP Transmission Control Protocol

TCO Total Cost of Ownership

TDD Time Division Duplex

ToD Time of Day

ToR Top of Rack

TSN Time‐Sensitive Networking

TX Transmitter

UC Unified Communication

UDP User Datagram Protocol

UE User Equipment

UL Uplink

URLLC Ultra‐reliable and low‐latency communications

vBBU Virtual Baseband Unit

VIP Very Important Person

VLAN Virtual Local Area Network

VNF Virtualised Network Function

vNIC Virtual NIC

WDM Wavelength Division Multiplexing

WRR Weight Round Robin


of 75




X2 Interface between eNB


of 75




Contents

1 Introduction .................................................................................................................................. 12

2 iCIRRUS use‐case and KPI review .................................................................................................. 13

Use cases and societal/business KPIs ................................................................................... 13

Ethernet evolved‐fronthaul use cases .......................................................................... 14

Unified Communications and Mobile Cloud/Clone use cases ...................................... 15

Device‐to‐Device (D2D) use case .................................................................................. 18

Performance KPIs and SLA overview .................................................................................... 19

KPIs for evolved‐fronthaul use‐cases ............................................................................ 19

KPIs for fronthaul plus RAN optimisation use‐case ...................................................... 21

KPIs for mobile cloud use‐case ..................................................................................... 21

KPIs for D2D communication ........................................................................................ 22

Architecture overview ........................................................................................................... 25

3 Verification through test results and analysis .............................................................................. 26

CPRI over Ethernet transport test and verification in live LTE network ............................... 27

Real‐time fronthaul platform using 60GHz wireless ............................................................. 29

Software‐based‐DU radio‐over‐Ethernet .............................................................................. 31

Testbed integration with sync‐enabled aggregator/switch .................................................. 34

Video streaming in C‐RAN with mobile cloud ....................................................................... 38

Mobile task offloading in C‐RAN with mobile cloud ............................................................. 41

Network controlled D2D use‐case ........................................................................................ 44

Ethernet transport lab tests of new higher‐layer functional split fronthaul ........................ 47

4 Description and evaluation of the final Showcase demonstrator ................................................ 49

Description of the Showcase demonstrator ......................................................................... 49

CPRI over Ethernet test and verification in the Showcase ................................................... 52

Real‐time fronthaul platform using 60GHz wireless with UPPER‐PHY LLS verification in the

Showcase .......................................................................................................................................... 55

Reference measurement (fronthaul link only).............................................................. 55

Inclusion of aggregator switches into 60GHz fronthaul link ......................................... 56


of 75




Inclusion of 3km installed fibre into fronthaul link ....................................................... 58

Software‐based‐DU radio over Ethernet with MAC‐PHY LLS validation in the Showcase .... 60

Femto‐cell fronthaul/backhaul aggregation validation in the Showcase ............................. 65

Aggregated Ethernet‐based fronthaul network validation in the Showcase ....................... 66

Evaluation of the Showcase demonstrator ........................................................................... 67

5 Conclusion ..................................................................................................................................... 70

6 References .................................................................................................................................... 71

7 List of figures ................................................................................................................................. 73

8 List of tables .................................................................................................................................. 75


of 75




1 IntroductionThe iCIRRUS project pioneered the proposed use of Ethernet in the fronthaul, and the use of the

intelligent monitoring of this fronthaul, largely enabled by the adoption of Ethernet‐based

technology, to enhance service provision. Of course, in the time since the inception of these ideas,

there has been a large effort globally on 5G, including in the European 5G PPP projects. With these

major developments, it has been necessary to continually place the iCIRRUS effort in context – in

order that it can positively make its own distinct contributions within European and global research

and innovation and standardisation efforts. For example, the iCIRRUS project proposal idea on using

“midhaul”, continued to develop and became part of the efforts in several standards bodies of

defining new Radio Access Network (RAN) functional splits, while intelligent control of these splits

became part of the global moves for 5G of functional virtualisation, orchestration and software‐

defined networking. Similarly, the proposals for using network intelligence for mobile “clone”

operation and offloading processing from mobile devices, and/or offloading traffic from the

network, can be seen in the context of 5G proposals for increased use of Mobile Edge Computing.

These examples demonstrate that iCIRRUS has retained relevance in the rapidly developing 5G

landscape, and it has made significant inputs by focussing on specific challenges (for example,

Ethernet, rather than the full scope of possible fronthaul technologies). Of course, to demonstrate

the significance of these inputs it is necessary to verify performance and, if possible, validate the

fundamental concepts that have been proposed. In order to do this, in this deliverable report, we

examine performance verification in terms of some of the key performance indicators (KPIs) that

have been proposed for the iCIRRUS architecture, in iCIRRUS deliverables D2.1 [2],D2.2 [3], D3.3 [4]

and D3.4 [5] . Further, to put our work in the context of the overall European 5G effort, we relate

these KPIs to those defined by the Euro‐5G project for the 5G PPP [1].

While the primary target has been to verify KPIs, and evaluate these in the context of 5G PPP KPIs to

provide a validation of the iCIRRUS concepts, it has also been important for the iCIRRUS project to

build a final, showcase demonstration. The integration of different, specific technological solutions

that is necessary for such a demonstration also provides a further validation of the architecture,

showing that one technological solution is not a hindrance to another, rather that they are

complementary. Due to verification experiments being available at different times during the

project, and, sometimes unforeseen, difficulties in putting theory and concepts into practice or in

transporting experiments to the final demonstration site in Telekom Slovenije’s labs in Ljubljana, the

showcase demonstration does not attempt to integrate every technological research advance made

in the project; it does, however, integrate both a range of fronthaul technologies over Ethernet, and

cloud/clone services.

In the following Section, the use‐cases explored in iCIRRUS are introduced, the societal and business

KPIs are considered and the performance the KPIs associated with validating their performance are

described, and the interfaces on which measurements are made to determine the KPIs are

illustrated with reference to the iCIRRUS high‐level architecture. The iCIRRUS project’s KPIs are


of 75




related to 5G PPP KPIs [1]. In Section 3, the test measurement results obtained to verify the

performance KPIs are described. Section 4 presents an overall evaluation of the showcase

demonstration and the verification results obtained, with a view to validating the iCIRRUS

architecture. Conclusions on the work are presented in Section 5.

2 iCIRRUSuse‐caseandKPIreviewIn this Section, we examine the use cases for the deployment of an iCIRRUS‐type network together

with a comparison with business and societal KPIs and the performance KPIs that have been

targeted by the project and their relation to KPIs defined by 5G PPP. Additionally, the interfaces and

resource clouds on which the quantitative performance KPIs may be measured are considered by

reference to the high‐level iCIRRUS architecture.

Usecasesandsocietal/businessKPIs5G PPP has identified several business and societal KPIs [1]. The first two business KPIs are specific to

the 5G PPP initiative. The only contribution that iCIRRUS can make is indirect through cooperation

with 5G PPP projects (which has been done in several cases, such as joint workshop coordination

and joint contributions to book chapters and other publications). The third business KPI (B3) targets

the market share for EU‐headquartered ICT companies for 5G equipment and services. This is seen

to be related to the third societal KPI (S3) on the “European availability of a competitive industrial

offer for 5G systems and technologies”. The 5G‐Euro project states that there are difficulties in

assessing these KPIs. However, among the assessment measures proposed are (1) the number of

proof of concept demonstrations, (2) contributions to standards, (3) patents, and (4) the costs of

ownership (TCO) and rollout for different coverage targets for operators when using the 5G

technologies developed.

The other societal KPIs include: “Enabling advanced user privacy” (S1); “Reduction of energy

consumption per service…” (S2); “Stimulation of new economically‐viable services of high societal

value…” (S4) and “Establishment and availability of 5G skills development curricula (in partnership

with the EIT)” (S5). In iCIRRUS, the reduction of energy consumption is certainly an important target.

5G‐Euro states that it is a difficult KPI to measure and suggests that it could be done through proxies,

such as spectrum efficiency, specific protocols to switch devices/cells off and energy consumption

reduction in hardware. The other societal KPI with relevance to iCIRRUS is that for the stimulation of

new services.

In our overview of these KPIs for the iCIRRUS project, we examine in turn three areas for the work in

the following sub‐sections.


of 75




Ethernetevolved‐fronthaulusecasesAs has been stated in [2] and [5], the Ethernet evolved‐fronthaul use‐case has several drivers: the

desire to exploit the ubiquity and potential for low‐cost implementation and management of

Ethernet; the need to create a fronthaul solution that is scalable to the throughputs required for the

5G enhanced mobile broadband (eMBB) use‐case; and the desire to exploit statistical multiplexing

for greater transport network efficiency (of use and of energy) . Early work in iCIRRUS and other

projects made it clear that strategies based solely on the transport of time‐domain IQ samples of the

radio baseband waveforms do not scale in an economically efficient manner; however, there still

exist use cases where such transport is required, due to backward‐compatibility with legacy

equipment and to the fact that new functional splits in the RAN do come with potential performance

penalties. The drivers for the new Ethernet evolved‐fronthaul can be summarised as follows:

Reduce fronthaul transport deployment costs

Reduce fronthaul transport management costs

Exploit the ubiquity of Ethernet

Minimise the need for extra cost and complexity in supporting fronthaul requirements

Reduce TCO and rollout costs for different coverage targets for operators

These can be seen very clearly to contribute to the business and societal KPIs (B3, S3) for reducing

cost of ownership and deployment costs. Our work in D2.3 [6] presented an analysis of technologies

to support low cost initial roll‐out of 5G. Factors considered included the expected increase in traffic

demand driven by video services, network sharing, and that the highest bandwidths are not required

to be serviced uniformly across the geography, according to subscriber modelling by Orange,

Telekom Slovenije and PrimeTel. Re‐purposing of existing fronthaul with PtP fibre and PON, and low‐

cost 100G links were addressed and system costs estimated. Also, business cases to estimate the

payback period for deployment of D2D and clone‐based technologies were developed that indicated

payback within 3 years.

The fronthaul work has also contributed to the following standards:

The iCIRRUS evolved fronthaul architecture and concepts have been presented both directly

and indirectly (through discussions during meetings) in several IEEE 1914 working group

meetings.

Many iCIRRUS concepts, and initial implementation have been presented in the

OpenAirInterface software alliance.

ETSI Zero Touch Service and network Management (ZSM) “RAN SON ZSM input iCIRRUS” [7]

Presented iCIRRUS concepts joint fronthaul‐RAN SON. The group requested this material is

used in work items on slicing, developing user stories and use cases and to help align

nomenclature.


of 75




The following patents have been filed by iCIRRUS partners in relation to the fronthaul work:

“Enhancing Network Topology Information for a Self‐Organising Network,” [8]

“Packet colouring for data stream monitoring,” [9]

“Techniques for providing front‐haul data awareness” [10]

“Method and apparatus for mitigation of packet delay variation,” [11]

Finally, the iCIRRUS fronthaul work has contributed to the business/societal 5G PPP KPIs related to

European proof‐of‐concept demonstrations. While individual partners have focussed on testbeds to

examine performance KPIs, the final showcase demonstration of the project integrating the work of

individual partners in Telekom Slovenije’s lab on Ljubljana is a major proof‐of‐concept for fronthaul

integration. The showcase demonstration integrates transport for two lower‐layer RAN functional

split points (one in upper PHY layer for a wide, 5G‐like bandwidth 60GHz system, the other at the

MAC‐PHY interface in an LTE system), for generic and CPRI IQ waveform transport, and for backhaul.

All are passed through a high‐speed aggregator with timing/synchronisation signals and using time‐

sensitive networking. In addition, an earlier proof‐of‐concept demonstrator was made in Orange’s

laboratory in Lannion, demonstrating a higher‐layer functional split (PDCP‐RLC split in LTE).

The performance KPIs for each of the fronthaul testbeds and the showcase demonstration are

discussed in Section 2.2.

UnifiedCommunicationsandMobileCloud/CloneusecasesLet us first consider what is generally understood by the term Unified Communications (UC):

“Unified communications (UC) is a business term describing the integration of enterprise

communication services such as instant messaging (chat), presence information, voice

(including IP telephony), mobility features (including extension mobility and single number

reach), audio, web & video conferencing, fixed‐mobile convergence (FMC), desktop sharing,

data sharing (including web connected electronic interactive whiteboards), call control and

speech recognition with non‐real‐time communication services such as unified messaging

(integrated voicemail, e‐mail, SMS and fax). UC is not necessarily a single product, but a set

of products that provides a consistent unified user interface and user experience across

multiple devices and media types” [12].

Consequently, from the business point of view it is appropriate that its performance measures and

goals are directed towards improving business targets such as competitiveness, excellence in the

attention to clients, and increase in network efficiency and effectiveness, while minimizing the

consumption of resources.


of 75




In the framework of ICT, the UC combines all the communications possibilities onto one platform,

instead of having them scattered and separate. UC is not a concrete product: Unified

Communications is a basic and strategic solution for any company. Some studies have revealed that

more than 60% of the employees of a company manage different communications devices. Having

to manage different media on different platforms considerably reduces the effectiveness of

communications within a company.

Unified Communications unifies each means of communication in a company (documents exchange,

calls, message, emails, etc.). These are grouped in an intelligent manner, increasing user

productivity, and offering better solutions with better quality that improves user’ experience. In

addition, the UC platform offers ubiquitous accessibility, so that UC systems may be provided for any

business model, regardless of its size, resources and location.

Concerning the iCIRRUS objective, this use case can be considered as a proof of concept that adds

value to 5G networking through communications in the cloud. The mobile cloud is also an added

value to the UC because it offers elasticity and agility, with companies able to scale up and down the

functionalities on offer, as needed. The potential advantages offered by UC with the mobile cloud in

comparison with traditional hosted services include:

Reduce IT infrastructure cost

Increase business elasticity and scalability: provisioning the resources as needed

Standards and flexibility

Multi‐tenancy

Disaster recovery

In this sense, the UC system provided by Wellness Telecom benefits from the iCIRRUS project with

the indirect benefits of a 5G communication system. In a voice and real‐time application, the

advantages presented by iCIRRUS are experienced by users in the QoE. To see the impact of this

pilot, different UC platforms are compared and the final value proposition of this use case is

presented.

Multiple platforms are available in the market, with a selection of these being summarised in Table

2‐1. As the feature terminology is not standardised, the Table can only provide a rough comparison

of feature comparability.


of 75




presence

chat

fax

voicemail

email

IM

Video

bridge

Meetings

Documen

t

man

agement

IP telephony

fixed m

obile

integration

Platform

sm

artphone

Platform

PC

Cloud support of

application

Elastix1 x x x x x X x

Cisco2 x x x x X X x

Mitel3 x x x X x x

Nortel4 x x x X x x x

Innova‐phone5 x x x x X x x

Telefonica6 x x x X x x X x

WT iCIRRUS x x x X x X x X

Table 2‐1 Comparison of Unified Communication offerings

Within iCIRRUS two mobile cloud/clone use cases were integrated over the fronthaul of the

showcase demonstrator: the “Video streaming in C‐RAN with mobile cloud,” and “Mobile task

offloading in C‐RAN with mobile cloud.” The associated performance KPIs are described in Section

2.2. The integration of these service‐type technologies with the lower‐layer networking technologies

of the fronthaul is a further indication of the value of the iCIRRUS showcase demonstration. These

are illustrated in Table 2‐1 in comparison with the other UC offerings, illustrating how use of cloud

support of applications by iCIRRUS is a differentiator that may help deliver 5G KPIs.

UC is indispensable in the business world, improving reachability, increasing efficiency and,

accelerating business process. However, such a UC service tends to consume a great quantity of the

resources available in a device, eg processing large amounts of video and audio data. The iCIRRUS

system directly benefits Unified Communications using the Mobile Cloud with the potential to

improve the QoS in two ways:

Improving the provisioning of voice and video quality in a 5G network by reducing latency

and increasing bandwidth was proved in the offloading task in D5.3 [13].

The consumption of resources in the devices is reduced because most of the computing

workload is carried out in the cloud (as explained later when examining the performance

KPIs).

1 https://www.elastix.org/.December 2017 2 https://www.cisco.com/ December 2017 3 http://www.mitel.com/ December 2017 4 http://www.minitel.ca/Files/CMFiles/46UnifiiedCommunications.pdf December 2017 5 http://www.innovaphone.com/en/produkte-unified-communications.html December 2017 6 http://www.movistar.es/grandes-empresas/soluciones/fichas/comunicaciones-unificadas/ December 2017


of 75




The Wellness Telecom UC service offering can certainly be an innovative, and competitive 5G service

(S4): WT expect benefits from iCIRRUS with this UC pilot of 70.000 € by 2020. There are aspects of

what can be provided, for example, disaster‐recovery, that undoubtedly have high societal value.

The media mixer implemented and demonstrated in D5.3 [2] processed all the video and image data,

resulting in improved battery life of the devices. Thus, there is a contribution to energy consumption

reduction and the 2nd societal KPI “reduction of energy consumption per service” [1] (S2). The

application demonstration proved the reliability of the 5G communications network connected with

the Mobile Cloud, one of the aims of the iCIRRUS project, and contributing also to the service

viability societal KPI (S4).

Contributions to standards and patents, were not considered appropriate to the chosen business

model which is to deliver the product via 3rd party software.

Device‐to‐Device(D2D)usecaseDevice‐to‐device (D2D) communications will serve to meet business/societal KPIs in the following

ways:

Improved spectrum and energy use for operators (S2)

Improved energy efficiency for user devices (S2)

Stimulation of new service offerings (S4)

Extension of coverage to facilitate reduced TCO (S3) and attainment of reliability and

accessibility KPIs

The work in iCIRRUS has led to filing of a patent application:

“High resolution angle of arrival estimation and dynamic beam nulling”, [14]

which discloses methods for establishing very high throughput communication links between

devices using novel millimetre wave signal angle of arrival estimation. The application also provides

practical procedures to minimize interference between devices using transmitter side beam

optimization techniques. The patent follows work done in the IEEE 802.11ay standards group on

millimetre wave channel characterisation. IDCC envisage this work as building on their previous D2D

communication platform operating at microwave/RF.

In iCIRRUS, the aim was that the above benefits would be further enhanced by examining

possibilities for using the remote radio units in D2D functions, such as enhancing user device

localisation and, with higher layer functional splits, moving decision‐making for D2D offloading from

the mobile network closer to the users, allowing it to be more responsive. Performance KPIs for D2D

are examined in Section 2.2.


of 75




PerformanceKPIsandSLAoverviewThe following Sections outline the performance KPIs for the iCIRRUS architecture, taking the

evolved‐fronthaul use cases, the mobile cloud/clone and D2D use cases in turn. KPIs for the joint

optimisation of the fronthaul and mobile RAN are also considered.

From Euro‐5G/5G PPP [1], 4 over‐arching performance KPIs were identified.

The first was in “providing 1000 times higher wireless area capacity and more varied service

capabilities”. Euro‐5G translated this into two distinct KPIs:

P1a – absorbing Tb/s traffic in a smart office (equivalent to 10Tb/s/km2 wireless area

capacity)

P1b – reaching a peak user data rate of between 1 and 10Gb/s for specific scenarios/use

cases

The second KPI was on reducing the average service creation time to 90 minutes. Euro‐5G proposed

to monitor the following for this:

P2a – level of automation of service processes

P2b – existence/range of autonomic network management functions

P2c – use of Open Source, SDKs etc., to facilitate deployments

The third performance KPI was on facilitating very dense deployments for large numbers of users

(people and devices). These were translated to:

P3a – handling up to 1 million devices per km2 in certain scenarios/use cases

P3b – facilitating deployment and operation of a large number of small cells (tens of sites

per km2)

The fourth performance KPI was in providing a secure, reliable and dependable internet. This was

translated to:

P4a – providing a latency of between 1 and 10ms (1 ms being for the air interface link)

P4b – reaching reliability >99% or more (in terms of packets delivered within latency budget)

P4c – reach an availability >99% (in terms of users being covered by a 5G service)

KPIsforevolved‐fronthauluse‐casesOne of the primary transport related use‐cases that iCIRRUS has investigated is that of the evolved‐

fronthaul for a RAN with different functional splits. Table 2‐2, adapted from D3.4 [5] with added


of 75




columns to include the backhaul case and the 5G KPIs, is a compilation of the fronthaul KPIs for the

different use‐cases that are, respectively: transport of legacy CPRI; two examples7 of transport with

a low‐layer‐split (LLS) with and without CoMP support; and an example of a higher‐layer‐split (HLS).

With the concurrent development of standards and quasi‐standards during the unfolding of the

iCIRRUS project, these respective use‐cases may be related with reasonable accuracy to

(1) the LLS that is being addressed by the eCPRI industry “standard,” that is designed to target

requirements for the LLS.

(2) the LLS split options that are being discussed within 3GPP

(3) the HLS that is being standardised by the 3GPP as the f18 interface between the centralised

unit (CU) and distributed unit (DU).

Different types of traffic supported by the evolved‐fronthaul

Fronthaul Requirements

Legacy traffic (CPRI

Upper PHY split in DL and UL (no CoMP)

Upper PHY split in DL lower PHY split in UL (CoMP)1

PDCP‐RLC split Backhaul (unified transport)

5G KPIs

User Data rate 100‐400Gbps

(P1) Area coverage

(P1a) 10TB/s/km2

(P1b) Per user 1‐10GB/s

FH data rate

User Latency (P4a) 1‐10ms 9

Max latency (round trip delay)

150µs (CoMP) 440µs (no CoMP)

440µs 150µs 60ms 60ms

Min frequency accuracy

+/‐ 2ppb (per hop)




Min phase and timing accuracy

+/‐ 10ns (MIMO & TX diversity)

+/‐ 30ns +/‐ 30ns

TBD (expected approx. +/‐

1.36s (LTE TDD)

Max latency imbalance

+/‐ 16ns +/‐ 163ns +/‐ 163ns Not yet defined

Max BER 10‐12 10‐6

10‐3 non‐priority data 10‐5

priority data

[1] The MAC‐PHY split has similar KPI requirements so they are not separately presented here

Table 2‐2 Compilation of Fronthaul KPI

7 One option is MAC-PHY and the other is UPPER-PHY 8 A parallel interface is being standardised for 4.5G called the v1 interface 9 1ms being for the air-interface link. User plane 1-way delay <4ms eMBB, <1ms URLLC


of 75




Table 2‐2 defines the KPIs that are applicable for the Ethernet fronthaul scenarios in Section 2.1.1,

including the transport of CPRI, HLS and LLS, and femto‐cell backhaul and by extension to any

aggregation in a 100Gb/s+ fronthaul scenario that would still have to meet these requirements. In

terms of higher level performance KPIs, fronthaul KPIs impact particularly on latency considerations.

The Euro‐5G document gives examples of processing delays, synchronisation and retransmissions as

steps that need to be quantified within User Plane latency as an illustration of performance

assessments that would have to be carried out.

KPIsforfronthaulplusRANoptimisationuse‐caseAs an extension of the evolved‐fronthaul use‐case, iCIRRUS has also addressed the use‐case for joint

optimisation of the fronthaul and RAN. This required an increase in the number of KPIs that should

be monitored; however, these are RAN performance indicators D3.3 [4], rather than minimum

acceptable criteria for operation, as in Section 2.2.1.

KPI Definition 5G KPI target

Accessibility Success rate of access attempts to the system

(P4a) >99%

Retainability 1 ‐ dropped call rate (P4b) >99%, URLLC 99.999%10

Throughput Median and 95th percentile throughput

(P1b) 1‐10GB/s

Application performance Per application, eg mean opinion score for voice

Fronthaul redundancy Measure of available alternative paths

Figure 2‐1 KPIs for fronthaul and RAN joint optimisation

These system‐level KPI goals are not easy to quantify directly without a large‐scale system simulation

or wide‐area deployment. However, the “retainability” KPI, which is a common performance KPI in

today’s mobile networks that stems from a circuit‐switched view of the world, can be mapped to

“reliability.” As, “reliability,” is defined as successful delivery percentage of packets within a latency

constraint, it allows us to relate the KPI, to some extent, to achievement of the enhanced fronthaul

latency KPIs. Additionally, it is useful to state these KPIs here, to provide context for the ultimate

goals of an iCIRRUS derived system deployed in a commercial network.

KPIsformobileclouduse‐caseThe mobile cloud use‐case in the iCIRRUS project is explored with two proof‐of‐concept (PoC)

applications as examples of possible components in an enhanced UC offering: video streaming

supported by the mobile cloud, and mobile application offloading supported by the mobile cloud.

10 Packets delivered successfully within the required latency


of 75




These use‐cases can be mapped to the business KPI B3 to the extent that they have potential to

impact TCO and there is a direct mapping to societal 5G KPIs related to S4 “stimulation of new

economically‐viable services…” and S2 “the reduction of energy consumption…” The mobile cloud

defined for these services has linkage to the 5G performance KPI P2 “reducing the average service

creation time cycle”

KPIsforD2DcommunicationD2D communication in iCIRRUS allows two devices in proximity to set up a direct D2D transmission

link underlaying the iCIRRUS network infrastructure, allowing offloading of traffic from this

infrastructure. However, to guarantee the quality of service (QoS) of D2D communications,

significant assistance is required from the C‐RAN architecture, including D2D link set up and resource

management. According to the heterogeneity of D2D communication, in which D2D communication

is a potential alternative to cellular communication, strict numeric KPIs may vary for different use

cases. However, several KPIs could be used to demonstrate the requirements and benefits of D2D

communication in the iCIRRUS architecture, as shown in Table 2‐3 and Table 2‐4. In previous works,

reported in D5.3, the tests for D2D set up along with the real‐time video transmission via D2D

communication were discussed [13].

KPI for D2D link set up Definition 5G KPI target

Set up latency The time consumption of the signalling overheads between D2D pair and BBU pool via fronthaul.

(P4a) CP latency < 10ms UP latency 1‐10ms

Discovery range The maximum transmit‐power of pilot signal, which determines the maximum D2D set up range.

(P3a) 10,000‐1,000,000 per km2 devices

Discovery efficiency The ratio of the number of successful D2D set‐ups to the number of D2D requests.

(P3a) 10,000‐1,000,000 per km2 devices, (P4b) >99%

Table 2‐3 KPIs for D2D link set up

D2D link set‐up in iCIRRUS is related to 5G KPIs for control plane. The detailed explanation of KPIs for

D2D link set‐up are shown below:

1. Set‐up latency: From the D2D set‐up framework mentioned in D2.2 [15], several handshakes

are required between the D2D pairs and BBU pool. For each handshake, the delay between

UE and BBU pool significantly impacts the D2D link set‐up latency; hence the number of

handshakes should be limited to guarantee the QoS of D2D communication. In iCIRRUS, a

location based D2D discovery and link set‐up strategy has been proposed. Compared to the

standard discovery strategy which involves the communications with the BBU pool at least

three times and includes 7 or more steps, our proposed strategy only needs to communicate


of 75




with the BBU pool once and has only 4 steps for D2D link set‐up. It makes our contribution

related to the 5G KPI P4a on “latency.”

2. Discovery range: The maximum transmit power of the pilot signal of the user device

determines the maximum D2D set‐up range. It could be adjusted dynamically according to

the specific service requirements of D2D communication, eg data rate, latency, etc.

3. Discovery efficiency: A key performance metric to evaluate the discovery strategy is the

successful set‐up ratio, which leads our contributions to the 5G KPI P4b “reliability.”

Furthermore, for real‐time D2D communication, the D2D discovery protocol proposed in

iCIRRUS is an asynchronous one. As the locations of UEs can be proactively estimated at

their access RRSs, the resource blocks (RBs) and power allocation for pilot transmission,

which is used for D2D link set‐up, could be optimized to increase the potential matching of

D2D pairs. Meanwhile, the spectral and energy efficiency of the discovery process can be

improved.

KPI for D2D communication Definition 5G KPI target

UE data rate Data rate received at the

receiver of D2D pair in bits per second.

(P1b) reaching a peak user data rate of between 1 and

10Gb/s

Transmission efficiency The packet delay and loss rate

between two devices.

Energy efficiency Power consumption at each

device in bits per joule.

Resource management latency The time consumption of one signalling packet for D2D resource management.

(P4a) CP latency < 10ms UP latency 1‐10ms

Table 2‐4 KPIs for D2D communication

D2D communication in iCIRRUS is related to 5G KPIs for both data plane and control plane. The

detailed explanation of KPIs for D2D communication in iCIRRUS are shown below:

1. UE data rate: Due to the short transmit distance of D2D communication, the received data

rate can be greatly enhanced. Therefore, D2D communication can be used for large volume

content delivery, such as high‐quality video transmission. The proposed semi‐distributed

resource management in iCIRRUS could improve the D2D data rate by several times through

the introduction of frequency reuse in a BBU’s coverage area, which can contribute to the

5G KPI P1b.

2. Transmission efficiency: The packet delay and loss rate of D2D communication are much

lower than in cellular communication, which makes D2D a vital network component to

enable some latency‐critical applications, eg online real‐time gaming, virtual reality, mobile


of 75




edge computing, etc, that may contribute to the 5G societal KPI to simulate new services S4

as well as latency with P4a.

3. Energy efficiency: With direct data transmission between two devices, the transmit power at

transmitter could be greatly reduced to achieve desired QoS in various use cases, which

contributes to societal 5G KPI S2.

4. Latency for resource management: It is essential to explore network assisted resource

management for the D2D communications to achieve optimal network performance, eg

throughput, outage probability, etc. Therefore, the duration of the signalling between D2D

pairs and BBU pool might exceed the maximum delay requirement of some real‐time

services. In iCIRRUS, it is considered that some of the MAC layer function blocks for D2D

resource allocation is shifted from the BBU pool to RRHs through the functional splitting in

the fronthaul, thus reducing significantly the resource management latency, which

contributes to the 5G KPI P4a.


of 75




ArchitectureoverviewThis Section maps the quantitative KPIs introduced in the previous Sections to the interfaces and

resource clouds of the iCIRRUS conceptual architecture, illustrated in Figure 2‐2, at which these

quantitative KPIs values may be measured. The actual means of determining the KPIs is described in

Section 2.1.

Figure 2‐2 iCIRRUS architecture overview

Considering the evolved‐fronthaul use‐cases outlined in Section 2.1.1,

IQ Transport Fronthaul, is shown in Figure 2‐2, emanating from the BBU pool, conveyed via

one or more Ethernet switches and delivered to the RRHs towards the top right of the

Figure. The respective end‐points for measuring the KPIs set out in Table 2‐2 are the BBU

pool and the concerned RRH.

NGFI‐type fronthaul, is shown in Figure 2‐2, emanating from the RCC pool, conveyed via one

or more Ethernet switches and delivered to the RRSs (which may themselves be split

between RAU and RRU) towards the bottom left of the Figure, the respective end‐points for

measuring the KPIs set out in Table 2‐2 are the RRC pool and the concerned RRS/RRU. The

Figure does not show the details of the functional split in the radio so LLS and HLS use‐cases


of 75




are not distinguished. The respective end‐points for measuring the KPIs set out in Table 2‐2

are the RCC pool and the concerned RRS/RRU.

Backhaul aggregation with fronthaul, is shown in Figure 2‐2, with backhaul emanating from

the RCC pool and conveyed via one or more Ethernet switches through the “backhaul to

core”. The end‐points for measuring KPIs are the RCC pool and the core network.

Aggregation of “midhaul” between the micro BTS at the bottom of the Figure and the RCC

pool with fronthaul and backhaul is also shown. The respective end‐points for measuring the

KPIs set out in Table 2‐2 are the Micro‐BTS and the RCC pool.

The aggregated Ethernet fronthaul is represented by the vertical trunks that interconnect

the Ethernet switches and the Ethernet links connecting Ethernet switches to the RCC pool

in Figure 2‐2. The performance of the aggregated fronthaul is verified by the simultaneous

meeting of the KPIs of the component use‐cases.

Considering the Unified Communication example use‐cases examples outlined in Section 2.1.2,

The mapping of the UC use‐case components to the elements of the iCIRRUS architecture in

Figure 2‐2, is the same for both “Video streaming in C‐RAN with mobile cloud,” and “Mobile

task offloading in C‐RAN with mobile cloud.” The application component is hosted in the

mobile cloud, that is in MB1 or MB2 on the cloud side and in the mobile device itself on the

other. The real‐time telemetry framework which functions as a virtual infrastructure

manager (VIM) for collecting the KPIs in these component in the Mobile Cloud is described

respectively in Sections 4.6 and framework collecting and storing phone related KPIs is

described in Section 3.5.

Considering the network controlled Device‐to‐Device example use‐case examples outlined in Section

2.1.3,

The network controlled D2D use‐case components are mapped to the elements of the

iCIRRUS architecture in Figure 2‐2 as follows: the network controlled element maps to the

MAC that may be in the RCC pool. Alternatively, the MAC element dealing with D2D may be

pushed nearer to the network edge at the RAU. The D2D element is mapped to the two or

more mobiles located at the right of the Figure.

3 VerificationthroughtestresultsandanalysisSection 3 summarises the results of the various testbed experiments conducted throughout the

iCIRRUS project and compares them with the iCIRRUS design target KPIs and also validates them in

the broader context set by the 5G business, societal and performance KPIs set out in [1].

Section 4 addresses the Showcase demonstration where a subset of the individual testbeds were

integrated and simultaneously supported on a 100G Ethernet fronthaul link. However, in cases


of 75




where the most complete validation of the individual testbeds was conducted at the same time and

location as the Showcase, those Showcase demonstrator results are reported in Section 3 together

with the individual testbed results. Section 4 then refers back to these results in the broader context

of the Showcase.

CPRIoverEthernettransporttestandverificationinliveLTEnetwork

The starting point of the CPRI to Ethernet test was a direct 10G Ethernet connection established

between 2 FPU Boards, see Figure 3‐1. The VIAVI MTS 5800 CPRI Tester acted as both master and

slave device within the CPRI link, using PRBS31 pattern for reasonable transmission tests. The

transportation of frequency information from master to slave was achieved by a synchronous

Ethernet connection (SyncE). With this, frequency recovering was enabled on the BBU side (recovery

of CPRI Master clock) as well as on the RRH side (recovery of Ethernet clock). The BBU side Ethernet

core was driven by a DSPLL clock derived from the recovered CPRI clock. In a comparable manner,

the RRH side’s CPRI clock was generated from the recovered Ethernet clock. As a result, both BBU

and RRH CPRI core clocks operated effectively at the same frequency, and the difference between

the clock frequency on the master side and on slave side couldn’t be measured. The required

minimum frequency accuracy according to Table 2‐2 was met (± 19.66Hz @9.83Gbit / ± 4.92Hz

@2.45Gbit).

iCirrus FPGA Board ‚BBU Side‘

VIAVI MTS 5800CPRI Tester

CPRI Master

Port

CPRI / Ethernet BridgeClock Unit(DSPLL)

RecoveredCLK CPRI

REF CLK

Eth REF CLK

CPRI Core

10G Eth Core

CPRI SlavePort

BERMeasurement

iCirrus FPGA Board ‚RRH Side‘

CPRI / Ethernet Bridge

Clock Unit(DSPLL)

Recovered CLK

CPRIREF CLK

Eth REF CLK

CPRI Core

10G Eth Core

CPRI CPRI

SYNC Ethernet

Figure 3‐1 Point‐to‐point with SyncE


of 75




Fronthaul KPI Requirement Measured

performance Comment

User Data rate 100‐400Gbps ‐

FH Data Rate

User Latency

Max latency (round trip

delay)


35µs direct link

Exceeds most demanding requirement



Better than (± 4.92Hz

@2.45Gbit)

Actual error was less than measurement capability




+/‐ 16ns Not measured

Max BER Below measurement sensitivity

Table 3‐1 Legacy 2.45Gbit CPRI traffic over Ethernet, requirement v performance


performance Comment

User Data Rate

FH Data rate 100‐400Gbps ‐

User Latency

Max latency (round trip delay)


10µs direct link

Buffer depth set by more latency consuming 2.45Gbit




@9.83Gbit)

Actual error was less than measurement capability




+/‐ 16ns ‐ Not measured

Max BER ‐ Below measurement sensitivity


A comparison with the target KPIs is provided in Table 3‐1 and Table 3‐2. The results indicate that

the CPRI over Ethernet can exceed most of the KPI requirements that were identified. The peak

timing accuracy performance as measured on the aggregated link indicates that performance may

not be sufficient for the most severe MIMO and TX diversity requirements.

The benefit of the presented test results is detailed knowledge of the CPRI over Ethernet functioning

as expected, the upcoming mobile generation of mobile fronthaul is expected be based on Ethernet

transport technology. In the network upgrading phase, the interconnection possibility of different

technologies (backward compatibility) is very important to operators. Consequently, demonstration

of CPRI over Ethernet can be related to the business and societal KPI B3/S3 related to cost of

ownership and rollout of 5G with its support of backward compatibility.

With the knowledge obtained of the CPRI/ETH conversion performance and its technical demands, it

is possible to consider it in all planned network maintenance upgrades (to be prepared in advance).


of 75




In this way, similar converters could be used for 2G, 3G and 4G remnants on the 5G fronthaul

segment.

Real‐timefronthaulplatformusing60GHzwirelessThis Section summarises the test results of the 60GHz fronthaul system and its integration with the

time sensitive Ethernet aggregated fronthaul link in the ADVA lab that was reported in D5.3 [13] and

compares them with the iCIRRUS design target KPIs and also validates them in the broader context

set by the 5G business, societal and performance set out in [1]. The analysis of the results in the

context of the Showcase demo is presented in Section 4.3.

Figure 3‐2: Test setup for Integration of DU, RU, UE and 60GHz

Figure 3‐2 shows the test set‐up used and Figure 3‐3 summarises the results over the link with

different and varying frame sizes. The lowest latency can be observed for the reference

measurement without the aggregator switch, but with the VIAVI Ethernet protocol analyser (yellow

curve) in all cases. The curves for the test series with the aggregator switch with or without PTP

traffic are almost indistinguishable (violet and green) and show a slight increase of the latency with

respect to the reference measurement. The red curve, with a slightly worse latency, shows the

results of the using a commercial switch instead of the aggregator, however, as the DU and RU

firmware was different, the difference is likely not significant. The highest latency is observed for the

setup with the UE device included.


of 75




Figure 3‐3: comparison of all setups for different frame lengths: 86bytes (a), 512bytes (b), 1024bytes (c), 1518bytes (d), random size 86…1518bytes (e)


of 75




Fronthaul KPI

Requirement Upper PHY split in DL and UL (no

CoMP)

Requirement Upper PHY split in DL lower PHY

split in UL (CoMP)

Performance 5G KPI target

User Data rate 100‐400Gbps 100‐400Gbps 2.2Gbit/s (P1b) 1‐10Gbit/s

FH data rate

User Latency (P4a) 1‐10ms on the air

interface


delay) 440µs 150µs

250µs @ 1Gbit/s 1200µs @ 2.5Gbit/s





+/‐ 30ns +/‐ 30ns


+/‐ 163ns +/‐ 163ns

Max BER 10‐12

Table 3‐3 lower layer split at the UPPER PHY, requirement v performance

The results are compared with the iCIRRUS KPI targets and the 5G KPIs in Table 3‐3, and the

performance meets the 5G performance KPI for throughput and air‐interface latency depending on

the scenario.

Furthermore, the testbed can also be seen as contributing to the business goal B3 considering the

number of PoC as a proxy for the 5G readiness of the European telecommunications sector, to the

societal goal S3 considering the assessment of the maturity and performance of the different

technical components and the potential impact to TCO, and to the goal S4 considering the potential

to stimulate new high‐value services given the availability of significant allocated bandwidth in the

millimetre wave band.

Software‐based‐DUradio‐over‐EthernetThis Section summarises the test results of the software‐based‐DU radio over Ethernet system that

was reported in D5.3 [13] and compares them with the iCIRRUS design target KPIs and validates

them in the broader context set by the 5G business, societal and performance set out in [1]. The

analysis of the results in the context of the Showcase demo is presented in Section 4.4.


of 75




Figure 3‐4 Indicative setup for software‐based‐DU radio over Ethernet

Figure 3‐4 is indicative of the set‐up used to generate the results summarised here11. Two sets of

results were generated, the first comprising non‐LTE results and the second based on an LTE system.

The non‐LTE results more fully test the capability of the fronthaul as arbitrary data is sent at user‐

defined sampling rates to and from the RRH. All tests were successful in utilising the available

fronthaul bandwidth without the occurrence of buffer underflows or overflows in the DU. For the

second set of results the SW eNodeB and EPC were both active.

11 Several different set-ups were used and this is exemplary as it includes all elements of interest

DU


of 75




Fronthaul KPI


CoMP)


split in UL (CoMP)


User Data rate 100‐400Gbps 100‐400Gbps 0.05Gbit/s (Non‐LTE) 1

0.02Gbit/s (LTE) 2 (P1b) 1‐10Gbit/s

FH Data rate N/A N/A 0.8Gbit/s (Non‐LTE) 3

0.4Gbit/s (LTE) 4

User latency N/A N/A N/A (P4a) 1‐10ms on the air

interface


delay) 440µs 150µs

40µs (Non‐LTE) 5 266µs (LTE) 5

N/A




N/A N/A


+/‐ 30ns +/‐ 30ns N/A N/A


+/‐ 163ns +/‐ 163ns N/A N/A

Max BER6 10‐12 10‐12 N/A 1. The effective use rate is achieved by scaling the LTE bandwidth by the relative fronthaul rates in D5.3 Table 2‐4 and Table 2‐5 =

800/370 * 20Mbit/s for the X310 radio, D5.3 page 34. 2. This is the user rate from Table 2‐5 for the X310 radio 3. This is the fronthaul data rate in D5.3 table 2‐4 for the X310 radio 4. This is the fronthaul data rate in D5.3 table 2‐5 for the X310 radio 5. The fronthaul latency is a function of the sample width and sample rate, these values are based on the value in D5.3 Figure 24

scaled to the used sample width and sample rate, that is, 173 * 5.76/25 and 173 * 5.76/3.75 6. Performance value based on specification of Ethernet equipment (i.e. not measured)

Table 3‐4 lower layer split at the MAC/PHY, requirement v performance

These tests help to demonstrate different iCIRRUS benefits and innovations, such as the intelligent

processing performed by the intelligence unit to achieve a self‐optimizing network, and data rate

reduction through Functional split(ting) and statistical multiplexing gains, which support the business

goal B3 considering the number PoC, and S3 considering quantified assessment of maturity and

performance of technological components.

In addition, the use of Ethernet in the fronthaul is validated, which is a potentially low‐cost

commodity technology with standardized carrier Class OAM and connection verification facilities. It

can aid the convergence of fronthaul and legacy backhaul (x‐haul), and the insertion of background

traffic was used to aid the evaluation of these convergence scenarios, which supports goal S3

considering TCO.


of 75




Testbedintegrationwithsync‐enabledaggregator/switchBoth local and remote 100G aggregators employed the Guaranteed Service Transport (GST) priority

class (developed by Transpacket12) to establish the PTP link. More details on the GST principle and

implementation can be found in D5.3 [13].

In the integration test, the PTP configuration was the same as that applied in D5.3, and the 1G client

port at the CO was used to forward and broadcast the PTP stream from the GM clock to all the

remote client ports. The PTP analysis at the remote 1G and 10G client ports was performed by the

VIAVI MTS‐5800 tester. Table 2‐1 summarises the PTP link stats for both ports. Note that the PTP

stream egressing from the remote 1G client port was eventually fed into the slave clock (OSA‐5410).

Remote port Master to Slave

delay (µs) Slave to Master

delay (µs) Two‐way

time error (µs)

1G 187.631 188.081 0.225

10G 115.350 115.826 0.238 Table 3‐5 PTP link stats at the remote node

The sync PDV and delay request PDV at the two remote client ports are depicted in Figure 2‐7,

respectively.

1G Port

12 http://www.transpacket.com/


of 75




10G port


of 75




Figure 3‐7 PDV analyses at the remote 1G and 10G ports

Compared to the results in D5.3 [13], apart from the additional propagation delay induced by the

field fibre ring, no other impact was observed in the field integration.

Figure 3‐8 depicts the calculated maximum time interval error (MTIE) and time deviation (TDEV) by

applying a 0.1 Hz first‐order low‐pass filter per ITU‐T G.8271.1 to the measured time error samples.

It can be seen that with the observation interval of wider than 2000s, the MTIE persists at a constant

value of around 110 ns corresponding to a frequency accuracy of less than 55ppt, proving that the

recovered PTP clock at the remote 10G client port was synchronised with the grand master clock

without any frequency offset. Given a shorter observation interval of 10s, the derived frequency

accuracy is then less than 3ppb, however, this value is considered as a pessimistic estimation.


of 75




Figure 3‐8 MTIE/TDEV analysis of PTP time error at remote 10G client port

Table 3‐6 summarises the measured synchronisation performance between two aggregators with

respect to the Fronthaul KPI.


Legacy traffic (CPRI)


Upper PHY split in DL lower PHY split in UL (CoMP)

PDCP‐RLC split

Measured performanc

e Comments






< 3ppb (2 nodes)



+/‐ 30ns +/‐ 30ns

Not yet defined (expected approx.

+/‐ 1.36s (LTE TDD)

Average 100ns

Better accuracies require additional means such as active delay measurement and compensation. Note that the GPS reference clock for the probe tester has a maximum time error of +/‐ 50ns.


+/‐ 16ns +/‐ 163ns +/‐ 163ns Not yet defined

Average 200ns

Table 3‐6 Fronthaul synchronisation requirement v performance

Therefore, the GST‐enabled 100G aggregators can broadcast the PTP stream without inducing

additional PDV, which allows the recovered PTP clock at the remote node to be precisely

synchronised with the grand master clock (eg the remote Ethernet‐to‐CPRI clock is recovered from

the PTP clock).


of 75




VideostreaminginC‐RANwithmobilecloudIn the video streaming application use‐case, a user of a mobile phone can connect to a mobile clone

that acts as a proxy for access to the video stream in the Internet, with the mobile clone providing a

video transcoding service. The components used in this use‐case are listed in Table 3‐5.

Component Description Implementation

Video server Stream video using RTMP protocol to a local port.

ffmpeg / Avconv

Video proxy Pull a live video stream from remote host, transcode the video stream, then publish the video stream to a local port.

Nginx Nginx RTMP module ffmpeg

Video client Consume the video stream. VLC player

KPI collector Collect the measurements of this use case, such as CPU, network interface traffic.

Snap13 framework Snap plugin collector‐psutil Snap plugin publisher‐influxdb

KPI storage Store the measurements of this use case. InfluxDB

KPI monitoring Publish the measurements of this use case for monitoring.

Grafana

Table 3‐5 Component of the video streaming use‐case

The sequence of steps performed in this use‐case is as follows:

1. The video server (eg ffmpeg) produced an RTMP video stream, which was live streamed and

accessible through a local port, as illustrated in Table 3‐5

2. The mobile phone of a user requested the video stream by sending a request to the mobile

clone

3. The mobile clone requested the video stream from the video server and transcodes the

video stream

4. The mobile phone received the transcoded video stream from the mobile clone

Figure 3‐5 Command line interface (CLI) of the video server generating RTMP video stream

13 Please see http://snap-telemetry.io/ for more information.


of 75




In the assessment of the mobile cloud use‐cases, several measurements defined and used in the

mobile clone framework to illustrate the benefits obtained, these are listed in Figure 3‐7. In the

video streaming application, the video stream, through the mobile clone, will be transcoded to adapt

to the network status or user devices. The video transcoding will consume a certain amount of CPU

capacity. The in‐going and out‐going video stream of the mobile clone will assume different traffic

volumes as the quality of video differs. Thus, the CPU capacity of the mobile clone and the traffic

through it are monitored. The monitoring of the CPU capacity and traffic can provide the mobile

clone service framework information about the available computation and communication resources

of the mobile clone. The monitoring data can also be used to illustrate the benefits obtained in the

use‐case, for example, if the traffic sent to the user is less than that received by the transcoder, the

video transcoding over the clone saves the bandwidth between clone and user’s mobile device.

Measurements for mobile clone (related to P2 (P2a, P2c), S2)

Definition

CPU Capacity CPU usage of mobile clone in percentile.

Transcoder Input Rate Traffic received from the network in Bytes per second.

Transcoder Output Rate Traffic transmitted to the user in Bytes per second.

Table 3‐6 Measurements for mobile clone

The real‐time telemetry framework Snap used to collect the measurements of this use‐case,

performed the function of VIM in a generic virtualised architecture. Figure 3‐6 shows the surge of

CPU usage of the mobile clone when a user requests a video stream through the video clone which

can provide video transcoding. As was expected, if the transcoding compresses the video stream, the

outgoing network traffic of the clone is less than the incoming traffic, which is shown in Figure 3‐7.

Also, the video proxy permits the live stream (it also could be the transcoded live stream) to be

shared for multiple mobile phones in the same domain, ie, the mobile phones whose clones are in

the same mobile cloud network. The advantage of this is to eliminate the duplicated video traffic

from the video server to the mobile clone network. This sharing functionality has been partly

implemented to show the benefits of a clone as a video proxy; the missing implementation being a

load balancer to automatically forward the video request from its own clone to the neighbouring

clone.


of 75




Figure 3‐6 Monitoring interface of the CPU usage of mobile clone (video proxy)

Figure 3‐7 Monitoring interface of the network interface traffic of mobile clone (video proxy)

Considering the “video streaming in C‐RAN with mobile cloud” application, the results in Table 3‐7

show that the KPI framework is collecting representative KPI values and that the benefits of mobile

clone transcoding are recorded as expected.

KPI for mobile clone Definition Measured performance Comment

CPU Capacity CPU usage of mobile clone in

percentile. Clone CPU usage rises to 25% when transcoding is initiated

The KPI collection framework is providing KPI data as expected

Traffic Received Rate Traffic received of the network interface in Bytes per second.

Received traffic is recorded at ~ 500KB/s

50% reduction in traffic through the mobile clone with video transcoding

Traffic Transmitted Rate

Traffic transmitted of the network interface in Bytes per second.

Transmitted traffic is recorded at ~ 250KB/s

Table 3‐7 KPIs for mobile clone

Considering the 5G business societal and KPIs, the results contribute to the business goal B3

considering the number PoC that demonstrate 5G related technologies, and S3 considering

quantified assessment of maturity and performance of technological components with respect to

cloud technology and complexity and to S4 with respect to stimulation of new economically feasible

applications related to cloud assistance of applications running in user equipment.


of 75




MobiletaskoffloadinginC‐RANwithmobilecloudIn the mobile task offloading use‐case, the mobile cloud provides the mobile devices with the

capability to offload demanding computational tasks to its corresponding mobile clone in the mobile

cloud. By offloading the computationally‐intensive tasks of the mobile device to the mobile clone,

the energy consumption of the mobile device can be reduced, as well as offering the possibility of

offloading those tasks that exceed the capabilities of mobile device. The components used in this

use‐case are listed in Table 3‐8. The KPI collector differs to the framework used in the video

streaming use‐case, because the mobile clone in this use‐case is running x86 Android OS, which does

not work for the real‐time telemetry framework such as Snap. The VMware vSphere RESTful API was

used to collect the KPIs of the mobile clone via the helper tool vSphere‐influxdb‐go, which runs

periodically using the Linux cron service. The helper tool saved the collected KPIs to the InfluxDB.

Component Description Implementation

Offloading framework – clone service

Serve the task offloading request from mobile phone; execute the offloaded task; send the result of task to mobile phone.

Android native Java code implemented in iCIRRUS

project.

Offloading framework – mobile phone

Offload a task to mobile clone; receive the result of task from mobile clone.

Android native Java code implemented in iCIRRUS

project.

KPI collector Collect the KPI of this use case, such as CPU,

network interface traffic. vSphere‐influxdb‐go14

Linux cron

KPI storage Store the KPI of this use case. InfluxDB

KPI monitoring Publish the KPI of this use case for

monitoring. Grafana

Table 3‐8 Component of the mobile task offloading use‐case

The sequence of steps performed in this use‐case is as follows:

1. The mobile task is sent to the mobile clone on the fly

2. The offloaded task is executed on the mobile clone

3. The result of the offloaded task is sent back to the mobile phone

4. CPU utilization changes on the clone are expected, while offloading requests are being

executed

5. Uplink and downlink network usage changes are observed, as the tasks are offloading and

when the results are sent back to the mobile phone, respectively

In the mobile task offloading application, the reduced energy consumption of a mobile phone is the

major motivation for the use‐case. Thus, the battery information of the mobile phone is collected as

the KPIs of this use‐case, which are illustrated in Table 3‐9. The instantaneous battery current

14 See https://github.com/Oxalide/vsphere-influxdb-go for more information.


of 75




illustrates the battery drain if the battery is not charging. When a task is executed locally on the

mobile phone, the battery shows significant battery drain. However, if the task is offloaded to the

mobile clone, the battery should show much reduced drainage. The battery voltage is also

monitored so that the instantaneous power consumption can be calculated using the current

multiplied by the voltage. The battery capacity is therefore another important KPI in the task

offloading framework, in which the offloading decision could be made based on the current battery

capacity, which is therefore monitored as well.

Measurements for mobile phone

Definition

Battery Current Instantaneous battery current in microamperes, as an integer. Positive values indicate net current entering the battery from a charge source; negative values indicate net current discharging from the battery

Battery Voltage The present battery voltage level, in volts

Battery Capacity Remaining battery capacity as an integer percentage of total capacity

Table 3‐9 KPIs for mobile phone

In a real‐world environment, the mobile phone state, battery capacity, voltage etc., may be required

by the network infrastructure controller/orchestrator to be used for resource allocation algorithms.

The mobile phone’s battery current, voltage and capacity are all useful in the task offloading use‐

case, so these metrics were collected via the offloading framework installed on the mobile phone.

These collected metrics were saved into InfluxdB and monitored via Grafana, and are shown in

Figure 3‐8 and Figure 3‐9. The energy consumption of the mobile phone was obtained based on the

product of the battery current and voltage. The capacity was only monitored here (ie, not acted

upon); but it could have been potentially integrated into the offloading framework of a decision‐

making algorithm. The computation and network usage of a mobile clone were monitored for

evaluation, and were illustrated in Figure 3‐10 and Figure 3‐11 respectively. When a task was

offloaded to the clone, the battery‐drain of the mobile phone became very minor. On the clone side,

the CPU usages shows significant increase. There was also some network traffic on the clone side

when the offloading request and result were transferred. This use‐case proved that the energy

consumption of a mobile phone could therefore be dramatically reduced by offloading the mobile

task to a clone in the mobile cloud.


of 75




Figure 3‐8 Monitoring interface of the mobile phone battery current

Figure 3‐9 Monitoring interface of the mobile phone's battery voltage and capacity

Figure 3‐10 Monitoring interface of the mobile clone's CPU usage


of 75




Figure 3‐11 Monitoring interface of the mobile clone's network traffic

Considering the “mobile task offloading in C‐RAN with mobile cloud” application, the results in Table

3‐7 show that the KPI framework is collecting representative KPI values and that the benefits of

mobile clone transcoding are recorded as expected.

KPI for mobile phone Definition Measured performance Comment

Battery Current Instantaneous battery current in microamperes, as an integer.

Positive values indicate net current entering the battery from a charge source, negative values indicate net current discharging from the

battery

Phone current shows activity with a narrow spike but overall

a small impact

The KPI collection framework is providing

KPI as expected

Battery Voltage The current battery voltage level in volt

The battery voltage is displayed

The KPI collection framework is providing KPIs as expected – this value could be used to help support offload

decisions

Battery Capacity Remaining battery capacity as an integer percentage of total

capacity

The battery voltage is displayed

CPU Capacity CPU usage of mobile clone in percentile

Clone CPU usage rises to 15% when application offload is

initiated

The KPI collection framework is providing KPI data as expected

Table 3‐10 KPIs for mobile task off‐loading

Considering the 5G business societal and KPIs, the results contribute to the business goal B3

considering the number PoC that demonstrate 5G related technologies, and S3 considering

quantified assessment of maturity and performance of technological components with respect to

cloud technology and complexity and to S4 with respect to stimulation of new economically feasible

applications related to cloud assistance of applications running in user equipment.

NetworkcontrolledD2Duse‐caseThis Section represents developments to the analysis and simulation results of network controlled

D2D use‐case that were reported in D4.4 [16].


of 75




In the network controlled D2D use‐case, the D2D pairs are divided into groups based on their

geographical locations. Each D2D group is covered by one RRS, and the area covering one D2D group

is called a sub‐cell. Orthogonal resource allocation for one D2D group is performed locally at its RRS.

By allowing frequency reuse among different D2D groups or sub‐cells, the spectral efficiency of D2D

communications could be improved by several folds, which contributes the societal goal S2 to

reduce energy consumption. The integration of D2D technology also offers potential for facilitating

development of new applications which support the S4 KPI on service “stimulation.”

Figure 3‐12 demonstrates the spectral efficiency achieved by all D2D pairs when frequency reuse is

allowed among multiple sub‐cells, each of which is covered by one RRS. In this figure, K is the

number of D2D pairs in the system. L represents the number of sub‐cells or D2D groups. Note that

when L is equal to one, there is no frequency reuse among D2D pairs. It can be seen that the highest

spectral efficiency can be 8 times that obtained in the scenario without frequency reuse.

Figure 3‐12 Spectral efficiency achieved by network controlled D2D communication with frequency reuse among L sub‐cells

In iCIRRUS, D2D communication is taken as a key component for reducing the latency of data

transmission. Through simulations, the latency of packet transmission via D2D communications has

been compared with the latency experienced by the packet transmission in traditional cellular

network, in which the traffic will have to go through the BS.

Figure 3‐13 compares the data transmission latency of the D2D communication and that of

traditional LTE communication. In the simulations, both light and heavy traffic load are considered.

Since D2D communication uses a direct link between the source and destination devices, the


of 75




transmission latency under D2D communication is about the half of the latency in the LTE system.

The latency gain is even better when the traffic load is high.

Figure 3‐13 Latency Comparison between D2D communications and LTE systems

The key KPI components involved in this use‐case are data rate and latency, which are listed in Table

3‐11.

Network Controlled D2D

KPI Definition


D2D data rate

The data rate achieved by

individual D2D pair

Compared with no frequency reuse case,

individual user’s data rate improved significant

when the number of sub‐cells is larger than one.

(P1b) 1‐10Gbit/s

System data rate

The total data rate achieved by all D2D pairs in the coverage of multiple RRHs

controlled by one BBU

Compared with no frequency reuse case, the whole system’s data rate improved significant

when the number of sub‐cells is larger than one

User Latency

The duration for a packet to be

transmitted from the source

(device) to the destination (device).

(P4a) UP latency 1‐10ms

Table 3‐11 D2D Communications performance


of 75




Ethernettransportlabtestsofnewhigher‐layerfunctionalsplitfronthaul

This Section summarises the test results of the PDCP/RLC higher layer split (HLS) demonstration that

was conducted at Orange’s facility in Lannion and was reported in D5.3 [13], and compares the

results with the iCIRRUS design target KPIs and validates them in the broader context set by the 5G

business, societal and performance KPIs set out in [1]. The analysis of the results in the context of

the overall project is presented in Section 5.

Figure 3‐14 Experimental setup (top) PtP, (bottom) PON shows the experimental set‐up with Point‐

to‐Point (PtP) Ethernet (top) and G‐PON (bottom).

Figure 3‐14 Experimental setup (top) PtP, (bottom) PON

At the transmitter side, generic servers running on OpenStack host a virtual EMS (Element

Management System), a virtual EPC (Evolved Packet Core) where the Mobile Edge Computing (MEC)

was operating, and a virtual BBU (vBBU). A 10G Top of Rack (ToR) Ethernet switch was used to

connect these nodes. At the receiver side, an intelligent RRH (iRRH) was connected to a User

Equipment (UE) (here, a computer) via a mobile dongle. We used a mobile signal based on LTE (Long

Term Evolution) with 15MHz bandwidth and 2x2 MIMO (Multiple Input Multiple Output). The

synchronization in this network is provided by Global Positioning System (GPS) devices connected to

each iRRH.


of 75




Fronthaul KPI PDCP‐RLC split Performance 5G KPI target

User Data rate 100‐400Gbps 0.1Gbit/s (LTE) (P1b) 1‐10Gbit/s

FH Data rate 0.12Gbit/s (LTE) 1

User latency (P4a) 1‐10ms on the air

interface


delay) 60ms 5‐10ms PtP 2


NA NA


NA +/‐ 900ns downstream +/‐ 48µs upstream


NA +/‐ 40ns downstream+/‐ 140ns upstream

Max BER 10‐3 non‐priority

data3 10‐5 priority data

NA

1. Fronthaul rate is 20% greater than backhaul rate. 2. Dynamic resource allocation in PON adds an extra 1ms to fronthaul latency, avoided with fixed allocation 3. The performance of the HLS experiment was measured at the application layer hence it is appropriate to compare

performance with that required for Ethernet backhaul

Table 3‐12 HLS split at the PDCP‐RLC layer, requirement v performance

The PDCP‐RLC higher layer split generated a fronthaul rate only 20% greater than the user data rate.

This creates an opportunity that existing optical transport deployments using either PtP or PON

architecture may be reused for initial 5G roll‐out. Also, throughput measurements show that

backhaul and FTTH infrastructures can be potentially used for the transport of this Ethernet

fronthaul, albeit with the need to perform adjustments jointly in the RAN and transport equipment

(eg queuing policies).

Considering the 5G performance KPIs P1b and P4a, the results for PDCP‐RLC HLS indicate that

existing fibre infrastructure with a 1G or 10G capability meets, or is close to meeting, the

requirements for per user throughput and latency except for the URLLC use‐case.

Also, considering the 5G business societal and KPIs, the results contribute to the business goal B3

considering the number of PoC that demonstrate 5G related technologies, and S3 considering

quantified assessment of maturity and performance of technological components, and especially

considering estimation of the TCO and roll‐out that would be greatly assisted by the potential to re‐

use existing fibre infrastructure. Additionally, as the RAN is software based with a vEPC, vBBU and

MEC platform, hosted on servers based on OpenStack the demonstration contributes to the 5G

performance goal P2a considering automation of service creation and P2c considering use of

opensource SDK.


of 75




4 DescriptionandevaluationofthefinalShowcasedemonstrator

DescriptionoftheShowcasedemonstratorThe overall setup of the demo testbed is depicted in Figure 4‐1, where five different front and

backhaul services were successfully aggregated and converged onto a single metro fibre link. Two

identical 100G aggregators were deployed at the CO and remote node. The metro link used in the

showcase demonstration test is shown in Figure 4‐2. It consists of an average length optical span

(3km) which is used for backhauling mobile connections and for C‐RAN fronthaul, and can cover the

range of a city C‐RAN cluster [D2.3] [6]. In the actual test scenario, the link was looped at the

terminal BS location, resulting in a doubled effective optical path of 6km. In this case, a typical

network link length was emulated, which introduced an additional 30μs of transport delay and

represents a realistic scenario for the equipment under test.

Figure 4‐1 iCIRRUS testbed setup at Telekom Slovenije

At the CO, the legacy CPRI (Option 3) traffic was generated by the VIAVI MTS‐5800 tester, and the

CPRI streams were then de‐mapped and packetized in the normal Ethernet frames. The size of an

Ethernet packet can be configured by software, while each data section of a MAC packet consists of

an internal header information field and 2n CPRI basic‐frames. Typical configurations used for the

tests were payloads of 4, 8 and 16 basic‐frames per MAC‐packet for 2.45Gbit CPRI and 4 basic‐

frames per MAC‐packet for 9.83Gbit CPRI. With these configurations, the effective packet‐sizes can

be held within the standard MTU size of 1492 bytes. The effective packet‐sizes can be found in Table

4‐1.


of 75




CPRI 2.45Gbit CPRI 9.83Gbit

Number of basic‐frames

Size (byte) Ratio Header/Packet

Size (byte) Ratio Header/Packet

1 80 20.00% 256 6.25%

2 144 11.11% 496 3.23%

4 272 5.88% 976 1.64%

8 528 3.03% 1936 0.83%

16 1040 1.54% 3856 0.41%

32 2064 0.78% 7696 0.21%

64 4112 0.39% 15376 0.10%

128 8208 0.19% 30736 0.05%

256 16400 0.10% 61456 0.03% Table 4‐1CPRI over Ethernet ‐ MAC payload‐sizes

On the second client port, an Ethernet stream with 9Gbit/s traffic load and a fixed frame size of 1518

Bytes was generated by the VIAVI ONT‐606 tester. The 60GHz UPPER PHY LLS was connected to the

third 10G client port. Incoming Ethernet frames were processed by the DU, and the fronthaul data

was passed to the aggregator. Due to physical layer limitations, the data rate for the 60‐GHz link was

limited to a gross rate of 2.5Gb/s. A flow control protocol between DU and RU triggered a

backpressure mechanism back to the data source, to avoid overflow of the fronthaul system.

On the fourth 10G client port, a software‐based base station (BS) running the SRS‐LTE software15 was

connected. Specifically, one port of the BS was connected to the aggregator access port through a

10GbE switch while a second port was connected to the backhaul (Evolved Packet Core, EPC). The

proof‐of‐concept application for cloud computing were also hosted over this link.

The last client port was connected to the TS core network via 10G Ethernet router, to provision the

4G‐LTE femto‐cell access at the remote node. To enable the synchronization between the CO and

RN, an ADVA OSA‐5410 was configured as a grand master clock in accordance with the ITU‐T

G8275.1 PTP profile, receiving the reference from GNSS satellites and establishing the PTP link via

the 1G client port on the aggregator. The 10MHz clock output was used to trigger the CPRI traffic

generation. The PTP packets were then multiplexed with higher priority over other SM traffic and

broadcast to all the client ports of the remote aggregator, more details can be found in D5.3 [13].

15 SRS-ENB, Software Radio Systems, http://www.softwareradiosystems.com/


of 75




Figure 4‐2 Metro fibre link (in red) used in the iCIRRUS integration demo

After aggregation at the 100G interface and transmission over 6km of TS metro fibre, different traffic

streams were disaggregated on the client side accordingly. On the 1G client port, another ADVA

OSA‐5410 terminated the PTP packets as a slave clock, of which both time and phase could be

synchronized to the grand master clock. The internal 10MHz clock was also locked to the recovered

PTP clock. The CPRIoEth transponder board was externally triggered by this 10MHz clock, to recover

the accurate CPRI data clock and depacketize the original CPRI stream. The recovered CPRI data was

then analysed by the VIAVI MTS‐5800. Similarly, the 9Gbit/s Ethernet traffic was looped back on the

client port to the CO for analysis. The 60‐GHz RU was connected to the third client port. The RU

processes the fronthaul data and generates the physical layer information. A 60GHz link was

connected to the RU downstream port in a loopback configuration. A commercial remote radio

head, used to transmit the LTE signal (generated by the software BS) over‐the‐air, was connected in

the fourth client port, with the LTE signal being received by a commercial UE phone (used during

cloud computing experiments).

The following Sections outline the adaptation to the individual testbeds undertaken to enable their

incorporation into the Showcase demonstrator. Then present the results obtained for each of the

components, under simultaneous operation. Finally, an evaluation of the Show case demonstrator is

provided that highlights the results obtained in the wider context of 5GPP business, societal and

performance KPIs.


of 75




CPRIoverEthernettestandverificationintheShowcaseThe individual testbed was described in Section 3.1 together with the test and verification of its

associated KPIs.

This Section presents the necessary adaptations the testbed to enable integration and results of the

test and verification of CPRI over Ethernet in the Showcase demonstrator.

For the 100G aggregator scenario (Table 4‐7), the frequency synchronisation mechanism slightly

different to that in Section 3.1 is required. Because the 100G aggregator did not support

synchronous Ethernet operation, the frequency adjustment had to be realized using ADVA’s OSA‐

5410 device, as already described in Section 3.1. From the FPU and clock distribution’s point of view,

only the RRH side software configuration had to be changed. The on‐board DSPLL was re‐

programmed to generate its internal CPRI and Ethernet Core clocks from the provided OSA reference

clock. It turned out that both CPRI clocks at the BBU and RRH sides could be effectively synchronized

via the highly prioritized PTP link. A frequency offset between the BBU side and RRH side could not

be determined; the requirements of Table 2‐2 demanding a minimum frequency accuracy of ±2ppb

were therefore met.

iCirrus FPGA Board ‚BBU Side‘

VIAVI MTS 5800CPRI Tester

CPRI Master

Port

CPRI / Ethernet BridgeClock Unit(DSPLL)

RecoveredCLK CPRI

REF CLK

Eth REF CLK

CPRI Core

10G Eth Core

CPRI SlavePort

BERMeasurement

iCirrus FPGA Board ‚RRH Side‘

CPRI / Ethernet Bridge

Clock Unit(DSPLL)

ReferenceCLK

CPRIREF CLK

Eth REF CLK

CPRI Core

10G Eth Core

CPRI CPRI

10G Eth

OSA OSAGPS Reference

Aggregator10GEth

1G Eth

100GEth

1G Eth. PtP Sync Packets

10 MHz Reference CLK

Aggregator10GEth

1G Eth

100GEth100G Eth

1G Eth. PtP Sync Packets

10G Eth

Figure 4‐3 CPRI/Ethernet bridge connected to aggregator without SyncE

As CPRI is a continuous data stream, every unexpected delay of incoming data would result in a total

failure of the CPRI link. Because of the number of sources connected to the aggregator and the

transmission of prioritized services, the Ethernet packets of the CPRI link will sometimes be affected

with temporary increased latency (packet delay variation). To improve the stability of the CPRI link

within the 100G network, additional buffer FIFOs were added to both CPRI RX data paths at the BBU

and RRH sides. At start‐up, the “playout” of CPRI data on the RX side will not start until a software


of 75




pre‐programmed buffer level is reached. These buffers introduce an additional static latency from

the CPRI Master to the CPRI Slave side, but prevent the RX side from running out of data when a

subsequent packet arrives with an unexpected delay offset. The additional static delay correlates

with the buffer level and the size of Ethernet MAC packets containing the CPRI data. Figure 4‐4 and

Figure 4‐5 show the measured one‐way latency (VIAVI CPRI master port to VIAVI CPRI slave port)

with FIFI buffer levels “2.45Gbit CPRI” and “9.83Gbit CPRI” respectively. The internal latency within

the VIAVI MTS5800 and the latency of the 100G aggregator have been eliminated.

Figure 4‐4 Latency vs. FIFO buffer level (2.45Gbit CPRI)

Figure 4‐5 Latency vs. FIFO buffer level (9.83Gbit CPRI)

The latency displayed at FIFO Fill Level “0” represents the system’s internal signal path delay through

all the involved CPRI and Ethernet cores and all the additional FPGA firmware‐modules (basic one‐

way‐latency). The 2.45Gbit mode is affected with a much higher basic one‐way‐latency (~17.5 µs)


of 75




than the 9.83Gbit mode, which is the result of higher CPRI core latencies and a higher internal data

acquisition time. The 2.45Gbit CPRI can therefore be taken as the worst‐case mode.

The tests showed that a small RX buffer‐level adding < 10µs of latency was enough to prevent the

Ethernet‐to‐CPRI module from running out of data. For the 2.45Gbit mode, the resulting delay has to

be increased by ~10µs to an overall one‐way‐latency of ~ 27.5µs. Because of the symmetric design of

the CPRI‐over‐Ethernet firmware, the round‐trip‐delay can be calculated as

~ 27.5µs*2 = ~55µs, which covers the requirements of Table 2‐2.

A comparison with the target KPIs is provided in Table 4‐2 and Table 4‐3 the Tables also include the

standalone performance reported in Section 3.1.


performance Comment

Data rate 100‐400Gbps ‐


delay)


35µs direct link 55µs aggregated

link

Exceeds most demanding requirement




@2.45Gbit)

Actual error was less than measurement capability in with direct connection and via

aggregated link



Aggregated link 30ns average, 1460ns max

Measured on aggregated link


+/‐ 16ns Not measured

Max BER Below measurement sensitivity



performance Comment

Data rate 100‐400Gbps ‐


delay)


10µs direct link30µs aggregated

link

Buffer depth set by more latency consuming 2.45Gbit




@9.83Gbit)

Actual error was less than measurement capability in with direct connection and via

aggregated link



Aggregated link 30ns average, 1460ns max

Measured on aggregated link


+/‐ 16ns ‐ Not measured

Max BER ‐ Below measurement sensitivity


The results indicate that CPRI over Ethernet can exceed most of the KPI requirements that were

identified. However, the peak timing accuracy performance as measured on the aggregated link


of 75




indicates that performance may not be sufficient for the most severe MIMO and TX diversity

requirements.

Extending what was stated in Section 3.1, demonstration of CPRI over Ethernet over an aggregated

fronthaul can be related to the business and societal KPI B3/S3 related to cost of ownership and

rollout of 5G with its support of backward compatibility.

Real‐timefronthaulplatformusing60GHzwirelesswithUPPER‐PHYLLSverificationintheShowcase

This Section describes the test results of the 60‐GHz fronthaul system, which have been gained

during the integrated trial in Ljubljana at the TS labs. A description and the measurement results of

the standalone 60‐GHz fronthaul platform can be found in D5.3 [13]. Due to export regulations by US

authorities, it was not possible to ship the UE device to the final test. Therefore, a loopback

configuration at the RU has been used. A block diagram of the tested 60‐GHz fronthaul system can

be found in Figure 4‐6.

Figure 4‐6: 60‐GHz fronthaul system with radio loopback (Ethernet analyser shown as data source and sink)

Referencemeasurement(fronthaullinkonly)In a first series of measurements, the fronthaul link was characterized.

Two general trends can be observed for the throughput measurement. First, the L1 throughput

decreases with increasing frame size. This can be explained with the control mechanism that

guarantee that at the 60GHz physical layer only transmits 2.5Gb/s. Incoming Ethernet frames will be

stripped of Preamble and FCS and forwarded via the fronthaul link to the 60GHz PHY. Since the PHY

rate at the 60GHz is always 2.5Gb/s, for short Ethernet frames more preambles and FCS information

will be put into the system and therefore a higher L1 rate is required.


of 75




Figure 4‐7: Throughput for 60GHz fronthaul link for different frame sizes at different layer 1…4 (L1…L2)

Second, for Layer 2 and above the opposite effect can be observed. Shorter Ethernet frames (or IP or

TCP packets) have more overhead which reduces the net data rate.

The analysis of the system latency has confirmed the previous results of latency values in the order

of 700µs. The jitter was measured with values between five and eight µs. Both results are shown in

Figure 4‐8 and Figure 4‐9.

Figure 4‐8: Latency vs. frame size for 60GHz fronthaul link

Figure 4‐9: Jitter vs. frame size for 60GHz fronthaul link

Inclusionofaggregatorswitchesinto60GHzfronthaullinkIn a next step, the two aggregator switches have been inserted into the link (Figure 4‐10) and the

measurements have been repeated. The aggregator nodes have also carried other traffic, eg PTP

traffic.


of 75




Figure 4‐10: 60GHz fronthaul system with radio loopback and two aggregator nodes

The throughput, latency and jitter measurements as shown in Figure 4‐11, Figure 4‐12, and Figure

4‐13 do not show any significant difference compared to the reference results discussed in Section

4.3.1. It must be noted that the latency is not increased, although two additional elements have

been integrated into the system. The latency is determined by the flow control algorithm, which

requires an input queue of a sufficient depth to avoid jitter in the output data.

Figure 4‐11: Throughput for 60GHz fronthaul link with aggregator switches for different frame sizes at different layer 1…4 (L1…L2)

Figure 4‐12: Latency for 60GHz fronthaul link with aggregator switches for different frame sizes

Figure 4‐13: Jitter (ave) for 60GHz fronthaul link with aggregator switches for different frame sizes


of 75




The measured latency figures of ~700µs are above the KPI requirement as discussed in Table 2‐2.

This can be explained by the requirement of the flow control algorithm, which adjusts the source

data rate to avoid an overflow of the 60GHz physical layer, which is limited to 2.5Gb/s currently.

Currently, the data source can send out bursts of data frames with a rate of 10Gb/s. The last frames

of such a burst must be delayed most, which causes this relatively high latency. The latency can be

reduced if the data source would send out data with a more even distribution.

Inclusionof3kminstalledfibreintofronthaullinkThe inclusion of a 3km fibre link into the testbed triggered an error in the fronthaul system, which

could not be fixed so far. To investigate this issue, the previous setup as shown in Figure 4‐10 was

modified insofar that the aggregation nodes have been removed, but the short link between DU and

RU was increased to a length of 3km now. Consequently, the measurement setup is very like the one

shown in Figure 4‐6 (except for the 3km link between DU and RU).

In this configuration, the following results for throughput, latency and jitter have been measured

(Figure 4‐14, Figure 4‐15, Figure 4‐16).

Figure 4‐14: Throughput for 60GHz fronthaul link with 3km fibre for different frame sizes at different layer 1…4 (L1…L2)

Figure 4‐15: Latency for 60GHz fronthaul link with 3km reach for different frame sizes


of 75




Figure 4‐16: Jitter for 60GHz fronthaul link with 3km reach for different frame sizes

When introducing a 3km link between DU and RU a loss of frames could be observed. The reason for

this behaviour can be again explained with the flow control algorithm. It works in the following way.

If the receiver of an Ethernet link gets more frames than it can process (detected by a threshold in

the input queue fill state), it sends out a PAUSE frame in direction of the data source. The source

should then stop the data traffic for the duration indicated in the PAUSE frame. If the algorithm

works correctly in a fully loaded system this means we get bursts of frames with a line rate of 10Gb/s

and a period with no data. In the case of a targeted rate of 2.5Gb/s the burst duration should be ¼ of

the time gap between burst.

If the link between DU and RU is long, there are many frames “stored” in the fibre. If the PAUSE

frame reaches the data source all frames stored in the fibre still reach the receiver. If the storage

capacity of the fibre and input queue are not correctly balanced and frames get dropped.

It is planned to modify the queue size and the thresholds for a final test to avoid the loss of data

frames.

Fronthaul KPI


CoMP)


split in UL (CoMP)


User Data rate 100‐400Gbps 100‐400Gbps 2.2Gb/s (P1b) 1‐10GB/s

FH Data Rate

User Latency (P4a) 1‐10ms on the air

interface


delay) 440µs 150µs 800µs



+/‐ 2ppb(per hop)


+/‐ 30ns +/‐ 30ns


+/‐ 163ns +/‐ 163ns

Max BER 10‐12

Table 4‐4 lower layer split at the UPPER PHY, requirement v performance


of 75




The results of the FPGA‐based DU radio in section 3.3 indicate that the described fronthaul solution

falls outside the latency requirements, which indicates that further optimisation is required. The

major reason for the latency in the order of 800µs is the queuing required for the flow control. The

observed jitter can also be explained by the flow control mechanism. The 60GHz wireless link also

operates using a low latency scheduler, however, due to legal issues the 60GHz mobile, the link was

operated in a radio‐loop back mode and representative performance data was not available.

Software‐based‐DUradiooverEthernetwithMAC‐PHYLLSvalidationintheShowcase

The set‐up for the functional split case and the generated packet types are described in detail in [17]

and [5], while the analysis presented here is based on the results from [18]. The validation set‐up is

shown in Figure 4‐17, together with the different interface points where measurement results can

be extracted. The generated packet types include the Physical Downlink Shared Channel (PDSCH),

Downlink Control Information (DCI), Random Access Response (RAR) and System Information (SI)

packet type. For these tests, the maximum packet size is for the DLSCH packet type and it is

approximately 1000 Octets long, but this value is shared by the number of recipients (users) within

the subframe. For these tests, a maximum number of 3 users were active within a subframe at any

given time. The transport network operates at 1Gbps (1GbE) and includes only Layer‐2 processing.

The results presented here include the mean and standard deviation (STD) of packet latency, which

is defined Table 2‐2 (albeit as two‐way latency, while the measurements presented here are for one‐

way latency), and subframe (LTE) latency. The latter is important as it relates to the length of time

that Ethernet packets that form LTE subframes must be queued at the RU prior to playout of the

subframe.

Figure 4‐17 Set‐up used for validation testing (left) and interface measurement points (right)

The packet latency was measured between interface points 1 and 3. It is an average of the delay

encountered by the 4 different packet types during the transmission of 9000 LTE subframes. The

subframe latency took into account the processing time of each subframe, and was measured at the

same interface points (1 & 3) but taking into account the first packet in each subframe (a DCI packet‐


of 75




type) at interface point 1 and the last packet for that same subframe (PDSCH, SI or RAR packet types)

at interface point 3. The results of Figure 4‐18 included the contention effects with bursty

background traffic of different burst sizes and different packet lengths, while for those in Figure

4‐19, the background traffic is constant‐packet rate (CPR). In both sets of results, the baseline case is

without contention. Table 4‐5 shows a summary of the average latency for the different networking

configurations, while Table 4‐6 extrapolates these results for a network with 10GbE.

Figure 4‐18 Fronthaul latency with two Ethernet switches over a 1Gbps Ethernet network, Mean (Left) and Standard Deviation (Right). The traces include packet latency (“per‐packet”) and subframe latency (“per‐subframe). The dotted lines

are baseline cases while the rest of the traces are with bursty background traffic.

Figure 4‐19 Mean and standard deviation for the fronthaul latency with two Ethernet switches over a 1Gbps Ethernet network. The traces include packet latency (“per‐packet”) and subframe latency (“per‐subframe). The dotted lines are

baseline cases while the rest of the traces are with constant packet‐rate background traffic.


of 75




KPI No Switch 1 Switch 2 Switches

Packet latency mean (µs) 45 53 60

Subframe latency mean (µs) 54 62 69

Packet latency STD (µs) 9 N/A N/A

Subframe latency STD (µs) 6.3 N/A N/A Table 4‐5 Measurement results for the packet and subframe latencies for the baseline cases (no contention) (x indicated not

measured)

KPI No Switch 1 Switch 2 Switches

Packet latency mean (µs) 43 48.2 53.5

Subframe latency mean (µs) 52 57.3 62.5

Packet latency STD (µs) 6.9 N/A N/A

Subframe latency STD (µs) 4.1 N/A N/A Table 4‐6 Estimated packet and subframe latencies for the baseline case (no contention) for 10GbE networking (x indicates

not measured)

Based on these results, an extrapolation of the latency for the set‐up with the iCIRRUS aggregator

can be carried out in a crude manner as follows: Assuming 10GbE access, the standard deviation will

be dominated by the processing delay in the DU, including consideration of a one‐way delay of 35 s for the aggregator. The result of this extrapolation is shown Table 4‐7. The measurement results

with contention can also be similarly extrapolated. However, for large aggregation levels the effect

of contention is insignificant. Based on the results in Figure 4‐18 and Figure 4‐19, for large

aggregation levels, for example, a ratio of split traffic data to total aggregated data of 1.25%

(assuming 80% utilisation of the aggregator link rate, this would correspond to a split data rate of

1Gbps) and a packet size of 1024 Octets (for the background traffic), the increase in packet latency

and STD will be insignificant.

KPI iCIRRUS Aggregator

Packet latency mean (µs) 78

Packet latency STD (µs) 6.9

Subframe latency mean (µs) 87

Subframe latency STD (µs) 4.1 Table 4‐7 Mean and standard deviation (STD) of the packet and subframe latencies with the iCIRRUS aggregation set‐up.

The values are extrapolated from the measured results.


of 75




Fronthaul KPI


CoMP)


split in UL (CoMP)

Extrapolated performance

Data rate 100‐400Gbps 100‐400Gbps 1Gbps


delay) 440µs 150µs 156µs




N/A


+/‐ 30ns +/‐ 30ns N/A


+/‐ 163ns +/‐ 163ns N/A

Max BER 10‐12 10‐12

Table 4‐8 MAC‐PHY LLS, requirement v performance

The results of the software‐based‐DU radio in Section 3.3 indicate that the MAC‐PHY LLS only slightly

exceeds the most stringent RTT latency requirements that were identified, which indicates that the

majority of CoMP approaches would be supportable. A large percentage of the quoted one‐way

latency (78 µs) is fixed by the aggregator latency while the rest is mainly due to the software

generation process in the end station. The latter has the potential of significant reduction with

hardware acceleration technologies. The remaining quoted KPI values in Table 4‐7 cannot be

compared to specific requirements as such requirements do not currently exist. The reason is that

the subframe‐level KPI requirements will be implementation dependent. That is, they will vary from

one implementation to the next and will need to be absorbed through buffering at the end stations.

It is also important to note, that the packet jitter KPI that is defined in Table 4‐8, does not apply

here. The reason is that with the iCIRRUS aggregator the PTP traffic is not contending with the

MAC/PHY split traffic. However, the case is made here that operators will need to monitor these

additional KPIs (subframe‐related) and constrain them to acceptable levels based on their specific

architecture requirements.

In addition to the societal KPIs that were discussed in Section 2.1, regarding the importance of the

evolved fronthaul for meeting these KPIs, several performance KPIs were also defined. The MAC‐PHY

LLS evolved fronthaul can be assessed in terms of adherence to these KPIs, in an indirect‐

quantitative manner (by extrapolating from benchmarking results) and in a qualitative manner.

Referring to Figure 4‐20, the data rates over the LLS fronthaul can be seen to scale with the cell load.

This is an important result when considering 5G use cases with increased data rate demands.

Extrapolating from the results shown here, assuming a 10GbE link and same levels of overhead (note

that these are based on 4G, optimisations may be expected for 5G), the LLS can potentially support

the user throughputs defined in KPI P1a, something that would be extremely challenging to achieve

with current transport technologies (generic IQ, CPRI). Note that benchmarking tests (summarised in

Table 3‐4) with software‐based DU implementations have shown that they can scale to the levels of

user throughput allowed by current 4G bandwidths. Similarly, with dense RU deployments, it will


of 75




potentially be possible to offer the traffic levels expected by KPI P1a. Furthermore, the efficient use

of available transport bandwidths (compared to current fronthaul transport technologies), means

that the system has the potential of meeting KPI P3b.

While the extrapolated latency values presented in Table 4‐7 do not consider the end‐to‐end latency

(that is, including the processing in user equipment), the latency contributions that are taken into

account are such that they should allow the system to meet KPI P4a.

The LLS is a software implementation, running on open‐source software modules and off‐the‐shelf

hardware (software‐defined radios and Ethernet equipment). As such it meets KPI P2c.

Currently the LLS implementation does not meet KPI P4b specifications. However, the system is

currently solely software‐based and it is expected that significant reductions in packet latency

variation can be achieved through hardware acceleration techniques.

The LLS fronthaul implementation has been presented in several peer‐reviewed conferences. The

initial benchmarking results were first presented in [10]. The performance of the system under

traffic contention was presented in an invited paper [14], while first experimental results using strict

priority scheduling in the LLS fronthaul were presented in [11]. There are further plans to improve

and optimise the system while its contribution to new research and innovation calls (national and

EU‐based) is considered as significant.

Figure 4‐20 Benchmarking results for the LLS evolved fronthaul. The fronthaul data rate scales with the application data rate (equivalent to the cell load) while the overhead remains approximately constant at around 42%.


of 75




Femto‐cellfronthaul/backhaulaggregationvalidationintheShowcase

Small/femto‐cell (eg indoor, home cell) deployment strategy is a general mobile operator strategy to

extend the network coverage to indoor and network edge. In the 5G fronthaul concept considered in

the iCIRRUS project, the high capacity network infrastructure could be shared with the fixed internet

access. In this case it can be used as a high quality backhaul for small cell deployment.

As was confirmed in the integration demo test, the small cell backhaul traffic could be efficiently

integrated in the 100G fronthaul segment as low priority traffic. It is based on an IPsec connection to

the mobile network core, which provides transparent, interoperable, and cryptography‐based

security services to ensure confidentiality, integrity, and authenticity of data and to provide anti‐

replay protection. The system itself provides automatic cell provisioning without operator needs for

manual configurations. Due to the limited required system (intercell) coordination, for handover

only (no CoMP), the backhaul performance is not critical (eg latency). For proper small cell operation

and good user experience, enough backhaul bandwidth and low latency is required, which is assured

by the 100G trunk. The measured 0.07 ms RTT of the ADVA aggregator switch is well below the most

severe upper limits values (1ms) for the backhaul as defined in the previous D2.1 [2] report.

In the test, a Cat 4 LTE femto‐cell with nominal values (300Mbps DL/50Mbps UL) was used. In our

speed‐test we measured 110Mbps DL, 42Mbps UL, and 30 ms RTT (ping). We can therefore conclude

that the traffic was not limited by the ADVA aggregator switch, because we measured practically the

same values without it.

The benefit of using a high capacity fronthaul network for small cell backhaul is the high‐grade

service QoS due to the carrier grade network segment used in the test. It is entirely under the

service provider’s management. In contrast, in the case of a customer fixed access network as the

small cell backhaul, the small cell QoS can vary depending on the fixed‐line personal use, and so is

not under the control of the mobile network operator.


performance Comment

Data rate 100‐400Gbps


delay) 60ms

30ms(0.07ms

aggregator performance)

RTT delay meets requirements,

not influenced by aggregator


N/A N/A


N/A N/A


N/A N/A

Max BER 10‐3 non‐priority

data No errors measured


of 75




10‐5 priority data

Table 4‐9 Backhaul (unified transport), requirement v performance

The experiment showed that the aggregated link could seamlessly support integration of backhaul

traffic. Performance degradation due to the inclusion of the aggregator was found to be negligible.

The result supports the business and societal KPI B3/S3 goal considering 5G enabling PoC and also in

understanding TCO the performance of RG enabling technology components. Additionally, the

aggregation of multiple 10G streams onto a 100G fronthaul link with TSM support for PTP operation

is also enabling for the performance 5G KPIs including P1b per user throughput, and P4a on latency.

AggregatedEthernet‐basedfronthaulnetworkvalidationintheShowcase

As depicted in Figure 4‐1, a 10GbE traffic flow was generated by a VIAVI network tester at the local

aggregator and looped back from the remote aggregator to evaluate the round‐trip transmission

performance of the fronthaul link. The traffic profile was configured with a fixed frame size of 1518

Bytes and a sustained a throughput of 9Gbit/s, and the transmission was kept running for 60 hours

without any bit error and packet loss. Table 2‐2 and Table 4‐6 summarise the measured round‐trip

transfer delay and packet jitter.

Average (µs) Minimum (µs) Maximum (µs) Variation (µs)

69.712 69.610 71.360 1.75 Table 2‐2 Round‐trip time delay of 10GbE traffic flow

Average (ns) Minimum (ns) Peak (ns)

30 0 1460 Table 2‐3 Instantaneous jitter of 10GbE traffic flow

It is worth pointing out that during the test, the PTP flow was always broadcast at the same time as

the GST class, which led to a higher delay variation and packet jitter on the statistically multiplexed

(SM) traffic (ie 10GbE). Similar results can be also found in the back‐to‐back case in D5.3, Section 2.5

[13]. In addition, compared to the back‐to‐back result, the 6km metro fibre link introduced another

round‐trip propagation delay of 63.2µs.

The performance comparison of different Ethernet traffic patterns can be found in D5.3, Section 2.5

[13]. There was no additional impact observed in the field integration.


of 75





Legacy traffic (CPRI)


Upper PHY split in DL

lower PHY split in UL (CoMP)

PDCP‐RLC split Backhaul (unified

transport)

Measured performance

Data rate 100‐400Gbps 100Gbps per network

(trunk) port

FH data rate 10 Gbps per

RRH


delay)

150µs (CoMP)

440µs (no CoMP)

440µs 150µs 60ms 60ms

6.5µs (2 nodes) +

approx.. 10µs per fibre km

Max BER 10‐12 10‐6

10‐3 non‐priority data 10‐5 priority

data

Error‐free

Table 4‐10 Fronthaul data plane requirement v performance

Table 4‐10 summarises the measured results in comparison with the fronthaul KPI. Considering the

most stringent latency requirement stated in Table 2‐2, which is a max round‐trip delay of 150µs in

case of enabling CoMP in the legacy CPRI link, the proposed fronthaul network is still able to

aggregate 10 times 10GbE traffics over a single 100G trunk link of up to 14.6 km distance. This

provides further support that the aggregation testbed is enabling for the performance 5G KPIs

including P1b per user throughput, and P4a on latency

EvaluationoftheShowcasedemonstratorThe aim of this Section is to draw together contributions that have been made towards validating

the iCIRRUS project’s outputs in the context set by the 5G business, societal and performance KPIs

set out in [1].

The Showcase simultaneously demonstrated many of the iCIRRUS use‐cases over a common

infrastructure in the laboratories of Telekom Slovenije. The Unified Communication use‐case also

demonstrated integration with cloud servers located at Telekom Slovenije.

As set out in the preceding subsections, the following technologies were incorporated, and

simultaneously operated in the Showcase,

CPRI over Ethernet (Section 4.2)

60GHz wireless with UPPER‐PHY LLS (Section 4.3)

Software‐based‐DU radio with MAC/PHY LLS (Section 4.4)

Common transport with femto‐cell fronthaul/backhaul aggregation (Section 4.5)

100G TSN Ethernet aggregated fronthaul (Section 4.6)

Unified Communication video application with mobile cloud support (Section 3.5)

Unified Communication application offload to mobile cloud (Section 3.6)


of 75




Other technologies were evaluated independently of the Showcase. However, it is useful to consider

the KPIs validated by these experiments to provide a thorough overview of the project.

Network controlled D2D simulation analysis (Section 3.7)

vRAN demonstration with PDCP/RLC HLS (Section 3.8)

The 5G KPIs are addressed in a sequential fashion with a short description of the contribution that

each element makes, but with B3 and S3 being dealt together due to the similarity of their

underlying objective,

B3: “Reach a global market share for 5G equipment and services delivered …,” with proxy

metrics as number of PoC developed and disseminated, contribution to standards and IPR

development and S3: “European availability of a competitive industrial offer for 5G systems

and technologies,” with proxy metrics, quantified assessments of maturity and performance

of technological components, of estimating TCO and rollout.

There were many significant contributions to KPI B3/S3. Foremost, is the Showcase demo itself as a

PoC that demonstrated many 5G enabling technologies with simultaneous operation of most of the

testbeds above. As described in the Sections for each individual testbed B3/S3 is supported in a

similar way by each. Also, considering the contribution to support of innovation in the wider

community, as set out above and detailed in D6.4 [19], many members attended standardisation

bodies representing iCIRRUS views and several iCIRRUS specific contributions, 5 patents were filed

and over 20 publications and presentations were made. Estimates of the impact of various

technologies on TCO and rollout were also made, for example, simulation and analysis in the case of

D2D and extrapolation from field data in the case of C‐RAN, see [6] for details. Additionally,

considering the initial roll‐out of 5G the PDCP/RLC HLS demo shows existing 1G and 10G transport

infrastructure meets requirements for providing support for TCO and initial roll‐out.

S2: “Reduction of energy consumption per service up 90%.”

Contributions to the S2, reducing energy consumption per service, are made by the cloud‐supported

Unified Communication applications that demonstrate reduction of energy consumption in mobile

devices and efficient processing of application data in the mobile cloud, and D2D communication

where enhanced spectral efficiency supports use of lower transmit powers. It is also made by the

vRAN/C‐RAN iCIRRUS architecture itself, and by the use of lower bit‐rate transmission in the

fronthaul, enabled by the RAN functional splitting, and the iCIRRUS Ethernet transport profiles, and

efficient, time‐sensitive multiplexing and aggregation which support these.

S4: “Stimulation of new economically viable services …”


of 75




Contributions to S4 were made by made by several testbeds including mobile cloud assisted

applications, enabling applications potentially inhibited by user equipment power and energy

considerations to be addressed, 60GHz wireless facilitated by opening up new wider bandwidths,

and network controlled D2D communication which facilitates new ways of delivering applications.

P1: Providing 1000 times higher wireless area capacity …” with proxy metrics of absorbing

P1a 10TB/s/km2, and P1b reach a peak user data rate of 1‐10Gbit/s.

Contributions to P1b are made by D2D where simulation confirms that short range between D2D

devices and enhanced spectral efficiency promote user data rate, and 60GHz wireless with UPPER

PHY LLS that demonstrates a 2.2Gbit/s user rate. The transport for new RAN functional split

interfaces also facilitates these KPIs, as (1) fronthaul bit rates become closer to user data rates,

instead of more than an order of magnitude greater as in technologies such as CPRI, and (2) the

aggregation levels demonstrated at 100 Gb/s will enable interconnection of numbers of mmW cells

in dense or office areas which permit the attainment of 10 Tb/s/km2 area capacity.

P2: “Reducing average service creation time…” with proxy metrics of P2a level of automation

of service creation, P2b existence of autonomic management framework and, P2c, usage of

opensource SDK.

Contributions to the P2a and P2c KPIs are made by the cloud assisted Universal Communication

applications, which relate to automated service creation and use of opensource software.

Additionally, both the MAC/PHY LLS and the PDCP/RLC HLS contribute to P2a considering RAN‐as‐a‐

service, and to P2c as they are based on opensource software.

P3: “Facilitate very dense deployments…” with proxy metrics handle P3a handling 10,000 to

1M devices/km2, P3b facilitate large numbers of small‐cells

Indication that D2D is a component technology that may help realise KPI P3a is provided by the

simulation and analysis of network controlled D2D which indicate potential facilitate support for

very large numbers of devices. Also, the results achieved by the use of different functional splits (HLS

PDCP/RLC, LLS MAC/PHY, LLS UPPER PHI), allowing efficient use of available transport bandwidth,

increases the potential the KPI P3b may be met when deploying these technologies. Additionally,

common transport carrying fronthaul and backhaul with TSN Ethernet fronthaul supports

deployment of femto‐cell, that is small‐cells P3b

P4: “Create a secure, reliable and dependent internet with “zero perceived” downtime…”

with proxy metrics P4a of provide a 1‐10ms latency over the air interface, P4b reach

reliability > 99% and, P4c reach availability > 99%

Support for KPI P4a related to latency is provided by the D2D simulation and analysis that optimises

the set‐up procedures and offers the possibility of direct communication not intermediated by the


of 75




network. And, additionally, support for this KPI is provided in the following testbeds; the 60GHz

wireless testbed, which provides latency in the range 250µs ‐ 1200µs dependent on throughput, the

vRAN PDCP‐RLC testbed that shows that existing installed fibre has potential to meet the less

stringent 5G use‐cases regarding latency, and finally, extrapolation based on results of the software‐

based radio with MAC/PHY split, indicate that the system also has potential to meet the KPI.

Additionally, the use of Ethernet based fronthaul also facilitates path redundancy that contributes to

availability P4c.

In summary, the Showcase demonstrator and its component testbeds, and the testbeds

demonstrated separately or examined through simulation, has made many contributions that are

relevant to the 5G KPIs defined by the 5GPPP [1].

5 ConclusionOne of the major highlights was the Showcase demonstration itself, which contributes to the 5G

business KPI B3, with respect to proof of concept demonstration of 5G technologies, and to 5G

societal KPI S3, with respect to maturity of potential 5G component technologies and impact on total

cost of ownership.

The 5G performance KPIs are very demanding, but in several cases the testbeds directly met them

them, for example, demonstration of a 2.2Gbit/s application rate with latency in the sub‐millisecond

range by the 60GHz wireless against a requirement of 1‐10GHz peak user rate and 1‐10ms latency

and. Additionally, investigation showed that the installed fibre base has the potential to meet

latency constraints for some of the less demanding 5G use‐cases, with 5‐10ms versus 1‐10ms

latency. Additionally, the role that other technologies may play in achieving these KPIs was

demonstrated, for example, the HLS and LLS that prevent fronthaul becoming a bottleneck for future

5G and C‐RAN/vRAN deployment, the use of D2D that may assist in achieving the reliability and

accessibility KPIs and energy saving, and the cloud‐assisted applications that facilitate energy saving

for the mobile device.

Other of the societal and performance KPIs are qualitative in nature, however, significant

contribution was shown in the potential to stimulate new services S4, and reducing average service

creation time with the deployment of virtualisation and extensive use of opensource software.

Thus, we can conclude that the iCIRRUS project, and particularly the Showcase demonstrator and its

component testbeds, made significant contributions to understanding and validation of the 5G

business, societal and performance KPIs as set out in [1].


of 75




6 References

[1] D. Kennedy, “D2.6 ICT ‐ Final report on programme progress and KPIs,” 5G PPP H2020‐ICT‐2014‐

2, 2017.

[2] P. Chanclou, Z. Tayq, P. Turnbull, V. Jungnickel, L. Fernandez‐del‐Rosal, P. Assimakopoulos, N.

Gomes, K. Yuan, H. Zhu, M. Parker, S. Walker, C. Magurawalage, K. Yang, H. Thomas, M.

Castaño‐Torres, I. Campos Rivera and F. Jesus‐Hidalgo‐Gonzalez, “D2.1 iCIRRUS – Intelligent C‐

RAN Architecture,” Horizon 2020 iCIRRUS Project, 2015.

[3] P. Asimakopoulos, N. Gomes, H. Zhu, L. Rosal‐del‐Fernandez, M. Hinrichs, P. Ritoša, D. Münch, J.

P. Elbers, C. Magurawalage, K. Wang, S. Delaitre, M. Castaño‐Torres and S. Blasco, “D5.2

iCIRRUS Tools, scenarios and results for testing analysis methods for validation test,” Horizon

2020 iCIRRUS Project, 2017.

[4] H. Thomas, L. Fernandez‐del‐Rosal, P. Asimakopoulos, Y. Kai, K. Wang, P. Chanclou and D.

Muench, “D3.3 SLA and SON Concept for iCIRRUS,” Horizon 2020 iCIRRUS Project, 2016.

[5] K. Habel, C. Kottke, M. Hinrichs, L. F. d. Rosal, D. Muench, N. Eiselt, P. Assimakopoulos, N.

Gomes, H. Thomas, M. parker, F. Ngobigha, G. Koczian, T. Quinlan, S. walker and P. Chanclou,

“D3.4 iCIRRUS Updated low‐cost, energy‐efficient fronthaul,” Horizon 2020 iCIRRUS Project,

2017.

[6] P. Ritoša, S. Delaitre, G. Flores, D. Capitán, P. Chanclou, R. Sušnik, H. Zhu, M. Polyvios, J. Zou, P.

Elbers and H. Thomas, “D2.3 iCIRRUS Business case analysis,” Horizon 2020 iCIRRUS Project,

2017.

[7] P. Kangru, “ZSM (18) 000037 RAN SON ZSM Input iCIRRUS,” in ETSI ZSM (18), 2018.

[8] W. Featherstone, D. Padfield, C. Murphy, H. Thomas, P. Claridge and M. Wang, “Enhancing

network topology information for a self‐organizing network”. Patent US201514929923

20151102, 10 05 2017.

[9] J. Haver, H. Thomas and J. Govert, “Packet colouring for data stream monitoring”. Patent Not

yet published, 8 9 2017.

[10] D. Padfield, C. Murphy and H. Thomas, “Techniques for providing fronthaul data awareness”.

Patent US201562147827P, 15 4 2015.


of 75




[11] P. Turnbull, “Method and apparatus for mitigation of packet delay variation”. USA Patent

US201414562062 20141205, 11 06 2015.

[12] Wikipedia, “Unified communications,” Wikipedia, 17 12 2018. [Online]. Available:

https://en.wikipedia.org/wiki/Unified_communications#cite_note‐1. [Accessed 17 01 2018].

[13] K. Habel, P. Assimakopoulos, N. Gomes, I. Laurinavičius, H. Zhu, M. Hinrichs, S. Weide, J. Zou

and J. Elbers, “D5.3 Validation test setup and execution report,” Horizon 2020 iCIRRUS Project,

2017.

[14] O. Sahin, P. Pietraski, A. Demir, W. Lawton, A. Roy, M. Fazili, K. Pan, M. Abou‐El‐Seoud and K.

Wanuga, “High resolution angle of arrival estimation and dynamic beam nulling”. USA Patent

PCT/US2017/040685, 05 07 2017.

[15] M. Georgiades, P. Michael, P. Chanclou, P. T. H. Ritosa, S. Delaitre, N. Gomes, H. Zhu, P.

Asimakopoulos, L. Fernandez‐del‐Rosal and K. Wang, “D2.2 Refined iCIRRUS architecture

definitions and specifications,” Horizon 2020 iCIRRUS Project, 2016.

[16] K. Wang, X. Du, C. Magurawalage, K. Yang, G. Flores, M. Castaño‐Torres, H. Thomas, P. Ritosa, Y.

Kai and H. Zhu, “D4.4 Joint resource management,” Horizon 2020 iCIRRUS Project, 2017.

[17] G. S. Birring, P. Assimakopoulos and N. Gomes, “An Ethernet‐based fronthaul implementation

with MAC/PHY split LTE processing,” in Globecom, Singapore, 2017.

[18] P. Assimakopoulos, G. S. Birring and N. Gomes, “Effects of contention and delay in a switched

Ethernet evolved fronthaul for future cloud‐RAN applications,” in ECOC, Gothenburg, 2017.

[19] PrimeTel, University of Kent, Orange, InterDigital, HHI, University of Essex, ADVA, Telecom

Slovenia, VIAVI, IAF, “D6.4 Final dissemination, communication, standardisation and

exploitation: Plan and report of activities,” Horizon 2020 iCIRRUS project 2018, 2018.


of 75




7 ListoffiguresFigure 2‐1 KPIs for fronthaul and RAN joint optimisation ............................................................................ 21

Figure 2‐2 iCIRRUS architecture overview ................................................................................................... 25

Figure 3‐1 Point‐to‐point with SyncE .......................................................................................................... 27

Figure 3‐2: Test setup for Integration of DU, RU, UE and 60GHz .................................................................. 29

Figure 3‐3: comparison of all setups for different frame lengths: 86bytes (a), 512bytes (b), 1024bytes (c),

1518bytes (d), random size 86…1518bytes (e) .................................................................................... 30

Figure 3‐4 Indicative setup for software‐based‐DU radio over Ethernet ...................................................... 32

Figure 3‐5 Command line interface (CLI) of the video server generating RTMP video stream ....................... 38

Figure 3‐6 Monitoring interface of the CPU usage of mobile clone (video proxy) ......................................... 40

Figure 3‐7 Monitoring interface of the network interface traffic of mobile clone (video proxy) ................... 40

Figure 3‐8 Monitoring interface of the mobile phone battery current ......................................................... 43

Figure 3‐9 Monitoring interface of the mobile phone's battery voltage and capacity ................................... 43

Figure 3‐10 Monitoring interface of the mobile clone's CPU usage .............................................................. 43

Figure 3‐11 Monitoring interface of the mobile clone's network traffic ....................................................... 44

Figure 3‐12 Spectral efficiency achieved by network controlled D2D communication with frequency reuse

among L sub‐cells ............................................................................................................................... 45

Figure 3‐13 Latency Comparison between D2D communications and LTE systems ....................................... 46

Figure 3‐14 Experimental setup (top) PtP, (bottom) PON ............................................................................ 47

Figure 4‐1 iCIRRUS testbed setup at Telekom Slovenije ............................................................................... 49

Figure 4‐2 Metro fibre link (in red) used in the iCIRRUS integration demo ................................................... 51

Figure 4‐3 CPRI/Ethernet bridge connected to aggregator without SyncE .................................................... 52

Figure 4‐4 Latency vs. FIFO buffer level (2.45Gbit CPRI) .............................................................................. 53

Figure 4‐5 Latency vs. FIFO buffer level (9.83Gbit CPRI) .............................................................................. 53

Figure 4‐6: 60‐GHz fronthaul system with radio loopback (Ethernet analyser shown as data source and sink)

.......................................................................................................................................................... 55

Figure 4‐7: Throughput for 60GHz fronthaul link for different frame sizes at different layer 1…4 (L1…L2) .... 56

Figure 4‐8: Latency vs. frame size for 60GHz fronthaul link ......................................................................... 56

Figure 4‐9: Jitter vs. frame size for 60GHz fronthaul link ............................................................................. 56

Figure 4‐10: 60GHz fronthaul system with radio loopback and two aggregator nodes ................................. 57

Figure 4‐11: Throughput for 60GHz fronthaul link with aggregator switches for different frame sizes at

different layer 1…4 (L1…L2) ................................................................................................................ 57

Figure 4‐12: Latency for 60GHz fronthaul link with aggregator switches for different frame sizes ................ 57

Figure 4‐13: Jitter (ave) for 60GHz fronthaul link with aggregator switches for different frame sizes ........... 57

Figure 4‐14: Throughput for 60GHz fronthaul link with 3km fibre for different frame sizes at different layer

1…4 (L1…L2) ....................................................................................................................................... 58

Figure 4‐15: Latency for 60GHz fronthaul link with 3km reach for different frame sizes ............................... 58

Figure 4‐16: Jitter for 60GHz fronthaul link with 3km reach for different frame sizes ................................... 59

Figure 4‐17 Set‐up used for validation testing (left) and interface measurement points (right) .................... 60

Figure 4‐18 Fronthaul latency with two Ethernet switches over a 1Gbps Ethernet network, Mean (Left) and

Standard Deviation (Right). The traces include packet latency (“per‐packet”) and subframe latency

(“per‐subframe). The dotted lines are baseline cases while the rest of the traces are with bursty

background traffic. ............................................................................................................................ 61


of 75




Figure 4‐19 Mean and standard deviation for the fronthaul latency with two Ethernet switches over a 1Gbps

Ethernet network. The traces include packet latency (“per‐packet”) and subframe latency (“per‐

subframe). The dotted lines are baseline cases while the rest of the traces are with constant packet‐

rate background traffic. ..................................................................................................................... 61

Figure 4‐20 Benchmarking results for the LLS evolved fronthaul. The fronthaul data rate scales with the

application data rate (equivalent to the cell load) while the overhead remains approximately constant

at around 42%. .................................................................................................................................. 64


of 75




8 Listoftables

Table 2‐1 Comparison of Unified Communication offerings ......................................................................... 17

Table 2‐2 Compilation of Fronthaul KPI ...................................................................................................... 20

Table 2‐3 KPIs for D2D link set up ............................................................................................................... 22

Table 2‐4 KPIs for D2D communication ....................................................................................................... 23

Table 3‐1 Legacy 2.45Gbit CPRI traffic over Ethernet, requirement v performance ...................................... 28


Table 3‐3 lower layer split at the UPPER PHY, requirement v performance.................................................. 31

Table 3‐4 lower layer split at the MAC/PHY, requirement v performance .................................................... 33

Table 3‐5 Component of the video streaming use‐case ............................................................................... 38

Table 3‐6 Measurements for mobile clone .................................................................................................. 39

Table 3‐7 KPIs for mobile clone .................................................................................................................. 40

Table 3‐8 Component of the mobile task offloading use‐case...................................................................... 41

Table 3‐9 KPIs for mobile phone ................................................................................................................. 42

Table 3‐10 KPIs for mobile task off‐loading ................................................................................................. 44

Table 3‐11 D2D Communications performance ........................................................................................... 46

Table 3‐12 HLS split at the PDCP‐RLC layer, requirement v performance ..................................................... 48

Table 4‐1CPRI over Ethernet ‐ MAC payload‐sizes ....................................................................................... 50



Table 4‐4 lower layer split at the UPPER PHY, requirement v performance.................................................. 59

Table 4‐5 Measurement results for the packet and subframe latencies for the baseline cases (no contention)

(x indicated not measured) ................................................................................................................ 62

Table 4‐6 Estimated packet and subframe latencies for the baseline case (no contention) for 10GbE

networking (x indicates not measured) .............................................................................................. 62

Table 4‐7 Mean and standard deviation (STD) of the packet and subframe latencies with the iCIRRUS

aggregation set‐up. The values are extrapolated from the measured results....................................... 62

Table 4‐8 MAC‐PHY LLS, requirement v performance .................................................................................. 63

Table 4‐9 Backhaul (unified transport), requirement v performance ........................................................... 66

icirrus contract no. 644526 1 jan 2015 – 31 dec 2017 · enb enhanced node b (radio base station...

Documents