delivarable d4.a - annex to d4.1 and d4.2 mucaps prototype and experimentation

Seventh Framework STREP No. 317846 D4.A Public

Version 1.0 Page 1 of 30 Copyright 2015, the Members of the SmartenIT Consortium

Socially-aware Management of New Overlay Application Traffic with

Energy Efficiency in the Internet European Seventh Framework Project FP7-2012-ICT- 317846-STREP

Deliverable D4.A (Annex to D4.1 and D4.2)

MUCAPS Prototype and Experimentation

The SmartenIT Consortium

Universitt Zrich, UZH, Switzerland Athens University of Economics and Business - Research Center, AUEB, Greece Julius-Maximilians Universitt Wrzburg, UniWue, Germany Technische Universitt Darmstadt, TUD, Germany Akademia Gorniczo-Hutnicza im. Stanislawa Staszica w Krakowie, AGH, Poland Intracom SA Telecom Solutions, ICOM, Greece Alcatel Lucent Bell Labs, ALBLF, France Instytut Chemii Bioorganicznej PAN, PSNC, Poland Interoute S.P.A, IRT, Italy Telekom Deutschland GmbH, TDG, Germany

Copyright 2015, the Members of the SmartenIT Consortium

For more information on this document or the SmartenIT project, please contact:

Prof. Dr. Burkhard Stiller Universitt Zrich, CSG@IFI Binzmhlestrasse 14 CH8050 Zrich Switzerland

Phone: +41 44 635 4331 Fax: +41 44 635 6809 E-mail: [email protected]

D4.A Seventh Framework STREP No. 317846 Public

Page 2 of 30 Version 1.0 Copyright 2015, the Members of the SmartenIT Consortium

Document Control

Title: D4.M MUCAPS Prototype and Experiments Type: Public Editor(s): Sabine Randriamasy, Frdric Faucheux E-mail: [email protected], [email protected] Author(s): Sabine Randriamasy, Frdric Faucheux Doc ID: D4.A-v1.0

AMENDMENT HISTORY

Version Date Author Description/Comments

V0.1 2015/10/19 Sabine Randriamasy First version

V0.2 2015/10/30 Frdric Faucheux Revised version after comments by Roman Lapacz and Jeremias Blendin

V1.0 2015/10/30 Sabine Randriamasy, Frdric Faucheux

Completed version after fixing formal issues

Legal Notices The information in this document is subject to change without notice. The Members of the SmartenIT Consortium make no warranty of any kind with regard to this document, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The Members of the SmatenIT Consortium shall not be held liable for errors contained herein or direct, indirect, special, incidental or consequential damages in connection with the furnishing, performance, or use of this material.



Table of Contents

1 Executive Summary .................................................................................................. 5 2 Introduction ............................................................................................................... 6 3 MUCAPS and VITALU in the SmartenIT Architecture............................................. 7

3.1 MUCAPS ..............................................................................................................................................7 3.1.1 Integrating MUCAPS with the SmartenIT architecture..................................................................8 3.1.2 MUCAPS implementation and execution: entities and interfaces .................................................9

3.2 Using VITALU to Evaluate MUCAPS .................................................................................................10

4 MUCAPS Prototype Design and Instrumentation ................................................. 11 4.1 MUCAPS Prototype Design................................................................................................................11 4.2 MUCAPS Prototype Set-up and Configuration...................................................................................11

4.2.1 TinyDNS......................................................................................................................................11 4.2.2 Service handling..........................................................................................................................12 4.2.3 MACAO Block .............................................................................................................................13

4.2.3.1 Newconfig_initial.cfg................................................................................................................13 4.2.3.2 Mappingconfig.cfg ...................................................................................................................13 4.2.3.3 user_endpoint_config.cfg ........................................................................................................13 4.2.3.4 Metric_weights.cfg...................................................................................................................13 4.2.3.5 Newconfig.cfg..........................................................................................................................14

4.2.4 ALTO Server ...............................................................................................................................15 4.2.4.1 Network-map.txt ......................................................................................................................15 4.2.4.2 Pidcosts.txt ..............................................................................................................................15

4.3 Prototype Instrumentation for Evaluation and Assessment ...............................................................16 4.3.1 QoE performance measurement points and KPIs ......................................................................17 4.3.2 The VITALU QoE analyzer..........................................................................................................17

5 Experiments Definition and Set-up for MUCAPS.................................................. 19 5.1 MUCAPS Experiments .......................................................................................................................21

5.1.1 Functional tests on the DIG command........................................................................................21 5.1.1.1 Parameters, Measurements, and Metrics ...............................................................................22 5.1.1.2 Functionality test documentation .............................................................................................23

5.1.2 Performance tests with QoE analysis .........................................................................................24 5.1.2.1 Parameters, Measurements, and Metrics ...............................................................................24

5.2 MUCAPS Showcase ..........................................................................................................................24 5.2.1 MUCAPS Showcase Scenario 1 Video Streaming on a High Resolution PC ..........................25

5.2.1.1 Scenario Topology...................................................................................................................25 5.2.1.2 Showcase Scenario.................................................................................................................26

5.2.2 MUCAPS Showcase Scenario 2 Video Streaming on a Mobile Phone....................................26 5.2.2.1 Scenario Topology...................................................................................................................26 5.2.2.2 Showcase Scenario.................................................................................................................26

6 Summary and Conclusions .................................................................................... 28 7 References ............................................................................................................... 29 8 Abbreviations........................................................................................................... 30



(This page is left blank intentionally.)



1 Executive Summary The SmartenIT project has selected two core mechanisms which are DTM and RB-HORST for full prototyping and experimentation in WP4. These mechanisms provide a full traffic management and transportation solution for the EFS and OFS scenarios. Multi-partner specifications of these mechanisms and their implementations started in Q1 2014 within SmartenIT. In Q1 2015, it was decided to add MUCAPS to the achievements of SmartenIT in an Annex deliverable. Contrary to DTM and RB-HORST, MUCAPS is not a fully fleshed multi-partner developed mechanism, but was proposed as an add-on to optimize application traffic management based on a transport network layer aware paradigm that is being standardized at the IETF as the ALTO (Application Layer Traffic Optimization) protocol, specified in RFC 7285. MUCAPS has been defined and prototyped by ALBLF. MUCAPS revises the initial Endpoints selection done by the application overlay by adding awareness on the underlying network topology and on transport costs through information not available by traditional end to end on-line measurement mechanisms. The selection criteria used in MUCAPS are an abstraction of end-to-end performances on application paths. This abstraction relies itself on an ISP-centric abstraction of the Internet topology, provided to applications via the IETF ALTO protocol and one of its extensions supporting several network cost metrics and constraints combining several logical operators and metrics, see [1]. MUCAPS also integrates the access technology used by the device receiving the application resource in its decision making. Besides MUCAPS, ALBLF had been developing a QoE measurement and analysis tool called VITALU that measures a set of QoE specific metric values at devices receiving video files and devices and derives a Video Quality Score (VQS) built in a similar way to a Mean Opinion Score. VITALU is in a pre-product stage and represents an effort of about 6 person years. VITALU has been customized within the SmartenIT project to support measurements done on mobile phones and further analysis functions, candidates to the QoE analysis entity pictured in the SmartenIT architecture. The purpose of this annex is to complement, for MUCAPS, the test-bed definition provided in D4.1 and experimentation definitions provided in D4.2 that were provided for the multi-partner implemented solutions DTM and RB-HORST.



2 Introduction MUCAPS has been defined and prototyped by ALBLF as an add-on mechanism to further optimize application traffic management based on a transport network layer aware paradigm that is being standardized at the IETF as the ALTO (Application Layer Traffic Optimization) protocol, specified in RFC 7285. MUCAPS stands for "MUlti-Criteria Application endPoint Selection" and has been specified in Section 4.11 of Deliverables D2.2 [4] and D2.4 [5]. MUCAPS addresses the EFS scenario, in particular the use cases Locality in UNaDa and Service and content placement for users. The MUCAPS added value to application traffic management and SmartenIT is a further optimization of overlay application traffic. To this end, MUCAPS revises the initial Endpoints selection done by the application overlay by adding awareness on the underlying network topology and on transport costs through information not available by traditional end to end on-line measurement mechanisms. The selection criteria used in MUCAPS are an abstraction of end-to-end performances on application paths. This abstraction relies itself on an ISP-centric abstraction of the Internet topology, that is available to applications via the IETF ALTO protocol, documented in [1]. MUCAPS leverages proposed ALTO protocol extensions, extending the base protocol set of ALTO metrics and allowing ALTO transactions with multiple metrics and constraints combined with multiple logical operators allowing richer compromises. So MUCAPS can help out any SmartenIT mechanism that supports applications having multiple candidate resources locations and that have no awareness of the underlying network. The MUCAPS design integrates an extension to the ALTO protocol called Multi-Cost ALTO that enables ALTO Clients to request path costs on several metrics jointly and also allows clients to place constraints combining several metrics and logical operators. Multi-Cost ALTO is now an IETF ALTO WG item, see [1]. MUCAPS also integrates awareness of the access technology used by a device receiving the application resource. In the remainder of this document, section 2 complements D4.1 with the prototype set-up and configuration for MUCAPS and section 3 complements D4.2 with the experiments definition and Set-up.



3 MUCAPS and VITALU in the SmartenIT Architecture While MUCAPS main outline is presented first, the use of VITALU to evaluate MUCAPS is described thereafter.

3.1 MUCAPS The ALTO protocol is integrated in the SmartenIT paradigm: the ALTO Server stores and provides abstracted transport network information to requesting ALTO Clients associated to applications. However, the ALTO Client alone cannot automatically decide what information to request and how to process it once received. MUCAPS builds the decision making around the ALTO Client. It assists the choice of the location from which to download content. Locations can be various entities including Video Server, Data Center, uNaDa and Peers. Content covers video streams as well as computing resources for virtualized or distributed applications. MUCAPS is suitable for applications having the choice among several feasible Application Endpoints (AEP) and supports multi layer and multi party incentive decision making by involving the following aspects in the AEP selection:

Topology distance: number of hops (inter or intra AS), impacting delay, Routing cost: reflecting the ISP policy in terms of AS peering agreements, ISP preferences in terms of resources availability (e.g., path bandwidth).

An ALTO Client has no intelligence to choose the AEP selection metrics and the ranking method, as ALTO is mainly a transport means for abstracted network information. The initial selection of overlay Endpoints, referred to in MUCAPS as Application Endpoints is usually produced by gathering functions such as DNS Servers and Clients, DHTs, Peer Trackers and referred to in MUCAPS as Application EndPoint Gathering (AEPG) functions. Figure 1 illustrates how MUCAPS is involved in a content downloading scenario, as detailed in SmartenIT deliverables D2.2 [4] and D2.4 [5]. In this figure, the term MACAO stands for Multi Access and Cost ALTO. Basically, the MACAO Client block does (or performs) the following actions:

decides for which metrics an ALTO Client must request values, decides which weight is applied to the metrics, and then uses the obtained values to rank the AEPs.

MUCAPS comprises three parts: a MACAO Service Client (MARC) that requires from the MACAO Service interface (MARS) an additional ranking of the initial AEP selection. The MARC is hooked to an AEPG that may call it for enhanced ranking and hands it from it the initial list of AEPs, identified in the current implementation, by their IP addresses. The MARS is just an interface to the MACAO block that does the actual selection and ranking. The MACAO block with its MARS is entirely deployed in the provider network. Finally, the MARC and the MARS communicate via a simple IP Socket carrying a light message. The message typically includes: a list of AEP IP addresses, the MUCAPS service ID pointing here to AEP Ranking and the number of AEPs to rank.



Figure 1 Involvement of MUCAPS components (MACAO, MARC and MARS) in a content downloading scenario.

3.1.1 Integrating MUCAPS with the SmartenIT architecture The impact of MUCAPS on the architecture is shown in Figure 2 with following modifications:

The ALTO component in the SBox is replaced by the set {MACAO block, MARS interface and ALTO Server}. The ALTO Client is now in the MACAO block.

There is no more need to have an ALTO Client in end systems involved in overlay, that is: Data Centers, uNaDas and End User Entities. Instead, the ALTO Client there is replaced by the MARC that sends to the MARS in the SBox, MUCAPS requests, that is requests to re-order the candidate AEP list wrt ALTO based network information.

The MARC is invoked by the Overlay Manager or the application of EUEs and communicates with the MARS located in the SBox via a simple IP socket.

An alternative mapping of MUCAPS to the SmartenIT architecture is also described in deliverable D3.3. It consists in replacing the ALTO (Client) component in the Overlay end systems by the MARC, thus keeping the current interfaces established with MUCAPS/ALTO replacing ALTO. This setting must however assume that the MARC can be hooked with an AEPG function located in the Overlay Manager of the Data Center and the uNaDa or in some appropriate place in the End User Entity. The following SmartenIT architecture extensions are needed:

A placeholder is needed in the EUE for an AEPG that could call the MARC. Interfaces should now be established between the MUCAPS/ALTO block and the

MARCs.



Figure 2 Entities and components involved in MUCAPS.

3.1.2 MUCAPS implementation and execution: entities and interfaces In the current MUCAPS deployment, the MARC is located in the ISP network. This is the case for instance when the ISP wants to ensure that the DNS selection of the application network is compliant to its policy in resources usage. In that case, the ISP captures the response of its DNS Resolver, processes it with the MACAO module and sends the MACAO selection to the DNS Resolver that sends it to the DNS Client. All this is done in a transparent way for end users and applications. As illustrated in Figure 3, the MACAO functional block has an external interface called MARS for MACAO Request handling Server allowing it to communicate with a MARS Client (called MARC) hooked to the AEPG. The network-side deployment enables a lightweight transaction between MUCAPS and the ISP DNS resolver by: adding to the MUCAPS block a MARS service interface, becoming thus a MACAO Request Server (MARS). This deployment includes the following steps: 1. The application client queries the DNS server for the list of content servers to

download/upload from. 2. The DNS Server,

2.1. selects the MACAO Service Endpoint Ranking



2.2. hands to its embedded MARC its query result, that is the list of the candidate Application Endpoint IP addresses, together with the IP address of the User Endpoint, that is the DNS Client associated to the user device,

3. sends this light information to the MARS IF via an established IP Socket, as a query to perform the multi-cost ranking of the AEPs,

4. The MUCAPS block performs the ranking and the MARS sends the reordered list of AEPs to the MARC.

The MARC passes it to the AEPG/DNS Server that sends it to the requesting AEPG/DNS Client.

Figure 3: MUCAPS entities and interfaces: MARC integrated in a network located AEPG, here a DNS server.

3.2 Using VITALU to Evaluate MUCAPS VITALU uses and analyses PCAP (Packet CAPture) messages either produced by tools such as: wireshark, tshark, tcpdump, tpacketcapture, or that it captures by launching a tshark instance, or gets on a non rooted Android mobile phone via tpacketcapture (with all related issues) The procedure to follow is to use disjoint networks for media download and PCAP sending (e.g. WiFi for video and Ethernet for PCAP sending ) to avoid interferences between pcap and media flow, or simply (and probably better) to wait until the end of the media flow and then transfer the captured PCAP to the VITALU machine for analysis.



4 MUCAPS Prototype Design and Instrumentation The MUCAPS test-bed has been designed to: Evaluate the ability of MUCAPS to revise the initial AEP selection whenever it was

appropriate, Evaluate the impact of the MUCAPS selection on the QoE. A MUCAPS intervention is qualified as appropriate whenever the initial AEP selection does not optimize ISP costs and other metrics that impact the application QoE. The applicable ISP is the one hosting the UEP. The other metric considered is a bandwidth score, rating the bandwidth performance of the end to end path between an AEP and the UEP. All metrics and their values are assumed to be set by the ISP. In the present evaluation, the initial AEP selection is assumed to be done by the ISP DNS resolver.

4.1 MUCAPS Prototype Design The design of the prototype for MUCAPS with the involved hardware is shown in Figure 4.

Figure 4 MUCAPS prototype and involved hardware

4.2 MUCAPS Prototype Set-up and Configuration

4.2.1 TinyDNS TinyDNS, a part of djbdns software package, is a popular open source DNS server. It has been modified in order to include the support of Macao API.



In order to install the customized version of tinydns, the following command must be run: cd /home/alto/workspace/tinydns ./install-tinydns-alto.sh tinydns-conf tinydns dnslog /etc/tinydns @IP_DNS_Server

where @IP_DNS_Server: represents the IP address of DNS Server or of host.

The next step is to fill the tinydns database with the IPs of the different video servers. This can be done using the following commands:

./add-ns alto 10.201.65.165 # add dns server

./add-host video.alto 10.201.70.1

./add-host video.alto 10.201.70.2

./add-host video.alto 10.201.170.133 make # recompilation to create file data.cdb

The first commands modify a file called /etc/tinydns/root/data that contains the link between the IP addresses and the domain names. It can also be modified manually. Here is an example of the data file:

.alto:10.201.65.165:a:259200

=video.alto:10.201.70.1:10 =video.alto:10.201.70.2:10 =video.alto:10.201.170.133:10 =server1.alto:10.201.70.1:10

=server2.alto:10.201.70.2:10

=server3.alto:10.201.170.133:10

The last three entries have been added in order to be able to address the three servers individually. After the data file is modified, the tinydns-data command should be run in order to validate the changes in the running tinydns server.

Another file to consider is /etc/tinydns/env/IP. It contains the IP address at which the tinydns server listens. If it is set to a specific IP (like 10.201.65.165 in the example) it will only listen from this address. As the server is using a different IP address for LAN access, Wi-Fi and Cellular, SmartenIT needs the tinydns server to be able to listen to all of these addresses. The simpler way to achieve this behavior is to put the IP address 0.0.0.0 in the file /etc/tinydns/env/IP. By doing this tinydns will answer dns queries coming from any network interface. Tinydns needs to be restarted after changing this file.

4.2.2 Service handling In order to automate the launching of tinydns it can be used as a linux service. In order to achieve this the following commands can be issued:

ln -s /etc/tinydns /service/ sudo svscan /service &



svc -u /service/tinydns svstat /service/* netstat -anpe | grep 53 # check that dns service is launched on port 53.

4.2.3 MACAO Block When no MACAO server is launched, the tinydns server returns the results directly from its internal database. When the server is launched, the tinydns transmits the results from the internal database to the MACAO block so that it sorts the results depending on the configuration. In order to install and run the server, the following commands should be run in root mode

cd ~/workspace/MACAO2012/ALTO-CLIENT-BLOCK/ALTO_Application_server/ unset http_proxy # remove proxy if there is. ./server # launch the MACAO block

The Macao block reads the following configuration files.

4.2.3.1 Newconfig_initial.cfg This file indicates is ALTO should be used or not and if the Automatic mode is enabled or not.

Status = 1; # 0=ALTO OFF, 1=ALTO ON Metric_Automatic = 1; # 0=AUTO mode OFF, 1=AUTO mode ON

4.2.3.2 Mappingconfig.cfg This file stores the IP addresses of the destination endpoint and identifies the type of server (video, audio or other) depending of its IP address :

vidservers = [ "10.201.170.133", "10.201.70.101", "10.201.70.102" ]; audservers = [ "10.201.70.5", "10.201.70.6", "10.201.70.7" ]; otherservers = [ "10.201.80.8", "10.201.180.8", "10.201.222.2" ];

4.2.3.3 user_endpoint_config.cfg This file is used to recognize on which link the client IP address (i.e. the address issuing the DNS request) is connected. An example of its content is the following:

LAN = [ "10.201.70.102", "10.201.70.103", "10.201.70.14" ]; WiFi = [ "10.201.70.52", "192.168.1.145", "192.168.1.104" ]; Cellular = [ "10.201.70.10", "192.168.42.208" ];

4.2.3.4 Metric_weights.cfg This file contains the weights associated to each metric that can be considered, and for each of the network interfaces.



The syntax is as follows: LAN :

{ routingcost = 1.0; hopcount = 1.0;

propadelay = 1.0;

bandwidth = 1.0; }; WiFi : { routingcost =2.3; hopcount = 4.0;

propadelay = 1.0;

bandwidth = 1.0; }; cellular :

{ routingcost = 3.0; hopcount = 2.0;

propadelay = 3.0;

bandwidth = 3.0; };

4.2.3.5 Newconfig.cfg This file is used to modify the weights of the different metrics depending on the chosen configuration (Routing Cost, Routing Cost + BW Cost). Its syntax is the following

nummetrics = 2; # number of metrics to use elements = ( { costtype = 1;

costmode = 1;

weights = 1.0; }, { costtype = 2;

costmode = 1;

weights = 1.0; } );



service = 1; topN = 0;

4.2.4 ALTO Server The ALTO server answers to the MACAO requests by giving the costs of the different metrics for the source and destination IPs considered. The software was originally designed to use Routing Cost and Hop Count as metrics, but the decision was later changed to Routing Cost and Bandwidth Score. However, the software was not updated and the name hopcount is still used in configuration files, but is now used to represent Bandwidth Score.

4.2.4.1 Network-map.txt This file allows identifying the equipments access type by their IP address. The sample file used during the experiments is the following:

###### UEP-PIDS ###### ## LAN mypidLAN 10.201.65.165/18 ## WIFI mypidWiFi 192.168.1.0/24 ## 3G mypid3G 192.168.42.0/24

###### AEP-PIDS ###### mypid101 10.201.70.1/32 mypid102 10.201.70.2/32 mypid133 10.201.170.133/32

4.2.4.2 Pidcosts.txt This file defines the costs between two equipments depending on the access type. An example of use is the following:

##### COSTS for routingcost ##### type=routingcost setall=999 ## LAN mypidLAN,mypid101,7 mypidLAN,mypid102,11 mypidLAN,mypid133,60 ## WIFI mypidWiFi,mypid101,7



mypidWiFi,mypid102,11 mypidWiFi,mypid133,60 ## 3G mypid3G,mypid101,7 mypid3G,mypid102,11 mypid3G,mypid133,60

##### COSTS for hopcount/BWScore ##### type=hopcount setall=1 ## LAN mypidLAN,mypid101,8 mypidLAN,mypid102,18 mypidLAN,mypid133,20 ## WIFI mypidWiFi,mypid101,8 mypidWiFi,mypid102,18 mypidWiFi,mypid133,20 ## 3G mypid3G,mypid101,8 mypid3G,mypid102,18 mypid3G,mypid133,20 defaultpid,defaultpid,0

4.3 Prototype Instrumentation for Evaluation and Assessment MUCAPS aims at jointly optimizing ISP transport costs and path bandwidth usage while maintaining or improving end user QoE. To do so, MUCAPS must Appropriately choose the AEP selection metrics wrt the type of application, Use the ISP defined values on the selection metrics, stored in the ISP managed ALTO

Server, Appropriately set the selection metric weights according to the access type of the UEP, Rank the candidate AEPs according to the above cited elements Once the UEP connects to the AEP selected with the MUCAPS support, the resulting impact must be measured in terms of User QoE as measured by a fully fleshed QoE measurement and analysis tool, Theoretical MUCAPS inferred ISP costs based in the ISP ALTO information



4.3.1 QoE performance measurement points and KPIs The following key performance indicators are considered to evaluate the performance of MUCAPS. The measurement points are summarized in Table 1.

Table 1 Key performance indicators and measurement points for MUCAPS

KPI End user mobile devices

End-user PC UEP

Cross-Layer Utility (CLU)

X

Video Quality Score Directed to PC X Start time delay Directed to PC X NFRz, DFRz (number and duration of freezes

Directed to PC X

Media Bit Rate Directed to PC X

The Cross-Layer Utility (CLU) is the scalar value that MUCAPS outputs on which its final ranking is based. It can be measured at the UEP or any device that receives the MUCAPS response information, including ALTO responses and MUCAPS vector based ranking. It is a theoretical value that assesses the MUCAPS influence on the AEP choice. The rest of the KPIs are measurements, done by the VITALU QoE analyzer on PCAP files associated to video streams.

4.3.2 The VITALU QoE analyzer VITALU has been developed in ALBLF and transferred to an Alcatel-Lucent Business Unit. The primary goal of the tool is to evaluate/monitor the QoE for root cause of multimedia delivery issues. VITALU handles multiple streaming protocols such as HTTP Live, RTSP, RTP Multicast, or Flash HTTP. VITALU is part of service offer portfolio for operator customers. VITALU has the following main functions for each monitored video flow: Collect/Analyze IP network info (Packet Lost, Jitter, Delay, Timestamp, Sequence

number), Compute video quality metrics ( Video Score, Freezes, Frame Errors) Provide frame by frame analysis, with jitter buffer simulation for RTP protocol The metrics measured by VITALU include: DateTime Capture (s), Date, Clip Name, Resolution, Operator, Session ID, RTT (Avg, Std, Max), Video Access Time (s), Video Duration (s), Video Bitrate (bps), VQS Avg (Real), VQS Avg (Theor.), VQS Std (Real), VQS Std (Theor.), Frame Error (%), Freeze (%), Nb of Freezes, Freeze Duration (Avg) (s), Freeze Duration (Max) (s), Nb of Error Frames, Nb. of Packets, Packets Retransmitted (%), Nw Video Jitter (Avg) (s), Nw Video Jitter (Max) (s), Nw Bitrate (Avg) (bps), Nw Bitrate (Std) (bps), Late & Err. Frame Ratio (%), Frame Rate (Avg) (fps), Frame Rate (Std) (fps), Nb of Frame Resolutions, Frame Resolution (px).



SmartenIT uses 6 metrics derived from the above measurements list which are: Media Bit Rate, Start Time, RTT, VQS, Freeze Nb, Freeze Duration (avg), Freeze Duration (max). VITALU is also able to evaluate these metrics on partially captured streams, if the source video file is provided. For this experiment the VITALU tool was used to automatically assess the file captured on the UEP (after transmission of the captured file to the VITALU server). The results are then stored to a .csv file that can be analyzed offline.



5 Experiments Definition and Set-up for MUCAPS The end-user focused (EFS) scenario aims at providing increased QoE and ISP costs and network resources efficiency. In particular, this goal is achieved by applying an in-network optimization of application endpoint selection that takes interests of both ISPs and application resources consumers into account. In this context, MUCAPS revises the initial endpoint selection done by applications by directly involving of ISP-defined network costs and performance rating together with the access-type of the end-user device in its final decision. The MUCAPS intervention is transparent to the user, as the functionality is embedded in the ISP network. In the SmartenIT use case, MUCAPS is hooked to the ISP DNS resolver. The EFS experiments type for MUCAPS will be: prototype validation. Their goal is to validate its set of functionalities, evaluate its performances and benefits to end-users and ISPs on a set of sample configurations. Functional tests will validate the ability of the VLC video reader to integrate MUCAPS

when searching the video, the ability of MUCAPS to catch context information like End User Endpoint (UEP) access, Application Endpoint (AEP) localization by tinyDNS, establish a connection with the ALTO Server and the MARS. All this for three UEP access types: LAN/FTTH, WiFi/ADSL, 3G

The performance evaluation will be done by the QoE analyzer VITALU developed by ALBLF. During a video stream, TcpDump (or other network analyzer software such as Wireshark) will be used to capture the PCAP files.

Figure 5 shows the topology of the MUCAPS experiments. Figure 6 shows this topology mapped to the prototyping environment.

Figure 5 Topology of the MUCAPS prototype experiments



Figure 6 MUCAPS topology mapped to the evaluation environment

Two types of experiments are being conducted Functional tests by:

o Launching a dig command on the UEP, to validate the prototype, o Leading a video with VLC (for human visual assessment) or WGET (for automated

tests) on an end-user system that is either a PC or a mobile phone, to see whether the MUCAPS processing was launched via VLC or WGET.

Impact tests by: o Capturing the PCAP files generated by TCPDUMP of Wireshark, o Computing and analyzing the QoE metric values and resulting VQS.

For LAN tests, a web proxy had to be used in order to be able to access external video servers on the internet. In this case the name resolution was done on the web proxy server and SmartenIT had to change its configuration so it uses our tinydns server. For Wi-Fi tests, the PCs had a direct internet connection. For 3G tests, the mobile phone was used as a usb-modem for the PC hosting the video player: this was done because SmartenIT has to use a specific DNS server and that is not possible with an unrooted smartphone. The link to the internet was through the 3G/4G access of the smartphone but the video player and DNS resolution was performed on a linux PC. Some of the problems are linked to the use of a fake name server for our experiments. In a near future ALBLF plans to use a real name server in order to be able to use the default DNS configuration and be able to use it directly on unrooted smartphone for example. In addition, the following Table 2 provides the hosts list and the services used to run the MUCAPS experiments.



Table 2 Hosts and the services running on them for MUCAPS experiments

Host Services Comment isp1-PC1 UEP receiving the video on a PC isp1-Phone UEP receiving a video on a mobile phone isp1-PC2 tinyDNS+MACAO isp1-PC3 ALTO Server isp1-PC4 VITALU QoE Analyzer

isp1-AEP1 Video Server 1 in ISP1 high BW Score, moderate Routing Cost

isp1-AEP2 Video Server 2 in ISP1 Low BW Score, low Routing Cost

isp2-AEP3 Private Video Server 3 in ISP2 High BW Score, high Routing Cost

Table 3 provides the IP addresses of the Hosts listed in Table 2 that are involved in MUCAPS-assisted video streaming. The QoE analyzer is not listed as it performs off line analysis. AEP3 is a private server that only has a public address, which will for confidentiality not be given in the SmartenIT documentation. Addresses of hosts other than LAN addressed are partially masked for the same reasons.

Table 3 Hosts involved in the MUCAPS experiments and their addresses

Host LAN VPN PUBLIC PC1-UEP LAN 172.alu.vpn.uep

WiFi 192.168.pc1.uep

Phone-UEP 3G 1982.198.mph.uep PC2-DNS+MACAO 172.alu.vpn.165 PC3-AOS 172.alu.vpn.153 AEP1 10.201.70.1 172.alu.vpn.142 sp1.pub.srv.113 AEP2 10.201.70.2 172.alu.vpn.51 sp1.pub.srv.111 AEP3 spB.prv.srv.37

Table 4 provides the ISP defined metric costs, for each AEP class.

5.1 MUCAPS Experiments The following tests are defined to validate functionality and estimate performance.

5.1.1 Functional tests on the DIG command The dig command performs a DNS name resolution taking as input the domain name from the URI of the requested video, composed of the domain name and video file name.



The dig command shows how the name resolution is performed by the VLC reader when the URI of the requested video is entered, using a system library function such as gethostbyname. On a PC, it can be launched as a standalone command with optionally the address of the desired DNS resolver. The functional tests consist in launching a dig command on a PC and on a VLC reader and verify the following elements: Response with MUCAPS OFF, Response with MUCAPS ON and both selection metrics RC and BW activated, for all

3 access types. The functionality tests ensure that the MUCAPS functionality works as expected when the dig command is triggered by VLC. The assumption to see the MUCAPS changing the initial selection is that the initial endpoint selection does not jointly optimize the ISP routing cost and path bandwidth performance.

5.1.1.1 Parameters, Measurements, and Metrics ALTO Values for the 3 AEPS The AEPs have been mapped to 3 classes depending on their end2end path performances on RC and BWS that is specified in the ALTO Server. MUCAPS performs a multi-objective vector based ranking of the AEPs, which is more reliable. To better illustrate the abilities of MUCAPS to do so, the 3 classes of AEP are built so as to be mutually non-dominated in terms of performances. The RC and BWS values on the AEPs used for the experiments are the values of the AEP classes utilized for the theoretical evaluation described in D2.5. The three AEP classes are characterized as follows: AEP1 Class C1: moderate cost, high path bandwidth: RC = 11, BWS = 18 AEP2 Class C2: low cost, low path bandwidth: RC = 7, BWS = 8 AEP3 Class C3: high cost, high path bandwidth: RC = 60, BWS = 20 Setting ALTO BWS and RC values Although ALTO values are supposed to be specified by the ISP, the BWS values are set so as to be mapped to the downlink rates measured in France with different access types. Sources [2] and [3] provide access-dependent value ranges (min-max) for e2e downlink path bandwidth, based on measurements performed from 2013 to today in France. SmartenIT, thus, considers the following ranges and access types: FTTH [45-160] Mb/s ADSL [6-9] Mb/s Cell 3G: [4-7] Mb/s, In the AOS used by MUCAPS, BWS is a score with values in [1, 20]. In order to mitigate to effect of FTTH (Fiber-to-the-Home) performance, the mapping is done so as to mitigate the effect of FTTH performance. This approach took FTTX as a mixture of FTTH, FTTB (Fiber-to-the-Basement), and TTS (Fiber-to-the-Street).



Likewise, RC is completely dependent of the ISP policy. SmartenIT nevertheless assumes that RC may be influenced by factors such as OPEX and peering agreements. The resulting mapping was, therefore, adopted as a basis for example AOS values.

Table 4: Typical values for metrics depending on access type

ACCESS BW Range BWS median RC basis Cell 3G [4-7] Mbs 6.7 15 WIFI/ADSL [4-9] Mbs 7.2 9 FTTX [45-140+] Mb 17.5 11

The ALTO values of the AEPs have, therefore, been chosen as defined in Table 5. Table 5: ALTO values for each AEP

RC BWS AEP1 11 18 AEP2 7 8 AEP3 60 20

Finally, Table 6 recalls weights associated to those metrics depending on the UEP access type.

Table 6: Weights of metrics depending on access type

FTTX WiFi/ADSL 3G RC 1 2.3 3 BWS 1 4 2

5.1.1.2 Functionality test documentation The results of functional tests on the dig command will be documented in 2 different ways: A screen copy of the result of a dig command will be provided, as an illustration Results of dig commands will be provided in terms of RC, BWS and CLU value

associated to the AEP selected in the different modalities of MUCAPS activation. MUCAPS test modalities: for each UEP access type

o G1: MUCAPS OFF o G2: MUCAPS ON + RC o G3: MUCAPS ON + BWS o G4: MUCAPS ON + (RC, BWS)



5.1.2 Performance tests with QoE analysis The performance tests quantify the impact of MUCAPS on the QoE

5.1.2.1 Parameters, Measurements, and Metrics Deployment Infrastructure The deployment infrastructure is the one depicted in Figure 6. The location of the different elements is as follows: The ALTO Server (AOS), AEP1, AEP2 are located in the ALBLF premises, The UEP and MACAO blocs are located in remote premises of the Paris area and

connect to AOS, AEP1 and AEP2 via a VPN Server AEP3 is a private server belonging to another ISP Assuming the initial selection is AEP3 the tests groups have been defined as follows. Video sessions (2 types wrt length, resolution, internals such as movements).

With UEP in different access types: LAN/FTTX, wifi/ADSL, cellular 3G With different devices (PC, Phone) AEP types: video public server, private server

QoE Measurements with VITALU: On each video session: the QoE metrics will be measured

MUCAPS test modalities: for each video class and each UEP access type, the tests will only consider:

G1: MUCAPS OFF G4: MUCAPS ON + (routingcost, BWscore)

For each test group Gi, the QoE performance is measured. The QoE results are compared and impact is derived.

5.2 MUCAPS Showcase MUCAPS can be showcased following the 2 scenarios proposed below: Scenario 1 as Video Streaming on a High Resolution PC and scenario 2 as Video Streaming on a Mobile Phone. When the ALBLF VPN is accessible to the UEP, the experimental topology as described in Figure 7 is suitable. If the ALBLF VPN is not accessible to the UEP and to minimize the involved hardware MUCAPS can be showcased using the topology depicted in Figure 8. The showcase topology could even be reduced to having PC1 and PC2 merged so as to host the whole (UEP, tinyDNS, MACAO, AOS) set as all its elements can be assumed to belong to the same ISP.



Figure 7 Initial MUCAPS showcase topology

PC2/ISP1

VM

tinyDNSresolver

VM

MACAO

PC 1/ISP1

UEP

VLC

VM

ALTO Server

VideoServerAEP3/ISP2

PHONE

VLC

eth



WLAN

RAN

eth

eth

Figure 8 MUCAPS showcase topology when the UEP is not accessing its VPN

5.2.1 MUCAPS Showcase Scenario 1 Video Streaming on a High Resolution PC

5.2.1.1 Scenario Topology The MUCAPS showcase topology on LAN is illustrated in Figure 9.



Figure 9 MUCAPS showcase topology on LAN

5.2.1.2 Showcase Scenario This scenario goes through the following steps

1. UEP on a PC requests a video V1 without MUCAPS a. DNS selects AEP3 b. Transfer at high routing cost (RC), high path BW score (BWS)

2. UEP requests V1 with DNS assisted by MUCAPS+RC a. DNS-MACAO points to AEP2 b. Transfer at low cost but low quality

3. UEP now requests V1 with DNS assisted by MUCAPS + (RC, BW) a. DNS-MACAO points to AEP1 b. Transfer at moderate RC AND high BWS

5.2.2 MUCAPS Showcase Scenario 2 Video Streaming on a Mobile Phone

5.2.2.1 Scenario Topology The MUCAPS showcase topology on 3G is illustrated in Figure 10.

5.2.2.2 Showcase Scenario This scenario goes through the following steps and follows showcase of scenario 1.

1. UEP on a mobile phone gets V1 on AEP1 2. It gets a bad QoE

a. because the mobile phone cannot sustain the HR bitrate 3. UEP requests video V1 with DNS assisted by MUCAPS + (RC, BW)

a. DNS-MACAO takes 3G-access type into account and points to AEP3 b. UEP gets a good QoE because AEP3 delivers at a sustainable bitrate c. Transfer at low RC



Figure 10 MUCAPS showcase topology on 3G



6 Summary and Conclusions This deliverable complements the SmartenIT Deliverables D4.1 and D4.2 by adding the respective details of the MUCAPS mechanism. In particular, this annex here Details how MUCAPS can be integrated in the SmartenIT Architecture Presents the MUCAPS prototype set-up and how it is configured during the showcase Analyses the results of experiments Defines the main showcase scenarios. The main outcomes from these experiments is that it was possible to build a prototype that is able to automatically direct an user to the best suiting server depending on the type of service requested (e.g. video, audio, web), and the type of access of the user (LAN, Wi-Fi, 3G). The use of an offline analyzer also shows an improvement in the quality experienced by the user thanks to MUCAPS mechanism rather than using static routing tables. The 2 different showcase scenarios help to understand different configurations possible with MUCAPS, taking into account different metrics and showing which choice will give the best result. Each of the metrics is attributed a different weight depending on the chosen access type, hence providing different results. The showcase, thus, allows to achieve the impact of each of the metrics and of the access type on the chosen server for a specific type. The selected application for the demo is video streaming, as the different quality levels can be easily distinguished and show that the choice of the server will impact the quality experienced by the user. In conclusion, thanks to the MUCAPS prototype used in the experiments and showcases, the benefits linked to this mechanism can be evaluated both objectively and subjectively. The deployment of this mechanism can bring advantages inside the SmartenIT solution or independently and proves to be a very good solution in terms of network performance and user experience.



7 References [1] draft-ieft-alto-multi-cost-01: Multi-Cost ALTO, (work in progress) S. Randriamasy

and W. Roome and N. Schwan, October 19th 2015, https://tools.ietf.org/html/draft-ietf-alto-multi-cost-01

[2] "Baromtre des connexions Internet mobiles en France mtropolitaine Deuxime trimestre 2015 ", nPerf report, 3 Juillet 2015, http://media.nperf.com/files/publications/2015-07-03_Barometre-connexions-mobiles-nPerf-2015-T2.pdf

[3] "Observatoire des dbits Internet", (1/05/2013 to 09/2015), http://www.ariase.com/fr/vitesse/observatoire-debits.html

[4] The SmartenIT project: Deliverable D2.2: Report on Definitions of Traffic Management Mechanisms and Initial Evaluation Results; October 2013.

[5] The SmartenIT project: Deliverable D2.4: Report on Final specifications of Traffic Management Mechanisms and Evaluation Results; October 2014.



8 Abbreviations ADSL Asymmetric Digital Subscriber Line AEP Application Endpoint AEPG Application Endpoint Gathering ALTO Application Layer Traffic Optimization AOS ALTO Server AS Autonomous System BWS BandWidth Score CLU Cross-Layer Utility DTM Dynamic Traffic Management EFS End-user Focused Scenario EUE End User Entity FTTB Fiber-to-the-Basement FTTH Fiber-to-the-Home FTTS Fiber-to-the-Street FTTX Fiber-to-the-X IETF Internet Engineering Task Force ISP Internet Service Provider MACAO Multi Access and Cost ALTO MARC MACAO Service Client MARS MACAO Service interface MUCAPS MUlti-Criteria Application endPoint Selection OFS Operator Focused Scenario OPEX Operational Expenditure PCAP Packet CAPture QoE Quality-of-Experience QoS Quality-of-Service RC Routing Cost RFC Request for Comments SmartenIT Socially-aware Management of New Overlay Application Traffic with Energy

Efficiency in the Internet UEP End User Endpoint uNaDa User Nano Data Center VQS Video Quality Score

delivarable d4.a - annex to d4.1 and d4.2 mucaps prototype and experimentation

Documents