MEASUREMENT STUDY OF P2P MULTIMEDIA STREAMING SYSTEMS 1

Measurement Study of Peer-to-Peer MultimediaStreaming Systems

Jose Lucas

Abstract—Large-scale and heterogeneous peer-to-peer (P2P)-based multimedia streaming systems, for the distributionof Live and time-shifted (or On-demand) contents, have emerged in last years. However, most, if not all, of currentsolutions are closed-source or proprietary in terms of both architecture and streaming protocols used, and hardly reachhigh-quality streaming levels. The work developed in this thesis advances with a proposal of a monitoring solution for P2Pvideo streaming systems that use the protocols being developed by the P2P Streaming Protocol (PPSP) working groupof IETF. The monitoring system provides traffic analysis, with end-users able to monitor quality and system informationon a web-based interface. Monitoring agents, implemented in peers, trackers and media serving nodes, are able tocollect adequate information about the received quality of adaptive and scalable 2D/3D streamed contents in the P2Pnetwork. This information is useful for helping service providers to assess the streaming system performance as well asthe quality of the service.

Index Terms—P2P Multimedia Streaming Systems; PPSP; Quality Assessment; Monitoring System.

F

1 INTRODUCTION

S TREAMING Media services became recentlyhighly globalized, with users from all over

the world not confined anymore to their “homeproviders” to get access to contents. Large-scale and heterogeneous CDN-based or P2P-based streaming systems, for the distribution ofLive and On-demand contents, have emergedto cope with this demand. These systems try toaccommodate the temporal and spatial dynam-ics of the demands, like location and networkheterogeneity and the increasing quality of thedistributed media, so far up to High Defini-tion (HD) videos but still with several qualityconstraints. However, most, if not all, of cur-rent solutions are closed-source or proprietaryin terms of both architecture and streamingprotocols used and hardly reach high-qualitystreaming levels. The Peer-to-Peer StreamingProtocol (PPSP) working group of the InternetEngineering Task Force (IETF) is developingstandard Peer-to-Peer (P2P) streaming proto-

• Jose Lucas, nr. 70685,E-mail: jose.lucas@tecnico.ulisboa.pt,Instituto Superior Tecnico, Universidade de Lisboa.

Manuscript received May 15, 2015.

cols aimed to cope with the aforementionedissues. While there is no complete implemen-tation of solutions using these open protocols,is important to monitor and evaluate theirscalability and effectiveness, starting with theexisting prototypes developed in the scope ofthe European projects SARACEN [1] [2] andP2PNext [3].

Therefore, a monitoring system capable ofproviding Quality Of Service (QoS) and Qual-ity Of Experience (QoE) metrics for this typeof solutions, is becoming essential to accesstheir performance and quality, providing ade-quate data for comparison with existing closed-source systems. In the monitoring system, thefollowing information is presented throughweb-based output: (1) Content quality infor-mation: the receiving quality of the end users,start-up time and re-buffer events, (2) Trafficinformation: the upload/download volume forboth media streaming peer and the end users,and (3) System information: CPU/memory us-age of the media streaming peer and end users.The paper is organized as follows. In section 2,we describe the related research of multimediastreaming monitoring system. Section 3 showsthe architecture of the proposed monitoringsystem and its relevant design. The implemen-

2 MEASUREMENT STUDY OF P2P MULTIMEDIA STREAMING SYSTEMS

tation of the proposed system and the perfor-mance evaluation are provided in section 4.Finally, some conclusions are presented in sec-tion 5.

2 BACKGROUND

2.1 The P2P Streaming Protocols of IETF

The core protocols, i.e., Tracker Protocol andPeer Protocol are being developed by IETFto standardize signaling operations in the P2Pstreaming system.

The PPSP design includes a signaling pro-tocol between trackers and peers (the PPSPTracker protocol) and a signaling protocolamong the peers (the PPSP Peer protocol), asillustrated in Figure 1. The two protocols enablepeers to receive streaming content within thetime constraints.

Figure 1. PPSP System Architecture.

The functional entities related to PPSP arethe Client Media Player, the service Portal, theService Tracker and Peers. The Service Trackeris a logical entity that maintains the lists ofPPSP active peers storing and exchanging seg-ments for a specific content. The tracker an-swers queries from peers, collects informationon the activity of peers, and stores the statusof peers to help in the selection of appropri-ate candidate peers for a requesting peer. Theservice Portal is a logical entity typically usedfor client enrollment and content informationpublishing, searching and retrieval. The ClientMedia Player provides a direct interface to theend user at the client device, and includes thefunctions to select, request, decode and render

contents. In PPSP the Client Media Player inter-faces with the peer using request and responsestandard formats for Hypertext Transfer Proto-col (HTTP) Request and Response messages.

The process used for streaming distributionrelies on a segment transfer scheme wherebythe original content is re-encoded using adap-tive or scalable techniques and then choppedinto small video segments corresponding toa short play-out duration (in the order of afew seconds). With this method the system cansupport the following streaming mechanisms[4]:

• Adaptive - alternate versions of the con-tent with different qualities and bit rates;

• Scalable description levels - multiple ad-ditive descriptions of the content (i.e., ad-dition of descriptions refine the quality ofthe video);

• Scalable layered levels - nested dependentlayers corresponding to several hierarchi-cal levels of quality, i.e., higher enhance-ment layers refine the quality of the videoof lower layers;

• Scalable multi views - views correspondto 2D and to stereoscopic 3D videos, withseveral hierarchical levels of quality.

2.2 Related Work

There are several researches which measureand monitor P2P live streaming systems.Among these measurements, there are mainlythree types of data collection methodologies:

• Passive Monitor• Active Crawler• Log CollectionThe passive approach uses devices to watch

the traffic as it passes by. A passive measure-ment of two commercial P2P video streamingsystems [5], concluded that P2P live stream-ing has an even greater impact on networkbandwidth utilization and control than P2Pfile transfer applications. In [6] a frameworkwas proposed, to analyze the traffic that P2Papplications generate. The selected metrics al-lowed to analyze some of the most popular P2Papplication nowadays, highlighting their mainsimilarities and differences.In [7] during the

LUCAS 3

2006 FIFAWorld Cup, an extensive measure-ment was performed, on network traffic gen-erated by the most common P2P IP Television(IPTV) applications, namely PPLive, PPStream,SOPCast, and TVAnts. They extracted severalstatistics, which help in having a better under-standing of the behavior of P2P IPTV systems.

An Active Crawler is able to take full advan-tage of the partial membership list maintainedby each peer, and recursively query onlinenodes in a P2P system [8] [9].Existing literaturehas shown that a good crawler can find 95%of peers for a channel within 5 seconds [8].Though the active crawler is able to monitora much larger set of peers than the passivemonitor, it cannot obtain the detailed statusof a peer, as only a few bytes of informationare allowed to be exchanged between eachother due to overhead concerns. In [9] the twotechniques are combined to provide reasonablyaccurate estimates of ongoing video playbackquality throughout the network. To inspect therun-time behavior of UUSee P2P streaming,detailed measurement and reporting capabil-ities were implemented within its P2P clientsoftware [10].

The Log Collection method requires that adedicated log server is placed in this system,and each peer periodically reports its activities,such user behavior events and internal status,to the log server. In [11] the information froma peer is compacted into several parametersof the Uniform Resource Locator (URL) string.The log server stores the reports received frompeers into a log file. In the log file, each logentry is a normal HTTP request URL stringreferred as a log string. In [12]–[14] an agent-based real-time monitoring system for largescale P2P video streaming platforms is pro-posed, which provides traffic, user receivingquality and system information through web-based presentation. The monitoring agents areinstalled in both servers and end-users to col-lect information and send the information tothe real-time monitoring system. The systemhas an Log server that collects statistics andnotification logs from the monitoring agents,parsing the information carried in the logs andstored them into database. In the meanwhile,the data in the database will be aggregated for

web-based output.

3 SYSTEM DESIGN

In order to provide traffic, system informationand content quality about P2P streaming appli-cations using the PPSP, a log collection method-ology is proposed. The target P2P streamingis illustrated in Figure 2, in this system thecentralized services should be interpreted haslogical entities in terms of their role.

Media Serving Peer (VoD)

Media Serving Peer (Live) Tracker

Monitoring Server

Web Based Data Presentation

Seed Seed

Peer

Peer Peer

Peer End-User

Under NAT

Peer

Monitoring Report

Figure 2. Monitoring system and target peer-to-peer streaming system.

There are three types of statistics in themonitoring report send by peers:

Traffic Statistics: In this monitoring system,traffic statistics are important in measuringQoS of multimedia streaming. For ServingNodes and Peer Nodes, the traffic statisticcontains the incoming and outgoing trafficfrom other Serving Nodes and Peer Nodes.

System Information Statistics: Systeminformation statistic includes the currentsystArchitecture Design Requirementsemstatus information of a device, a Serving Nodeor Peer Node, such as CPU utilization andmemory consumption.

Content Quality Measurement Statistics:The content quality measurements statisticsincludes the number of re-buffer events andits duration, start-up time and receiving

4 MEASUREMENT STUDY OF P2P MULTIMEDIA STREAMING SYSTEMS

quality for the allowed streaming mechanisms.The information provided about streamingmechanisms, within the log interval, shouldidentify the maximum and average bit rate ofan adaptive stream and the maximum layer ofa stream encoded with Scalable Video Coding(SVC). Regarding 3D streaming, the log reportshould identify the number of received viewsand their quality levels. These informationare closer to the user’s service experience andreflect user’s player status and therefore playan important role in the QoE measurement.

Additionally, peers send notifications abouttheir actions, e.g join a swarm or send statusreport to tracker.

The log classification scheme in Figure 3was adopted, in order to adapt the monitoringsystem to the P2P distribution systemdeveloped by the PPSP working group ofIETF.

Figure 3. Log Classification.

The Monitoring System, illustrated in Fig-ure 4, is the combination of three servers: 1) logserver, 2) database server and 3) web server.To dynamically monitor the entire streamingsystem, detailed measurement and reportingcapabilities will be implemented within the

client application.

Figure 4. Monitoring System Architecture.

3.1 Monitoring Agents

The monitoring agents are responsible for col-lect and parse the logs from the P2P nodes. Thegenerated data is stored containing informationof one minute of log and the report is sendto the Monitoring Server periodically, with aconfigurable time interval that aggregates allthe generated files within that period. The com-munication between this two entities relies onHTTP request messages POST and response asillustrated in Figure 5.

Figure 5. Monitoring System message se-quence.

LUCAS 5

3.2 Log Server

The log server is responsible for receiving allthe statistics reports and to handle the filesposted to the server by the Monitoring Agents,validating and storing them. This storage areais continuously waiting for new files trans-ferred from all the Monitoring Agents in theP2P streaming system, and calls the the Ex-traction, Transformation and Load (ETL) pro-cedures from the Database Server when newfiles arrive into the server.

3.3 Database Server

In the database design, the data generated bythe streaming system was analyzed to build aLogic Entity-Relationship (E-R) model. Entity-relationship model is an abstract and con-ceptual representation of the data in the P2Pstreaming system.

Figure 6. E-R Model of the database.

Each rectangle shown in Figure 6, is anentity. A relationship, which is a diamond inthe figure, describes the entities related to theother entities. The content requested by a client,“End-user”, represent a “Service” witch can beLive or On-Demand. The service is availablefor consuming at “Media Server Peer”. Basedon the E-R model, we create the tables to storedata in the database. The data comes from flatlog files generated by Peers and is stored inthe database after the ETL process. This processincludes three steps:

• Extract stage: the data is parsed into thedesired format for transforming;

• Transform stage: applies a series of rulesto the extracted data from the source suchas data aggregation;

• Loading stage: stores data into end target.

These steps have the objective of transform-ing received information into stable, valid, con-sistent and periodically updated data, capableof being used for reporting, data visualizationand further analysis of streaming conditions.

3.4 Web Server

The web server is responsible for presentingmeasurement data to an administrator or sys-tem operator. Therefore, the server has a WebUI engine, capable of retrieving data from thedatabase, and generate charts and tables inHTML format for presentation to the user.

4 EXPERIMENTS AND EVALUATION

The P2P streaming network was implementedin laboratory using the Common Open Re-search Emulator (CORE)1, an emulator thatallows the deployment of network topologiesrunning in real-time [15]. Each host on the net-work is a virtual machine that runs a simplifiedversion of a Linux operating system, capable ofrunning the Monitoring Agent, the Peer Agentand a Player Emulator. The emulated networkis connected to a live network where is theMonitoring Server.

All the components of the Monitoring Serverare external to the CORE emulator, but run-ning on the the same host system. The COREemulated network provides connection to ex-ternal networks, allowing all elements in theemulated network, to communicate with theMonitoring Server.

The UI of the monitoring system is shown inFigure 7

Figure 7. Web UI of the monitoring system.

1. http://cs.itd.nrl.navy.mil/work/core/

6 MEASUREMENT STUDY OF P2P MULTIMEDIA STREAMING SYSTEMS

We monitored the streaming system in ascenario where height peers try to get contentin the presence of one tracker and one mediaserver peer.

Peers bandwidth is 12 Mbps for down-linkand 1 Mbps for up-link, and latency was set to5 ms.

The video content is streamed in SVC, andhas the characteristics described in tables 1 and2.

Table 1Video Characteristics

Streaming # Bandwidth Bandwidth VideoType Layers Min. Layer Max. Layer LengthSVC 4 0.25 Mbps 2 Mbps 80 seconds

Table 2SVC Layers Size

Layer File Min. Size File Avg. Size File Max. SizeL0 28140 bytes 73027 bytes 119251 bytesL1 45935 bytes 131314 bytes 213735 bytesL2 121506 bytes 351068 bytes 560592 bytesL3 234691 bytes 673065 bytes 1155548 bytes

In Figure 8 is illustrated the scenario wherepeer joins the swarm and start receiving con-tent, sending statistics reports in periods ofthirty seconds. When the streaming ends, thisinterval return to the default period of tenseconds, showing that the peer stayed in theswarm after receiving all chunks from theseeder.

0 20 40 60 80 100 120 140

Time (s)

JOIN

STAT REPORT

Figure 8. Peers actions

In peer mode notifications, the peer sendsto the monitoring system, the percentage ofavailable chunks from the streaming content, asillustrated in Figure 9. This type of notificationis only possible in Video On Demand (VoD)contents where the duration of the streamingis well known.

0 10 20 30 40 50 60 70 80

Time (s)

0.0

0.2

0.4

0.6

0.8

1.0

Avai

labilit

y

Peer Mode

Figure 9. Peer Mode

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

Time (m)

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Mem

ory u

tiliza

tion

(%

)

Peer Agent

Player

Monit. Agent

Figure 10. Memory consumption

The memory utilization of PPSP peer agent,player and monitoring agent are negligible (amaximum of around 8 MB), as illustrated inFigure 10.

The CPU load of PPSP peer agent, playerand monitoring agent is illustrated in Figure 10.The average CPU load of monitoring agent isonly four percent, and lower than the peeragent witch is the process with a central rolein streaming mechanism. In terms of CPU loadthe monitoring agents presents a good perfor-mance.

For traffic statistics, the monitoring agentcollects upload and download values for each

LUCAS 7

0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0

Time (m)

0

2

4

6

8

10C

pu loa

d (

%)

Peer Agent

Player

Monit. Agent

Figure 11. CPU load

peer and media server in the swarm. Figure 12illustrate the cumulative sum of download traf-fic of all peers for each time step of stream-ing emulation. Regarding the overhead trafficgenerated by monitoring reports, the averagesize of reports is 190 KB for one minute oflog. Taking into account that peers are preparedto wait a random period, between one andten seconds, to send the report, the traffic iseffectively spread in time.

0 20 40 60 80 100 120 140 160

Time (s)

0

5

10

15

20

25

30

Tra

ffic

(M

b)

DL from Seeders

DL from Peers

Figure 12. Download Traffic from Seeders

The time elapsed, between the client request-ing the content and the first frame displayed bythe player is computed as the start-up delay ofthe streaming. This means that the cumulativedownloading time of all consumed layers atfirst chunk, plus the read delay of the player,represent the start-up delay for each peer ina given swarm, as illustrated in Figure 13. Forall peers in the streaming network, the start-uptime was inferior to one second.

0 1 2 3 4 5 6 7 8 9Peer ID (swarm size 9)

0.00

0.05

0.10

0.15

0.20

Dela

ys (s

)

Join Channel delayStart-up delay

Figure 13. Start-Up time

The number of quality layers received bypeers can be correlated to a good quality ofexperience by the end-user. In this streamingscenario, peers receive the maximum layer ofquality in the overall streaming duration, asillustrated in Figure 14.

0 5 10 15 20 25 30 35 40

Chunk Id

0

1

2

3

4

5

Rec

eived

Lay

ers

Avg Layer

Max Layer

Figure 14. Downloaded Layers

PEER06 PEER08

Peer Id

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Num

ber

of Even

ts

Number of Events

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Avg

Dura

tion

(s)

Avg Duration

Figure 15. Re-buffer events

The SVC video content is streamed in two

8 MEASUREMENT STUDY OF P2P MULTIMEDIA STREAMING SYSTEMS

second period chunks. This means that, in or-der to avoid a freezing image, the next chunkmust arrive in less than two second. There-fore, the metric is computed by finding thenumber of periods in chunks arrivals that aresuperior to this interval, and also its averagetime. With the network links described above,no re-buffer events occurred, so we decreaseavailable bandwidth to 512 Kbps for down-linkand 128 Kbps to up-link. Figure 15 illustratethe the occurrence of re-buffer events in thisconditions.

5 CONCLUSION

The PPSP working group of the IETF is devel-oping standard P2P streaming protocols, andwhile there is no complete implementation ofsolutions using these open protocols, it was es-sential to develop a monitoring system capableof providing QoS and QoE metrics for this typeof solutions, to access their performance andquality, providing adequate data for compari-son with existing closed-source systems.

This paper proposed a monitoring system forP2P streaming systems based on PPSP proto-cols of IETF and the log classification adoptedis capable of describe this streaming solutionsregarding, system, QoS and QoE metrics, pro-viding the necessary statistics for comparisonwith other commercial systems.

The evaluation of the implemented proto-type demonstrated a system capable of re-trieving the proposed metrics, with monitoringagents implemented in peers, trackers and me-dia serving nodes.

REFERENCES[1] SARACEN Consortium, “System Architecture and Spec-

ifications,” (Restriced Access), SARACEN Consortium,Technical Report D2.2, 2011.

[2] ——, “Socially Aware, collaboRative, scAlable CodingmEdia distributioN project Home Page,” 2010. [Online].Available: http://www.saracen-p2p.eu/

[3] P2P-NEXT Consortium, “Next generation peer-to-peercontent delivery platform project Home Page,” 2010.[Online]. Available: http://p2p-next.org/

[4] M. Leszczuk, D. Juszka, L. Janowski, M. Grega, R. S. Cruz,M. S. Nunes, C. Patrikakis, and S. Papapanagiotou, “Qual-ity Aware, Adaptive, 3D Media Distribution over P2Parchitectures,” in Proceedings of the Workshop on Qualityof Experience for Multimedia Communications, GLOBECOM2013. IEEE, 2013.

[5] S. Ali, A. Mathur, and H. Zhang, “Measurement of com-mercial peer-to-peer live video streaming,” . . . in RecentAdvances in Peer-to-Peer Streaming, 2006.

[6] D. Rossi and E. Sottile, “Sherlock: A framework for P2Ptraffic analyis,” in 2009 IEEE Ninth International Conferenceon Peer-to-Peer Computing. IEEE, Sep. 2009, pp. 321–330.

[7] T. Silverston and O. Fourmaux, “Measuring p2p iptvsystems,” Proc. of ACM NOSSDAV, 2007.

[8] K. Ross, “A Measurement Study of a Large-Scale P2PIPTV System,” IEEE Transactions on Multimedia, vol. 9,no. 8, pp. 1672–1687, Dec. 2007.

[9] K. W. Ross, “Inferring Network-Wide Quality in P2PLive Streaming Systems,” IEEE Journal on Selected Areas inCommunications, vol. 25, no. 9, pp. 1640–1654, Dec. 2007.

[10] C. Wu, B. Li, and S. Zhao, “Diagnosing Network-WideP2P Live Streaming Inefficiencies,” in IEEE INFOCOM2009 - The 28th Conference on Computer Communications.IEEE, Apr. 2009, pp. 2731–2735.

[11] B. Li, S. Xie, Y. Qu, G. Y. Keung, C. Lin, J. Liu, andX. Zhang, “Inside the new Coolstreaming: Principles,measurements and performance implications,” in Proceed-ings - IEEE INFOCOM, 2008, pp. 1705–1713.

[12] N.-F. Huang, Y.-M. Chu, and Y.-R. Chen, “Design of a P2PLive Multimedia Streaming System with Hybrid Pushand Pull Mechanisms,” in 2010 International Conference onCommunications and Mobile Computing, vol. 1. IEEE, Apr.2010, pp. 541–545.

[13] N.-F. Huang, M.-H. Wang, T.-C. Wang, and S.-S. Peng,“Measuring QoE/QoS of large scale P2P IPTV service,” in2011 13th Asia-Pacific Network Operations and ManagementSymposium. IEEE, Sep. 2011, pp. 1–8.

[14] N.-F. Huang, T.-C. Wang, M.-H. Wang, and S.-S. Peng,“ARMS: An agent-based real-time monitoring system forlarge scale P2P video streaming platforms,” in The 17thAsia Pacific Conference on Communications. IEEE, Oct. 2011,pp. 405–410.

[15] J. Ahrenholz, “Comparison of CORE network emulationplatforms,” in Proceedings - IEEE Military CommunicationsConference MILCOM, 2010, pp. 166–171.