measuring quality of user experience in communication networks€¦ · measuring quality of user...

Brussels 05 June 2012

Measuring quality of user experience in communication

networks

Pravir Chawdhry

European Commission - Joint Research Centre

ITU Regional Conference on Broadband Mapping

Warsaw, Poland – 11-12 April 2016


Quality of User experience

One of the most important aspects that affects the Internet consumer experience is the quality of network performance. Traffic growth is dominated by on-line video and audio-streaming so low latency and high speed add great value to user experience.



One of the most important aspects that affects the Internet consumer experience is the quality of network performance. Traffic growth is dominated by on-line video and audio-streaming so low latency and high speed add great value to user experience. Speed test applications have become widely used tools for customers.




But how reliable are these tools?




Are these tools reliable?

What do these tools really measure?


Measuring network performance

ICT Technicians can use advance tools and

use them with their domain competences.





Know-how to check

network activity.





Know-how to check

network activity.

Know-how to

inspect network

characteristics

with command

like tracert and

nmap. And run specialized

applications like

speedtest-cli, curl, etc..



Nontechnical citizens search for a Speed test

tool over Internet.



But results can be very confusing.



How can we know if these tools are reliable?



How can we know if these tools are reliable?

App Ping

ms

Download Speed

Mb/s

Upload Speed Mb/s

SpeedTest 14 204,49 83,36

mLab 22 9,75 13,19

netBravo 71 15,74 59,63

RTR-NetTest 18 200,00 42,00

Here is an example of speed test measures over the

Internet from the same Workstation using 4

different tools



How reliable are these tools?

And what are they measuring?





UDP





UDP

TCP





UDP

TCP

HTTP



The discrete behaviour of the NETWORK is very complex and fairly

unpredictable

Server Switch Router Firewall Internet

Compression

Every transition can have effect on the final resulting speed due to:

• High bandwidth usage on any route segment

• Router, firewall or switch misconfigurations

• CPU overhead on any used device

• Client network activity

• Compressible or uncompressible content

• ….


What about benchmarking?

To reduce the complexity of the network and test the reliability of

Speed Test tools we can create a reference framework to test and

benchmark the applications using a simulated simplified network in

which we can operate on various parameters:

• Up and Down stream Bandwidth

• Latency

• Packet loss

• Round time trip

• ….

Following the directive of IPPM (IETF)


Benchmarking: an example

Previous speed test tools tested on a simulated network that cut

download and upload bandwidth to 1, 2, 5, and 10 Mb/s

WebApp Latency Download Upload

1Mbs

NetBravo 23 ms 0,90 Mb/s 1,02 Mb/s

M-Lab 410 ms 0,88 Mb/s 0,91 Mb/s

Ookla 30 ms 0,97 Mb/s 0,98 Mb/s

Z-Test 72 ms 0,94 Mb/s 0,94 Mb/s

2Mbs


M-Lab 230 ms 1,84 Mb/s 1,88 Mb/s



5Mbs


M-Lab 102 ms 4,63 Mb/s 3,84 Mb/s



10Mbs


M-Lab 76 ms 9,22 Mb/s 7,91 Mb/s




JRC experience: netBravo

JRC developed a mobile crowd sourcing application that measures

quality of user experience:

Speed test: Download and upload data transfer rates.

Latency, ping and DNS lookup test

Local GSM / UMTS / LTE signal strength

Local WiFi 2.4 and 5 GHz access point signal strength, sid and

characteristics (used channel, security protocols, sid, etc..)

Network neutrality test (voip, nat, p2p, etc..)


iOS Application screen-shots


Android Application screen-shots


netBravo Mapping: netbravo.org


netBravo Application optimizations

The application is: Optimized for battery usage

Optimized for Bandwidth consumption

Takes measurements only when a location is available

Measurements are made based on battery state and location

availability

Uploads measurements to the server when the preferred connection is available

Application user data is fully anonymized.


Thank you

Further information:

www.netbravo.eu

[email protected]

+39 0332 78 58 23

http://publications.jrc.ec.europa.eu/repository/

http://publications.jrc.ec.europa.eu/repository/

measuring quality of user experience in communication networks€¦ · measuring quality of user...

Documents