Dr. Anna Inverso

AGCOM – Consumer Protection Directorate

Cartagena de Indias, April 13-15, 2015 AECID/ BEREC - REGULATEL Meetings

Quality of telecommunications services

Quality of service is one of the basic requirements for a market based on full

competition

Knowledge and ability to compare quality offered by different service providers

(SPs) is a fundamental right of end users

Yet information about quality of service is hard to find and competition may not be

effective in ensuring that end users’ needs are taken into account

Sector-specific measures on QoS may then be necessary to both protect end-

users rights (by setting minimum requirements and by reducing the information

asymmetry between SPs and end-users) and to prevent service deterioration,

constraints in accessing to services and the slowing of traffic over networks

Quality

2

QUALITY OF SERVICE (QoS) REQUIREMENTS 2002/22/EC Directive (USD) as amended by the 2009/136/EC Directive (Citizens’ Rights)

• Article 22(1) – Transparency and publication of information

Member States shall ensure that national regulatory authorities are able to require undertakings that provide publicly available electronic communications networks and/or

services to publish comparable, adequate and up-to-date information for end-users on the quality of their services

• Article 22(2) – Quality Performance Indicators and Certification NRAs may specify, inter alia, the quality of service parameters to be measured and the

content, form and manner of the information to be published, including possible quality certification mechanisms, in order to ensure that end-users, including disabled end-users,

have access to comprehensive, comparable, reliable and user-friendly information

• Article 22(3) – Minimum QoS levels in order to prevent the degradation of service and the hindering or slowing down of traffic

over networks, (…) NRAs shall be able to set minimum quality of service requirements on undertakings providing public communications networks

The EU Framework

3

The General Directive on Quality of Telecommunications Services

Resolution n° 179/03/CSP

Article 10 – Quality of service

• set the general criteria for the quality of services

• establishes the minimum requirements for the adoption of service charters

• defines the minimum service level to achieve in order to guarantee end users’ rights

• specifies compensation and refund arrangements

The AGCOM Framework

4

The General Directive on Quality of Telecommunications Services

Service providers should introduce a system for measuring the quality of services and shall publish promptly and comprehensively the results in such a way that users have

adequate comparison tool to select best services

Quality as a leaver to

The AGCOM Framework

5

Increase awareness of final users and strengthen market competition

strengthen the user – oriented approach of service providers

According to provisions of the General QoS Directive, a series of directives on quality have been approved over time for each

different sector

The AGCOM QoS Framework

General Directive Res. n°179/03/CSP

Pay TV

Res. n°. 278/04/CSP

6

Mobile Phone Res. n° 154/12/CONS

Fixed Internet

Access Res. n° 154/12/CONS

Mobile Internet

Access

Res n°. 131/06/CSP

Res. n° 244/08/CSP

Res. n°. 656/14/CONS

Universal service

Res. n°.

328/10/CONS

Each year

a Resolution

sets QoS for US

Fixed phone

Res. n° 254/04/CSP

Call Center Res. n° 79/09/CSP

The system for measuring quality is based on a set of "indicators", which are

aspects of services deemed most relevant to the liking of the user (for example, the

activation time, the accuracy of the billing, the rate of malfunction , the rate of fall of the

line on mobile networks, and so on).

For each "indicator" operators must set the annual objectives to be

achieved.(minimum and target level)

The results of these measurements must be made public, in the service charters

and on websites of service providers, and should be sent to the NRA.

For some services, namely the Universal service and the Call center services ,

AGCOM sets a "target value" that both the NRA and final users could monitor to

assess if a service can be judged of quality.

Quality Control and Assurance

7

Publication of QoS results

8

Under the general directive on quality, SPs are required to

publish, on their web sites:

service charters

semi-annual reports on the results of the quality of service achieved

annual reports on quality of services, containing indicators,

measurement methods, the target set for each indicators and the

actual results achieved during the calendar reference

NRA collects and publishes results on QoS performance

every half-year. To let end users to easily compare data on QoS

Performance, AGCOM makes available on its website, and updates

them periodically, links to web pages in which the SPs publish

information on quality. The results of the performance of "traditional"

services are published for each operator; while for broadband

services NRA publishes the results compared

A differentiated approach

9

Traditional fixed and mobile services (voice, SMS,

narrow band services)

KPI measurements made by operator network

counters

Broadband services (both fixed and mobile)

also live KPI measurements made by a third party

(Fondazione Ugo Bordoni ), selected from Agcom

and financed by operators.

What does QoS?

10

ASSURANCE

Preventive

maintenance

Corrective

maintenance

BILLING

CUSTOMER

ASSISTANCE

DELIVERY

PERFORMANCES

1. Supply time for initial telephone connection

2. Failure rate per line

3. Time fault repair

4. Response time for operator services

5. Response times to calls to customer services

6. Percentage of public payphones in service

7. Disputed invoices

8. Billing accuracy

9. Response times for directory enquiry services

10. Percentage of failed calls (optional)

11. Call setup time (optional)

12. Time supply of carrier pre-selection service

QoS indicators for fixed phone

11

QoS indicators for Universal Service

12

"a defined minimum set of services of specified quality available to all users

regardless of their geographical location and, taking account of specific

national conditions, at an affordable price" (Directive 97/33 / EC)

1. Supply time for initial telephone connection

2. Failure rate per line

3. Time fault repair

4. Response time for operator services

5. Response times to calls to customer services

6. Percentage of public payphones in service

7. Disputed invoices

8. Billing accuracy

The Global

Quality Index of

the Universal

Service

weighted average

of the values

assumed by these

8 indicators

QoS indicators for other services

13

Activation time for voice service, Time of renewal of the credit,

Response times for directory enquiry services, Complaints about

charges, Billing accuracy, Accessibility of GSM service, Likelihood

of maintaining the connection, Failure in SMS transfer, SMS

delivery time from the SMS message center

Mobile voice

Activation time for service, Failure rate, Disputed invoices, Billing

accuracy, Response time to calls to customer service Pay TV

Response time to calls to customer service (waiting time in the IVR until the choice to speak with a human operator and waiting time spent until the answer of the operator); Percentage of one call solution

Call

centers

Supply time for the service activation, Time fault repair, Supply

time for the initial connection, Response times to calls to customer

services, Disputed invoices, Unavailability of service in dial-up

Broadband

fixed access

Focus on Quality: rationales

14

Targets End users’ benefits

Allow users to select suitable offer taking into account also QoS factor Better visibility of QoS

Gradual improvement of QoS

Allow customers to enjoy of a service of growing quality and satisfaction

Minimum QoS threshold to be matched

Allow customers to withdraw from the contract without penalties in case of unsatisfactory QoS

Agcom Broadband QoS monitoring An Italian good practice

15

MisuraInternet Project: a tool for measuring

quality of fixed broadband internet access with

far reaching implications

Empower end-users

Stimulate competion between ISPs

Increase comparability of main wired accesses with

headline speed up to 20Mbit/s

Capture a picture of broadband speed evolution

Identify Digital Divide areas

Misura Internet Description

16

Two types of measures:

• ISP Measurements based on dedicated “probes” located in regional

capitals

• End-user measurements allows users to measure the line on their

own through a software called Ne.Me.Sys to be downloaded on their PC.

• Both measurement methods employ the same Network Measurement

System, based on a software agent running on a standard Personal

Computer.

• http://www.agcom.it/misura-internet-da-postazione-fissa

Torino

Roma

Bari

Napoli

End point for ISP measurement

Measurement Client :

Central system managed by FUB (Rome)

Measurement Server:

Neutral Acess Point (NAP)

ISP Measurement architecture: location of measurement sites

Milano

Headquarter AGCOM (Naples)

Agcom’s Broadband QoS monitoring Parameters to measure QoS of fixed access

18

Data transmission speed

data transmission rate, achieved separately for downloading and

uploading, measured by transmitting specified test files between a

remote server and a user's computer

Packet Delay (one way transmission time)

time needed for an ICMP Echo Request/ReplyPacket (Ping

Command) to a valid IP address

Packet loss ratio

the ratio between the number of not replied Ping commands to the

total number of sent Ping commands

Unsuccessful data transmission ratio

the ratio of unsuccessful data transmissions to the total number of

data transmission attempts in a specified time period

Agcom’s Broadband QoS monitoring Usage of Misura Internet (2010-2014)

19

Number of website visitors 2.118.500

Registered Users on www.misurainternet.it 165.286

No. Released Ne.Me.Sys Certificates 28.891

No. MisuraInternet Speed Test Downloads 227.623

No. MisuraInternet Speed Test Measures 63.506

*Source: Agcom Annual Report (April, 2014)

Last year, Agcom launched a trial version of MisuraInternet Speed Test, that allows users to perform one measurement without prior registration

Agcom’s Broadband QoS monitoring An Italian good practice

20

MisuraInternet Project: a tool for measuring

quality of fixed broadband internet access with

far reaching implications

Empower end-users

Increase comparability of main wired accesses with

headline speed up to 20Mbit/s and Stimulate competion

between ISPs https://www.misurainternet.it/charts

Capture a picture of broadband speed evolution

Identify Digital Divide areas

Misura Internet Evolution

21

Resolution n° 656/14/CONS

Broadened scope: measurements to wireless

connections from fixed locations, and certify more

robust results for IAS over 30 Mbit/s

Consumer empowerment: free commercial

downgrade in addition to the right to withdraw with

no penalties

Measurement upgrade: compliance with new ETSI

standard ES” and use of the HTTP protocol instead

of the FTP protocol for the execution of measures of

throughput

MisuraInternetMobile.it Project

22

Evaluation of performances of the mobile networks

operating in Italy, in front of mobile broadband QoS.

Publication of result statistics to be available to service

subscribers, in order to ensure transparency of the

performances really provided compared with the

features offered from the operators.

A Project to measure the quality of mobile

Internet access service through a multi-year plan

of periodic Drive Test measurement campaigns

MisuraInternetMobile.it Project

23

Tested the 4 mobile access networks available in Italy: H3G,

Telecom Italia, Vodafone and Wind

Each test session (22 minutes) is executed, at the same time over

the 4 networks, within a test pixel (elementary area of 500 m x 500

m) selected among the most populated ones (> 1,800 inhabitant

per km²) of the visited cities.

For year 2013, 20 big cities were selected, covering all geographic

regions of the Country.

One campaign per half-year planned.

First two officials campaign executed in January 2013 and March

2014. Results were available and published by end of July 2014.

The third campaign executed in the timeframe May – September

2014. Results under processing.

Main features of MI drive Test

24

Data transmission throughput

Data transmission unsuccessful rate

Packet delay

Packet loss

Jitter

MisuraInternetMobile.it Project Parameters to measure QoS

25

MisuraInternetMobile.it Project Some comparative results

KPI First Trial 2012 1st

campaign 2013 1H13 vs 2012

2nd campaign

2013 2H13 vs 1H13

FTP Upload rate (Kbps)

1393 1573 +12,92% 1707 +8,52%

HTTP Download

rate (Kbps) 5043 6268 +24,29% 6738 +7,50%

HTTP Browsing – Keplero page

download time

(sec)

5.66 3.58 +36,75% 2.90 +18,99%

Average Throughput

KPI 1st

campaign 2013 2nd campaign 2013 2H13 vs 1H13

Packet delay (round trip

time) in ms 54.51 55.37 - 0,16%

Packet loss 0.60% 0.44% +26,67%

Jitter (in ms) 5.88 6.10 - 3,74%

Packet transmission performance

Thanks for your attention

a.inverso@agcom.it