standards - etsi compliance(2)

8/2/2019 Standards - ETSI Compliance(2)

1/17

The present document describes the compliance of SwissQual

measurement system (data collection and post processing) with the

requirements given in the ETSI documents TS 102250 point by point. Each

sheet contains the table of contents of the corresponding document with the

compliance comments in the same line.

The ETSI document TS 102250 is a multipart deliverable and is Technical

Specification (TS) that has been produced by ETSI Technical Committee

Speech Processing, Transmission and Quality Aspects (STQ). It covers the

QoS aspects for popular services in GSM and 3G networks, as identified

below:

The contents of the different parts is outlined below:Part 1: "Identification of Quality of Service aspects";

Part 2: "Definition of Quality of Service parameters and their computation";Part 3: "Typical procedures for Quality of Service measurement equipment";

Part 4: "Requirements for Quality of Service measurement equipment";

Part 5: "Definition of typical measurement profiles";

Part 6: "Post processing and statistical methods".

Part 1 identifies QoS aspects for popular services in GSM and

3G networks. For each service chosen QoS indicators are listed.

They are considered to be suitable for the quantitatively

characterization of the dominant technical QoS aspects asexperienced from the end-customer perspective.

Part 2 defines QoS parameters and their computation for

popular services in GSM and 3G networks. The technical QoS

indicators, listed in part 1, are the basis for the parameter set

chosen. The parameter definition is split into two parts: the


2/17

Part 3 describes typical procedures used for QoS

measurements over GSM, along with settings and parameters

for such measurements.

Part 4 defines the minimum requirements of QoS measurement

equipment for GSM and 3G networks in the way that the values

and trigger-points needed to compute the QoS parameter as

defined in Part 2 can be measured following the procedures

defined in part 3. Test-equipment fulfilling the specified minimum

requirements, will allow to perform the proposed measurements

in a reliable and reproducible way.

Part 5 specifies test profiles which are required to enable

benchmarking of different GSM or 3G networks both within and

outside national boundaries. It is necessary to have these

profiles so that when a specific set of tests are carried out then

customers are comparing "like for like" performance.

Part 6 describes procedures to be used for statistical

calculations in the field of QoS measurement of GSM and 3G

networks using probing systems.


3/17

Section Indicator Name compliant comment

5 QoS Indicator List5.1 Overview

5.2 Service independent5.2.1 QoS Aspect: Network Access

5.2.1.1 QoS Indicator: Network Accessibility (NA) defined in Part 102250-2

5.3 Telephony

5.3.1 QoS Aspect: Service Access

5.3.1.1 QoS Indicator: Service Accessibility Telephony (SA-T) defined in Part 102250-2

5.3.1.2 QoS Indicator: Setup Time Telephony (ST-T) defined in Part 102250-2

5.3.2 QoS Aspect: Service integrity

5.3.2.1 QoS Indicator: Speech Quality (SpQ) defined in Part 102250-2

5.3.3 QoS Aspect: Service retainability5.3.3.1 QoS Indicator: Call Completion Ratio circuit switched (CCR-CS) defined in Part 102250-2

5.4 Short Message Service


5.4.1.1 QoS Indicator: Service Accessibility SMS MO (SA SMS MO) defined in Part 102250-2

5.4.1.2 QoS Indicator: Access Delay SMS MO (AD SMS MO) defined in Part 102250-2

5.4.2 QoS Aspect: Service integrity SMS

5.4.2.1 QoS Indicator: End-to-end Delivery Time SMS (DT SMS) defined in Part 102250-2

5.4.2.2 QoS Indicator: Completion Rate SMS circuit switched (CR SMS CS) defined in Part 102250-2

6 Multimedia Messaging Service (MMS)6.1 Void

6.1.1 Preconditions Reservation due to implementation

6.1.2 Overview of indicators for QoS measurement of MMS


6.1.3.1 QoS Indicator: MMS Send Failure Ratio (MO) [%] defined in Part 102250-2

6.1.3.2 QoS indicator: MMS Send Time (MO) [s] defined in Part 102250-2

6.1.3.3 QoS Indicator: MMS Retrieval Failure Ratio (MT) [%] defined in Part 102250-2


4/17

6.1.3.4 QoS Indicator: MMS Retrieval Time (MT) [s] defined in Part 102250-2

6.1.4 QoS Aspect: Service retainability

6.1.4.1 QoS Indicator: MMS Notification Failure Ratio [%] defined in Part 102250-26.1.4.2 QoS Indicator: MMS Notification Time [s] defined in Part 102250-2

6.1.4.3 QoS Indicator: MMS End-to-end Failure Ratio [%] defined in Part 102250-2

6.1.4.4 QoS Indicator: MMS End-to-end Delivery Time (MO/MT) [s] defined in Part 102250-2


5/17

Section Indicator Name compliant (corresponding KPI No) possible

4.2 Service Independent

4.2.1 Network Accessibility Circuit Switched (NA - CS) -

4.2.2 Network Accessibility Packet Switched (NA - PS) -

4.3 Telephony

4.3.1 Service Accessibility-Telephony (SA-T) 10100

4.3.2 Setup Time Telephony (ST-T) 10100

4.3.3 Speech Quality on call basis (SpQ-C) X

4.3.4 Speech Quality on sample basis (SpQ-S) 30100

4.3.5 Call Completion Rate Circuit Switched Telephony (CCR-CS-T) 20100

4.4 Short Message Service (SMS)

4.4.1 Service Accessibility SMS MO (SA SMS MO) 20900

4.4.2 Access Delay SMS MO (AD SMS-MO) 209004.4.3 End-to-end Delivery Time SMS (DT SMS) 30900

4.4.4 Completion Rate SMS Circuit Switched (CR SMS CS) 30900

4.5 Circuit Switched Data Service

4.5.1 Service Accessibility, Circuit Switched Data (SA - CSD) -

4.5.2 Set-up Time (ST - CSD) -

4.5.3 Data Quality (DQ-CSD) -

4.5.4 Completion Rate Circuit Switched Data (CR-CSD) -

4.6 Packet Switched Data Services

4.6.1 Key Performance Indicators Method A

4.6.1.1 {Service} Service Accessibility Ratio (SA-PSD) [%] X

4.6.1.2 {Service} Setup Time (ST-PSD) [seconds] X

4.6.1.3 {Service} IP-Service Access Ratio (IPSA-PSD) 10200,10201,10300,103001,10400

4.6.1.4 {Service} IP-Service Setup Time (IPST-PSD) 10200,10201,10300,103001,10400

4.6.1.5 {Service} Completed Session Ratio (CoSeR-PSD) [%] 10211,10310,10311,10410

4.6.1.6 {Service} Session Time (SeT-PSD) 10211,10310,10311,10410

4.6.1.7 {Service} Mean Data Rate (MDR-PSD) [kBit/s] 30201,30306,30405


6/17

4.6.1.8 {Service} Data Transfer Cut-off Ratio (DTCoR-PSD) [%] 20201,20301,20306,20400,20405

4.6.2 Key Performance Indicators Method B

4.6.2.1 {Service} Service Accessibility Ratio (SA-PSD) [%] 10320,10321,104204.6.2.2 {Service} Setup Time (ST-PSD) [seconds] 10320,10321,10420

4.6.2.3 {Service} IP-Service Access Ratio (IPSA-PSD) 10307,10308,10407

4.6.2.4 {Service} IP-Service Setup Time (IPST-PSD) 10307,10308,10407

4.6.2.5 {Service} Completed Session Ratio (CoSeR-PSD) [%] 10211,10310,10311,10410

4.6.2.6 {Service} Session Time (SeT-PSD) 10211,10310,10311,10410

4.6.2.7 {Service} Mean Data Rate (MDR-PSD) [kBit/s] 30307,30308,30407

4.6.2.8 {Service} Data Transfer Cut-off Ratio (DTCoR-PSD) [%] 20307,20308,20407

4.6.3 Performance Indicators

4.6.3.1 Unavailability [%] 162004.6.3.2 Attach Failure Ratio [%] 15100

4.6.3.3 Attach Setup Time [seconds] 15100

4.6.3.4 {Service} PDP Context Activation Failure Ratio [%] 15200

4.6.3.5 {Service} PDP Context Activation Time [seconds] 15200

4.6.3.6 {Service} PDP Context Cut-off Ratio [%] 25200

4.6.3.7 {Service} Round Trip Time [milliseconds] 21000

4.7 Multimedia Messaging Service (MMS)

4.7.1 MMS send failure ratio (MO) [%] 20800

4.7.2 MMS retrieval failure ratio (MT) [%] 20801

4.7.3 MMS send time (MO) [s] 20800

4.7.4 MMS retrieval time (MT) [s] 30804

4.7.5 MMS notification failure ratio [%] 30801

4.7.6 MMS notification time [s] 30801

4.7.7 MMS end-to-end failure ratio [%] 30800

4.7.8 MMS End-to-end Delivery Time (MO/MT) [s] 30800

4.8 Streaming

4.8.4 Streaming Service Non-Accessibility [%] 106004.8.5 Streaming Service Access Time [s] 10600

4.8.6 Streaming Reproduction Cut-off Ratio [%] 20600

4.8.7 Streaming Audio Quality [] SquadLQ

4.8.8 Streaming Video Quality VQuad

4.8.9 Streaming Audio/Video De-Synchronisation lipsync

4.8.10 Streaming Reproduction Start Failure Ratio [%] 30600

4.8.11 Streaming Reproduction Start Delay [s] 30600


7/17

4.9 Video Telephony

4.9.3 VT Service Non-Accessibility [%] 101004.9.4 VT Service Access Time [s] 10100

4.9.5 VT Audio/Video Setup Failure Ratio [%] (10105)

4.9.6 VT Audio/Video Setup Time [s] (10105)

4.9.7 VT Cut-off Call Ratio [%] 20100

4.9.8 VT Speech Quality on Call Basis [MOS-LQO] (Call Quality)

4.9.9 VT Speech Quality on sample basis [MOS-LQO] SquadLQ,P.862.1

4.9.10 VT Video Quality VQuad

4.9.11 VT End-To-End Mean One-Way Transmission Time [s] RTT/2

4.9.12 VT Audio/Video Synchronization [%] lipsync

Notes:

Numbers represent KPI numbers (see KPI Users Guide)

Numbers in brackets represent KPIs that do not match exactly the definition

compliant: Used definition matches exact ETSI spec

possible : Not defined but possible within the KPI framework

Failure ratio and success rate is considered equivalent


8/17


5 General aspects for all types of services -

5.1 Set-up and control yes6 Telephony measurements -6.1 General aspects void

6.2 Speech telephony void

6.3 Video telephony -

6.3.1 Measurement set-up and control yes

6.3.2 Call Types yes

6.3.3 Call transaction phases -

6.3.3.1 Call set-up partly network accessability

6.3.3.1.1 Service accessibility yes

6.3.3.1.2 Setup time yes

6.3.3.2 Call connect yes

6.3.3.2.1 Speech quality yes

6.3.3.2.2 Call completion rate yes

6.3.3.3 Call clear-down -

6.3.3.4 Pause yes Idle phase

7 SMS measurements -7.1 General aspects -

7.1.1 SMS void

7.1.2 General aspects of SMS measurement yes7.2 Testing modes -

7.2.1 SMS-MT -

7.2.1.1 Event flow yes

7.2.1.2 Specific QoS objects partly network accessability

7.2.2 SMS-MO -

7.2.2.1 Event flow yes


9/17

7.2.2.2 Specific QoS objects no

7.3 Common QoS objects partly bit error detection

7.3.1 MMS void

8 Data measurements -8.1 Common aspects -

8.1.1 Target for data access yes

8.1.2 Test data content yes

8.1.3 Outer conditions to be kept constant for testing yes

8.1.4 Definition of transaction yes

8.1.5 Assessing transactions partly no indication of the kind of error

8.2 Circuit switched no no circuit switched tests8.3 Packet switched (GPRS) yes

9 Information to be logged for reproducibility of measurements9.1 Speech measurements yes

9.2 SMS measurements partly no notification type logged

9.3 Data measurements partly no OS,no MTU, no logical Location, no DNS

9.3.1 HTTP yes

9.3.2 FTP partly no client type

9.3.3 E-mail (SMPT/POP3) yes

9.3.4 WAP partly no access type9.3.5 UDP yes

Notes:

void not defined yet

- no definition

partly missing condition in the comment


10/17


5 General requirements -

5.1 General requirement for data logging yes5.2 Required information for logging yes

6 Mobile QoS Test-equipment (MQT) -6.1 Overview yes

6.2 Test-MS yes

6.3 Antennas -

6.4 Controller/processor/storage yes

6.5 Man Machine Interface (MMI) yes

6.5.1 MQT-LC yes

6.5.2 MQT-RC yes

6.6 Geographical positioning yes

6.6.1 Format of geographical co-ordinates yes

6.6.2 Accuracy yes

6.7 Time sources yes

6.8 Environnemental conditions yes

7 Fixed QoS Test-equipment (FQT) -7.1 General yes

7.2 FQT for user-to-user services yes

7.3 FQT for store-and-forward services yes

7.4 FQT for information (download) services yes7.5 FQT for push services no no push services

7.6 Controller yes

7.7 Time-sources yes

7.8 Environmental conditions yes

8 QoS parameter export


11/17

8.1 Overview

8.2 Description of standard parameter set

8.2.1 Geo data yes

8.2.2 QoS data yes

8.2.2.1 Telephony yes

8.2.2.2 SMS yes

8.2.2.3 CSD no no circuit switched data

8.2.2.4 PSD yes

8.2.3 Measurement system info yes

8.3 Data selection and file format

8.3.1 Data selection box no export tool neccessary8.3.2 Export File 1 no report neccessary

8.3.3 Export File 2 no report neccessary

8.3.4 Export File 3 no report neccessary

Notes:


- no definition


no missing condition in the comment


12/17


4 Measurement profiles -4.1 Classification of measurement environments yes collectcion name

4.2 Service profiles -

4.2.1 Telephony yes call windows definable4.2.1.1 Voice telephony yes call windows definable

4.2.1.2 Video Telephony yes call windows definable

4.2.2 Messaging Services yes

4.2.2.1 SMS partly no E2E timeout defined

4.2.2.2 MMS partly no E2E timeout defined

4.2.3 Data services -

4.2.3.1 Circuit switched -

4.2.3.2 Packet switched partly

4.2.3.3 Streaming -4.3 Usage Profiles for Data Sessions partly not all timeouts definable

4.3.1 Web browsing using HTTP yes

4.3.2 E-Mail access yes

4.3.3 File Transfer using FTP yes

4.3.4 File Sharing using UDP -

4.3.5 Synthetic tests -

4.3.5.1 UDP void

4.3.5.2 ICMP void

4.3.5.3 TCP void

Notes:


- no definition


no missing condition in the comment


13/17


4 Important measurement data types in mobile communications

4.1 Data with binary values4.2 Data out of time-interval measurements

4.3 Measurement of data throughput

4.4 Data concerning quality measures

5 Distributions and moments

5.1 Introduction

5.2 Continuous and discrete distributions

5.3 Definition of density function and distribution function

5.3.1 Probability Distribution Function (PDF)

5.3.2 Cumulative Distribution Function (CDF)

5.4 Moments and quantiles5.5 Estimation of moments and quantiles

5.6 Important distributions

5.6.1 Continuous distributions

5.6.1.1 Normal distribution

5.6.1.1.1 Standard normal distribution

5.6.1.1.2 Central limit theorem

5.6.1.1.3 Transformation to normality

5.6.1.2 Log-Normal distribution

5.6.1.2.1 Use-case: transformations5.6.1.3 Exponential distribution

5.6.1.4 Weibull distribution

5.6.1.5 Pareto distribution

5.6.1.6 Extreme distribution (Fisher-Tippett distribution)

5.6.2 Testing distributions

5.6.2.1 Chi-square distribution with n degrees of freedom


14/17

5.6.2.1.1 Further relations5.6.2.1.2 Relation to empirical variance

5.6.2.2 Student t-distribution

5.6.2.2.1 Relation to normal distribution

5.6.2.3 F distribution

5.6.2.3.1 Quantiles

5.6.2.3.2 Approximation of quantiles

5.6.2.3.3 Relations to other distributions

5.6.3 Discrete distributions

5.6.3.1 Bernoulli distribution

5.6.3.2 Binomial distribution5.6.3.3 Geometric distribution

5.6.3.4 Poisson distribution

5.6.4 Transitions between distributions and appropriate approximations

5.6.4.1 From binomial to Poisson distribution

5.6.4.2 From binomial to Normal distribution

5.6.4.3 From Poisson to Normal distribution

5.6.5 Truncated distributions

5.6.6 Distribution selection and parameter estimation

5.6.6.1 Test procedures5.6.6.1.1 Chi-square test

5.6.6.1.2 Kolmogorov-Smirnov test

5.6.6.1.3 Shapiro-Wilk test

5.6.6.2 Parameter estimation methods

5.7 Evaluation of measurement data

5.7.1 Statistical tests

5.7.1.1 Formulation of statistical tests

5.7.1.2 Classes of statistical tests

5.7.1.3 Tests for normal and binomial data

5.7.1.3.1 One-sample tests for normal data5.7.1.3.2 Two-sample tests for normal data

5.7.1.3.3 Test for binomial data

5.7.1.4 Distribution-free tests for location

5.7.1.4.1 Sign tests

5.7.1.4.2 Sign rank test

5.7.1.4.3 Wilcoxon rank sum test


15/17

5.7.2 Confidence Interval

5.7.2.1 Binomial distribution

5.7.2.2 Normal (Gaussian) distribution

5.7.3 Required sample size for certain confidence levels

6 Visualization Techniques

6.1 Visualization of static data

6.1.1 Histograms

6.1.2 Barplots

6.1.3 QQ-Plots

6.1.4 Boxplots

6.2 Visualization of dynamic data6.2.1 Line Diagrams

6.2.2 Temporal changing Boxplots

6.2.3 MMQ-Plots

7 Time series modelling

7.1 Descriptive characterization

7.1.1 Empirical moments

7.1.2 Decomposition of time series

7.1.3 Determination of the trend component

7.1.3.1 Trend function types7.1.3.1.1 Linear trend function

7.1.3.1.2 Polynomial trend function

7.1.3.1.3 Non-linear trend models

7.1.3.2 Trend estimation

7.1.3.3 Transformation of time series by filtering

7.1.3.3.1 Linear filters

7.1.3.3.2 Exponential filters

7.1.4 Seasonal component

8 Data aggregation

8.1 Basic data aggregation operators8.2 Data sources, structures and properties

8.2.1 Raw data

8.2.1.1 Performance data

8.2.1.2 Event data

8.2.2 Key Performance Indicators / Parameters

8.3 Aggregation Hierarchies


16/17

8.3.1 Temporal Aggregation

8.3.2 Spatial Aggregation

8.4 Parameter estimation methods

8.4.1 Projection method

8.4.2 Substitution method

8.4.3 Application of estimation methods

8.4.4 Attributes of aggregation operators

8.5 Weighted aggregation

8.5.1 Perceived QoS

8.5.2 Weighted quantiles

8.6 Additional data aggregation operators8.6.1 MAWD and BH

8.6.2 AVGn

9 Assessment of performance indices

9.1 Estimation of performance parameters based on active service probing systems

9.2 Monitoring concepts

9.2.1 Control charts

9.2.1.1 Shewhart control charts

9.2.1.2 CUSUM and EWMA charts

9.2.2 Other alarming rules9.3 Methods for evaluation of objectives

9.3.1 Desirability functions

9.3.2 Loss functions


17/17

No Issue Comment

1 Network Accessability Network Accessability is not independently measured

2 Timing pattern identical number of attempts (no extra attempts in case of failure)

standards - etsi compliance(2)

Documents