kpi measurement method v3a

KPI measurement methodology. Orange Optimisation.

Document reference: aa.bb.ccc.xxxxDocument issue: 01.01 / ENDocument status: DraftDate: 1 July 2004

External Document

Copyright 2001 Nortel Networks, All Rights Reserved

Printed in France

NORTEL NETWORKS CONFIDENTIAL:

The information contained in this document is the property of Nortel Networks. Except as specifically authorized in writing by Nortel Networks, the holder of this document shall keep the information contained herein confidential and shall protect same in whole or in part from disclosure and dissemination to third parties and use same for evaluation, operation and maintenance purposes only.

The content of this document is provided for information purposes only and is subject to modification. It does not constitute any representation or warranty from Nortel Networks as to the content or accuracy of the information contained herein, including but not limited to the suitability and performances of the product or its intended application.

The following are trademarks of Nortel Networks: *NORTEL NETWORKS, the NORTEL NETWORKS corporate logo, the NORTEL Globemark, UNIFIED NETWORKS. The information in this document is subject to change without notice. Nortel Networks assumes no responsibility for errors that might appear in this document.

All other brand and product names are trademarks or registered trademarks of their respective holders.


PUBLICATION HISTORY

1 July 2004

Issue 01.01 / EN, Draft

Creation

aa.bb.ccc.xxxx 01.01 / EN Draft 1 July 2004 /32


CONTENTS

1. CS VOICE – LONG CALLS............................................................................................5

1.1. XCAL CONFIGURATION...............................................................................................5

1.2. METRICS..................................................................................................................5

1.2.1 NCOUP..............................................................................................................51.2.2 DL BLER............................................................................................................61.2.3 UL BLER............................................................................................................7

2. CS64 VIDEO – 2 MIN CALLS.........................................................................................9

2.1. XCAL CONFIGURATION...............................................................................................9

2.2. METRICS..................................................................................................................9

2.2.1 CR......................................................................................................................92.2.2 DL BLER..........................................................................................................102.2.3 UL BLER..........................................................................................................10

3. PS 128 DL FTP – 1M FILE DOWNLOAD.....................................................................12

3.1. XCAL CONFIGURATION.............................................................................................12

3.2. METRICS................................................................................................................13

3.2.1 DL Throughput.................................................................................................133.2.2 DL BLER..........................................................................................................163.2.3 CR....................................................................................................................18

4. PS 64 UL FTP – 0.5M FILE UPLOAD..........................................................................20


4.2. METRICS................................................................................................................20

4.2.1 UL throughput..................................................................................................204.2.2 CR....................................................................................................................214.2.3 UL BLER..........................................................................................................20

5. CS / PS SHORT CALLS...............................................................................................22


5.2. METRICS................................................................................................................22

5.2.1 CR....................................................................................................................22

6. CS 3G/2G HHO.............................................................................................................26

7. ANNEX - EXCLUSION ZONE.......................................................................................27

8. ANNEX – DROP AND FAIL CLASSIFICATION...........................................................28

9. ANNEX - RF CONFIGURATAION................................................................................30

9.1. MOBILE DANS VALISE D'ATTENUATION........................................................................30



9.2. MOBILE SEUL (TESTS OUTDOOR SANS CHARGE OU OUTDOOR AVEC CHARGE DL) :......30

9.3. MOBILE SEUL (TESTS INCAR) :...................................................................................30

10. ANNEX - OCNS.............................................................................................................31



1. CS VOICE – LONG CALLS

1.1. XCAL CONFIGURATION

XCAL Version 1.44

PhoneIdle Time: 10SetUpTime: 20Traffic Time: 99999Call Count: 99999

Voice CallCall Type : OriginationDialed digit : 04 37 50 15 98 (or 08 00 12 34 56 when not available)AMR Rate : 12.2 kbpsCall Flow : Using Alert Msg

1.2. METRICS

1.2.1 NCOUP

NCOUP = number of call drops per hour

1. Use XCAP to generate and export the VOICE CALL EVENT table. 2. Calculate Total Call Duration (T). There are two methods that can be applied to do so:

a. Sum up the time difference between the “Call Drop” and “Call Start” events (i.e. do not consider the Call Fail events).

i. If the call ends normally (as in the case of a “Stop by User”) use the normal end event timestamp.

b. Count the number of TX Power samples (binning is done every 1s).

16:28:00 Call Start16:28:01 Connection Request16:28:01 Connection Setup Complete16:28:06 Call Connected16:28:06 Call Establishment16:28:11 Call Connected16:42:51 Call Drop 00:14:5116:42:51 Call End16:43:02 Call Start16:43:03 Connection Request16:43:03 Connection Setup Complete16:43:08 Call Connected16:43:08 Call Establishment16:43:13 Call Connected16:48:49 Call Drop 00:05:4716:48:50 Call End



3. Select only the “Call Drop” events. 4. Analyse each one of the drops identifying RF, UE and System issues.5. Report all the drops and their causes after investigation. Example :

16:42:51 Call Drop 00:14:51 Pas de mesures sur le 144 (com sur 113) mesures16:48:49 Call Drop 00:05:47 No ASU Remove 82 ASU

6. Count the number of DROPs EXCEPT (cf. 8 ANNEX – Drop and Fail Classification for details on the Fail/Drop classification):

a. Drops occurred in exclusion areasb. Drops due to hardware failures. Document the failure reporting the city, area,

drive (out/indoor), measurement protocol, date, site and type of issue.c. Drops/Fails due to calls setup on another frequencyd. Drops due to UE issues approved/validated by Orange teamse. Failures on the tool’s Call Scenario setupf. Events that are wrongly identified as drops by XCAP/XCAL (Normal Call

Clearing)

NCOUP = DROPs / T

1.2.2 DL BLER

1. Use XCAP and the filtering option using the adequate threshold (cf. 7 ANNEX - EXCLUSION ZONE) to generate a table containing :

a. scanner Top 1 RSCP datab. BLER info / Channel Record Count / Channel 1c. BLER info / Channel Record Error / Channel 1

2. Use XCAP to generate and export to Excel the table showing Qualcomm signalling data (layer 3 messages)



3. For each call filter out the BLER data generated before the “CC DTAP CONNECT” message and after the “CC DTAP DISCONNECT” message, as well as the last point before a drop. For instance, in the example below, exclude the yellow samples :

20:38:58.517 Call Start20:39:01.505 Call Connected20:39:04.000 101 5 -58.620:39:06.000 100 2 -57.620:39:08.000 100 1 -59.120:39:08.479 DL DCCH downlinkDirectTransfer CONNECT20:39:08.489 UL DCCH uplinkDirectTransfer CONNECT ACK20:39:10.000 101 2 -57.920:39:12.000 100 1 -51.7

…20:40:56.000 100 6 -57.720:40:58.000 100 0 -62.120:41:00.000 100 0 -59.420:41:01.517 Call Success20:41:02.000 100 0 -54.820:41:02.128 Call End20:41:03.213 UL DCCH uplinkDirectTransfer DISCONNECT20:41:04.000 54 29 -55.720:41:13.165 Call Start

…

4. Sum up all transmitted blocks on the traffic channel (Channel Record Count / Channel 1) = BLOCKS

5. Sum up all erroneous blocks on the traffic channel (Channel Record Error / Channel 1) = ERROR

DL BLER = 100 * ERROR / BLOCKS

Additional analysis (optional):

1. with XCAP generate a map showing the DL BLER geographical distribution2. generate the cumulative distribution of DL BLER values

1.2.3 UL BLER

1. Collect the RNC call trace data corresponding to the drive test being analysed2. Import and post-process the call trace data with Nortel RFO tool3. Generate the average UL BLER value with the RFO UL BLER Analyser.4. Report Average UL BLER on DCH 1



Figure 1 - Example of UL calculation in a PS call

Important Note:

It is not possible to exclude the UL BLER samples measured in the exclusion zones with this method. A complex and time consuming method is needed to synchronise UL and DL traces. A tool enhancement has been requested in order to be able to produce BLER distributions.



2. CS64 VIDEO – 2 MIN CALLS


XCAL Version 1.44

PhoneIdle Time: 10SetUpTime: 20 Traffic Time: 120Call Count: 99999

CSCall Type: OriginationOption: at+cbst=116,1,0DialNum: atd 06 31 31 11 69 (number of B-Party phone)

2.2. METRICS

2.2.1 CR

CR = number of calls successfully setup and maintained for 2 minutes / number of call attempts

1. Use XCAP to generate and export the CS CALL EVENT table. 2. Select the “Call Drop” and “Call Fail” events. Do not take into account non voice calls:

location/routing area updates …

yyyy-06-dd 20:01:36.357 Call Startyyyy-06-dd 20:01:39.479 Call Connectedyyyy-06-dd 20:03:39.358 Call Successyyyy-06-dd 20:03:39.968 Call Endyyyy-06-dd 20:03:50.999 Call Startyyyy-06-dd 20:04:20.999 Call Failyyyy-06-dd 20:04:21.609 Call Endyyyy-06-dd 20:04:32.650 Call Startyyyy-06-dd 20:04:37.374 Call Connectedyyyy-06-dd 20:04:49.256 Call Dropyyyy-06-dd 20:04:49.866 Call Endyyyy-06-dd 20:05:00.910 Call Startyyyy-06-dd 20:05:04.965 Call Connectedyyyy-06-dd 20:07:04.910 Call Successyyyy-06-dd 20:07:05.519 Call End

3. Analyse each one of the drops and fails identifying RF, UE and System issues.



4. Report all the drops and fails and their causes after investigation. Example :

16:42:51 Call Drop pas de mesures sur le 144 (com sur 113) mesures16:48:49 Call Fail Exclusion Zone EZ

5. Count the number of DROPs and FAILs EXCEPT (cf. 8 ANNEX – Drop and Fail Classification for details on the Fail/Drop classification):

a. Drops/Fails occurred in exclusion areasb. Drops/Fails due to hardware failures. Document the failure reporting the city,

area, drive (out/indoor), measurement protocol, date, site and type of issue.c. Drops/Fails due to calls setup on another frequencyd. Drops/Fails due to UE issues approved/validated by Orange teamse. Failures on the tool’s Call Scenario setupf. Events that are wrongly identified as Drops/Fails by XCAP/XCAL (Normal Call

Clearing)

6. Count all the “Call Start” events = EVENTs

CR = 100 * ( 1 – (DROPs + FAILs) / EVENTs)

2.2.2 DL BLER

The methodology to calculate the DL BLER is the same as the one described in section 1.2.2 except that the transport channel to be used for a CSD RAB is Channel 4.

BLER information is given in groups of 200 blocks.

2.2.3 UL BLER





Important Note:




3. PS 128 DL FTP – 1M FILE DOWNLOAD


XCAL Version 1.44

PhoneIdle Time: 10SetUpTime: 60 T.SetUpTime: 30Traffic Time: 99999Call Count: 99999

FTP CallHost: 10.66.216.193Login ID: urm (urce URCE)Password: xxxxxxChange Dir: downloadsGet File: 1MoRetry Count: 1

Dormant: untickedWait PPP Release: tickedLogging FTP data: tickedPending: unticked



3.2. METRICS

3.2.1 DL THROUGHPUT

1. Use XCAP and the filtering option using the adequate threshold (cf. ANNEX - EXCLUSION ZONE) to generate a table containing :

a. scanner Top 1 RSCP data b. Throughput info / FTP / Total Rx Bytesc. Throughput info / FTP / FWD Throughput

The resulting table will show:

2. Report the Average throughput (115.796 kbps in the example above).

DL Throughput (kbps) = Average Throughput



3.2.1.1 ZERO THROUGHPUT SAMPLES AFTER A DROP

In the calculation of the average Throughput, we count all the samples except those under Exclusion Zones including those that correspond to the calls that dropped (the transfer of the file has not completely finished). We observe that before each drop, the tool will report throughput values equal to 0 during some seconds. This will make that, taking into account our methodology; the average throughput will be biased by those null samples. Thus, the final KPI is “degraded”. Indeed so many zero samples before a drop are not representative of the average throughput of successful FTP sessions.

See below an example taken from a UL FTP (Upload PS):

Zero throughput samples are reported from 20:22:30.000 BCCH channel is listened by the mobile at 20:23:20.892 (around 50 seconds later) The Drop is detected by the tool at 20:23:20.892 (when BCCH is received)

20:22:19.872 UL CCCH rrcConnectionRequest originatingHighPrioritySignalling20:22:20.215 DL CCCH rrcConnectionSetup cell_DCH20:22:20.334 UL DCCH rrcConnectionSetupComplete20:22:20.340 UL DCCH initialDirectTransfer SERVICE REQUEST20:22:21.335 DL DCCH downlinkDirectTransfer AUTHENTICATION AND

CIPHERING REQ.20:22:21.513 UL DCCH uplinkDirectTransfer AUTHENTICATION AND

CIPHERING RESP.20:22:23.029 DL DCCH securityModeCommand20:22:23.031 UL DCCH securityModeComplete20:22:23.306 UL DCCH uplinkDirectTransfer ACTIVATE PDP CONTEXT

REQUEST20:22:25.115 DL DCCH radioBearerSetup sf16 / sf1620:22:26.315 UL DCCH radioBearerSetupComplete20:22:27.265 DL DCCH downlinkDirectTransfer ACTIVATE PDP CONTEXT

ACCEPT20:22:27.544 IPCP Complete20:22:27.544 PPP Logon Start20:22:27.984 PPP Logon End20:22:27.992 Traffic Setup Start20:22:27.992 FTP Connecting20:22:28.909 FTP Connected20:22:29.000 0 020:22:29.393 FTP Load CMD Upload20:22:29.393 Traffic Start20:22:29.733 FTP Work Upload20:22:30.000 41.632 520420:22:31.000 63.488 793620:22:32.000 58.4 7300

…20:23:08.000 58.4 730020:23:09.000 58.4 730020:23:10.000 46.72 584020:23:11.000 70.08 876020:23:12.000 23.36 292020:23:13.000 0 020:23:14.000 0 020:23:15.000 0 0



20:23:16.000 0 020:23:17.000 0 020:23:18.000 0 020:23:19.000 0 020:23:20.000 0 020:23:20.892 DL BCCH systemInformationBlockType1120:23:20.892 Drop20:23:20.892 DUN Release Start20:23:20.936 DL BCCH systemInformationBlockType-m20:23:20.940 DL BCCH systemInformationBlockType1120:23:20.951 DL BCCH systemInformationBlockType11

3.2.1.2 SAMPLES OUTSIDE THE ACTUAL DOWNLOAD

The Call Events given by XCAP of a successful Data session are the following ones:

TIME_STAMP Data Call Eventyyyy-06-dd 13:22:57.230 Idleyyyy-06-dd 13:23:07.231 Setupyyyy-06-dd 13:23:07.274 ATyyyy-06-dd 13:23:07.550 Call Startyyyy-06-dd 13:23:07.550 ATDTyyyy-06-dd 13:23:07.589 AT Connectyyyy-06-dd 13:23:07.617 LCP Startyyyy-06-dd 13:23:07.626 LCP Startyyyy-06-dd 13:23:07.633 LCP Completeyyyy-06-dd 13:23:07.635 Auth Startyyyy-06-dd 13:23:07.644 Auth Completeyyyy-06-dd 13:23:07.645 IPCP Startyyyy-06-dd 13:23:16.162 IPCP Completeyyyy-06-dd 13:23:16.162 PPP LogOnyyyy-06-dd 13:23:16.488 PPP LogOnyyyy-06-dd 13:23:16.493 Trafficyyyy-06-dd 13:23:16.493 FTP Connectingyyyy-06-dd 13:23:17.445 FTP Connectedyyyy-06-dd 13:23:18.214 FTP download /upload Startyyyy-06-dd 13:25:44.493 FTP Successyyyy-06-dd 13:25:44.494 Releaseyyyy-06-dd 13:25:49.125 Call End

Actual FTP Download occurs between the “FTP Download/upload Start” and “FTP Success” events.

It has been observed that Couei’s average FTP when a filter is applied is sometimes pessimistic with regards to a manual calculation using Microsoft Excel. If the average throughput is not within the targets do a detailed throughput analysis between the above two call states taking into account the Exclusion Zones.



The Throughput calculation would consist in accounting the bytes downloaded between those two events once the samples recorded in the exclusion zones are removed, and dividing the result by the number of samples (with the binning used, there is one throughput sample per second).

3.2.2 DL BLER

1. Use XCAP and the filtering option using the adequate threshold (cf. 6 ANNEX - EXCLUSION ZONE) to generate a table containing the following fields:

a. scanner Top 1 RSCP datab. BLER info / Channel Record Count / Channel 4c. BLER info / Channel Record Error / Channel 4

BLER information is given in groups of 800 blocks.

2. Use XCAP to generate and export to Excel the table showing the Data Call Events. Actual FTP Download occurs between the “FTP Download/upload Start” and “FTP Success” events.

TIME_STAMP Data Call Eventyyyy-06-dd 13:22:57.230 Idleyyyy-06-dd 13:23:07.231 Setupyyyy-06-dd 13:23:07.274 ATyyyy-06-dd 13:23:07.550 Call Startyyyy-06-dd 13:23:07.550 ATDTyyyy-06-dd 13:23:07.589 AT Connectyyyy-06-dd 13:23:07.617 LCP Startyyyy-06-dd 13:23:07.626 LCP Startyyyy-06-dd 13:23:07.633 LCP Completeyyyy-06-dd 13:23:07.635 Auth Startyyyy-06-dd 13:23:07.644 Auth Completeyyyy-06-dd 13:23:07.645 IPCP Startyyyy-06-dd 13:23:16.162 IPCP Completeyyyy-06-dd 13:23:16.162 PPP LogOnyyyy-06-dd 13:23:16.488 PPP LogOnyyyy-06-dd 13:23:16.493 Trafficyyyy-06-dd 13:23:16.493 FTP Connectingyyyy-06-dd 13:23:17.445 FTP Connectedyyyy-06-dd 13:23:18.214 FTP download /upload Startyyyy-06-dd 13:25:44.493 FTP Successyyyy-06-dd 13:25:44.494 Releaseyyyy-06-dd 13:25:49.125 Call End



3. For each call, filter out the BLER data generated before the “FTP Download/upload Start” event and after the “FTP Success” event as well as the last point before a drop. For instance, in the example below, exclude the yellow samples :

TIME_STAMP Record CountChannel 4

Record ErrorChannel 4

Top 1RSCP(dBm)

yyyy-06-dd 13:23:18.000 45 0 -72.6yyyy-06-dd 13:23:20.000 670 0 -74.9yyyy-06-dd 13:23:22.000 790 0 -72.7yyyy-06-dd 13:23:24.000 800 0 -73.7yyyy-06-dd 13:23:26.000 800 0 -73.3yyyy-06-dd 13:23:28.000 741 0 -72

…yyyy-06-dd 13:25:42.000 714 0 -66.5yyyy-06-dd 13:25:44.000 673 0 -66.5yyyy-06-dd 13:25:46.000 139 10 -67yyyy-06-dd 13:25:47.000 36 36 -66.3

4. Sum up all transmitted blocks on the traffic channel (Channel Record Count / Channel 4) = BLOCKS

5. Sum up all erroneous blocks on the traffic channel (Channel Record Error / Channel 4) = ERROR

DL BLER = 100 * ERROR / BLOCKS

Additional analysis (optional):

1. with XCAP generate a map showing the DL BLER geographical distribution2. generate the cumulative distribution of DL BLER values



3.2.3 CR

CR = number of calls successfully setup and maintained until download is completed / number of call attempts

1. Use XCAP to generate and export the DATA CALL EVENT table. 2. Select the “Call Drop” and “Call Fail” events. Do not take into account non data calls:

location/routing area updates …

yyyy-06-dd 13:54:09.407 Idleyyyy-06-dd 13:54:19.408 Setupyyyy-06-dd 13:54:19.452 ATyyyy-06-dd 13:54:19.752 Call Startyyyy-06-dd 13:54:19.752 ATDTyyyy-06-dd 13:54:19.765 AT Connectyyyy-06-dd 13:54:19.792 LCP Startyyyy-06-dd 13:54:19.797 LCP Startyyyy-06-dd 13:54:19.804 LCP Completeyyyy-06-dd 13:54:19.806 Auth Startyyyy-06-dd 13:54:19.816 Auth Completeyyyy-06-dd 13:54:19.818 IPCP Startyyyy-06-dd 13:55:19.752 Setup Failyyyy-06-dd 13:55:19.752 Releaseyyyy-06-dd 13:55:21.356 Call Endyyyy-06-dd 13:55:22.353 Idleyyyy-06-dd 13:55:32.353 Setupyyyy-06-dd 13:55:32.405 ATyyyy-06-dd 13:55:32.711 Call Startyyyy-06-dd 13:55:32.711 ATDTyyyy-06-dd 13:55:32.725 AT Connectyyyy-06-dd 13:55:32.753 LCP Startyyyy-06-dd 13:55:32.757 LCP Startyyyy-06-dd 13:55:32.764 LCP Completeyyyy-06-dd 13:55:32.766 Auth Startyyyy-06-dd 13:55:32.775 Auth Completeyyyy-06-dd 13:55:32.777 IPCP Startyyyy-06-dd 13:55:40.619 IPCP Completeyyyy-06-dd 13:55:40.619 PPP LogOnyyyy-06-dd 13:55:40.941 PPP LogOnyyyy-06-dd 13:55:40.947 Trafficyyyy-06-dd 13:55:40.947 FTP Connectingyyyy-06-dd 13:55:42.388 FTP Connectedyyyy-06-dd 13:55:43.191 FTP download /upload Startyyyy-06-dd 13:56:49.429 Dropyyyy-06-dd 13:56:49.429 Releaseyyyy-06-dd 13:57:52.569 Call Endyyyy-06-dd 13:57:53.570 Idle

3. Analyse each one of the drops and fails identifying RF, UE and System issues.



4. Report all the drops and fails and their causes after investigation. Example :



a. Drops/Fails occurred in exclusion areasa. Drops/Failures due to hardware failures. Document the failure reporting the city,

area, drive (out/indoor), measurement protocol, date, site and type of issue.b. Drops/Fails due to calls setup on another frequencyc. Drops/Fails due to UE issues approved/validated by Orange teamsd. Failures on the tool’s Call Scenario setupe. Events that are wrongly identified as Drops/Fails by XCAP/XCAL (Normal Call

Clearing)

6. Count all the “Call Start” events = EVENTs




4. PS 64 UL FTP – 0.5M FILE UPLOAD


XCAL Version 1.44

PhoneIdle Time: 10SetUpTime: 60 T.SetUpTime: 30 Traffic Time: 99999Call Count: 99999

FTP CallHost: 10.66.216.193Login ID: urm (urce URCE)Pasword: xxxxxxChange Dir: uploadsPut File: 500kRetry Count: 1

Dormant: untickedWait PPP Release: tickedLogging FTP data: tickedPending: unticked

4.2. METRICS

4.2.1 UL THROUGHPUT

Use the same methodology as described in section 3.2.1. Just export the “Total Tx Bytes” instead of the “Total Rx Bytes”. Therefore, the first step would be:

1. Use XCAP and the filtering option using the adequate threshold (cf. ANNEX - EXCLUSION ZONE) to generate a table containing :

a. scanner Top 1 RSCP datab. Throughput info / FTP / Total Tx Bytesc. Throughput info / FTP / RVS Throughput

4.2.2 UL BLER





Important Note:


4.2.3 CR

CR = number of calls successfully setup and maintained until upload is completed / number of call attempts

Refer to section 3.2.3.



5. CS / PS SHORT CALLS


XCAL Version 1.44

CSPhone

Idle Time: 10SetUp Time: 20Traffic Time: 10Call Count: 1

Voice CallCall Type: OriginationDialed digit: 04 37 50 15 98 (or 08 00 12 34 56 when not available)AMR Rate: 12.2 kbpsCall Flow: Using Alert Msg

PSPPP scenario

5.2. METRICS

5.2.1 CR

CR = number of calls successfully setup and maintained following the call scenario / number of call attempts

To compute the CR for the CS calls, it is possible to follow the same methodology as described in 2.2.1, i.e. using XCAP’s Voice Call Events. However, XCAP does not generate any Data Call Events when the PPP scenario has been used for the PS calls. This means that the process described in 3.2.3 cannot be fully applied here.

In order to calculate the CR for this test scenario, it is needed to look at the Layer 3 messages in order to identify the Drops and the Fails of the PS call (calculation of a “PS CR”). We can use the same methodology to detect the Drops and the Fails for the CS call (“CS CR”).

In order to simplify the procedure it is proposed here to calculate the CS CR and the PS CR using the same table generated once with XCAP. CS CR can also be calculated following the method described in 2.2.1.



1. Use XCAP to generate a table with the “Logging Message” information and filter the Qualcomm Signalling Msg

a. In order to save time during the export, you can unselect messages that are not being used during the first steps of the analysis like:

i. DL BCCH:BCHii. DL DCCH activeSetUpdate

iii. UL DCCH measurementReport

2. Export to Excel all the messages of the resulting “Filtered Message Window” (do not select any Export Option). The resulting table will contain the following sequence of messages:

b. CS Call is initiated at 23:00:51.980c. PS Call is initiated at 23:01:20.121

23:00:51.980 rrcConnectionRequest originatingConversationalCall 23:00:52.301 rrcConnectionSetup cell_DCH 23:00:52.421 rrcConnectionSetupComplete 23:00:52.429 initialDirectTransfer CM SERVICE REQUEST 23:00:53.075 downlinkDirectTransfer AUTHENTICATION REQUEST 23:00:53.268 uplinkDirectTransfer AUTHENTICATION RESPONSE 23:00:53.524 securityModeCommand 23:00:53.524 securityModeComplete 23:00:53.719 uplinkDirectTransfer SETUP 23:00:53.768 downlinkDirectTransfer IDENTITY REQUEST 23:00:53.770 uplinkDirectTransfer IDENTITY RESPONSE 23:00:54.126 downlinkDirectTransfer CALL PROCEEDING 23:00:54.644 radioBearerSetup sf128 / sf64 23:00:55.705 radioBearerSetupComplete 23:00:57.278 downlinkDirectTransfer ALERTING 23:00:57.643 downlinkDirectTransfer CONNECT 23:00:57.653 uplinkDirectTransfer CONNECT ACKNOWLEDGE 23:01:07.568 uplinkDirectTransfer DISCONNECT 23:01:07.828 downlinkDirectTransfer RELEASE 23:01:07.871 uplinkDirectTransfer RELEASE COMPLETE 23:01:08.075 rrcConnectionRelease-DCCH normalEvent 23:01:08.115 rrcConnectionReleaseComplete 23:01:08.202 rrcConnectionRelease-DCCH normalEvent 23:01:08.239 rrcConnectionReleaseComplete 23:01:08.435 rrcConnectionReleaseComplete 23:01:08.572 rrcConnectionReleaseComplete 23:01:20.121 rrcConnectionRequest originatingHighPrioritySignalling 23:01:20.453 rrcConnectionSetup cell_DCH 23:01:20.557 rrcConnectionSetupComplete 23:01:20.562 initialDirectTransfer SERVICE REQUEST 23:01:22.387 downlinkDirectTransfer AUTHENTICATION AND CIPHERING REQ. 23:01:22.556 uplinkDirectTransfer AUTHENTICATION AND CIPHERING RESP. 23:01:24.312 securityModeCommand 23:01:24.313 securityModeComplete 23:01:24.512 uplinkDirectTransfer ACTIVATE PDP CONTEXT REQUEST 23:01:27.647 radioBearerSetup sf16 / sf16 23:01:28.804 radioBearerSetupComplete 23:01:30.433 downlinkDirectTransfer ACTIVATE PDP CONTEXT ACCEPT 23:01:31.563 uplinkDirectTransfer DEACTIVATE PDP CONTEXT REQUEST 23:01:33.267 downlinkDirectTransfer DEACTIVATE PDP CONTEXT ACCEPT 23:01:33.471 rrcConnectionRelease-DCCH normalEvent 23:01:33.475 rrcConnectionReleaseComplete 23:01:33.476 rrcConnectionRelease-DCCH normalEvent



23:01:33.535 rrcConnectionRelease-DCCH normalEvent 23:01:33.631 rrcConnectionReleaseComplete 23:01:33.788 rrcConnectionReleaseComplete 23:01:33.948 rrcConnectionReleaseComplete

3. Use Excel’s AutoFilter feature to display only lines that containing, in their third column, the messages CM SERVICE REQUEST and DISCONNECT.

initialDirectTransfer CM SERVICE REQUESTuplinkDirectTransfer DISCONNECTinitialDirectTransfer CM SERVICE REQUEST fail/dropinitialDirectTransfer CM SERVICE REQUESTuplinkDirectTransfer DISCONNECTinitialDirectTransfer CM SERVICE REQUEST fail/dropinitialDirectTransfer CM SERVICE REQUESTuplinkDirectTransfer DISCONNECTinitialDirectTransfer CM SERVICE REQUESTuplinkDirectTransfer DISCONNECT

4. If a CM SERVICE REQUEST is not followed by a DISCONNECT, the call is classified as having suffered either a CS Call Fail or a CS Call Drop

5. Use Excel’s AutoFilter feature to display only lines that containing, in their third column, the messages SERVICE REQUEST and DEACTIVATE PDP CONTEXT ACCEPT.

initialDirectTransfer SERVICE REQUESTdownlinkDirectTransfer DEACTIVATE PDP CONTEXT ACCEPTinitialDirectTransfer SERVICE REQUESTdownlinkDirectTransfer DEACTIVATE PDP CONTEXT ACCEPTinitialDirectTransfer SERVICE REQUEST fail/dropinitialDirectTransfer SERVICE REQUEST fail/dropinitialDirectTransfer SERVICE REQUESTdownlinkDirectTransfer DEACTIVATE PDP CONTEXT ACCEPTinitialDirectTransfer SERVICE REQUESTdownlinkDirectTransfer DEACTIVATE PDP CONTEXT ACCEPT

6. If a SERVICE REQUEST is not followed by a DEACTIVATE PDP CONTEXT ACCEPT, the call is classified as having suffered either a PS Call Fail or a PS Call Drop

7. Analyse each one of the CS and PS DROPs and FAILs identifying RF, UE and System issues.8. Report all the CS and PS DROPs and FAILs and their causes after investigation. Example :





f. Drops/Fails occurred in exclusion areasg. Drops/Fails due to hardware failures. Document the failure reporting the city,

area, drive (out/indoor), measurement protocol, date, site and type of issue.h. Drops/Fails due to calls setup on another frequencyi. Drops/Fails due to UE issues approved/validated by Orange teamsj. Failures on the tool’s Call Scenario setupk. Events that are wrongly identified as Drops/Fails by XCAP/XCAL (Normal Call

Clearing)

10. Count all the “SERVICE REQUEST” and “CM SERVICE REQUEST” messages = EVENTs




6. CS 3G/2G HHO


XCAL Version 1.44

QCT mobile is to be used WITHOUT the external antenna in a INCAR configuration (no attenuation box).

Put the test mobile always in the same position inside the car during all the tests. Use a 2G mobile with network monitoring capabilities in order to check the availability of

GSM coverage.

Setup a voice call under 3G coverage and drive outside it towards a 2G covered area. Use the long call setup scenario for XCAL.

PhoneIdle Time: 10SetUpTime: 20Traffic Time: 99999Call Count: 99999

Voice CallCall Type : OriginationDialed digit : 04 37 50 15 98 (or 08 00 12 34 56 when not available)AMR Rate : 12.2 kbpsCall Flow : Using Alert Msg

6.2. METRICS

6.2.1 HHO SUCCESS

HHO Success = percentage of calls initiated under 3G coverage that have been successfully handed over and maintained at least 20 seconds in 2G coverage.

TBC



7. ANNEX - EXCLUSION ZONE

All acceptance tests are done with simulated DL and UL load. Exclusion zones are defined as:

Outdoor: scanner Top1 RSCP < -98dBm Indoor: scanner Top1 RSCP < -81dBm ( = -98 -(- 17) )

Samples within exclusion zones can be filtered out using exports to Excel (UE data is synchronised with Scanner Anritsu Top 1 RSCP values) or thanks to the FILTERING option in XCAP.

Filters can be defined as follows:

Figure 4 - Filtering option Outdoor Exclusion Zone

To apply a filter in a table, we select the field to drag on the right hand side of XCAP window, right click and “tick” the filter we want to apply. The table will contain only samples that follow the filter rules.

Figure 5 - Example of applying Filter EZ in XCAP



8. ANNEX – DROP AND FAIL CLASSIFICATION

If the column COUNT has a “N”, the Drops/Fails classified in that category are not to be accounted in the KPI formula after agreement with Orange teams.

Class Sub ClassCount (Y/N)

Failure Name

Failure Description

RF Design

RSCP < -98 Outdoor RSCP < -81 Indoor

NExclusion

Zone

Top 1 RSCP from Scanner Threshold is below -98 dBm for Outdoor and -81 dBm for

Indoor.

Antenna Location / Type

YAntenna choice

Moving the antenna location or changing the antenna type may improve coverage and provide better dominance in pilot polluted

area. Change has been approved by Orange but not implemented during the audit.

Radio Fails

Pilot Pollution

Y

Pilot Pollution

An Area defined by high number (> 3) of SC with equal RSCP and Ec/Io Values with no

dominant SC

Missing NeighbourMissing

NeighborSC not included in the reference SC's

neighbor list

Overshoot OvershootOvershoot is defined as a cell who's

coverage extends beyound the second tier of the cell

No Overlap No Overlap

There is no adequate overlap between the sectors. The efect is strong SC seen by UE very abruptly causing fast interference that

causes a drop

Other Other Other Radio causes not listed before.

Missing PA siteRSCP>-98

YDrops due to Missing

PA site

Includes All fails (R1-R4) that would not otherwise happen if the PA site was present.

Document properly this type of fail.

Network Fails

Site Down/Hardware Fails

N

LL Down / RF ON

Loss of communication from RNC to Node-B. The Cell is still radiating causing interference

Site Down / RF OFF

Cell is not radiating

Island CellCell is radiating but UE can not hand-in or

Hand-out

Bad TMATMA is off or alarmed. Usually indicated by a

high UL_Interference value in the SIB 7 Message on the BCH channel

Bad antenna/RF

Cable

Poor Ec/Io and or RSCP in line of sight to cell

System Fails

NL Full Y Neighbor List Full

Neighbor List has 16 neighbors



ASU Y

Missing Active Set

Update

Measurement report indicates a SC that meets the add criteria but no active set

update is received by the UE

Incorrect ASU

decision

Wrong ASU message received taking into account what the UE is sending in the

Measurement Reports.

Datafill YWrong Cell

LAC

The location areas code of the cell is not correct. The result is a call drop or access failure. The cell may behave like an island

cell

Other YDescribe and document every DROP/FAIL

not classified

Mobile Fails & Tools

Wrong Frequency N

SFR Frequency

The mobile goes to the SFR frequency.

Integration Frequency

The mobile goes to the integration frequency

Mobile Issues Pending

Y

UE SC Detection

Fail

The UE does not find a strong SC yet the scanner finds it. Found with software release

5.1.48High DL BLER in good RF

not in SHO

UE has only 1 radio link, CPICH RSCP and Ec/Io are good, UE Tx power is OK, but DL BLER increases. Most likely unexplained

QCT defect.

Mobile Issues Approved

NFOR bit to

0

PDP Context Activation fails when done just after a PS Attach or RAU Request with FOR

bit set to 0.

XCAL/XCAP N

Dial Fail XCAP reports dial fail

No rrcConnect

Request

During a UE mobile origination there are no rrcConnect messages. Data collection has to

be stopped and computer re-booted

Not Classe

d

Investigation in progress

Y

Need further

investigation

All Failures that can not be classified and require further study

RRC Releas

e Unspecified

rrcReleaseUnsp Y

RRC Release Cause

unspecified

RRC Release Cause unspecified probably due to UL SYNCH lost.

Normal Call

Clearing

End by UserN

End by User

Normal end by user call clearing

Distant clearingDistant clearing

Normal call Termination: DTAP UL/DL DISCONNECT sent/received



9. ANNEX - RF CONFIGURATAION

9.1. MOBILE DANS VALISE D'ATTENUATION

Scanner configuration:

Scanner User Correction Value : -3dB (*) (pour enlever 3dB a la mesure du scanner pour compenser les pertes de la boite)

Scanner Antenna Adjust : DISABLED (pour ne pas corriger le gain d'antenna car le mobile ne corrige pas cette valeur)

(*) If the attenuation box has around 3dB losses. Put here the losses of the box.

Chaine att. UE : +5 (antenne) -2 (cable) -3 (valise) = 0Chaine att. Scanner : +5 (antenne) -2 (cable) -3 (correction) = 0

Tests Outdoor sans charge : Tous les attenuateurs de la valise a 0dB Tests Outdoor avec charge : Attenuateur path UL a 3dB Tests Indoor sans charge : Attenuateur path UL a 17dB + Attenuateur path DL a 17dB Tests Indoor avec charge : Attenuateur path UL a 20dB + Attenuateur path DL a 17dB

9.2. MOBILE SEUL (TESTS OUTDOOR SANS CHARGE OU OUTDOOR AVEC CHARGE DL) :

Scanner configuration:

Scanner User Correction Value : 0dB Scanner Antenna Adjust : DISABLED (pour ne pas corriger le gain d'antenna car le mobile ne

corrige pas cette valeur)

Chaine att. UE : +5 (antenne) -2 (cable) = +3Chaine att. Scanner : +5 (antenne) -2 (cable) = +3

Avec cette configuration nous mettons le mobile et le scanner sous les memes conditions. Cette configuration n'est pas actuellement utilise en UR CE.

Inconvenient : les mesures du scanner du parcours fait avec la valise seront differentes de celles faites sans la valise. Les signaux sont "theoriquement" amplifies de 3dB. L'information est a traiter avec precaution.

9.3. MOBILE SEUL (TESTS INCAR) :

Ce n'est pas une configuration "standard" donc a traiter cas par cas. L'attenuation incar est difficile a caracteriser donc nous ne pourrons pas compenser correctement la mesure scanner. Il est donc recommende d'utiliser :

Scanner User Correction Value : -3dB



10. ANNEX - OCNS

The OCNS is activated during the test under loaded conditions and set at 50% of the maximum Node B transmitted carrier power.

Following the agreed recommendations we use the activity factor hypothesis indicated below when calculating the power used in average by the Common Channels.

Example (50% load):

dBm WmaxTxPower 42.1 16.218

Common Channels dBm W Activity Factor WattPrimary CPICH Power 33 1.995 1 1.995P-CCPCH Power / BCH 31 1.259 0.9 1.133P-SCH Power 28 0.631 0.1 0.063S-SCH Power 28 0.631 0.1 0.063AICH Power 26 0.398 0.1 0.040PICH Power 28 0.631 0.96 0.606SCCPCH Power 31 1.259 0.1 0.126

CommonChannelsPower 4.026 = Pcc

CommonChannelsPower to be used 4.026

Traffic Power 40.9 12.192

% LOAD vs maxTxPower 50%LOAD Power Watt 8.109PowerOCNS Watt 36.1 4.083 = LOAD Pwr - Pcc

%OCNS Power 33% = PowerOCNS vs Traffic Power

The power actually reserved (but not necessarily radiated) for the common channels at the BTS is:

Pcc = Pcpich power + max {Pccpch power ; (Psch power + Ssch power)} + aichPower + pichPower + sccpchPower



END OF DOCUMENT


kpi measurement method v3a

Documents

excels autofilter

ftp downloadupload

logging ftp

kpi measurement

produce bler

call start

dcch rrcconnectionreleasecomplete

ftp call host