TEMS Automatic - Script Design Guide

, , 1

2 , ,

© Ascom. All rights reserved.

TEMS is a trademark of Ascom. All other trademarks are the property of their respective holders.

Contents

1 Introduction1.1 General limitations.............................................................................................................

2 General2.1 Failure handling..................................................................................................................

2.2 Logfile recording.................................................................................................................

2.3 Device configuration...........................................................................................................

3 Data services3.1 Network connect................................................................................................................

3.1.1 APN.................................................................................................................................

3.2 IP services..........................................................................................................................

4 Voice services4.1 Mobile to mobile voice service.........................................................................................

4.1.1 Mobile Originated Script................................................................................................

4.1.2 Mobile Terminated script...............................................................................................

4.2 Mobile to fixed voice service............................................................................................

4.3 Default phone number......................................................................................................

4.4 Multi RAB.........................................................................................................................

5 Optimization5.1 Optimizing LTE coverage.................................................................................................

5.2 Optimizing LTE performance...........................................................................................

5.2.1 Other optimization recommendations............................................................................

6 Appendix6.1 References.......................................................................................................................

6.2 Abbreviations...................................................................................................................

, , 3

1 Introduction

This document describes the general service control script design recommendations for TEMS Automatic.

This document is intended for any user of TEMS Automatic.

1.1 General limitationsOnly one device per script is supported.

4 , ,

2 General

2.1 Failure handlingIn each measurement activity the “On Failure” parameter shall be set to “Continue script execution” otherwise Service Control will stop the script execution if the activity fails.

2.2 Logfile recordingOne logfile is automatically recorded per script.

2.3 Device configurationEach script shall set the RAT lock conditions for the script in the first configuration sequence. If no RAT lock is desired a RAT lock activity with the “Technology” parameter set to “Not locked” shall be inserted. The reason is to reset any previous setting of the device from a previous script that might have left the device in an unknown state.

After each RAT lock activity it is required to insert a wait activity to allow the device to renegotiate capabilities with the network. This time is device dependent. There is currently no way to detect that the device has finished its renegotiation with the network.

If band lock is used it is required to perform a sequence of band lock, RAT lock and band lock. The first band lock shall reset any previous band lock. The RAT lock is required to guarantee the function of the final band lock. The final band lock sets the wanted bands.

For device configuration activities it is recommended to set the “On failure” parameter to “Stop script execution” to avoid running the script in wrong configuration mode.

, , 5

3 Data services

3.1 Network connectBefore each IP activity it is recommended to insert a Network Connect activity. The reason is that the device should reconnect to the network if the network connection has been lost in a previous IP activity. If the network connection is still present no new attempt will be executed, i.e. no network connect attempt will be visible in the logfile.

The “Succeed if already connected” parameter shall be set to “True”. I.e. if the device is already connected the Network Connect activity will be identified as “successful” and no additional connect attempt will occur.

The “Abort” parameter shall be set to “Disabled”.

Every network connect activity shall have the “Mode” parameter set to “NDIS”. The “Best Available” option is not recommended since it allows RAS connection which is not supported.

Network connect is not used for smartphone testing since the phone is always connected to the network.

6 , ,

3.1.1 APN

It is recommended to configure the APN per SIM in the Operator Console as depicted below.

It is then possible to use the pre-defined APN in the work order as depicted below.

, , 7

When designing the service control script it is possible to leave the APN field empty. The APN used when performing network connect will be the one specified in the Work Order action, which in turn is defined for the SIM in the RTU properties.

This way it is possible to use the same script for multiple work orders with different SIM/operators.

The APN is retrieved from the SIM when using smartphones. Due to a limitation in Operator Console the APN has to be defined for smartphones if executing data services. The defined APN will not be used in the script though.

8 , ,

3.2 IP servicesAny IP service, like FTP or HTTP, should be configured to transfer more data than one IP packet otherwise no KPIs will be generated.

For each IP service activity it is recommended to have a parallel wait activity with a timer with the same setting as the expected duration of the test case. The reason for this is to control the amount of IP service activities per time unit even if network connect or IP service activities fail. The parallel activity can be replaced with an if-else activity based on the previous network connect activity if the user does not want to the IP service activity to be executed.

All IP service has an “Abort” parameter that determines under what conditions the activity will be aborted.

Never: The activity will execute as long as it takes to complete.

On Timeout: The activity will be aborted after a fixed period of time, unless it has already completed before that time. What you indicate here is thus a maximum duration for the activity.

On Event: The activity will be aborted if and when one of the specified Event(s) occurs; otherwise it will run to completion.

On Timeout and/or Event: A combination of the previous alternatives.

For example, if you want to abort a HTTP Get service whenever an IP connection is lost the Abort Property should be set to “On Event” and the Events property should be set to “Network Connection Lost”. All IP service activities shall at least set the “Abort” parameter to “On Timeout” with a reasonable timeout value greater than the expected duration of the IP service activity. This will abort the IP service activity if the activity hangs or takes too long time.

, , 9

10 , ,

4 Voice services

4.1 Mobile to mobile voice serviceIt is recommended not to perform upload on devices assigned to mobile to mobile voice service testing. The reason is that the Mobile Terminating device will continuously record logfiles while waiting for the incoming call. If the Mobile Originating device is performing upload there may be occasions where it initiates a call by the time the Mobile Terminating device has decided to close its logfile and start a new Mobile Terminated voice activity. In this case the recording of the incoming call on the Mobile Terminated side may be split into two logfiles causing post processing being unable to decode the signaling correctly.

4.1.1 Mobile Originated Script

When designing the mobile to mobile Mobile Originated script it is recommended to use the script depicted below. It is recommended to end the script with a Wait activity to ensure that the Mobile Terminated device is ready to receive a new voice call.

The parallel wait activity will reduce the number of call attempts if the probe enters a bad coverage area. Without the parallel activity multiple failed call attempts may be executed in short time which will skew the statistics. The duration of the wait activity should be the same as the duration of the voice quality activity.

, , 11

4.1.2 Mobile Terminated script

When designing the mobile to mobile Mobile Terminated script it is recommended to use the script depicted below. Please note that no Hangup activity is required since the call is terminated on the Mobile Originated side. A one second wait activity is required after the Answer activity to ensure that the answering device can indicate the open call to the system.

The answer activity should have an abort after timeout with a time less than the duration of the voice quality activity to ensure that the script is not stuck if the Mobile Originated call fails for any reason.

12 , ,

4.2 Mobile to fixed voice serviceFor Mobile to Fixed calls it is required to use the “Synchronized Voice Call sequence Snippet” to setup a call sequence of alternating Mobile Originating and Mobile Terminating.

For Mobile Originating calls only it is possible to run a script similar to the Mobile to Mobile - Mobile Originated script, see section 4.1.1.

For Mobile to Fixed calls it is required to configure the SIM phone number in Operator Console, see below. The phone number is used for synchronization of uplink and downlink AQM samples in post processing.

, , 13

4.3 Default phone numberIn the Operator Console it is possible to define the phone number to call when running a voice service script when the dial activity is configured to use the default phone number.

This way it is possible to use the same script for multiple work orders with different SIM/operators.

14 , ,

The “Voice” number is used for the Synchronized Voice Call sequence Snippet.

The “MTM receiver” is used for Mobile to Mobile - Mobile Originating Calls.

If using the default phone number, +0123456789, in the dial activity the default phone number will be replaced with the predefined phone number.

Please note when running a “Mobile to Fixed – Mobile Originated call only” the default phone number is not supported, i.e. it is required to configure the phone number to call in the dial activity.

, , 15

4.4 Multi RABSome devices support multiple radio access bearers and can hence do voice and data calls concurrently. Mostly this is true for WCDMA but in some cases 2G may also be supported.One of the common test cases is to verify if the data call does have any negative impact on the voice call. The simplest script to achieve this is to start with a Network Connect activity and use a Parallel activity for setting up voice and some data transfer concurrently.

In the script displayed the ftp connection will be set up in parallel to the Dial activity and will likely start downloading before the call is established. The scenario here is basically to have a data transfer continuously being done in parallel with the voice call. A Wait activity can be inserted before the Dial activity to allow the FTP download some time to be setup before doing the voice call.Notice that there is no Network Disconnect activity, this activity is optional if we don’t have a specific need to disconnect.

16 , ,

5 Optimization

5.1 Optimizing LTE coverageSome devices prefer changing technology from WCDMA to LTE only in idle mode. In order to increase the likelihood of the device operating in LTE it is therefore recommended to insert an idle period of 20 seconds between the IP activities or after a sequence of activities.

For some devices it might be necessary to perform a network disconnect as well before the idle period and consequently a network connect before the next IP activity.

5.2 Optimizing LTE performanceAll of the new features render the RTU-5 more powerful but also more sensitive, requiring more attention to configuration and to test script design in order to deliver optimum performance.

For stress tests it is recommended to break up your measurement into sessions of fixed and moderate duration rather than conduct a single long session (for example, transferring a single huge FTP file). After each task, schedule a brief pause to ensure that the system has sufficient time to finish the session properly.

After a test script is executed the logfile is processed, including compressing the logfile. This will consume some of the CPU resources. It is therefore recommended to begin any script with an idle period. This ensures that the CPU load does not reduce the achievable throughput of the measurement.

5.2.1 Other optimization recommendations

For demanding tasks like stress tests or lab tests, it is strongly recommended that you disable any RTU-5 devices not needed for that particular test, freeing up system resources. This can be done remotely from the Operator Console. For such tasks it is also recommendable to use PCI-e slots #3 and #4, which have their own USB controller bus, not shared with any other system resources.

For Qualcomm devices (MC77x0), you can choose between two device APIs for data recording which have different characteristics:

o DIP (Direct IP). This API supports concurrent tests with multiple devices, while having the drawback of sometimes limiting the maximum throughput.

o QMI (Qualcomm API). This API usually gives a higher average throughput; it is however currently limited to testing with one device at a time

, , 17

6 Appendix

6.1 References

6.2 Abbreviations

18 , ,