a new approach to available bandwidth measurements for wireless networks

Prepared by: Date: Document: Ascom Network Testing 2/1/2013 NT13-16812, v2.0 Ascom (2013) TEMS is a trademark of Ascom. All other trademarks are the property of their respective holders.

No part of this document may be reproduced in any form without the written permission of the copyright holder.

The contents of this document are subject to revision without notice due to continued progress in methodology, design and manufacturing. Ascom shall have no liability for any error or damage of any kind resulting from the use of this document.

A New Approach to Available Bandwidth Measurements for Wireless Networks

Ascom (2013) Document: NT13-16812, v2.0

Contents

1 Introduction ................................................................ 1

1.1 Background ...............................................................................1

1.2 Aspects of Present-day Mobile Networks That Must Inform ABM Design ..................................................................1

1.3 Requirements on ABM for 4G Mobile Networks .....................2

2 ABM with TEMS Products ......................................... 3

2.1 Overview of ABM Solution for 4G Mobile Networks ...............3

2.2 Measurement Procedure ..........................................................3

2.3 Adapting ABM to Network Configuration and UE Capabilities ...............................................................................5

2.4 Comparison with Traditional ABM: Ramping Up the Data Rate ...................................................................................7

2.5 Comparison with Using FTP for ABM .....................................8

2.6 TWAMP Measurement Protocol ...............................................9

3 Examples .................................................................... 9

4 Future Work .............................................................. 10

5 Conclusions ............................................................. 10

6 References ................................................................ 10

Ascom (2013) Document: NT13-16812, v2.0 1(10)

1 Introduction

This technical paper describes Ascom Network Testings Blixt technology for available bandwidth measurement (ABM). Blixt enables mobile operators to dramatically reduce the cost and time required to test network quality and deploy new capacity.

1.1 Background

Mobile networks are in the process of becoming the worlds leading medium for data traffic. As ever faster data rates are offered by mobile network technologies, the use of real-time applications such as media streaming in such networks is becoming increasingly commonplace.

Now, as is well known, mobile network performance depends crucially on the radio environment, which is subject to very rapid fluctuations. For example, Rayleigh fading conditions change on a millisecond basis, as do scheduling and cross-traffic (such as data from other users). Nonetheless, mobile network operators are expected to be able to maintain uniform bandwidth availability to all customers who are paying for a given service level (or class, or experience). Accomplishing this requires metrics and measurement tools designed specifically for the wireless environment.

As such measurements are performed in live commercial networks with paying subscribers, it is important to prevent the measurements from affecting the subscribers quality of experience. Ascoms patent-pending approach to Available Bandwidth Measurements (ABM), trademarked as Blixt, solves this problem by keeping the level of test and measurement intrusiveness to an absolute minimum. ABM identifies the throughput that can be delivered over the measured wireless link at a given place and at a given point in time.

1.2 Aspects of Present-day Mobile Networks That Must Inform ABM Design

Previous work on ABM has focused mostly on fixed IP networks, resulting in algorithms such as pathChip, TOPP, and SLoPS. These algorithms are designed for routers and bit-pipes whose performance is fairly constant over time, varying only with the amount of cross-traffic.

To date, ABM has been applied only sparingly to wireless communication, and the methods traditionally used to measure available bandwidth in wireless networks have been comparatively simple and have involved the downloading and uploading of files via FTP. While by no means ideal having only limited mechanisms for adapting to changes in the radio environment, for one thing these methods have been sufficient for technologies such as WCDMA Release 99 and older.

However, the latest generations of mobile telecommunications systems, such as LTE and HSPA, have a number of features that render traditional ABM methods inadequate. The most salient of these features are as follows:


In LTE and HSPA, the radio channel is a shared resource between all users in a cell. An FTP file transfer to one user in a cell (for example, the testing device) will significantly affect other users in that cell, as will any other traditional drive test activity.

It is also possible for multiple operators (carriers) to share the same radio access network. This puts requirements on parallel testing, as subscribers of different network operators might, for example, share the radio network but use separate core networks.

High data rates. To pick a typical state-of-the-art configuration, using a Category 3 user equipment (UE) in an optimal, unloaded LTE network with 20 MHz system bandwidth, it is theoretically possible to attain transfer rates of up to 100 Mbit/s. Just filling up such a large channel with data in order to measure the channels true bandwidth can be a challenge; every part of the system, all the way from the server to the FTP client, must be carefully tuned to manage such transfer rates. UE-based performance testing applications, especially, will have problems handling all the data and filling the bit-pipe due to the UEs limited CPU performance, which in turn is constrained chiefly by the performance of the UE battery.

Rich configuration possibilities. An LTE network can employ a large array of different MIMO configurations, and the scheduler used in this technology has very powerful and flexible mechanisms for maximum utilization of the radio path (both uplink and downlink). Traditional ABM techniques do not adapt to such rapid variations in the link capacity.

1.3 Requirements on ABM for 4G Mobile Networks

Taken together, the points in section 1.2 above offer enough good reasons to devise a new method for measuring the available bandwidth: a method specifically designed for state-of-the-art wireless technologies. The essential requirements on such an ABM method can be stated as follows:

1. Network load. To be able to probe the limits of bandwidth availability, the method must be capable of loading the bit-pipe up to the maximum. At the same time, however, it must have low intrusiveness meaning that it must keep down the time-averaged network load as far as possible to minimize interference with regular network users.

2. Fast adaptation in time domain. The method must take into account the properties of a radio link with Rayleigh fading conditions varying on a millisecond time scale.

3. Adaptation to network and user equipment configuration. The method must take into account different MIMO configurations, channel bandwidths, and UE capability categories1.

4. Adaptation to scheduling. The method must take into account the network schedulers mechanisms for maximizing the utilization of the

1 UE capabilities are not taken into account in current TEMS product ABM implementations.


radio path. The network scheduler adapts the resource allocation to traffic patterns, quality of service settings, and load.

2 ABM with TEMS Products

2.1 Overview of ABM Solution for 4G Mobile Networks

We begin with a summary of how Ascom addresses the requirements on ABM posed in the previous chapter. It is worth underlining at the outset that since we are pioneering work in the area, our current solution deals chiefly with fundamentals, and our use of ABM will be refined in future releases of TEMS products.

Data is sent in short, intense bursts (chirps) with pauses in between. The peak load is high enough to reach the networks theoretical maximum, while the average load is kept low. This scheme allows us to sound out the available bandwidth while still making minimum use of network resources, thus avoiding a negative impact on regular network users.

Using short bursts meets the requirement of a high temporal resolution. That is to say: at least once in a while, we can expect optimal radio conditions to prevail throughout a data burst (provided that the network configuration and the devices position permit this in the first place).

The algorithm adapts to network configuration parameters: the amount of data sent is adjusted according to the networks maximum throughput while keeping the level of intrusiveness to a minimum at all times.

The packet train transmissions are designed to make full use of the maximum bandwidth, without the throughput rate being limited by slow-start or low-load scheduling mechanisms.

The whole design is based on a device communicating with a server, where the server reflects the packets back to the device, including timestamps and other information included in the packets. The device can then easily be configured to test the performance of different parts of the network by accessing different servers. For more details of the protocol and device-to-server communication setup, see section 2.6.

2.2 Measurement Procedure

Data bursts are sent at 0.5 second intervals. In between these bursts, nothing is sent.

Each data burst consists of a number of packets sent back-to-back, collectively referred to as a packet train.


Figure 1 ABM data bursts (symbolic representation).

2.2.1 Example: LTE

Suppose we want to measure available bandwidth in an LTE network with 20 MHz bandwidth using a Category 3 device, whose maximum achievable downlink throughput in optimal radio conditions is 100 Mbit/s on the physical layer.

In order to fully load the bit-pipe and be able to attain this maximum throughput rate, we need to transmit 100 kbit in each Transmission Time Interval (TTI), since the TTI length in LTE is 1 ms. For the sake of obtaining a reliable measurement, as further discussed in section 2.2.3, we want to make use of several consecutive TTIs. To be precise, in this case we will send 58 packets each of size 1,500 bytes on the application layer, resulting in about 750,000 bits in total on the physical layer (58 1500 8 = 696,000 bits plus a protocol overhead of about 7%).

Assuming the networks full capacity is available to our ABM-testing UE, the measurement will be finished in just above 8 ms, meaning that the level of intrusiveness (the fraction of time occupied with taking the ABM) is as low as 1.5% if the available bandwidth is measured two times per second ([2 750,000] / 100,000,000 = 1.5%).

The uplink in this configuration has a maximum throughput close to half of the downlink, or 50 Mbit/s. Consequently, when doing ABM on the uplink, using the same packet train, the level of intrusiveness will be about twice as high, but still as low as 3% (see the note on asymmetric load in chapter 4).

2.2.2 Side Benefits: Packet Loss and Delay

As an added bonus of Ascoms approach to measuring ABM, packet loss rate and delay measurements are obtained for free from the packet timestamps and sequence numbering. Making use of the information in the packets, it is also possible to separate the uplink (UE to server) from the downlink (server to UE), so that both uplink and downlink packet loss and trip times can be calculated. By removing the queue delay in the server, we can calculate the effective round-trip time as well.

One packet train time


2.2.3 Accuracy Considerations

The accuracy of the ABM method is determined by the number of packets in the packet train and the packet size, as well as by the instantaneous data rate, which is chosen to correspond to the maximum bandwidth according to current System Information parameters, UE capabilities, and other settings (see section 2.3 below).

To safeguard measurement accuracy, it is necessary to send not just a single packet but a sequence of packets that are contiguous in time. The reason for this is that if only one packet were sent, it would most likely not fill up one TTI, or it would be scheduled across two TTIs, meaning that the full available bandwidth would not be utilized in any TTI. On the other hand, with multiple packets sent back-to-back and scheduled in consecutive TTIs, it is ensured that the ABM service has the networks full available capacity allotted to it at least for some TTIs in the middle of the burst.

Assuming that one TTI can accommodate 100,000 bits, the maximum size of one IP packet is 1,500 bytes (= 12,000 bits). So in this case it takes at least 8.3 packets (100,000 / 12,000) to fill one TTI. It is important to transmit at least a few times this number of packets to ensure that a reasonable number of TTIs are filled with ABM traffic. However, note the trade-off here: the level of intrusiveness of the measuring activity rises in direct proportion to the number of packets sent.

Figure 2 Distribution of one ABM data burst across TTIs. The bandwidth allocated to other users is not represented in this figure; furthermore, optimal radio conditions are assumed. The point illustrated here is that at the beginning and end of the burst, the ABM transmission is not competing for the whole of a TTI.

2.3 Adapting ABM to Network Configuration and UE Capabilities

The amount of data sent in performing ABM must be adapted to the fundamental network capacity (radio access technology). Further improvements can be made to optimize the level of intrusiveness; see chapter 4.

Available bandwidth

One TTI

time

bandwidth


Below are some use cases with their associated ABM setups, designed to achieve a good trade-off between level of intrusiveness and measurement accuracy as discussed in section 2.2.3.

Technology # Packets in Packet

Train

Level of Intrusiveness (Typical) (%)

ABM Maximum Error (%) DL UL

LTE, 20 MHz bandwidth, 2 TBs, Category 3 UE

58 1.5 3.0 4.5

LTE, 10 MHz bandwidth, 2 TBs

58 2.0 4.0 1.7

LTE, 10 MHz bandwidth, 1 TB

58 4.1 4.1 1.7

LTE, 5 MHz bandwidth, 1 TB

58 8.2 8.2 1.8

HSPA, 64-QAM, dual carrier or MIMO with 2 TBs

30 1.8 6.7 5

HSPA, non-MIMO 16-QAM, 15 codes

30 5.5 26 6.2

WCDMA Rel. 99 15 10 60* 0.6

EGPRS 15 8.7 68* 8.7

GPRS 10 17 68* 7

EV-DO 20 2 33* 4.7

CDMA (1x) 20 16.5 16.5* 3.6

Wi-Fi 30 1.4 1.4 2.1

* The level of intrusiveness is inevitably much higher in these cases (and would be high

even if just a single packet were sent) because the data transfer is so slow.

The ABM packet train properties (packet size and interval) are selected to suit the particular radio bearer configuration. Consequently, different ABM setups will typically be used for different networks/operators. Likewise, as a testing session proceeds, the ABM setup will frequently vary over time as the UE moves between cells, or to another carrier, or switches to a different radio access technology (for example, between a 3G WCDMA and a 4G LTE network).

In future Ascom solutions, the plan is to use an even more flexible implementation which continuously adjusts to the level of cross-traffic and to the radio environment, further increasing the measurement accuracy while maintaining the same level of intrusiveness, or in some cases even reducing it. See chapter 4 for more on this topic.


2.4 Comparison with Traditional ABM: Ramping Up the Data Rate

Below we describe one feature of traditional ABM methods that we avoid using in our ABM implementation.

Traditional ABM methods used in fixed-line networks often start out by probing the bit-pipe between the server and the client at a low data rate, then ramp up the data rate until the bottleneck (the maximum bandwidth or data transfer rate) of the bit-pipe is reached. The load is kept at that threshold level for a short time so that the connection is just about overloaded and the available bandwidth is sampled. Finally, the load is released until the next measurement is made (which may be, for example, once every second). When this procedure is iterated and its output filtered, a reliable estimate of the available bandwidth is obtained.

Figure 3 Ramp-up of ABM data rate. The "knee" in the graph is where the inter-packet separation starts to exceed the interval at which the packets were sent; that is, the point where the network can no longer provide the bandwidth requested.


By contrast, in Ascoms implementation, there is no ramping up of the amount of data until the knee is encountered. Rather, the bit pipe is loaded to its maximum just as in an FTP session but for a much shorter time, down to a few milliseconds. In other words, in terms of the diagram above, we always stay to the right of the knee.

2.5 Comparison with Using FTP for ABM

Traditionally, ABM in mobile networks has been conducted by running FTP sessions. Throughput is then typically averaged over one-second intervals and reported once every second at the application layer. There is no way to obtain higher-resolution performance metrics from the application layer; that is, without drilling down into RF data.

Now it is highly unlikely that a one-second throughput average will ever reflect the full available bandwidth, since that would require perfect radio conditions to have prevailed throughout the one-second interval. As the radio environment typically undergoes substantial change on a millisecond time scale, such a scenario is highly improbable.

ABM as implemented by Ascom, by contrast, samples much shorter time intervals (down to 8 ms for LTE, as described in section 2.3) and is therefore able to hit the maximum bandwidth, or somewhere very close to it. For this reason, ABM as implemented by Ascom can be expected to give a more accurate (though also more varying) estimate of the available bandwidth than an FTP-based method.

Figure 4 Comparison of approaches to ABM. The black line curve indicates the true available bandwidth as a function of time. The red bars represent TEMS ABM data bursts. Near-maximum bandwidth is attained for the second ABM data burst. The blue area represents ABM performed by means of an FTP data transfer (1 s segment). The average throughput over one second is substantially below the maximum throughput reached.

There is, in fact, an additional and grave shortcoming to using FTP with currently available UEs: it has proven impossible during LTE network testing to reach bit rates higher than about 60 Mbit/s (one-second average) even in perfect radio conditions and with no other users present. The bottleneck here is the UE processor, whose performance is hampered by the tasks imposed on it by the UE operating system (running applications, background processes, etc.). Since the packet trains used in Ascoms ABM

time

available bandwidth

1 second


approach minimize the load on the UE processor, measuring and reporting on the networks full bandwidth is now possible.

2.6 TWAMP Measurement Protocol

Ascoms ABM technique relies on a time-stamping protocol commonly known as Two-Way Active Measurement Protocol or TWAMP. Other time-stamping protocols could have been used; our reason for selecting TWAMP was that it is a standard protocol in the field which has a simple implementation and is easily extendable. See IETF RFC 5357 for more details.

3 Examples

The picture below shows a live test with two devices, one running Ascom ABM (red line for downlink, green for uplink) and the other one running FTP download (blue line). Both devices are in the same cell, using HSPA with 10 codes. The cross-traffic is unknown. As can be seen, the available bandwidth correlates very well between the two devices.

Remember that Ascom ABM and FTP download measurements are comparable only in static conditions (same RF, resource allocation and cross-traffic). In real-life conditions, ABM has a number of clear advantages; most importantly, it is much less intrusive and therefore much less likely to affect subscriber quality of experience.


4 Future Work

A number of areas can be identified where Ascoms ABM method will undergo further refinement:

To further reduce the impact on the shared radio resources, the ABM procedure should be made adaptive to the rapidly varying radio environment. For example, the algorithm might adapt to the channel rank controlling MIMO usage, or to other channel state information measured by the UE and reported to the network.

For HSPA, just as for LTE, the ABM procedure should adjust to the highly dynamic radio configuration. This is very important in order to lower the level of ABM intrusiveness. At one extreme, an HSPA cell using higher-order modulation (64-QAM), dual carriers, and MIMO over the air interface, can provide up to 82 Mbit/s downlink throughput if the RF environment is good enough. Compare this to a cell where the uplink is HSUPA with 10 ms TTI, offering a maximum throughput of 2 Mbit/s: that is, 40 times less.

ABM should adapt to the asymmetrical performance of the uplink compared to the downlink in certain systems and setups.

5 Conclusions

The current implementation of the Ascom ABM method clearly demonstrates its advantages over existing methods such as FTP download. Ascoms approach to ABM has proven to be a good and flexible solution for current, as well as future, products in the wireless network test and measurement area.

6 References

IETF RFC 5357: http://tools.ietf.org/rfc/rfc5357.txt

a new approach to available bandwidth measurements for wireless networks

Documents