white paper - pc architecture for mobile data services testing

7/24/2019 White Paper - PC Architecture for Mobile Data Services Testing

1/8

PC Architecture ForMobile Data ServicesTesting

White PaperSeptember 2010

SwissQual License AGAllmendweg 8CH-4528 Zuchwil

Switzerland

Internet: http://www.swissqual.comOffice: +41 32 686 65 65Fax: +41 32 686 65 66

Part Number: 10-070-201035-3
http://www.swissqual.com/http://www.swissqual.com/


2/8

Copyright 2000 - 2010 SwissQual AG. All rights reserved.

No part of this publication may be copied, distributed, transmitted, transcribed, stored in a retrieval system,or translated into any human or computer language without the prior written permission of SwissQual AG.

SwissQual has made every effort to ensure that eventual instructions contained in the document areadequate and free of errors and omissions. SwissQual will, if necessary, explain issues which may not be

covered by the documents. SwissQuals liability for any errors in the documents is limited to the correction oferrors and the aforementioned advisory services.

When you refer to a SwissQual technology or product, you must acknowledge the respective text or logotrademark somewhere in your text.

SwissQual, Seven.Five, SQuad, QualiPoc, NetQual, VQuad as well as the following logos areregistered trademarks of SwissQual AG.

Diversity, NQDI, VMon, NiNA, NiNA+, NQView, NQComm, NQTM, QualiWatch-M,QualiWatch-S, NQAgent, NQWeb, QPControl, SystemInspector, Diversity Unattended are

trademarks of SwissQual AG.

SwissQual acknowledges the following trademarks for company names and products:

Adobe, Adobe Acrobat, and Adobe Postscript are trademarks of Adobe Systems Incorporated.

Apple is a trademark of Apple Computer, Inc.

DIMENSION, LATITUDE, and OPTIPLEX are registered trademarks of Dell Inc.

ELEKTROBIT is a registered trademark of Elektrobit Group Plc.

Google is a registered trademark of Google Inc.

Intel, Intel Itanium, Intel Pentium, and Intel Xeon are trademarks or registered trademarks of IntelCorporation.

INTERNET EXPLORER, SMARTPHONE, TABLET are registered trademarks of Microsoft Corporation.

Java is a U.S. trademark of Sun Microsystems, Inc.

Linux is a registered trademark of Linus Torvalds.

Microsoft, Microsoft Windows, Microsoft Windows NT, and Windows Vista are either registeredtrademarks or trademarks of Microsoft Corporation in the United States and/or other countries U.S.

NOKIA is a registered trademark of Nokia Corporation.

Oracle is a registered US trademark of Oracle Corporation, Redwood City, California.

SAMSUNG is a registered trademark of Samsung Corporation.

SIERRA WIRELESS is a registered trademark of Sierra Wireless, Inc.

TRIMBLE is a registered trademark of Trimble Navigation Limited.

U-BLOX is a registered trademark of u-blox Holding AG.

UNIX is a registered trademark of The Open Group.


3/8

PC Architecture For Mobile Data Services Testing White Paper

ii |Contents

Contents

1 Introduction........................................................................................................................1

2

Dial-Up Data Sessions in Microsoft Windows ................................................................2

Establishing a Dial-Up Connection......................................................................................2

Windows Sockets ................................................................................................................2

Routing ................................................................................................................................2

Name Resolution .................................................................................................................3

3 Test System Architectures ...............................................................................................4

Single OS System................................................................................................................4

Distributed OS System ........................................................................................................5


4/8


Chapter 1 Introduction | 1

1 IntroductionThe PC Architecture For Mobile Data Services Testing White Papercompares the SwissQualDiversity drive test measurement solution to competing solutions.

SwissQual was the first company in the world to deliver a mobile network services drive testsystem that was designed to meet the needs of 3G high speed data measurements. This systembecame an instant success among operators around the world.

Over the years since that time, SwissQual measurement systems have produced reliable andaccurate results and have been highly accepted for benchmarking and optimizing mobile andfixed-line networks.

The key rule at SwissQual has been to design the systems to perform measurements and collectdata from a customer experience perspective. To this end, each channel runs independently withdedicated and predictable processing power as well as optimum hard disk I/O performance.

Note: If these performance criteria are not met, the number of channelsin the system will inevitably bias the measurement results for each channel.

To further match the customer experience, SwissQual uses a standard and unmodified MicrosoftWindows distribution, including the Windows IP-based data services, to run the SwissQualDiversity measurement platform.

The Diversity measurement platform is a unique and scalable hardware and software architecturethat supports multiple technologies without performance limitations. Diversity is also notdependent on the number of channels that are actively running measurements. SwissQual isconstantly developing and optimizing Diversity in response to technology advances and changes.

Competing products claim that a single Personal Computer (PC) and Operating System (OS) canbe used as a benchmarking solution for data services. In such a setup, multiple measurementdevices are connected to a single PC and run tests simultaneously.

This White Paper highlights the advantages of the SwissQual Diversity solution with respect to thesingle PC and Operating System solution.


5/8


2 |Chapter 2 Dial-Up Data Sessions in Microsoft Windows

2 Dial-Up Data Sessions inMicrosoft Windows

This chapter describes how Microsoft Windows creates dial-up connections and resolves hostserver names.

Establishing a Dial-Up ConnectionThe dominant method to establish a mobile phone or mobile data modem connection from aWindows PC is to use the dial-up networking interface with mobile networks packet data servicessuch as GPRS and CDMA 1xRTT and subsequent standards.

The mobile device connects to the PC through a serial interface, for example, a USB port: The

device uses the serial IP protocol PPP to set up and establish an IP connection between theWindows TCP/IP stack on the PC and the data access point of the mobile network, most often toprovide access to the Internet. Windows recognizes this physical connection as a networkinterface or network adapter.

Windows SocketsTo provide a Windows application access to the network adapter, Windows Sockets (Winsock)creates an interface with the TCP/IP stack.

If a PC has several mobile devices with dialup connections to the Internet and an application onlyneeds to use one of these devices, the application must bind the socket to force Winsock to use aspecific interface. However, to fully control the flow of the packets, you also need to edit the

routing table for the TCP/IP stack.

Important: You cannot control the flow of packets for Windowsapplications such as Email clients, Web browsers (Internet Explorer,Firefox), and Media players (RealPlayer and QuickTime). These applicationsuse the default interface and the internal binding and name resolutioncannot be changed.

Windows also does not have a policy algorithm to ensure that all sockets receive an equalamount of resources at any given time. Internal priority decisions by the operating system canbias the actual resource allocation to the available sockets.

RoutingThe Windows TCP/IP stack, which is responsible for IP routing, compares the destination addressof a packet to the routing table to determine the appropriate network interface for the packet.

By default, the stack only supports one default route. However, since each new dialup connectionthat you add has a different default route, a mechanism has to be implemented to tell the stackwhich network interface to use for the packets. This mechanism needs to dynamically check if theroutes and destinations are already occupied and to delete entries after use.

The quality and accuracy of the multiple data testing implementation, along with the assurancethat the data test application IP packets only use the intended dialup interface, depends heavilyon the routing mechanism.


6/8


Chapter 2 Dial-Up Data Sessions in Microsoft Windows | 3

Name ResolutionName resolution in Windows is a complex task that uses the default network interface andinvolves protocols and technologies such as NetBIOS, WINS, local files, and DNS as well as localcaches for names that have been resolved.

Note: Windows can only have one default network interface.

Each application uses the default name resolution mechanism as soon as packets need to besent to a destination that uses a hostname instead of an IP address.

A custom mechanism that can handle name resolution on a specific dialup connection is requiredfor an implementation that uses multiple data testing devices. Otherwise Windows assigns thedefault network interface to each connection.


7/8


4 |Chapter 3 Test System Architectures

3 Test System ArchitecturesThis chapter compares the single OS measurement systems with the SwissQual Diversitydistributed OS solution.

Single OS SystemSince the beginning of the GSM era in early nineties, laptop-based test tools have been popularand widespread for optimizing, troubleshooting, and tuning radio networks.

These tools, which incorporate mobile phone devices, are compact and portable. Unlike thededicated signal receivers that were used in analogue mobile network measurements in theprevious decade, the cost of these new measurement units was greatly reduced.

When GSM started to mature and GPRS was launched, an increased demand for the ability to

test service quality led manufacturers to develop support for speech quality and data integritytesting. As a result, some tools started to support multiple devices for benchmarking 2G networks.

This trend continued with the introduction of UMTS, CDMA 1xRTT, HSDPA, and EV-DOtechnologies and remained based on a single PC and operating system.

Such a system is adequate for testing voice services with RF scanners and other devices so longas the PC processing performance remains within acceptable limits. However, the limitations ofthis architecture become apparent during the testing of high speed data services.

As described in the "Dial-Up Data Sessions in Microsoft Windows" chapter, problems can and willarise from routing accuracy, the impossibility to predict priority decisions within the operatingsystem, and the flexibility of assigning host names in test cases.

If the operating system routing table is not edited and a custom name resolution mechanism is not

implemented, the configuration of multiple data tests will be complex and will have the followinglimitations:

Unpredictable IP routing during FTP tests: To avoid this problem, each device thatperforms FTP tests must have a dedicated FTP server IP address. Furthermore, the filesthat are used in the tests must have unique names to avoid conflicts.

Unpredictable IP routing during HTTP tests: To avoid this problem, each device thatperforms HTTP tests must have a dedicated URL. This setup limits the possibilities to usecommon national or ISP reference bandwidth test servers, which are very useful forbenchmark tests.

Tests cannot use Windows applications, such as Internet Explorer, that use the defaultnetwork interface instead of a specific dial-up interface.

A single OS system also runs the risk of not fully supporting the device drivers for multiple mobilephones and data devices. The manufacturers do not guarantee that the drivers work correctly ifmore than one instance of the same driver application runs in parallel on the PC.

In summary, a single OS solution cannot adequately recreate a multi-channel data testingenvironment from a customer viewpoint. The integrity of the test results will be negatively affectedby the system architecture, by the number of active measurement devices, and by the type ofapplications. Furthermore, if the TCP/IP stack is not modified, several configuration limitations willalso restrict the measurement performance.


8/8


Chapter 3 Test System Architectures | 5

Distributed OS SystemTo avoid the problems that arise in a single OS system, SwissQual uses a distributed operatingsystem architecture for the Diversity platform. As outlined in the "Dial-Up Data Sessions inMicrosoft Windows" chapter, this architecture provides the ultimate solution for customer

experienced multi-channel data testing. More specifically, each PC in a Diversity system runs adedicated operating system to control one data device. That is, the system resources of each PC,for example, RAM and CPU, are only available to one test device, which is exactly the way anormal subscriber would use the device.

In this SwissQual solution, Windows cannot route an application to the wrong data connection aseach Diversity PC only has one default network interface. This approach also avoids the devicedriver conflicts as each PC only has one data device.

To prevent interference with the measurement engines and to keep the measurementperformance balanced, the monitoring and controlling software for a Diversity system runs on aseparate laptop or backend server.

As a result, a SwissQual Diversity system does not have any test configuration limitations. All

data channels in the system can connect to the same FTP or HTTP server through the same IPaddress or HTTP URL and can download or upload the same file. The Diversity softwareautomatically assigns the file names to avoid file naming conflicts occur on the server.

In summary, the SwissQual Diversity solution, which uses a distributed OS system, moreaccurately simulates the customer experience for multi-channel data testing as compared to thesingle OS solutions that competing products offer. The integrity of the test results is much morepredictable in a Diversity system as each measuring channel has dedicated and controlled PCresources. Lastly, IP packet routing and name resolution effects do not limit the test configurationas on a single OS system.

white paper - pc architecture for mobile data services testing

Documents