an exact digital network replica for testing, training and

An Exact Digital Network Replica for Testing, Training and Operations of Network-centric Systems

S C A L A B L ESEIGOLONHCET KROWTEN

What is an Exact Digital Replica?

EXata Best of Breed Network Emulator

Three Planes of Emulation

Case Studies in Abstraction

Case Study 1 - Approximating MAC Protocol Functionality in Terms of Delay and Throughput

Case Study 2 - Approximating Routing Protocol Functionality and Mobility in Terms of Routing Tables

Case Study 3 - Approximating Real Background Traffi c in Terms of La-tency and Bottlenecks

Case Study 4 - Cross-Layer Application Performance Based on Network Size and Node Mobility

Summary

Technical Brief

IntroductionNew wireless networks are being designed with a wide variety of goals in mind, such as im-proving worker productivity, delivering mission-critical information to troops in combat, or just enhancing mobile lifestyles. Wireless networks require different capabilities than their wired counterparts, such as specialized middleware, service-oriented architectures, net-centric services, and mobile on-the-go applications. Unfortunately, these technologies haven’t yet reached the level of maturity needed to support the demands of the wireless users of tomor-row.

To ready network infrastructure for next generation wireless, new technologies must go through extensive and costly design, testing, analysis and evaluation. This process is prohibi-tively expensive both in terms of equipment and personnel resources, as well as time invest-ment.

Emulation serves two purposes:

• Reduce the cost of equipment

• Speed the process of testing, evaluation and integration of new technologies, speeding time to market

To greatly streamline and reduce the cost of rolling out new wireless technologies, emulation provides an exact digital replica of a real network or part thereof. Emulations using EXata soft-ware allow for the whole or a part of a real network to be replaced by a counterpart represen-tation in software form. EXata is a faithful representation of the system being replaced, and interfaces seamlessly with the rest of the system, even for the most complex of networks.

This white paper will

i. Describe what constitutes an exact digital replica,

ii. Explain in technical terms the specifi c requirements of next generation wireless networks and how that translates into requirements of emulators, and

iii. Provide four use cases that demonstrate the need for highest fi delity network stand-ins.

www.scalable-networks.com Technical Brief: EXata - an Exact Digital Network Replica, Page 1


What is an Exact Digital Replica?Emulation is the use of a software stand-in for real network components. Until now, there has been no effi cient, scalable, and low cost method for design and analysis of mobile wireless network systems. Emulation provides the same functionality as physical testbeds at a lower cost, while being easier to operate.

There are a number of tools on the market that claim to be network emulators. In this white paper, we will explain why EXata is the only tool capable of acting as an exact digital replica of a network. Strictly speaking, an emulated network shares the following with the real network it replaces:

Same Behavior / Logic

When an emulated and real network exhibit the same behavior or logic, a real router, for instance, can’t discern between emulated network nodes and real ones. To achieve this, the protocol stacks and signal waveforms must be identical. Also, the channel environment, such as pathloss, fading, modulation, etc., must affect the wireless signal in the same way it does the physical environment. Finally, middleware, network-centric services and applications in an emulated environment must be equivalent to the real network systems.

Same Interaction / Language

Same interaction means the response from an emulated network has the correct content, or language, including byte order, packet contents, packet headers, etc. An exact digital replica will allow the real network to interact with it on whatever layer of the protocol stack is called for. In other words, emulators must support cross-layer technologies.

Achieving same interaction means packets and headers are in the correct format (including byte and bitfi eld ordering). Protocol model logic must also be fully represented so that packets are issued identically to those generated by a physical network,.

Another requirement for same interaction is that the emulated network accept packets from, and transmits packets to, the physical hardware so that the physical network cannot discern if it is communicating with a EXata node or another piece of real hardware.

Same Response / Timing

Response time is critical in emulation. In order to respond like a physical network, the emula-tion must process network events no slower than real time. EXata’s discrete event simulation engine is uniquely designed for parallel computing environments to achieve real-time1.

On the other hand, the emulation must process network events no faster than real time. The concept of timing synchronization granularity means that the emulation must represent pack-ets and time at a level fi ne enough to perfectly synchronize with the external world.

EXata Best of Breed Network EmulatorTo achieve the experiential requirements described earlier, an emulation must fulfi ll the fol-lowing technical requirements :

• Highest fi delity models for both the protocol stack and the wireless channel environment

1 For more details on achieving speedup and scalability through parallel computing platforms, download the SNT publication entitled “Parallel Execution White Paper” at http://www.scalable-networks.com/publications/whitepapers.php.


Figure 1 – Emulation Model Architecture. QualNet can support high or highest fi delity models in all 3 spheres simultaneously. Others can support highest fi delity models in only one sphere at a time.

Real Network

Network Layer

Application Layer (real or emulated)

Transport Layer

Routing Layer

Link Layer

MAC Layer

3

1

2

Physical Layer

Physical LayerHigh Fidelity Simulation

Protocol LayerHighest Fidelity Emulation

• Highest fi delity interfaces between the real system components and emulated ones

• Real time constrained execution

Benefi ts Associated with Emulator Technical Requirements

EXata can accommodate highest fi delity in both protocols and interfaces while still operating in real-time Not until now has it been possible to weave and integrate real protocol model code and real applications seamlessly into the emulation environment.

EXata supports the same behavior, interaction, and response. A software or hardware system developed or evaluated in EXata is guaranteed to plug into a real network without a hitch. This is because the code that runs in EXata can also be migrated into the deployment envi-ronment. That means that no additional man-hours are needed to rebuild a prototype system in EXata so that it can run in a real network.

There is a wide range of ways an emulation can interact with the outside world, including:

• Human-in-the loop, where humans watch and manipulate an emulated network,

• Software-in-the-loop, such as with a Semi-Automated Forces (SAF) application such as VR-Forces2 instructing how the emulated network components move,

• Hardware-in-the-loop, such as a prototype radio interacting with a fl eet of perfect digital replicas of themselves in EXata, and

• Application-in-the-loop, where for instance, legacy applications and applications currently under development are evaluated together over a network provided by EXata.

Emulation Model ArchitectureWe’ve described in technical terms what consti-tutes emulation in terms of behavior, interaction, and response. Now let’s examine the architec-ture of the network emulation platform.

Figure 1 breaks down the emulation environ-ment into three spheres. Sphere 1 relates to the traffi c going through the emulation. True emulation accepts real traffi c from real physical networks and transports real packets through-out the emulated network and back out again.

Sphere 2 is the protocol models. Highest fi delity emulation means there is real protocol code

2 VR-Forces is a registered trademark of MÄK Technologies.

running in the emulation environment, OR, the protocol models are high enough fi delity to provide the requisite behavior, interaction and response defi ned in the previous section.

Sphere 3 is the physical layer models. There is no way to perfectly represent the physical layer short of disbursing radios and conducting fi eld tests in the physical world. We’ve already established that this is where physical testbeds become prohibitively costly.

The best approach, then, is to use the most high fi delity simulation models available of the physical layer. EXata has extremely detailed physical models, including propagation models that calculate pathloss, fading, and shadowing in urban, suburban, and rural environments. EXata’s physical layer models also include effects such as weather, mobility, terrain, and radio power.

How Is Emulation Used?

Next let’s paint a picture of ways that emulation is used. Below are some example projects requiring highest fi delity network emulation.

Performance evaluation and analysis of a target network. In the design phase, the applica-tions that will be run on the network are examined, ensuring better predictability of the ap-plication performance.

Interoperability testing. Existing networks and systems can be interfaced with emulations of next generation networks via hardware-in-the loop. Without exact digital replicas of the network, testing means setting up large numbers of devices for detailed testing of the physical testbeds.

To see a breakdown of costs for physical testbed experiments versus a blended emulation-physical experiment, see the SNT white paper entitled “Test and Evaluation of Network-cen-tric Systems”.


Figure 2 – The Three Planes of Emulation. QualNet can support all 3 levels; others can only support emulation on a Network Ef-fects level only.

Transport

Routing

MAC

File Transfer Discovery Web

Location Collaboration

Network Services Plane

Protocol Effects Plane

Network Effects Plane

Protocol AwareHardware-in-the-loopInterfaces

process packets for applications (receive, send, decode and encode)

topology management, routing, medium access control, data rate control, traffic flow shaping, etc.

aggregate performance: delay, throughput, channel (pathloss,

fading, mobility, terrain, weather)


Controlled, repeatable scalability tests. Without the highest fi delity digital replicas of the de-vices, scalability test on prototypes are prohibitively expensive and resource-intensive. Using emulation, the network designers only need a few real devices and interface them with a large number of emulated devices.

Three Planes of EmulationModeling complex wireless networks takes a tremendous amount of computing resources. The bigger and more complex the network, the more diffi cult it is to build an exact digital replica.

When emulating a network, you can look at an network performance on a number of levels.

Network Effects Plane

The Network Effects Plane looks at the network in terms of aggregate performance, such as end-to-end delay and throughput. In this case, the emulated network is a black box whose responsibility is limited to determining latency for each packet transmitted by the applications. This mode is useful for embedding an emulated network between two or more physical hosts running applications. The most common use case of this mode is observing the performance of applications or network services under the diverse and varying network conditions.

The Networks Effects Plane is the only plane through which you can view network perfor-mance with some model abstraction. An abstracted protocol model is a lower fi delity approxi-mation of the true protocol. The fi rst two case studies we present later in this white paper will involve abstracted models vs. their highest fi delity counterparts and how the performance is infl uenced in terms of network effects. The Network Effects Plane supports the Same Behav-ior requirement of emulation.

Protocol Effects Plane

The Protocol Effects Plane looks at network performance based on protocol logic. In the Pro-tocol Effects Plane, network performance is determined by routing, Medium Access Control, data rate control, topology management, and traffi c fl ow shaping.

A typical Protocol Effects Plane emulation scenario involves only a few real prototypes, say, 2 nodes running the physical OLSR routing agent while 100 emulated nodes run the highest fi delity emulated OLSR protocol. The requirement for this mode of emulation is that the two variants of protocols be able to interact with each other seamlessly.

Under a Protocol Effects Plane emulation, the emulated environment can generate traffi c and send it to the physical hosts. Thus, it is possible in this model to have traffi c sessions at any layer in the protocol stack that originate at a physical host and terminate at emulated node or vice-versa.

The typical uses for this approach include testing, validation, and evaluation of a design in a large scale or complex network settings where it is diffi cult to achieve with physical testbeds.

Another benefi t of this model is in designing cross layer interaction mechanisms. As the physi-cal node can have multiple layers of the protocol stack that are simultaneously physical and emulated, it offers the possibility of cross layer interactions across these layers that is not possible in Network Effects Plane emulations. Both these benefi ts are demonstrated in the case studies in the next section.

The Protocol Effects Plane supports the Same Behavior requirement of emulation. It also requires the same protocol functionality and same packet header formats between physical and emulated environments. SNT is the only company to achieve emulation on the Protocol Effects Plane.


Network Services Plane

The Network Services Plane takes emulation to an even higher level. On this plane, the emu-lation processes packets for applications and middleware by receiving, sending, decoding and encoding, all without the rest of the network noticing that emulation is taking place.

EXata runs services or middleware in their native source code within the emulator. The source code of these services can be ported, with very little to no modifi cation, into the emu-lator. This approach provides the design and evaluation environment for middleware services development, and an evaluation environment for applications that utilizes services. In the next section, we demonstrate an example of this mode with emulated HTTP web servers.

The Network Services Plane supports the Same Behavior, Same Response, and Same Interaction requirements of emulation. SNT is the only company to achieve emulation on the Network Services Plane.

Why don’t other emulators support Protocol Effects or Network Services planes? Oth-ers can’t. They don’t have the ability to simulate in real-time while at highest fi delity. EXata achieves this through highly optimized network models that run on parallel architectures3. At the same time, the protocol models in EXata are so accurate that they can actually be replaced for real implementations in some cases.

As an example, using SNT technology, an emulation of a network running twice as slow as real-time on a 2-processor machine was made to be about twice as fast as real-time with 16 processors.

Case StudiesThe following case studies were designed to demonstrate the importance of highest fi delity models in emulation, and how abstracting any single detail of network dynamics can lead to signifi cant loss of accuracy in the prediction of network performance.

The basic elements that infl uence the dynamics of wireless system operations and also dif-ferentiate it from its wired network counterparts are:

1. The broadcast nature of communication and the role of the Medium Access Control (MAC) protocol in coordinating access.

2. Mobility of nodes and the role of routing protocols in dynamic topology creation and maintenance.

3. Background traffi c that competes and interferes with application traffi c and network control traffi c.

4. The dynamic nature of the wireless channel environment.

These are the `building blocks’ of the network dynamics; that is, how a wireless network infl u-ences or shapes an application data stream is a composite effect of the above factors.

In this section we shall explore the role of high fi delity modeling of each of these factors for an accurate and meaningful representation of the network as an emulation.

We will show you two examples on the Network Effects Plane, one on the Network Services, and one that combines Network Services and Protocol Effects Planes.

3 For more details on achieving speedup and scalability through parallel computing platforms, download the SNT publication entitled “Parallel Execution White Paper” at http://www.scalable-networks.com/publications/whitepapers.php.

Case Study 1 - Approximating MAC Protocol Functionality as Delay and Throughput

In this case study, we demonstrate the effect of MAC layer modeling on application perfor-mance. The experimental setup consists of two physical hosts that serve as the two end points for communication, while the intermediate wireless network was emulated on a third computer. One host served as a streaming video server, while the other received the stream.

The Abstract Emulation was identical to the SNT Emulation except for the MAC layer protocol that was stripped out and replaced with static routing, average throughput and average delay throughout the emulated network. The average values were derived from the highest fi delity experiments (SNT Emulation).

Even though this case study is abstracted in the MAC layer model, our main focus for evalu-ation was at the application layer. That way we could observe the effects of application or user-perceived performance of the network.

The network level metrics of average throughput and delay were identical for both SNT Emu-lation and Abstract Emulation, yet we observed a signifi cant difference in performance when viewing video quality. Figure 3 summarizes the fi ndings of this study. In the SNT Emulation, the video at the destination alternated between periods of perfect quality and periods with some glitches in the video. The Abstract Emulation, however, was always corrupted and the extent of corruption varied over the time. This difference in performance can be attributed to how the packet delivery latency was calculated by each emulation.

In the case of the Abstract Emulation, each packet went through the abstract `pipe’ of given throughput and delay independent of other packets. This leads to the consistent poor perfor-mance of the streaming video.


0 10 20 30 40 50 600

50

100

SSIM

(%)

0 10 20 30 40 50 600

50

100

Time(sec)

SSIM

(%)

SNT Emulation

Abstract Emulation

Vide

o Fr

ame

Exac

tnes

s

No Video Corruption

Complete Video Corruption


No Video Corruption

Low Fidelity - Static Routing with

constant throughput and delay

Highest Fidelity - delays and throughputbased on 802.11 MAC

Figure 3 – SNT Emulation (top) shows a higher SSIM (Structural Similarity Index Measurement) vs Abstract Emulation, where the MAC layer protocol logic is replaced with static routing, and a constant throughput and delay. In this case study, abstraction leads to an overly pessimistic estimation of video quality.

Figure 4: Example video Frames with associated Structural Similarity Index Measurement (SSIM). Left: 100% SSIM or No Video Corruption. Right: 50% SSIM or 50% Video Corruption.


Figure 5: Abstract Emulation updates routing tables at discrete points in time and vastly overestimates video quality. Highest Fidelity Emulation updates routing tables the way real routing protocols do. In high-est fi delity emulations, video quality varies greatly based on the routing protocol in use.

0 20 40 600

50

100

SSIM

(%)

Abstract Emulation

0 20 40 600

50

100

Highest Fidelity Emulation: AODV

Time(sec)Time(sec)

SSIM

(%)

0 20 40 600

50

100

Highest Fidelity Emulation: DSR

Time(sec)

SSIM

(%)

0 20 40 600

50

100

Highest Fidelity Emulation: OLSR

Time(sec)

SSIM

(%)

Video Frame Exactness

Video Frame Exactness

No Video Corruption



No Video Corruption

In the case of SNT Emulation, however, the actual 802.11 MAC protocol determined the packet dynamics, and since the emulated network had multiple hops, there was a correlation due to the hidden terminal effect and buffering of packets while the channel was busy. In other words, the 802.11 MAC model logic created short periods of bursty loss of packets, as op-posed to periods of continuous loss.

To summarize this case study, it is best to use the actual MAC protocol and PHY operation or a highest fi delity representation, rather than abstracting them as pipes of specifi ed throughout and delay.

Thus, even for an extremely simple network confi guration, using aggregate values to predict fi ne granularity network results just doesn’t work.

Case Study 2 - Approximating Routing Protocol Functionality and Mobility in Terms of Routing Tables

In this case study, the effect of mobility and routing protocol logic are shown to greatly affect application performance. Mobility in ad-hoc networks can lead to change in connectivity topology, and routing protocols are constantly updating routing tables accordingly.

In Abstract Emulation, the operation of the routing protocol is removed and instead routing tables are updated at specifi ed times in the experiment and periods of transition, when the route discovery is taking place, are ignored.

The network in this case study consisted of three nodes, one being the video server, another the video client, and a third as a potential intermediate node. Only the destination client was mobile. Initially the source and the destination clients were close to each other so that they could communicate over a single hop. During the course of the experiment, the destination node moved too far away from the source and needed a third node to act as an intermediate forwarding node.

Figure 5 summarizes the results. Three of the four graphs are labeled as “Highest Fidelity Emulation” followed by the routing protocol responsible for topology discovery and mainte-nance. The Abstract Emulation, on the other hand, had the routing protocol stripped out and in its place were pre-computed static routes from a highest fi delity emulation. In the case of Abstract Emulation, the video performance was optimal; there is no ef-fect of the route change.

In all cases, the ex-periment started with the routing table showing the source and the des-tination nodes being separated by a single hop path. For the Abstract Emulation, new routing tables were introduced at a few points during the experiment.

Note that the rest of the network confi guration (MAC protocol, IP network protocol, transport

protocol and the wireless channel) were identical for all four trials.

With the Highest Fidelity Emulation by SNT, the routing protocols created video corruption not only during the interval when the route was changing, but also at later points in time. This can be attributed to the robustness and agility (or lack thereof) of a given routing protocol. Every routing protocol requires some time before it can detect the loss of link. During this period the source node continues to send packets to stale and unusable routes, leading to dropping of packets and corruption of video.

After the routing protocol is informed of link breakage, it initiates a route rediscovery pro-cedure, and until it can converge on an alternate stable route, there is possibility for further packet loss. Hence, video corruption was observed during periods after the initial route discovery.

Also seen in the data is the varying performance based on the class of routing protocols. While the reactive routing protocols AODV and DSR displayed similar behavior, the proactive routing protocol OLSR fared poorly in terms of video quality. This can be explained as the OLSR protocol operation sending periodic hello messages to discover topology changes and peri-odic topology control messages to disseminate this information. As a result, it takes a much longer time for the OLSR network to stabilize on the new route, allowing more packets to drop and a lower SSIM measure.

In conclusion, the dynamics of topology caused by mobile nodes can only be properly under-stood if the details of routing protocol agents are fully represented in the network emulation.

Case Study 3 - Approximating Real Background Traffi c in Terms of Latency and Bottlenecks

In this case study, we look at the impact of background traffi c on application performance in a wireless network. Traditionally, for wired networks, background traffi c is viewed as a compet-ing process that reduces the available bandwidth for a route. Therefore, for this case study, the Abstract Emulation represents background traffi c as reduced throughput of the application data stream and/or increased latency.

In actuality, in multiple hop wireless networks, the background traffi c not only competes with the application traffi c to alter its throughput and delay, but also interferes with the network control traffi c, such as the routing messages. In addition, the background traffi c can also induce topology changes when the routing protocols attempt to divert the application traffi c away from regions with excessive external traffi c and interference. If an emulation does not model the routing protocol, these effects and their impact on the applications will not be ac-curately represented.

The emulation scenario is a multi-hop network of 30 nodes distributed uniformly in a terrain of 1000 meters by 1000 meters. A single video stream fl ows between the source and destination, and those nodes are three hops apart.

Figure 6 shows that initially there is no background traffi c in the emulated network; after 15 seconds each node in the network periodically broadcasts a 512 byte packet to generate uni-form background traffi c. The interval of the background broadcast traffi c was varied to control the intensity of the background traffi c load.

As seen in the graphs, the background traffi c had no effect until the load reached 10 packets per second per node. However, as the traffi c load was increased to 20 packets/second, the video quality was affected, and the two different emulations predicted them differently.

In the SNT HiFi Emulation, there is an intermittent period when the quality was poor but it improves eventually. Upon examination of the routing protocol logs, we found that the AODV routing agent had selected another route that was less sensitive to the background traffi c.


Figure 7: Architecture of replicated web server case study

Web ClientWeb Client

Replicated Web Servers

Primary Web Server

Web Browser

OLSR Routing

Agent

G

E

T

i

n

d

e

x

.

h

t

m

l

G

E

T

i

n

d

e

x

.

h

t

m

l

S

e

r

v

e

r

_

s

e

l

e

c

t

.

j

s

1

4

2

3

The Abstract Emulation, however, only reduced the bandwidth of the route currently in use, leading to suboptimal performance at the application layer and completely missing the adapt-ability of AODV.

Again, we highlight with this set of results that modeling the details of the protocol stack in an emulation environment is essential for a meaningful evaluation.

Case Study 4 - Cross-Layer Application Performance Based on Network Size and Node Mobility

In this case study we highlight the benefi ts of SNT highest fi delity Emulation over simulation, abstract emulation and physical testbed based approaches for design and evaluation of mobile wireless based services. The primary focus of this case study is to describe the capabilities of SNT Emulation during the design and development phase of mobile applications.

Figure 7 shows a schematic of the case study. We built a simple application for disseminating information via web-based client-server communication in a mobile ad-hoc network. The application assumes the existence of a web server whose identity (IP address) is known to all other nodes in the network. Any client can connect to this server and request information that will potentially traverse over multiple hops under the infl uence of topology changes due to mobility. To improve the connection latency as well as resiliency, we borrow the concept of replicated servers from wired web services.

The purpose of the application is to allow nodes to locate a replica server that provides the best service in terms of web page download time.


Figure 6: Background traffi c inserted into the network affects throughput, delay and can interfere with network control traffi c and induce topology changes. Top: Highest fi delity emulation shows the network adapting well to the changes. Bottom: Abstract Emu-lation predicts poorer performance by approximating additional background traffi c as reduced throughput and increased latency.

0 10 20 30 40 50 600

50

100

SSIM

(%)

0 10 20 30 40 50 600

50

100

Time(sec)

SSIM

(%)

HiFi Emulation

Abstract Emulation

Vide

o Fr

ame

Exac

tnes

s

No Video Corruption



No Video Corruption

background traffic of20 packet/sec/node injected



This problem has been solved in wired networks, by algorithms based on geography, DNS, etc., as well as manually. However, this problem becomes more complex for MANETs due to the volatility of the network topology and lack of services such as DNS.

Our solution to the problem of fi nding the best server replica in a Mobile ad-hoc network environment is through cross-layer interaction between the application and routing layers. The nodes in this case study are running a link state routing protocol (OLSR) and each node stores the entire network topology.

When a particular node requests a web page, the application layer client consults the routing table at the network layer to fi nd the path quality metric for each of the replicated web servers and selects the best one.

Figure 7 shows the steps in the solution. The process starts with a client initiating a web page request to the main server (step 1). Since the address of this server is already known, there is no need for service discovery at this point. Instead of responding with the requested web page, the server delivers an html page with an embedded script that lists the addresses of the available replicated web servers based on cross-layer interaction (step 2). The embedded script executes at the client host, where it inspects the local routing table (step 3) and fi nally fetches the page from the selected server (step 4). This redirection is completely transparent to the user.

In the emulation trials, the number of nodes was varied from 30 to 100, with varying mobility patterns. The objective was to observe the impact of mobility and the number of replicated servers on the overall performance of the mobile web browsing application. For each evaluation, we compared the cross-layer interaction scheme against a scheme based on random server selection (i.e., no cross layer interaction) as well as a scheme with no replication.

Figure 8 shows the impact of the number of replicated web servers on the average web page load time. In the emulated network, the nodes were uniformly distributed over a terrain of 1000m by 1000m for a 30 node network, 1400m by 1400m for a 60 node network, and 1750m by 1750m for a 100 node network. In other words, the density of the nodes was kept constant for all three scenarios. One node at the edge of the network was selected as the client host. This client requested one web page per minute. The primary and replicated web servers were randomly selected. There is no node mobility in the example.

Emulation results showed the average load time versus the number of replication servers for three different sizes of networks. The 30- and 60-node network showed an improvement in the load time with replicated web servers, though the marginal benefi ts of increasing the number of servers diminished after two servers. The 100-node network, however, did not stabilize for even 5 replicated servers.

Figure 9 shows the impact of mobility on the cross layer application. Using EXata’s random waypoint mobility model, we ran 3 trials with the following input parameters:

(a) 10 m/sec speed with 60-second pauses throughout,

(b) 20 m/sec speed with 60 second pauses throughout, and

(c)10 m/sec speed without pause intervals.


Figure 8: Time to load vs. number of replicated servers for a 30, 60 and 100 node network.

Figure 9: Time to load web page vs. number of replicated servers for varying mobility schemes.

0 1 2 3 4 50.5

1

1.5

Number of Replicated Servers

Tim

e to

load

web

page

(sec

)

10 m/s, 60 sec pause20 m/s, 60 sec pause10 m/s, 0 sec pause

0 1 2 3 4 50.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Number of Replicated Servers

Tim

e to

load

web

page

(sec

) 30 Nodes60 Nodes100 Nodes

Feature Example in case

study

Simulation Physical

Testbed

Abstract

Emulation

SNT

Emulation

Real Applica-tions

web browser

Multiple Instances of Services

http server

Cross-layer Interaction

application and OLSR

Mobility random waypoint

Scale 100 nodes

Figure 10: Capabilities of Highest Fidelity SNT Emulation vs. Simulation, Physical Testbeds and Abstract Emulation.

The data show that the fewer the replication servers, the longer the response time, and the slower the mobility, the longer the load time. This can be explained easily since higher mobility implies greater probability of the source and destination being close to each other, but also greater overhead of routing and latency in route discovery. However, as the number of replicated servers increased, the differences in load time for the various mobility speeds also diminished. As the number of replicated reached 5, the benefi ts of locating a web server close by were outweighed by the routing protocol overhead.

Thus, this case study demonstrated the value of the SNT emulation environment. Highest fi delity emulation is an extremely effi cient tool for the design and testing of complex cross-layer services and applications, as well as the evaluation process for diffi cult-to-realize channel and network confi gurations.

SummaryEXata allows users to be more productive by offering capabilities not possible with simulation or testbeds. Following are some of the most challenging technical requirements for tools to design and develop wireless systems. Figure 10 compares how various methodologies fare with respect to the above requirements:

1. Support for real applications. There must be real components in the design or emulation framework; these could be systems under design or systems too complex to model.

2. Multiple instances of services or servers. New mobile applications will be built over middleware services, such as peer-to-peer overlay, web or ftp servers, discovery protocols, data replication and caching services, etc. A feature of these services is that they run as multiple instances over several, if not all, nodes in the network.

3. Cross-layer interactions. Network technologies are increasingly employing cross-layer interactions. Highest fi delity protocol models must be used to evaluate complex cross-layer interactions.

4. Mobility. Physical testbeds are a poor choice if the implementation needs to be tested or validated for scenarios with mobility in the network, Confi guring, executing, monitoring and repeating mobility in the network using a physical testbed is a diffi cult and costly affair.

5. Large scale networks. All the reasons provided for mobility are even more important if the network is even moderately large.


In conclusion, emulation is a poweful capability and EXata is software that provides this. In the four case studies, we showed how highest fi delity models must be used to build exact digital replicas of networks.

EXata is a better emulator than any other tool on the market and is state-of-the-art. The other emulators can produce erroneous results if they are not highest fi delity throughout. Only EXata has the parallel execution capability to run highest fi delity models in real-time, satisfying the Same Behavior, Same Interaction, Same Response requirements.


References[1] J. Zhou, Z. Ji, and R. Bagrodia, \TWINE: A hybrid emulation testbed for wireless networks

and applications,” in IEEE INFOCOM, 2006.

[2] M. Varshney, Z. Xu , S. Mohan, Y. Yang, and R. Bagrodia, \WHYNET: An Extensible Frame-work for In-Situ Evaluation of Heterogeneous Mobile Wireless Systems,” WiNTECH 2007: The Second ACM International Workshop on Wireless Network Testbeds, Experi-mental evaluation and Characterization

[3] D. Maltz, J. Broch, and D. Johnson, \Experiences Designing and Building a Multi-Hop Wire-less Ad-Hoc Network Testbed,”, CMU Technical Report TR99-116, 1999.

[4] H. Lunndgren H. et al. \A Large-Scale Testbed for Reproducible Ad Hoc Protocol Evalua-tions,” Proceedings of IEEE WCNC 2002.

[5] G. Judd and P. Steenkiste, \Using emulation to understand and improve wireless networks and applications,” Proceedings of NSDI, 2005.

[6] S. Sanghani, T. Brown, S. Bhandare and S. Doshi, \EWANT: The Emulated Wireless Ad Hoc Network Testbed,” Proceedings of IEEE WCNC 2003, 2003 .

[7] “Propsim. propsim c8 wideband multichannel simulator.” [Online]. Available: http://www.propsim.com/

[8] QualNet http://www.scalable-networks.com

[9] Tcpdump/libpcap http://www.tcpdump.org/

[10] “OLSR daemon.” [Online]. Available: http://www.olsr.org/

[11] “Shunra Network Emulator.” http://www.shunra.com/

[12] Libnet http://www.packetfactory.net/libnet/

[13] B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar, “An integrated experimental environment for distributed systems

and networks,” Fifth Symposium on Operating Systems Design and Implementation (OSDI), 2002.

[14] “Netbed.” [Online]. Available: http://www.netbed.org/

[15] J. Bicket, D. Aguayo, S. Biswas, and R. Morris, “Architecture and evaluation of an un-planned 802.11b mesh network,” In IEEE ACM MobiCom, 2005.

[16] J. Kaba and D. Raichle, “Testbed on a Desktop: Strategies and Techniques to Support Multihop MANET Routing Protocol Development,” Proceedings of ACM MobiHoc 2001.

[17] D. R. et al., “Overview of the orbit radio grid testbed for evaluation of next-generation wireless network protocols,” In WCNC, 2005.

[18] P. De, A. Raniwala, S. Sharma, and T. Chiueh, “Mint: A miniaturized network testbed for mobile applications,” In IEEE INFOCOM, 2005.

[19] P. Mahadevan, A. Rodriguez, D. Becker, and A. Vahdat, “Mobinet: A scalable emulation infrastructure for ad hoc and wireless networks,” In MobiSys Workshop on Wireless Traffi c Measurements and Modeling (WiTMeMo), 2005.

[20] P. Zheng and L. M. Ni, “Emwin: Emulating a mobile wireless network using a wired net-


work,” Fifth International Workshop on Wireless Mobile Multimedia (WoWMoM), 2002.

[21] Z. Wang, A,. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.

[22] R. Bagrodia, K. Tang, S. Goldman, and D. Kumar, “An Accurate, Scalable Communication Eects Server for the FCS System of Systems Simulation Environment,” Proceedings of the 2006 Winter Simulation Conference.

About EXataReal-time emulation is the solution for rapidly maturing the robust network infrastructure technology on which mobile network operations will depend. EXata from Scalable Network Technologies is the only commercially-available emulation tool built on a fully parallel kernel, enabling scientists and engineers to test networks of up to thousands of nodes, at real-time speed in virtual space, with the highest fi delity.

To take your fi rst step to realizing the full potential of network-centric systems, please call 310.338.3318 or email [email protected].

About Scalable Network TechnologiesHeadquartered in Los Angeles, California, Scalable Network Technologies is the leader in parallel processing technology for network performance evaluation. The company develops and supports high fi delity evaluation software tools used for predicting the performance of computing and communications networks and network devices.

SNT has created a new category of evaluation tools for today’s sophisticated networks that meets the demand for real-time, real-network performance testing. Widely recognized for its fl agship product, QualNet, the company’s customers include major aerospace and defense contractors, the US Department of Defense, mobile network operators, as well as research agencies and universities.

For more information, please visit:

http://www.exata.com

Scalable Network Technologies, Inc.

6100 Center Drive

Suite 1250

Los Angeles, CA 90045

http://www.scalable-networks.com

Copyright © 2008, Scalable Network Technologies, Inc. All Rights Reserved.


an exact digital network replica for testing, training and

Documents