[ieee milcom 2005 - 2005 ieee military communications conference - atlantic city, nj, usa (17-20...

7
DISRUPTION TOLERANT NETWORKING FOR HETEROGENEOUS AD-HOC NETWORKS Kevin Fallt Intel Research Berkeley, CA ABSTRACT Heterogeneous ad-hoc networks, for both military and scientific applications, may be far removedfrom communi- cations infrastructure such as the Internet. Yet, to be maxi- mally useful, these networks must ultimately be connected to data storage and analysis facilities. Providing connectivity for such networks may involve exotic and unusual methods of data transfer, including acoustic, free space optical, satellite, and data mule forms of network links. Problems of intermittent connectivity due to power scheduling, node fail- ure, and packet losses from unpredictable external factors are frequently encountered. In addition, due to the myriad of assets employed, a high degree of network heterogeneity is encountered. As an approach to dealing with these two major issues, we propose the use of the Disruption Tolerant Networking (DTN) architecture, which provides reliable asynchronous data communication across heterogeneous, failure-prone networks. INTRODUCTION Ad-hoc networks have been used for a number of military and civilian applications including environmental monitor- ing, hazard detection, target tracking and exploration of un- known terrain. Providing a common communication archi- tecture that spans such a wide range of possible applications is difficult due to the considerable heterogeneity among these applications in addition to the strong possibility that communications may be intentionally or unintentionally interrupted. The most common unifying protocol architecture, TCP/IP, is not well-equipped to deal with these issues, yet it remains the obvious first choice for integrating systems based on COTS technologies. Given this tension, we wish to explore ways of overcoming the shortcomings of TCP/IP as a common communication architecture, yet use it in places where it performs efficiently. The problems with TCP/IP arise when it is used with communication technologies characterized by different properties than are typical for Internet-like "always-on" t This document is based, in part, on work performed by Michael Demmer and Melissa Ho while at Intel Research, Berkeley. See refer- ences [2] and [7] for more details. networks. Many of the military ad-hoc networks are of this type- so-called challenged networks [6] that may require approaches other than the basic interoperable packet switch- ing supported by IP. In particular, a common data network architecture for communicating with a large, heterogeneous, forward-deployed force should be able to handle the follow- ing communication issues: lack of infrastructure, scheduled and predicted connectivity, preferential forwarding for spe- cial traffic classes, intermittency due to node or communica- tion link failure, significant heterogeneity among the various subnetworks in terms of architecture and protocols, and security (privacy, authentication, and resistance to attack). In consideration of these issues, we suggest the recently- proposed Delay Tolerant Networking architecture [6] as a basis for the communication fabric used in such net- works. The DTN architecture supports an exceptionally wide variety of underlying communication technologies, multiple (simultaneous) routing paths, hop-by-hop and end- to-end reliability and security, class-of-service scheduling, and tolerance to intended and unintended disconnection. It can be used with the TCP/IP protocols where connec- tivity is good (i.e. wired networks) or can run directly on link-layer (or other) protocols using store-and-forward procedures in difficult and error-prone environments. DTN capabilities may be provided for existing applications by the construction of appropriate proxies, or directly for newly- developed "DTN-aware" applications. Several implementa- tions of DTN presently exist, having been developed as a cooperative effort between academia, government, and industry. NETWORKING CHALLENGES Many of the challenges described so far have been addressed, with various levels of success, by combining the TCP/IP protocols with Performance Enhancing Proxies (PEPs). These devices typically modify the transport layer packet contents and/or timing structure to help compensate for the poor performance induced by small bandwidth or high latency links. If end-to-end contemporaneous con- nectivity is possible between sender and receiver, these approaches may be adequate. However, when no end-to- end path exists these systems encounter problems. They generally depend on the correct operation of IP, which 1 of 7

Upload: k

Post on 16-Apr-2017

213 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

DISRUPTION TOLERANT NETWORKING FOR HETEROGENEOUS AD-HOC NETWORKS

Kevin FalltIntel ResearchBerkeley, CA

ABSTRACT

Heterogeneous ad-hoc networks, for both military andscientific applications, may be far removedfrom communi-cations infrastructure such as the Internet. Yet, to be maxi-mally useful, these networks must ultimately be connected todata storage and analysis facilities. Providing connectivityfor such networks may involve exotic and unusual methodsof data transfer, including acoustic, free space optical,satellite, and data mule forms ofnetwork links. Problems ofintermittent connectivity due topower scheduling, node fail-ure, and packet losses from unpredictable external factorsare frequently encountered. In addition, due to the myriadof assets employed, a high degree of network heterogeneityis encountered. As an approach to dealing with these twomajor issues, we propose the use ofthe Disruption TolerantNetworking (DTN) architecture, which provides reliableasynchronous data communication across heterogeneous,failure-prone networks.

INTRODUCTION

Ad-hoc networks have been used for a number of militaryand civilian applications including environmental monitor-ing, hazard detection, target tracking and exploration of un-known terrain. Providing a common communication archi-tecture that spans such a wide range of possible applicationsis difficult due to the considerable heterogeneity amongthese applications in addition to the strong possibility thatcommunications may be intentionally or unintentionallyinterrupted.The most common unifying protocol architecture,

TCP/IP, is not well-equipped to deal with these issues, yetit remains the obvious first choice for integrating systemsbased on COTS technologies. Given this tension, we wish toexplore ways of overcoming the shortcomings of TCP/IP asa common communication architecture, yet use it in placeswhere it performs efficiently.

The problems with TCP/IP arise when it is usedwith communication technologies characterized by differentproperties than are typical for Internet-like "always-on"

t This document is based, in part, on work performed by MichaelDemmer and Melissa Ho while at Intel Research, Berkeley. See refer-ences [2] and [7] for more details.

networks. Many of the military ad-hoc networks are of thistype- so-called challenged networks [6] that may requireapproaches other than the basic interoperable packet switch-ing supported by IP. In particular, a common data networkarchitecture for communicating with a large, heterogeneous,forward-deployed force should be able to handle the follow-ing communication issues: lack of infrastructure, scheduledand predicted connectivity, preferential forwarding for spe-cial traffic classes, intermittency due to node or communica-tion link failure, significant heterogeneity among the varioussubnetworks in terms of architecture and protocols, andsecurity (privacy, authentication, and resistance to attack).

In consideration of these issues, we suggest the recently-proposed Delay Tolerant Networking architecture [6] asa basis for the communication fabric used in such net-works. The DTN architecture supports an exceptionallywide variety of underlying communication technologies,multiple (simultaneous) routing paths, hop-by-hop and end-to-end reliability and security, class-of-service scheduling,and tolerance to intended and unintended disconnection.It can be used with the TCP/IP protocols where connec-tivity is good (i.e. wired networks) or can run directlyon link-layer (or other) protocols using store-and-forwardprocedures in difficult and error-prone environments. DTNcapabilities may be provided for existing applications by theconstruction of appropriate proxies, or directly for newly-developed "DTN-aware" applications. Several implementa-tions of DTN presently exist, having been developed asa cooperative effort between academia, government, andindustry.

NETWORKING CHALLENGES

Many of the challenges described so far have beenaddressed, with various levels of success, by combiningthe TCP/IP protocols with Performance Enhancing Proxies(PEPs). These devices typically modify the transport layerpacket contents and/or timing structure to help compensatefor the poor performance induced by small bandwidth orhigh latency links. If end-to-end contemporaneous con-nectivity is possible between sender and receiver, theseapproaches may be adequate. However, when no end-to-end path exists these systems encounter problems. Theygenerally depend on the correct operation of IP, which

1 of 7

Page 2: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

was designed for supporting non-mobile "always-on" typesof infrastructure where the probability of packet loss isrelatively low (at least for TCP). The types of mobilead-hoc networks of present interest are clearly not fixed,and are certainly not "always-on." We therefore focus onthe following particular problems in more detail to betterappreciate the difficulty in applying the TCP/IP protocolsalone to heterogeneous ad-hoc networks:

interruptions: Scheduled down time, interference, or envi-ronmental hostility may cause the interruption of otherwise-operable communication links. When TCP/IP protocolsobserve link failures, they typically terminate transportlayer connections and may delete routing state that takessignificant time and network activity to restore.

heterogeneity: Highly heterogeneous networks consistingof assets procured from a wide range of sources at differenttimes and for different mission requirements may not berunning a common set of protocols, thereby requiring someadditional mechanisms to support interoperable communi-cation. With TCP/IP, address allocation is frequently cen-tralized and all networks must carry entire IP packets whichmay cause an unnecessary amount of overhead (e.g. sensornetworks). With its larger header, IPv6 only exacerbates thisproblem.

reliable delivery of data: is generally accomplished usingerror correcting codes or data retransmission. End-to-endreliability approaches such as TCP/IP utilize end-to-endretransmission, and perform very poorly when the packetloss rate is non-negligible. This problem is especially acutein networks where the loss is not due to congestion or whereend-to-end latency is high (e.g. above 10OOmsec).

security: The preferred methods of asymmetric key distri-bution frequently involve the ability to communicate witha centralized repository of key material (i.e. public keysin a PKI), or to require senders to provide certification oftheir public keys which must be sent across the network asdata. When network connectivity is rare relative to networkisolation, such approaches may not work well. In addition,authentication approaches that allow erroneous data to beforwarded through the network before it is discarded at anendpoint (i.e. IPSec) can be wasteful of network capacityand vulnerable to denial-of-service attacks.

class of service: When a large variety of network linkassets are available, choosing which ones to use to transferwhat data at what time may be a mission-critical issue. Mostconventional TCP/IP based networks are either unable todifferentiate between traffic classes at all, or do so on apacket stream basis. This may present a problem when,for example, large messages should be sent in a fashioncompletely different from smaller messages (e.g. later usinga high-delay message ferrying technique versus now using

a low-bandwidth satellite channel).As an architecture for so-called challenged internets,

DTN incorporates many features that can be of great usein addressing these problems. Some of these mechanisms,such as in-network buffering, have been employed foryears in ad-hoc ways as needed to meet needs of spe-cific networks. We believe it would be better, however, tostandardize on a complete architecture for the problemsof challenged networks, rather than building each featureindependently for each new system. In the next sections,we present an overview of the DTN architecture and a briefdiscussion of some network operational concepts that couldbenefit from adopting it, in whole or in part.

DTN ARCHITECTURE OVERVIEW

The DTN architecture builds upon the abstraction ofreliable asynchronous communication of application-layermessages. It is similar, in spirit, to electronic mail but differsit its approach to routing, reliability, and security. Appli-cations can specify an abstract class of service for eachmessage they send, as well as a deadline which indicatesthe time at which the message should be considered to beworthless (and subject to being deleted inside the network).In support of this abstraction, DTN implementations includecomponents for multi-path routing, hop-by-hop reliabilityand retransmission, a set of convergence layer protocolswhich provide adaptation to underlying transport and linklayer protocols, a naming scheme that can incorporateaddresses defined by other protocols families, and a two-layer security model that supports both hop-by-hop and end-to-end security in frequently-disconnected and disruptedenvironments.

ASYNCHRONOUS MESSAGES

DTN's asynchronous, store-and-forward message deliv-ery model addresses the problems of interruptions andlack of infrastructure. Because DTN-aware applications aregenerally tolerant to delay, data may be stored in a queuefor long periods, persistently if necessary, without adverselyaffecting application operations. This flexibility allows thenetwork to cope with a lack of available connectivity causedby interference, congestion, low-duty power cycling or lackof infrastructure. Figure 1 shows an extreme case in whichan isolated sensor network must transport its data usinga data mule [12] between the sensors and the Internetgateway. In-transit messages (called bundles) within thisnetwork may be in storage for long periods, including thetime that elapses while waiting for the next mule to arrive.

FRAGMENTATIONAND ROUTING

The DTN architecture supports the development of so-phisticated approaches to routing, including the use of

2 of 7

Page 3: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

UiEl~late_= =S

intermeddata stor

7I

| mobile transport

liaterage

bundle protoIrn/rrar 1-icniyenceU iayer convergence 1a

sensor transport -- sensor transl

sensor routing - ..- sensor sta

sensor mac _ .+ Wi

bundletransfer

bundletime storage

connectivityoertime

gateway

intermediate intermediatedata storage data storage

.-...-...-..-....-.-..-.--

col6 bundle protocolayer convergence layerport sensor TCPatic static P/iFi WiFi satellite s

Internet

IPatellite |...data collected

data sent

bundle relay

bunde delivery

Fig. 1. DTN transmission of data bundles from a sensor node (left)to a destination application on the Internet (right) through a collectionof heterogeneous underlying protocols. With no contemporaneous end-to-end path between the sensors and the Internet, the DTN bundleprotocol must store the bundles until the next available connectionarrives, allowing eventual delivery of the sensor data to its destinationas indicated by the time graph in the bottom of the picture.

multiple simultaneous paths and scheduled connectivity.Using store-and-forward, DTN routing computations gener-ally take place over a time-evolving graph where a sourceand sink may never have a contemporaneous end-to-endpath available. Generally, long messages are split into parts(called fragments, see below) to achieve high bandwidth uti-lization. While optimal joint routing/scheduling algorithmscan be employed, they are not practical for networks ofany significant size. Rather, suboptimal solutions based onenhancements of common routing algorithms (e.g. Dijkstra)can offer reasonable performance if schedules are known inadvance [8].

The architecture supports message splitting in the formof proactive and reactive fragmentation. In proactive frag-mentation, if communication is scheduled, queued messagesare split into appropriate-sized segments ahead of timeand when a communication opportunity becomes available,exactly the correct quantity of data is transferred. Whenunexpected failures are encountered, DTN employs reactivefragmentation, which essentially amounts to re-packagingdata received across a link so that it may be deliveredas an independent fragment. Fragments are eventually re-assembled by the final receiver in both cases. The com-bination of these features, in conjunction with multi-pathrouting, provides better utilization of scarce communicationresources and robustness against unexpected interruption.

CONVERGENCE LAYERS

Computer networking has evolved from isolated net-works running various protocols (some proprietary) to a

largely interoperable set of networks based on TCP/IP.As mentioned, however, TCP/IP is not always ideal forchallenged networks. For these networks, including mo-bile store-and-forward routers (data mules), satellite linkslike INMARSAT or Iridium, and recently-developed sensornetwork protocols [5], it may be advantageous to carrydata directly on the link-layer protocols optimized for theparticular communication media. 1

To make this possible in DTN, two mechanisms areused: a naming scheme (see below) capable of embeddingthe names/addresses used by other protocol families intoits own names, and a convergence layer abstraction thatprovides a standardized way to adapt existing protocols tobe used as underlay protocols for hop-by-hop asynchronousDTN message delivery. Convergence layers are essentiallya structured framework for constructing application-layergateways between different protocols or protocol instances.

The convergence layer abstraction helps to solve theproblem of adapting different underlying protocols (trans-port or otherwise) to be useful for supporting the DTNdelivery protocol (called bundle delivery). A convergencelayer for a specific protocol adds functionality to supportdelivery ofDTN messages over the corresponding transportlayer (e.g. augmenting TCP with message boundaries). Con-vergence layers also handle the signaling for fragmentationand connection re-establishment, if appropriate. Overall, aDTN router acts as a form of proxy among differing net-work types and is agnostic regarding the protocol layeringemployed in the networks it interconnects.

NAMING

DTN's flexible approach to naming uses variable-lengthendpoint identifiers based on Uniform Resource Identifiers(URIs) [11]. URI's include two primary parts: a schemename which identifies a namespace, and a scheme-specificpart, whose syntax and semantics are defined specificallyby the scheme. Both are variable-length, allowing for exten-sibility in the scheme namespace, as well as the ability toembed names or addresses defined by other protocol fam-ilies in the scheme-specific part. DTN routing agents mayprovide routing for more than one endpoint identifier. Forexample, a laptop computer may have identifiers for eachof its types of connectivity: 802.11, BlueTooth, InfraRed,and Ethernet in addition to some abstract endpoint (e.g."guid://80c243 6A9").

If an instance of a DTN is created using only abstractnames, it is often possible to route messages some dis-tance toward their destination without resolving them to

1Sensor networks have been important for the military for years,although the move toward joint network-centric operations has servedto accelerate the need for interoperability of such systems.

3 of 7

sensors

Page 4: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

underlying network protocol addresses. Thus, name-basedrouting may be used while forwarding a message until it isnecessary to resolve the name to an address. This providesa late binding capability, which allows name-to-addressmapping to be executed topologically near the target of acommunication as the message transits between DTN rout-ing agents, as opposed to the early binding performed bythe DNS operation typically executed by today's Internet-style applications. In such cases, the scheme name and someportion of the scheme-specific part provides the minimumamount of information required to make an initial routingdecision. This can help to improve operational performanceby reducing the number of round-trip exchanges requiredto resolve names prior to communication.

RELIABILITY

DTN message delivery takes place in a hop-by-hopfashion among DTN overlay nodes deployed by networkadministrators. It supports DTN-implemented retransmis-sion as a last resort if messages appear to be lost, but itsown retransmission mechanism is really designed only tohandle failure of the underlying transport layer's reliabilitymechanisms. DTN does offer another service to enhancereliability, called custody transfer. It is the transfer ofeventual message delivery responsibility from one node toanother, and represents a form of delegation. The node(s) towhich custody has been transferred (now called custodians)assume responsibility for reliable delivery, and are expectedto be at least as reliable as the originating custodian. Inintermittent networks, this is a formalized notion of the typ-ical scenario in which data is buffered at intermediate hopsbetween the source and destination. When power constraintsor other factors make end-to-end retransmission impossible,custody transfer can shift the data source progressivelycloser to the destination, while also ensuring that data is notlost in transit. In most cases, it also handles the situationwhen the original sending node catastrophically fails (e.g.is destroyed).

SECURITY

The DTN approach to security suggests a two-levelscheme for providing both hop-by-hop authentication atthe DTN layer, and an optional end-to-end set of securityfeatures for the user (authentication, error detection, and pri-vacy, with support for Type 1 modules if required). Becauseof the assumption of delay-tolerant operations, significantprocessing can be employed at intermediate routing nodes,and in particular can be used for enforcing authenticationand access control early in the delivery process.

Providing a hop-by-hop authentication capability isaimed at minimizing the ability of unauthorized users from

E2E ..... . . . . . . ..-.

HOP ........... .._ ...

Fig. 2. Experimental setup showing the end-to-end and hop-by-hop con-figurations. End-to-end experiments (denoted E2E) involve intermediatenodes forwarding IP datagrams through their operating system kernels.Hop-by-hop experiments (denoted HOP) involve bringing data to userspace and terminating transport connections prior to forwarding.

passing data across the network at the DTN layer. Inparticular, even if an unauthorized user is able to injecttraffic into the network at its edge, the first router configuredto enforce authentication will cause it to be discarded. Thisis in contrast with the Internet's packet delivery architecture,for example, that typically forwards data much (sometimesall) of the way to its destination before access controlchecks are performed and unwanted traffic discarded.

LIFETIMES AND CLASS OF SERVICE

Each DTN message ("bundle") includes a Class of Ser-vice (CoS) designator. The designator is a coarse grainapplication-specified request, and corresponds roughly tothe classes of service available via the US Postal System.The currently-defined service classes are: high priority,medium priority, low priority, and bulk. Routers generallyarrange for high priority messages to be sent as early aspossible, while lower priority messages may await connec-tions of high bandwidth to become available (e.g. even ifthey are of high delay, such as mules).

In addition to the CoS designator, DTN messages carryan origination timestamp and a useful lifetime indicator(expressed as an amount of time after the origination time).Given that DTN nodes are assumed to be roughly time-synchronized, this capability allows for intelligent discard-ing of messages that cannot be delivered before their usefullives expire.

EVALUATION

The primary goals of the DTN effort are to supportcommunications across massively heterogeneous networksand to offer good performance even in the face of disruption(and reasonable performance with no disruption). Whilethe first goal is difficult to evaluate quantitatively, thesecond goal can be measured in several ways. Here, weuse the DTN2 reference implementation [3] and evaluateits operational overhead in benign networks as well asits ability to perform well when network interruptions areprevalent. We ran several experiments on the University ofUtah's Emulab [4] testing facility to compare three messagedelivery protocols: DTN (with the TCP/IP convergence

4 of 7

Page 5: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

Fig. 3. Bandwidth utilization percentage of different protocols sending a

2MB data file. SFTP illustrates the effective performance of TCP/IP. Notethat DTN performs at least as well as the other protocols in almost allcases, and performs almost three times better than its nearest competitorfor 1KB messages in the HOP configuration.

layer), electronic mail (using sendmail/SMTP), and a simplefile transfer program (called SFTP).2 Note that the HOPconfigurations of DTN and sendmail/SMTP make use of a

large persistent storage at each hop, while SFTP/HOP uses

only the system memory allocated to TCP socket buffers.In the first set of experiments, we examine ability of each

protocol to utilize the network bandwidth made availableto it. This experiment is conducted with no disconnections,and we are able to verify that our implementation does notadd significant overhead, achieving about 90% utilizationwith modest message sizes. In the second set of experi-ments, we add disconnection cycles and show that whenusing the DTN architecture, message delivery performancein the presence of disconnection is vastly improved over

the other approaches.

MESSAGE TRANSFER OVERHEAD

In our first set of experiments, we use an uninterruptednetwork to compare DTN with the SMTP and SFTP pro-

tocols, in terms of overhead. We configured a five nodeEmulab network in a linear topology, connected via linksof 100kbps capacity and 5ms delay (approximately a 40msround-trip-time end-to-end). As shown in Figure 2, we ran

each protocols in two configurations. In the first (end-to-end or E2E), we only ran applications at the two endhosts; intermediate hosts simply performed kernel-level IPforwarding. In the second configuration (hop-by-hop or

HOP), all five nodes ran applications, with the intermediatesacting as application-layer proxies. In the case of SFTP,each intermediate host ran a simple user space TCP proxy

("plug proxy") to forward TCP data to the next hop.

2We wrote our own simple file transfer program because the standardFTP protocol proved to be rather intolerant of disconnections and was

challenging to configure to use a connection proxy at intermediate hostsdue to the use of a separate data port.

Figure 3 shows the six protocol configurations whenforwarding 2MB of data split into messages of size 1KB,10KB, 40KB and 100KB. The graph plots the bandwidthutilization percentage (i.e. given the time of the last messageto be delivered, we plot the ratio of the utilized bandwidthversus the total available bandwidth). We see that in theHOP configuration, for almost all of these message sizes,DTN has comparable or better overhead than its competi-tors. In particular, for the moderate message sizes up to40KB, DTN outperforms all the other protocols.

These results indicate that for the protocols dependenton end-to-end exchanges (i.e. all protocols in the E2Econfiguration plus SFTP/HOP), the per-message end-to-endexchanges dominate the cost of the transmission due to the40ms total round trip time. This is true even though all pro-

tocols maintain a single open TCP connection for the entireexperiment. When using store-and-forward in this case, theper-message latency is reduced because only the next hopis involved. This eventually leads to better pipelining andbetter utilization of available network bandwidth. Note thatSMTP is worse than DTN primarily due to its larger numberof round-trip exchanges (SMTP is "chatty"). It also has a

higher per-message header overhead.

Fig. 4. Bandwidth utilization percentage of different protocols withnetwork disruption. Disruption is induced as a periodic up/down intervalof 1 minute on followed by 3 minutes off. Aligned represents a networkwith end-to-end path present (no disruption). Shift represents the link up

start times are shifted by 10 seconds left-to-right. Sequential representsthe case where not more than one link is active at any one time. Randomis like shift, but with a random phase offset for the link up start time.The protocols all perform similarly for the aligned case. For the othercases, the hop-by-hop approaches perform significantly better, with theextremum being the sequential case where the end-to-end protocols can

make no forward progress.

DISRUPTION TOLERANCE

In our second set of experiments, we evaluate the claimedbenefits of using DTN to tolerate communication disrup-tions. For this set of experiments, we modified the previousconfiguration to introduce periodic dis-connectivity in eachof the four links. All disruptions are cyclical, whereby each

5 of 7

100%

00% L N (HOP)r_ E~~~~~~~~~~~VlMAL(HOF)

600OFTP(HOP)400/ U DT (E2E)

m 0 FMAL(E2E)

1 KB 10 KB 40 KB 100 KB

Message Size

100%

U MAX80%

CM ~~~~~~~~DTN(HOP)

60 ElMAIL(HOP)

N..... 0 SFTP (HOP)D 40o DTN(E2E)

OMAIL(E2E)

00

Aligned Shift Sequential Random

Link Pattern

Page 6: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

of the four links are up for one minute, then down forthree. We fixed the message transfer size at 40KB, as ourprevious experiments showed that the protocols performedmost similarly with that size.

Figure 4 shows the bandwidth utilization for four exper-iments in which we varied the phase of the cycles for eachlink. In this case, the maximum link bandwidth is calculatedto take into account the fact that each link is only active forone quarter of the time. Note that we also plot the maximumachievable rate for each of the experiments to reflect the factthat some of the bandwidth is necessarily wasted in someexperiments because a node cannot send data that it has notyet received (i.e. the pipeline must be initially filled).

In the aligned experiment, all four links were up at thesame time. As is shown in the graph, the relative perfor-mance of the protocols matches their relative performancein the fully connected case described above. Note that toachieve these results for the non-DTN protocols, we hadto tune the TCP retry parameters to be more aggressive inestablishing connections. Its performance would not evenbe close without this modification.

In the shift experiment, we moved the start offset phasefor each link progressively forward 10 seconds. As a result,the effective throughput of all the end-to-end protocolsis correspondingly halved. 3 These effects are even moreobvious in the sequential configuration, where each link isup one after another and no end-to-end contemporaneouspath ever exists. Because none of the end-to-end protocolscan establish a connection, their throughput drops to zero.Finally, in the random experiment, the phase of each linkstarts at a random offset. Note that in all these experiments,both DTN and hop-by-hop sendmail achieve notably betterperformance than all the others, with DTN in particular re-maining within 6% of the maximum achievable bandwidthin all cases.

DEPLOYMENT SCENARIOS

The DTN effort has its roots within NASA as an approachto the Interplanetary Internet (which suffers from longdelays and episodic connectivity), and has evolved to en-compass a wide variety of other challenged networks. In thissection we present three scenarios involving such networkswhere we believe DTN could provide substantial benefits.The first two were under development before the DTNarchitecture was specified. The third scenario is largelyhypothetical, and is based on a number of publicly-availablebriefings. It is provided as a vision to stimulate discussionas to the applicability of DTN for such environments. Weare necessarily brief due to space considerations.

3An end-to-end path is available thirty seconds into the experiment,and lasts for thirty seconds.

intermediltedata storaqe

bundlerelay

.....

Packbotwith

microserver

Packhbotfpa3th of trav elr-nKNS 7-.,e eNroua.e

;EM,.6- ,,r

t _/1WW*.

Multi--hopIDTN Forarding

sensors

Fig. 5. Researchers at UCLA use the packbot as a data mule to travelthrough a field of fixed sensors. Sensors far away from the packbot's pathuse multi-hop sensor network routing protocols to deliver data to nodeslocated nearby the mule's path where it is stored until the mule passesin range. The system uses a DTN-style store-and-forward approach,employing message ferrying as an alternative to a more expensive, lowlatency, "always-on" network infrastructure.

ZEBRANET

ZebraNet [9] uses custom tracking collars in a peer-to-peer sensor network to track the mobility patterns of zebrasin Kenya. Each collar collects a zebra's location periodicallyusing GPS and communicates the data in an epidemicfashion to other collars and a mobile base station that isonly available when researchers are in the field. The systemuses store-and-forward asynchronous messaging and payscareful attention to scheduling communications in orderto conserve battery life. Network connectivity betweenthe zebras and the base station (and between the zebrasthemselves) is intermittent and opportunistic.

Zebranet is a natural match for DTN, and it provides aninteresting scenario for development of techniques used tocombat data loss (i.e. replication and erasure coding). Thisidea is explored further in [13].

PACKBOT AS A DATA MULE

Packbot is an unmanned ground vehicle (UGV) devel-oped by iRobot with funding from DARPA. It was usedin Afghanistan by the US Army for carrying a videocamera used in cave exploration. Researchers at UCLAuse the packbot as a data mule for collecting data in theirheterogeneous sensor network [10]. Figure 5 illustrates itcarrying a mobile microserver (embedded Linux system) tocollect data from widely distributed stationary sensors thatare otherwise unreachable due to radio range limitations.

The DTN architecture serves as an appropriate frame-work for this diverse scenario. In an environment whereend-to-end paths are often not available, for example, when

6 of 7

1.11"'

Page 7: [IEEE MILCOM 2005 - 2005 IEEE Military Communications Conference - Atlantic City, NJ, USA (17-20 Oct. 2005)] MILCOM 2005 - 2005 IEEE Military Communications Conference - Disruption

the packbot is not currently present, the DTN architecturesupports buffer management in addition to multi-hop de-livery of data. DTN bundles travel as far as the path isavailable, and are buffered until the data mule arrives. Ifthe path and schedule of the packbot is known in advancewith some degree of precision, even more efficient routingcan be achieved [8].

US NAVY FORCENET

ForceNet is the US Navy's "operational construct and ar-chitectural framework for Naval Warfare in the InformationAge..." [1]. It is designed to support Navy and Marine Corpstransformation to network-centric operations. We considertwo example ForceNet scenarios of an unmanned under-water vehicle (UUV) operating in both friendly (local) andhostile waters. In the former case, a set of undersea sensorsand surface expressions provide relaying infrastructure forthe UUV, while in the second scenario, no infrastructure isassumed to be present other than the submarine (or surfaceship) launch platform.Mapping and oceanographic exploration of local terrain

are primary activities of UUVs in friendly waters. In suchcases, acoustic communications and navigation infrastruc-ture may be used while underwater and satellite RF com-munication may be used for reachback, if UUVs remain inpre-planned areas. However, their greatest utility is arguablyin cases where response to unusual events is required (e.g.bringing high-performance imagers or sonars in proximityto an event), and infrastructure at those particular locationsmay be limited. Such applications will require the UUV toact as a data mule, ferrying data back to its launch platformor other assets capable of returning the data back to shore.In making such communications, issues of intermittentcommunication in the water and heterogeneity in the airbecome relevant. For UUVs deployed in potentially hostilewaters, covert operation may be required, and the number ofcommunication assets to which the UUV may communicatemay be limited. If the area to be explored is unknown, large,and hostile, the corresponding volume of sensor data maybe significant.

For both scenarios, the DTN architecture provides somesupport. Data ferrying in the UUV is supported directlyby DTN scheduled routing, while opportunistic commu-nications with surface expressions or undersea relays thatmay be subject to interruption can be enhanced withproactive and reactive fragmentation. Further, the networkheterogeneity in these scenarios may be handled by theDTN approach to naming and addressing. While manydetails of these scenarios have been elided here, they bothshare several common elements: data ferrying, intermittentconnectivity, and possible non-existence of end-to-end data

paths. In such environments, end-to-end protocols such asTCP/IP will not work well, and alternatives such as DTNmust be considered.

CONCLUSION

The Disruption Tolerant Networking (DTN) architectureis based on previous work supported by NASA and DARPAfocusing on communicating with spacecraft deployed indeep space. Since then, DTN has evolved to encompass awider variety of networks and its development continues un-der the auspices of the Internet Research Task Force (IRTF)and DARPA's Disruption Tolerant Networking Program.

In this paper, we make the case that for challengednetworks, an architecture such as DTN is important, asTCP/IP alone is unlikely to meet basic performance require-ments. We have provided an overview of the main pointsof the DTN architecture, and in measuring our referenceimplementation, we find that its overhead relative to TCP/IPis small, and that it excels in handling communications innetworks where end-to-end contemporaneous connectivityis not available. We also provide a brief description of somenetwork operational scenarios that could potentially benefitfrom using some of its techniques, in hopes of initiating adialog with respect to its applicability.

REFERENCES

[1] CNO's Strategic Study Group. Xxi definition. July 2002.[2] M. Demmer, E. Brewer, K. Fall, S. Jain, M. Ho, and R. Patra.

Implementing Delay Tolerant Networking. Technical Report IRB-TR-04-020, Intel Research Berkeley, Dec. 2004.

[3] M. Demmer and DTNRG. DTN Reference Implementation -Version 2, Dec. 2004. Available at http:Hwww.dtnrg.org.

[4] E. N. Emulation. http: //www. emulab. net.[5] D. Estrin et al. Embedded, Everywhere: A Research Agenda for

Networked Systems of Embedded Computers. National AcedemyPress, Washington, DC, USA, 2001.

[6] K. Fall. A Delay-Tolerant Network Architecture for ChallengedInternets. In Proc. SIGCOMM 2003, Aug. 2003.

[7] M. Ho and K. Fall. Poster: delay tolerant networking for sensornetworks. In Proceedings ofIEEE Secon, Oct. 2004.

[8] S. Jain, K. Fall, and R. Patra. Routing in a Delay Tolerant Network.In Proc. SIGCOMM 2004, Aug. 2004.

[9] P. Juang, H. Oki, Y Wang, et al. Energy-Efficient Computing forWildlife Tracking: Design Tradeoffs and Early Experiences withZebraNet. In Proceedings ofASPLOS-X, Oct. 2002.

[10] A. Kansal, A. A. Somasundara, D. D. Jea, M. B. Strivastava, andD. Estrin. Intelligent fluid infrastructure for embedded networks.In Proceedings ofACM MobiSys, June 2004.

[11] T. Lee, R. Fielding, and L. Masinter. Uniform Resource Identifier(URI): Generic Syntax. In Internet RFC 3986, Jan. 2005.

[12] R. C. Shah, S. Roy, S. Jain, and W. Brunette. Data MULEs:Modeling a Three-tier Architecture for Sparse Sensor Networks. InProceedings of the IEEE Workshop on Sensor Network Protocolsand Applications, May 2003.

[13] M. M. Yong Wang, Sushant Jain and K. Fall. Erasure coding basedrouting in opportunistic networks. In ACM SIGCOMM Workshopon Delay Tolerant Networking, August 2005.

7 of 7