Software Architecture-Based Development Support for Ubiquitous Systems

Computer Science DepartmentUniversity of Southern California

Los Angeles, CA 90089-0781 U.S.A.{marija,neno}@usc.edu

Abstract. Over the past several decades software researchers and practitionershave proposed various approaches, techniques, and tools for developing large-scalesoftware systems. A new set of challenges has arisen with the emergence of inex-pensive, small, heterogeneous, resource-constrained, possibly embedded, highly-distributed, and highly-mobile computing platforms that are permeating virtuallyevery facet of life (i.e., ubiquitous systems). This paper presents an approach tosoftware development for ubiquitous systems whose intent is to address these chal-lenges. Our approach to Prism directly leverages the principles and solutions fromthe emerging body of work on software architectures. We refer to software develop-ment in this setting as programming in the small and many (Prism), in order to dis-tinguish it from the commonly adopted software engineering paradigm ofprogramming in the large (PitL). The centerpiece of our approach is a softwarearchitectural style and its middleware that support the key aspects of Prism applica-tions: self-awareness, distribution, heterogeneity, dynamism, mobility, and gracefuldegradation. Our approach has been successfully applied on a number of ubiquitousapplications. While several issues pertaining to Prism remain areas of future work,our experience to date has been very positive.

1. IntroductionSoftware is ubiquitous. It permeates virtually every facet of life, from commercialenterprises to infants’ toys, from space shuttles to electric shavers. The software sys-tems of today are rapidly growing in size, complexity, amount of distribution, hetero-geneity of constituent components, and numbers of users. We have recently witnesseda rapid increase in the speed and capacity of hardware, a decrease in its cost, the emer-gence of the Internet as a critical worldwide resource, and a proliferation of hand-heldconsumer electronics devices (e.g., mobile phones, PDAs). In turn, this has resulted inan increased demand for software applications, outpacing our ability to produce them,both in terms of their sheer numbers and the sophistication demanded of them. Onecan now envision a number of complex software development scenarios, ranging fromenabling tight integration of large numbers of heterogeneous devices in “smarthomes,” to real-time content delivery onto distributed and heterogeneous platforms insupport of distance education, to fleets of mobile devices used in environment andland-use monitoring, freeway-traffic management, fire fighting, and damage surveysin times of natural disaster. Such scenarios present daunting technical challenges:effective understanding of existing or prospective software configurations employed inthe ubiquitous systems; rapid composability and dynamic reconfigurability of the soft-ware; mobility of hardware, data, and code; scalability to large amounts of data, num-bers of data types, and numbers of devices; and heterogeneity of the softwareexecuting on each device and across devices. Furthermore, ubiquitous system softwareoften must execute in the face of highly constrained resources, characterized by lim-ited power, low network bandwidth and patchy connectivity, slow CPU speed, andlimited memory and persistent storage.

Marija Mikic-Rakic Nenad Medvidovic

In this paper, we present an approach to software development for ubiquitous sys-tems whose intent is to address several of the above challenges. We refer to softwaredevelopment in this setting as PRogramming In the Small and Many (Prism),1 in orderto distinguish it from the commonly adopted software engineering paradigm of Pro-gramming in the Large (PitL) [5]. Our approach to Prism directly leverages the princi-ples and solutions from the emerging body of work on software architectures [22,27].This choice is based on the observation that several aspects of architecture-based soft-ware development (component-based system composition, explicit software connec-tors, architectural styles, early system analysis and simulation, and support fordynamism) make it a particularly good fit for the needs of Prism. Our work inter-weaves a number of ideas in the area of software architectures that we had explored inthe past with several new ideas. In the process, we have developed several novel solu-tions to the Prism challenges, spanning two key areas:

• explicit design idioms comprising an architectural style [27] for applications in thePrism setting, and

• a light-weight, tailorable middleware [6] to support implementation, deployment,and reconfiguration of Prism applications.The key hypothesis of our work is that the architectural style and middleware in

concert provide a sufficiently rich basis for developing applications “in the small andmany” (i.e., for ubiqutious computing). To this end, we have performed analytic eval-uations of the Prism style [17] and extensive benchmark tests of the middleware [20];recently, Prism’s middleware was successfully evaluated by a major industrial organi-zation for use in one of their key embedded systems. We have also applied our workon a number of applications executing on a variety of desktop and mobile computingplatforms. While we do not yet possess sufficient evidence to make the claim that ourapproach addresses all of Prism’s challenges, our initial results are very promising. Wehighlight a number of those results in this paper.

The remainder of the paper is organized as follows. Section 2 presents the back-ground and motivation for the work described in this paper. Section 3 presents anexample application used to illustrate the concepts throughout the paper. Section 4describes the Prism architectural style, while Section 5 describes its middleware.Section 6 illustrates several applications built using techniques described in this paper.The paper concludes with overviews of related and future work.

2. Background and MotivationSoftware architectures provide high-level abstractions for representing the structure,behavior, and key properties of a software system [27]. These abstractions involvedescriptions of the elements from which systems are built, interactions among thoseelements, patterns that guide their composition, and constraints on those patterns [22].In general, a system is defined as a set of components (elements that embody computa-tions and state in a system), connectors (elements that embody interactions), and con-figuration or topology (overall organization of components and connectors).

Software architectures are further characterized by several additional concepts. Asoftware architectural style defines a vocabulary of component and connector typesand a set of constraints on how instances of those types may be combined in a systemor family of systems. When designing a software system, selection of an appropriatearchitectural style becomes a key determinant of the system’s success. Styles alsoinfluence architectural evolution by restricting the possible changes an architect is

1 The word “small” refers to the size of devices on which the software executes, and not to the size or com-plexity of the software itself.

allowed to make. Examples of styles include pipe and filter, blackboard, client-server[27], GenVoca [1], and C2 [28].

To date, many architecture description languages (ADLs) have been developed toaid architecture-based development [19]. ADLs provide formal notations to describesoftware systems and are usually accompanied by various tools for parsing, analysis,simulation, and code generation of the modeled systems. Examples of ADLs includeC2SADEL [18], Darwin [16], Rapide [15], UniCon [27], and Wright [27]. A numberof these ADLs also provide extensive support for modeling behaviors and constraintson the properties of components and connectors, which can be leveraged to ensure theconsistency of an architecture (e.g., by establishing conformance between the servicesof interacting components).

Several aspects of architecture-based software development make it a particularlygood fit for the needs ubiquitous systems, thus directly motivating our choice of anarchitecture-based approach to Prism. These aspects include

• a component-based perspective, which allows developers to focus on coarse-grainsoftware building blocks with well-defined interfaces, thus aiding system under-standability, compositionality, scalability, and evolvability;

• explicit software connectors, which separate computation from interaction in a sys-tem, decouple components, and directly facilitate system evolution, distribution,and heterogeneity;

• architectural styles, which help to codify and exploit effective system configura-tions for specific development situations, aiding design and code reuse; and

• ADLs, which facilitate system understanding, early analysis, and simulation.However, in order to effectively support development of ubiquitous systems, sev-

eral additional capabilities are required. These include support for decentralization,system’s self-awareness, mobility, trust, robustness, and graceful degradation. Further-more, architectural styles provide only design-level guidelines for composing softwaresystems. For these guidelines to be truly useful in a development setting, they must beaccompanied by support for their implementation (i.e., architectural middleware). Tothis end, we have developed an architectural style, Prism, and middleware, Prism-MW,that incorporate the above capabilities, as discussed in Sections 4 and 5 respectively.

3. Example Application To illustrate the concepts introduced in this paper, we use an application for distrib-uted, “on the fly” deployment of personnel, intended to deal with situations such asnatural disasters, military crises, and search-and-rescue efforts. The specific instanceof this application depicted in Figure 1 addresses military Troops Deployment and bat-tle Simulations (TDS). A computer at Headquarters gathers all information from thefield and displays the complete current battlefield status: the locations of friendly andenemy troops, as well as obstacles such as mine fields. The Headquarters computer isnetworked via a secure link to a set of hand-held devices used by officers in the field.The configuration in Figure 1 shows three Commanders and a General; two Com-manders use Palm Pilot Vx devices, while the third uses a Compaq iPAQ; the Generaluses a Palm Pilot VIIx. The Commanders are capable of controlling their own quad-rant of the battlefield. The General can see a summarized view of the entire battlefield(shown) or detailed views of each quadrant. The General can issue direct troopdeployment orders to individual Commanders or request transfers of troops among theCommanders. General can also request for deployment strategy suggestions fromHeadquarters. Finally, the General can issue a “fight” command, resulting in a battlesimulation that incrementally determines the likely winner.

The TDS applicationhas been designed,analyzed, imple-mented, deployed,migrated, and dynam-ically evolved usingthe techniquesdescribed in thispaper. While itsdegrees of scale,complexity, distribu-tion, and mobilitymay not be represen-tative of all ubiqui-tous systemapplications, the TDSapplication providesa sufficiently expres-sive platform forillustrating a numberof Prism concepts.

Specifically, several aspects of TDS embody the concept of multiplicity inherent inPrism (“many”). The application is implemented in four dialects of two programminglanguages: Java JVM, Java KVM, C++, and Embedded Visual C++ (EVC++). Theapplication is deployed on five devices, four of which are mobile. The TDS subsystemon each device can run using an arbitrary number of threads of control. The devices areof three different types (Palm Pilot, iPAQ, PC), running three OSs (PalmOS, Win-dowsCE, and Windows98, respectively); in addition, several analysis components usedin support of code mobility and graceful degradation run on a fourth platform (Sun)and OS (Unix). In the instance of TDS shown in Figures 1 and 2, sixteen softwarecomponents deployed across the five devices interact via fifteen software connectors.

4. The Prism Architectural StyleWe have developed an architectural style that is capable of effectively capturing thecharacteristics of application architectures found in the Prism setting. The style isintended to address the key characteristics of ubiquitous applications by providingnative support for architectural monitoring and analysis, distribution, dynamism,mobility, and graceful degradation. 4.1 Topological RulesAn architecture in the Prism style is modeled as a set of components, connectors, andthe topology into which they are composed. Prism-style components maintain stateand perform computations. The components may not assume a shared address space,but instead interact with other components solely by exchanging messages via theirthree communication ports (named top, bottom, and side). A message consists of aname and a set of typed parameters. A message in the Prism style is either a request fora recipient component to perform an operation, a notification that a sender componenthas performed an operation, or a peer message enabling symmetric communicationbetween components. Request messages are sent through the top ports, notificationsare sent through the bottom ports, and peer messages are sent through the side ports ofcomponents. The distinction between requests and notifications in particular, ensures

Figure 1. TDS applicationdistributed across multipledevices.

HEADQUARTERS

COMMANDER 3

COMMANDER 1 COMMANDER 2

GENERAL

the principle of substrate independence [28] in Prism, which mandates that a compo-nent in an architecture may have no knowledge of or dependencies on componentsbelow it. Connectors in the Prism style mediate the interactions among components bycontrolling the distribution of all messages. A connector in the Prism style may be hor-izontal (supporting the asymmetric request-notification interactions), or peer (support-ing the symmetric peer message interactions).

Due to the high degrees of distribution of ubiquitous applications, the Prism styleplaces special importance on connectors that span device boundaries. Such connectors,called border connectors, enable the interactions of components residing on onedevice with components on other devices. A single border connector may service net-work links to multiple devices (e.g., BottomBorderConnector on the PC in Figure 2).A border connector marshals and unmarshals data, code, and architectural models; dis-patches and receives messages across the network; and monitors the status of the net-work links. It may also perform data compression for efficiency and encryption forsecurity. Finally, it can monitor message traffic on each network link in support ofgraceful degradation as discussed in Section 4.4.

Coupling asymmetricand symmetric inter-actions in a singlestyle demands thattheir roles and rela-tionships be clearlydelineated. This hasresulted in an addi-tional set of Prismstyle rules. Two Prismcomponents may notengage in interactionvia peer messages ifthere exists a verticaltopological relation-ship between them.Allowing such inter-action would violatethe principle of sub-strate independence.For example, DataRe-pository on the PCand G_Scenario-Edi-tor on the Palm-1 in

Figure 2 may not exchange peer messages since one component is above the other; onthe other hand, no vertical topological relationship exists between C_iPAQ_Available-Troops and G_AvailableTroops and they may communicate via peer messages(through the shared peer connector). For a similar reason, the Prism style disallows thepossibility of exchanging messages between a peer and a horizontal connector (whichwould, in effect, convert peer messages into requests/notifications, and vice versa).Additional details on the Prism architectural style can be found in [17].

Figure 2 shows the architectural configuration of the TDS application in the Prismstyle, distributed across five devices. The subarchitecture on the PC device maintains amodel of the system’s overall resources: terrain, personnel, as well as the current andstandard deployment strategies. The StrategyAnalyzer, DeploymentAdvisor, and War-

Figure 2. Architecture of the TDS application, displayed in thePrism deployment environment. Unlabeled circles connectingcomponents across the hand-held devices represent peerconnectors.

Manager components, respectively, (1) analyze the deployments of friendly troopswith respect to enemy troops and obstacles; (2) suggest deployments of friendly troopsbased on their availability and positions of enemy troops and obstacles; and (3) incre-mentally simulate the outcome of the battle based on the current situation in the field.The subarchitecture on the Palm-1 device provides the General’s functionality, whilePalm-2, Palm-3, and iPAQ provide the three Commanders’ functionalities. TheG_AvailableTroops component in General’s subarchitecture is able to make directorders (by sending peer messages through peer connectors) to the Commanders’C_AvailableTroops components to reposition troops across battlefield quadrants. 4.2 Architectural Self-AwarenessIn order to address the need for adapting highly dynamic and mobile ubiquitous appli-cations to changes in their execution context, the Prism style supports architectures attwo levels: application-level and meta-level. The role of components at the Prismmeta-level is to observe and/or facilitate different aspects of the execution, dynamicevolution, mobility, and graceful degradation of application-level components. Meta-level components may be application-independent or application-specific. Applica-tion-level and meta-level components exist side-by-side in Prism (see Figure 3). Meta-level components are aware of application-level components and may initiate interac-tions with them, but not vice versa. The Prism style rules apply to both component cat-egories: meta-level components also engage in connector-mediated, message-basedinteractions with each other (and with application-level components).

In support of this two-level architecture, Prism currently distinguishes among threetypes of messages. ApplicationData messages are used by application-level compo-nents to communicate during execution. The other two message types, Component-Content and ArchitecturalModel, are used by Prism meta-level components.ComponentContent messages contain mobile code and accompanying information(e.g., the location of a migrant component in the destination configuration), whileArchitecturalModel messages carry information needed to perform architecture-levelanalyses [19] of prospective Prism configurations.4.3 Deployment, and Mobility

We have extensively employed special-purpose meta-level components, calledAdmin Components, whose task is to exchange ComponentContent messages andfacilitate the deployment and migration of application components across devices.Another meta-level component is the Continuous Analysis component, which lever-ages ArchitecturalModel messages for analyzing the (partial) architectural models dur-ing the application’s execution, assessing the validity of proposed runtimearchitectural changes, and possibly disallowing the changes. An example of an archi-tecture modeled in Prism’s ADL, called simply Prism-ADL, is given in Figure 4. Anal-ysis of Prism architectures is further discussed in [17].4.4 Support for Graceful DegradationDue to the nature of mobile devices, their network connections are intermittent, withperiods of disconnection. A goal of our work is to maximize the availability of anapplication during disconnection. The technique we have adopted is simple in princi-ple: migration of components to a local host before the disconnection occurs. Weleverage the topological constraints of the Prism style in achieving this goal.

The set of components to be migrated is chosen such that it maximizes the auton-omy of the local subsystem during disconnection, stays within the memory constraintsposed by the device, and can be migrated within the time remaining before disconnec-tion occurs. Component migration may require the ability to synchronize, after the con-nection has been restored, any updates made to the different replicas of a component

during disconnection. In order to simplify the synchronization of component replicasupon reconnection, Prism-ADL attaches a degraded mode indicator to each operationexported by a component, as depicted in Figure 4. The indicator reflects an operation’sdependence on component state: some operations do not depend on component stateand are fully accessible during disconnection (allowed); other operations are delayeduntil the connection is restored; finally, access to yet other operations is disallowed.For example, in the TDS application, operations such as AnalyzeStrategy are allowedto be executed locally during disconnection; we defer operations such as the Deploycommand issued from the General to the Commanders; finally, we disallow operationssuch as Fight (the positions of enemy troops might change during disconnection).

In order to determine the best set of components to be migrated, three key pieces ofinformation are needed:A. Message Frequencies. Our goal is to minimize the message traffic that would gothrough the disconnected link(s). This is accomplished by migrating the componentsresponding to these messages from remote host(s) to the local host. Border Connectorshave the capability to monitor message frequencies on each network link they control. B. Dependency. Dependency of an operation (di) is defined as the number of opera-tions provided by other components in the system needed for this operation’s comple-tion. Migration of components whose operations have low di values (“independentcomponents”) is likely to reduce the message traffic present on the network link, whilemigration of “dependent components” may in fact increase the message traffic alongthe link. As shown in Figure 4, Prism-ADL associates a dependency tag with eachoperation provided by a component, allowing easy calculation of the di values. C. Benefit of Migration. For each candidate component the benefit can be estimatedby comparing the component’s static description (such as that shown in Figure 4) withthe list of messages exchanged across the link. For all operations invoked on the com-ponent as a result of these messages, the benefit (initially zero) is calculated as fol-lows:

Benefit f Benefit + fi * (1 - di) where fi is the frequency of the message and di is the dependency factor of the corre-sponding operation. This formula states that the benefit of migrating “independent”components increases; the benefit remains unchanged in the case of component opera-tions that depend on exactly one external operation; it decreases in all other cases.

Figure 3. Layeredconstruction of anapplication usingthe Prism-MW. Theapplication isdistributed acrossfive devices, eachof which is runningmiddleware. Meta-level componentsare highlighted inthe figure.

In order to select the best setof components for migration,we employ a variant of the 0-1-Knapsack algorithm [3].Namely, given the benefit andrequired memory of N compo-nents, we select a subset of thecomponents that maximizesthe total benefit TB (as thesum of benefits of individualcomponents) and stays withinthe total available memoryTAM on the given device.This algorithm is solvable inO(N*TAM). It assumes thatthe benefits of individualcomponents are mutuallyexclusive, thus becoming anapproximation in the case ofhighly-coupled components.However, the algorithm guar-antees that the actual benefitof the resulting migration setis at least TB: the benefit ofmigrating two or more com-ponents that share a commu-nication link is greater than orequal to the sum of their indi-vidual benefits due to themessage traffic along their(migrated) link. A more exactsolution is obtainable by usingthe minimum k-cut algorithm[4] (where k would representthe number of devices). How-ever, this algorithm runs in

exponential time, which may be computationally too expensive if the number of candi-date components and/or hosts is high, as may be the case in a ubiquitous system.

In addition to the above three parameters, the ability to migrate a set of componentsto a device before disconnection occurs will, of course, also depend upon the networkbandwidth and the time to disconnection (which may be known in the cases of antici-pated disconnection [29], or estimated based on historical data in the cases of unantic-ipated or sudden disconnection [20]).

5. The Prism MiddlewarePrism’s middleware, Prism-MW, comprises an extensible framework of implementa-tion-level classes representing the key elements of the Prism style (e.g., components,connectors, messages) and their characteristics (e.g., a message has a name and a set ofparameters).

The design of Prism-MW is depicted in Figure 5. The classes shown are those ofinterest to the application developer. Multiple components and connectors in an archi-

architecture TDS is {component_types {

component PC_StrategyAnalyzer is extern { C:\spec\PC_StrategyAnalyzer.prism;} ... component C_AvailableTroops is extern { C:\spec\C_AvailableTroops.prism;} ...}} architectural_topology { component_instances { pcStratAnalyzer : PC_StrategyAnalyzer; pcDataRepository : PC_DataRepository; cAvailableTroops: C_AvailableTroops; cAvailableTroops1: C_AvailableTroops;} connector_instances { BottomBorderConn : RegularConn;} peer_connector_instances {

PeerCon : PeerConn;} connections { connector BottomBorderConn {

top pcStratAnalyzer;bottom pcDataRepository;}}

peer_connections{peer_connector PeerCon {

side cAvailableTroops,cAvailableTroops1;}}

}}

component PC_StrategyAnalyzer is {state { map: Map;}invariant {}interface { req ipGetMapInfo : MapInfo(): FILE; prov ipAnalyzeStrategy: AnalyzeStrategy(r:Map); dep: ipGetMapInfo; deg: allowed;}operations { req opGetMapInfo: { let PH: STATE_VARIABLE; post (\result = PH);} prov opAnalyze: { let m: Map; post (map = m);}}map { ipGetMapInfo -> opGetMapInfo(); ipAnalyzeStrategy -> opAnalyze(r-> m);}}

Figure 4. Partial architectural specification of the TDSapplication and the PC_StrategyAnalyzer component inPrism-ADL. The component’s provided service (denotedby the prov keyword) has associated dependency anddegraded mode tags (dep and deg keywords, highlighted)used in our support for graceful degradation.

tecture may run in a single thread of control (Component and Connector classes), orthey may have their own threads (ComponentThread and ConnectorThread). TheArchitecture class records the configuration of its constituent components and connec-tors, and provides meta-level facilities for their addition, removal, replacement, andreconnection, possibly at system runtime. A distributed application is implemented asa set of interacting Architecture objects. Components in an architecture communicateby exchanging Request, Notification, or Peer Messages via the Component class’ssend and handle methods. Messages are routed by Connectors. Finally, IScaffold is aninterface exported by every Brick (Component, Connector, or entire Architecture).IScaffold directly aids architectural awareness by allowing probing of the runtimebehavior of a Brick. To date, Prism-MW has been implemented in Java JVM and KVM[30], C++ and Embedded Visual C++ (EVC++), and Python. Different implementa-tions of the middleware have been ported to four computing platforms, two of whichare mobile: Compaq IPaq, Palm Pilot, Unix, and PC. Each implementation of the mid-dleware is quite small, averaging 1,750 SLOC.

The first step a developer(or tool automatically generat-ing an implementation from anarchitectural description [18])takes is to subclass from theComponent or Component-Thread classes for all compo-nents in the architecture and toimplement the application-spe-cific functionality for them.The next step is to instantiatethe Architecture classes foreach device and define theneeded instances of thus cre-ated components, and of con-nectors selected from thereusable connector library.Finally, attaching component and connector instances into a configuration is achievedby using the weld and peerWeld methods of the Architecture class.

We have extended Prism-MW with several properties that are specifically intendedto support development of ubiquitous software applications. These include efficiency,mobility, dynamic reconfigurability, awareness, scalability, security, distributeddeployment, and graceful degradation. In the remainder of this section we primarilydiscuss the properties that are the direct implementation-level counterparts to the stylecharacteristics discussed in Section 4. An in-depth treatment of all listed properties ofPrism-MW is given in [20].5.1 EfficiencyWe directly exploit Prism’s topological rules to facilitate efficient system implementa-tions. Since ubiquitous applications frequently run on resource-constrained devices,with low amounts of memory (e.g., 256 KB of dynamic heap memory on the PalmPilot) and processing power, we have performed several optimizations on Prism-MW.A large amount of dynamic system memory usage in Prism-MW is a result of theexchange of messages among components and connectors. We minimized the requiredmemory for message passing by replacing message queues at the component level, asrequired by the style, with a single, centralized message queue per each address space(i.e., Architecture object). A pool of shepherd threads handles messages sent by any

Figure 5. Class design view of the Prism middleware.

Brick

Architecture

IArchitecture

Message

Notification PeerRequest

PeerConnectorConnector

ConnectorThread

PeerConnectorThread

IScaffold

IComponent

ComponentThreadIConnector

Component

#top #bottom

ISideConnector

#side

Serializable

BorderConnector

PeerBorderConnector

component in a given address space, as depicted in Figure 6. We further optimizedmemory usage by adopting a fixed-sized, circular array for storing messages (i.e., forthe centralized queue). The concurrency management of the circular array slightlyreduces the speed of processing by applying the producer-consumer algorithm to sup-ply shepherd threads with a constant stream of messages to process.

To process a message,a shepherd thread removesthe message from the headof the queue. For commu-nication that spans addressspaces, the message istransported via a borderconnector to a recipientaddress space, and addedto its message queue. Forlocal communication, theshepherd thread is runthrough the connectorattached to the sendingcomponent; the connectordispatches the message to relevant components using the same thread of execution forprocessing their handle methods (see Figure 6). If a recipient component generates fur-ther messages, they are added to the end of the message queue, and different threadsare used for dispatching those messages to their intended recipients.

The optimizations described above have resulted in very light-weight middlewareimplementations that have shown reasonable performance. For illustration, wedescribe the results from one simple experiment used to measure the size and perfor-mance of the Java middleware implementation. The benchmarking application con-sisted of 50 identical components above and one component below a single connector.The application used a pool of 50 shepherd threads and a queue of 30 messages.100,000 simple (parameter-less) messages were sent by the bottom component to alltop components, resulting in 5,000,000 handled messages. When run on an Intel Pen-tium III 500 MHz processor with 256 MB of RAM running JDK 1.4 beta 2 onMicrosoft Windows 2000, this benchmark yielded the time to complete the messageexchange of 4.7 seconds. Memory usage of the middleware, recorded at the time ofarchitecture initialization, is 1.35 KB. The overhead of a “base” Prism component,supporting message passing but without any application-specific methods or state, is0.8 KB. Memory overhead of creating and sending a single message can be estimatedusing the following formula, obtained empirically:

message_overhead (in KB) = 0.2 + 0.02 * number_of_parameters The formula assumes that the parameters do not contain complex objects, but maycontain simple objects (e.g., Java Integer or String). Therefore, for example, the maxi-mum memory overhead (assuming the message queue is full) induced by using Prism-MW in our benchmark application described above is approximately

1.35 + (51 * 0.8) + (30 * (0.2 +(0.02 * 0))) 48 KB5.2 AwarenessAs discussed in Section 4, we support awareness in Prism via meta-level components.We have extensively used meta-level components, called Admin Components, whosetask is to exchange ComponentContent messages and facilitate distributed deploymentand mobility of application-level components across devices, further discussed in Sec-tions 5.3 and 5.4. The Admin Components are very light weight: at instantiation timethey occupy under 3 KB.

Component BComponent A

Component D

R1 N1 R2 R3 N2

send

Req

uest

R3

Thread Pool

hand

leR

eque

st

R

Connector C1

2

34 5 6

7

Figure 6. Message dispatching in the Prism middleware.Steps (1)-(7) are performed by a single shepherd thread.

≈

Another meta-level component is the Continuous Analysis component, whichleverages ArchitecturalModel messages for analyzing the Prism-ADL descriptionsduring the application’s execution, assessing the validity of proposed runtime architec-tural changes, and possibly disallowing the changes. We have recently implementedseveral Continuous Analysis components that encapsulate different subsets of ourDRADEL environment [18]. Depending on the kind of analysis it has to perform (e.g.,topological constraint checking or component behavioral compliance checking), thesizes of the unoptimized Continuous Analysis components (each of which includes anoff-the-shelf parser) range between 72 KB and 95 KB.

We are currently investigating several other aspects of awareness, including self-optimization and self-adaptation. 5.3 Deployment SupportOur support for deployment directly leverages Prism-MW’s services. We have inte-grated and extended the MS Visio tool to develop Prism-DE, the deployment environ-ment for Prism applications (see Figure 2). Prism-DE contains several toolboxes (leftside of Figure 2). The top toolbox enables an architect to specify a configuration ofhardware devices by dragging their icons onto the canvas and connecting them. Theremaining toolboxes supply the software components and connectors that may beplaced atop the hardware device icons. Once a desired software configuration is cre-ated in Prism-DE, it can be deployed onto the depicted hardware configuration with asimple button click. In order to deploy the desired architecture on a set of target hosts,we assume that a skeleton (meta-level) configuration is preloaded on each host. Theconfiguration consists of the Prism-MW’s Architecture object that contains a BorderConnector and an Admin Component attached to the connector. The skeleton configu-ration is extremely lightweight. For example, in our Java implementation, the skeletonuses under 8 KB of dynamic memory. Since Prism-MW, Architecture object, and Bor-der Connector are also used at the application level, the actual memory overhead ofour basic deployment support (i.e., the Admin Component) is only around 3 KB.

The Admin Component on each device contains a pointer to its Architecture objectand is thus able to effect run time changes to its local subsystem’s architecture: instan-tiation, addition, removal, connection, and disconnection of components and connec-tors. Admin Components are able to send and receive from any device to which theyare connected the meta-level ComponentContent messages through Border Connec-tors. Each Admin Component can request the components that are to be deployed in itssubsystem’s architecture. Finally, each Admin Component has the knowledge of the setof components that are available in the local configuration (via a pointer to the localArchitecture object).

The informationabout a system’scurrent and desiredconfigurations caneither be stored on asingle host (central-ized ownership) oreach subsystem mayhave the knowledgeof its current anddesired configura-tions (distributedownership). In thefirst scenario, the

Figure 7. Partial description of the configuration created byPrism-DE for the TDS application, with component sources anddestinations shown. Source devices are denoted as “none” in thecase of connectors that use the base implementation(asynchronous message broadcast) provided by Prism-MW.

add(DataRepository: source PC): PCadd(DeploymentStrategiesRepository: source PC): PCadd(DataConnector: source none): PCadd(C_IPAQAvailableTroops: source local): iPAQadd(C_IPAQScenarioEditor: source PC): iPAQadd(SaConnector: source none): iPAQweld(DataRepository,DataConnector): PCweld(DeploymentStrategiesRepository,DataConnector): PCweld(C_IPAQAvailableTroops,SaConnector): iPAQpeerWeld(G_AvailableTroops,SideBorderConnector):Palm-1

Admin Component on the host storing the descriptions of configurations will initiatethe deployment on all hosts. In the second scenario, each Admin Component will ini-tiate the deployment on its local host.

In order to support centralized ownership of the application’s architecture, the skel-eton configuration on the central host should also contain a Continuous Analysis com-

Table 1: Deployment Process

Centralized Ownership Distributed Ownership

1. The central Continuous Analysis componentreceives the desired configuration as part of anArchitecturalModel message, analyzes it, andafter ensuring that the configuration is valid,invokes the Admin Component on the local host.

1. Each Continuous Analysis component receivesthe desired configuration of the local subsystem,analyzes it, and, after ensuring that the configu-ration is valid, invokes the Admin Component onthe local host.

2. The Admin Component packages the compo-nents (e.g., .class files in Java) to be deployedinto a byte stream and sends them as a series ofComponentContent messages via its local Bor-der Connector to the target hosts.

2. (a) Each Admin Component issues a series ofrequests to other Admin Components (usingeither request or peer messages) for a set ofcomponents that are to be deployed locally.(b) AdminComponents that can service therequests package the required component(s) intobyte streams and send them as a series of Com-ponentContent messages via their local BorderConnectors to the requesting device.

3. Once received by the Border Connector on each destination device, the ComponentContent mes-sages are forwarded to the Admin Component running locally. The Admin Component reconstitutesthe migrant components from the byte stream contained in each message.

4. Each Admin Component invokes the add, weld, and peerWeld methods on its Architecture object toattach the received components to the appropriate connectors (as specified in the ComponentContentmessage) in its local subsystem.

5. Each Admin Component sends a message toinform the centralized Continuous Analysiscomponent that the deployment has been per-formed successfully. The central ContinuousAnalysis component updates the model of theconfiguration accordingly. Update to the modelis made only after successful deployments.

5. Each Admin Component sends a peer message toinform its local Continuous Analysis component(see Figure 3) that the deployment has been per-formed successfully. The local ContinuousAnalysis component updates the model of thelocal configuration accordingly.

6. Finally, each Admin Component invokes the start methods of all newly deployed components to ini-tiate their execution.

Figure 8. Deployment process usingcentralized ownership

BorderConnector

AdminComponent

3

D

E F

4

6

Central host

AdminComponent

ContinuousAnalysis

BottomBorderConnector

1

TopBorderConnector

AdminComponent

2

3

5

A

C

B6

BorderConnector

AdminComponent

3

D

E F

4

6

BorderConnector

AdminComponent

3

D

E F

4

64

Figure 9. Deployment process usingdistributed ownership

Host 1

AdminComponent

ContinuousAnalysis

PeerBorderConnector

Host 2

PeerBorderConnector

AdminComponent

ContinuousAnalysis

1

1

Host 3

PeerBorderConnector

AdminComponent

ContinuousAnalysis

1

4 4

A

C

B D

E F

G

H

4 5

5 5

3

3

3

6 6

6

2a

2a

2a

2b

2b

2b

ponent. The centralized Continuous Analysis component has the knowledge of thecurrent and desired configurations for all subsystems. Figure 7 shows the partialdescription of the TDS application configuration used by the central Continuous Anal-ysis component. Distributed ownership of the application’s architecture requires thatthe skeleton configuration on each host contain a Continuous Analysis component(attached to the local Border Connector), which is aware of its subsystem configura-tion. Each Continuous Analysis component is capable of analyzing the validity ofarchitectural configurations either by performing the analysis locally or by requestingthe analysis of a given configuration remotely (e.g., on a more capacious host that canperform such analysis). Additionally, the Continuous Analysis component is capableof storing the desired (but not yet deployed) local configuration.

Table 1 and Figures 8 and 9 describe the deployment process in Prism-MW forboth centralized and distributed ownership of the application’s architecture.5.4 Mobility and Dynamic ReconfigurationWe use the same approach for mobility and dynamic reconfiguration as we do fordeployment. For example, if during the application’s execution a desired component-or system-level property is violated (as indicated by a meta-level component), thearchitecture may decide to reconfigure itself in the following manner: 21. If necessary, the migrant component is disconnected from its attached connectors

using the Architecture object’s unweld and peerUnweld methods. 2. The sending Admin Component may unload the migrant component from the local

subsystem using the Architecture object’s remove method, or it may access the com-piled image of the migrant component from a local file.

3. The sending Admin Compo-nent serializes the migrantcomponent (or the DLLcontaining the component,in the case of Prism-MW’sC++ implementation) into abyte stream, and sends it asa ComponentContent mes-sage via its local BorderConnector to the destina-tion devices.

4. Once received by the Bor-der Connectors on the des-tination devices, theComponentContent mes-sage is forwarded to theAdmin Component runningon each device. Each AdminComponent reconstitutesthe migrant componentfrom the byte stream con-tained in the message.

5. Each Admin Componentinvokes the add, weld, and peerWeld methods on its Architecture object to attach the

2 Several implementation-level details of this process are elided for brevity. Also elided are the issues ofensuring application integrity during the dynamic adaptation (see [21]). Application integrity is achievedin our approach by exchanging ArchitecturalModel messages along with ComponentContent messages andleveraging Continuous Analysis meta-level components.

Table 2: Message traffic

Message Processing Component fi di

AnalyzeStrategy Strategy Analyzer 0.06 2

Simulate War Manager 0.16 0

Advise Deployment Advisor 0.18 0

Deploy Strategy Analyzer 0.37 0

Table 3: Candidate componentsStrategyAnalyzer

War Manager

Deployment Advisor

Required memory (KB) 13 9 5

Benefit 0.31 0.16 0.18

Table 4: Resulting migration setsTime to

DisconnectionConnection

SpeedAvailable Memory

Resulting Set

1 s 13 KB/s 50 KB Strategy Analyzer

0.5 s 40 KB/s 15 KB War MangerDeployment Advisor

1s 30 KB/s 40 KBStrategy Analyzer

War ManagerDeployment Advisor

received migrant component to the appropriate connectors (as specified in the Com-ponentContent message) in its local subsystem.

5.5 Graceful DegradationWe have implemented the gracefuldegradation facilities described inSection 4.4 in a Prism-MW meta-levelcomponent called Disconnection Con-troller. Disconnection Controllersreside on each device and collaboratein estimating the best set of migratingcomponents by calculating their bene-fits of migration. We illustrate ourapproach using the TDS application.Let us assume that General’s Palm(recall Figures 1 and 2) is going to getdisconnected within a given period,and that we know how much dynamicmemory remains unused on the Palm.Also, let us assume that the connectionspeed between the Palm and Head-quarters PC is known. The goal is tomaximize the functionality of the application running on General’s Palm until the con-nection is restored. Depending on the time to disconnection, message frequencies, andavailable memory on the device, the set of components selected by DisconnectionController for migration will vary, as shown in Tables 2, 3, and 4.

6. ApplicationsTo date, we have implemented over a dozen applications in the Prism style and on topof Prism-MW, involving traditional desktop platforms, PalmOS- and WindowsCE-compatible devices, digital cameras, and motion sensors. These applications includedistributed digital image processing, map visualization and navigation, location track-ing, and instant messaging for hand-held devices. In this section we briefly illustratethree of these applications.

6.1 Attention System With Multiple CamerasFigure 10 shows the high-level architecture of a system capable of detecting changesin different physical locations using multiple cameras, and possibly taking follow-onactions (e.g., turning on the alarm in the case of a surveillance system). Each Local

Figure 10. Architecture of the attention systemwith multiple cameras. The internal architectureof each Local Attention Module is highlighted.

GlobalAttention Module

Display and ActionComponent

Local AttentionModule 1

Local AttentionModule N

Local AttentionModule 2 ...

ImageAcquisition

GrayscaleConversion

ChangeDetection

RegionLabeling

Saliency MapCreation

InternalRepresentation

Timer

ReportGenerator

Figure 11. Two consecutive images taken by a camera located in a single attention module, andthe resulting feature map.

Attention Module is responsible for observing a given location and reporting anychanges to the Global Attention Module. A Timer triggers Image Acquisition at eachsite. The gathered image is stored in the Internal Representation. Grayscale conver-sion, change detection, and region labeling are performed, and finally, the feature mapis created. The Report Generator is responsible for reporting any significant changesto the Global Attention Module. Figure 11 shows the feature map (image containingthe changes) resulting from two consecutive images taken by a single Local AttentionModule camera. The feature map is sent to the Global Attention Module component,which then decides whether the change is important or should be ignored. The archi-tecture is flexible in that it allows easy addition and/or replacement of Display andAction Components (or, indeed, entire subsystems, such as home security).

6.2 Map Browser Application Figure 12 shows the Prism-style architecture of a distributed map browser system formilitary usage. This application integrates the mapping capabilities of a publicly avail-able internet mapping service, and allows the acquisition and exchange of militarilyimportant data (e.g., locations of enemy soldiers, artillery, and mechanized machin-ery). Map Cache component caches the frequently used maps for faster retrieval, andretrieves the newly requested maps from Tiger Maps. Map Navigation componentserves requests such as zooming in and out and scrolling of the map. Resource Locatorcomponent maintains resources of interest (i.e., military data). Map Cache, Map Navi-gation and Resource Locator components are deployed on Map Host (see Figure 13).Each Mapping Client component renders the received information from a Map Hostonto the handheld display, and issues remote requests in response to the end user input(i.e., requests for maps and militarily important data). Figure 13 shows hardware con-figuration onto which the Map Browser Application was deployed using Prism-DE(recall Section 5.3).6.3 Location Tracking SystemFigure 14 shows the architecture of a system for location tracking of a set of movingobjects in a confined area (e.g. a patient in a nursing home or a forklift in a factory).Moving Objects are RF identification tags, which send the data (using the reflection ofradiation emitted by the Location Sensors) to the Location Sensors that are in range.

Figure 12. The map browserapplication architecture. Thetwo connectors that wereimplemented and added toPrism-MW as a result of thisapplication have been labeled.

Figure 13. The hardware configuration of the Map Browserapplication.

census.gov

Map Hostmap.gif

map.gif

map

intermediary cache

proprietary compression

map retrieval

Tiger Mapshttp://tiger.census.gov

MappingClient 1

MappingClient N...

wireless conn

MapCache

www conn

MapNavigation

ResourceLocator

MappingClient 2

Each Local Location Module keeps track and can display the list of objects that arepresent at its location. Finally, Base Station Control can display locations of all mov-ing objects simultaneously, as shown in the window in Figure 15. This display can beaugmented or replaced simply by manipulating the Base Station Control component.The Base Station Control component itself has no knowledge of the Local LocationModule components (due to Prism’s principle of substrate independence, discussed inSection 4) allowing for addition and removal of location tracking subsystems, even atsystem run-time.

We are currently investigating the possibility of integrating the three applicationsdescribed above (and developed completely independently of one another) into a sin-gle surveillance system.

7. Related Work Our work on Prism has been primarily influenced by four research areas: architecturalstyles, middleware, code mobility, and graceful degradation. Architectural styles werediscussed in Section 2. Below we discuss related approaches in the latter three areas.Middleware. The research and use of middleware can be classified into six distinctgenerations on the basis of the achieved level of component reuse: (1) Module inter-connection languages [5] enabled the reuse of components implemented in a singlePL. (2) Remote procedure calls and platform-neutral data representations (e.g., [2,24])enabled distribution and reuse across PLs. (3) Platform-neutral runtime environmentsand dynamic component loading (e.g., [9,14]) enabled dynamism and reuse acrosscomputing platforms. (4) Domain-specific and GUI frameworks (e.g., [10,23]) enabledreuse across applications. (5) Provision of infrastructure services such as naming,threading, reflection, persistence, and transaction management (e.g., [12,26]) intro-duced the possibility of reuse of architecture-level abstractions. (6) Reuse of architec-ture-level abstractions became an explicit focus of architectural style-basedmiddleware (e.g., [28]). While it exhibits properties of middleware spanning severalgenerations, the Prism middleware is most closely related to the sixth generation.Code Mobility. A detailed overview of existing code mobility techniques is given in[8]. Fuggetta et al. describe three code mobility paradigms: remote evaluation, mobile

Figure 14. High-level architecture of locationtracking system.

Base StationControl

Local LocationModule 1

Local LocationModule N...

LocationSensor 2

LocationSensor 1

Local LocationModule 2

LocationSensor N...

Moving Object1

Moving ObjectK

RF RF

Figure 15. Display generated by the BaseStation Control.

agent, and code-on-demand. Remote evaluation allows the proactive shipping of codeto a remote host in order to be executed. Mobile agents are autonomous objects thatcarry their state and code, and proactively move across the network. In the code-on-demand paradigm, the client owns the resources (e.g., data) needed for the executionof a service, but lacks the functionality needed to perform the service. In this para-digm, the desired component can be retrieved from a remote host, which acts as a coderepository, and then executed on the client. As described in Section 5.4, our work pri-marily supports the code-on-demand technique, where each device is potentially acode repository.

Existing mobile code systems offer two forms of mobility. Strong mobility allowsmigration of both the code and the state of an execution unit to a different computa-tional environment. Weak mobility allows code transfers across different environ-ments; the code may be accompanied by some initialization data, but the executionstate is not migrated. Our approach discussed in Section 5.4 supports both forms ofmobility.Graceful Degradation. Ensuring availability of a system during disconnection hasbeen explored primarily in the domain of file systems. The approach is to make themobile computer more autonomous (i.e., less dependent on the network) by using suchmethods as file caching or prefetching, and lazy writeback. Example systems such asCoda [13], D-NFS [7], and Ficus [11] use optimistic replication for file caching, andreconciliation of replicas to resolve conflicting updates. In optimistic replication,updates can be made concurrently to different file replicas, resulting in multiple ver-sions of a file. To recover from conflicting updates, after-the-fact conflict resolution(i.e., reconciliation) actions are required to recombine multiple versions into one. Con-flict resolution can be automated [25], but it may also require the intervention of the(human) owner of the file. Our approach to disconnected operation is more similar inits nature to FarGo-DA [29], which has recently added support for migrating compo-nents as computational elements, rather than as files, in response to disconnection.However, while FarGo-DA handles only anticipated disconnection, we have devel-oped a more general, risk-based approach, which can be also used in cases of suddendisconnection.

8. Conclusions and Future Work This paper has presented a description and evaluation of an architectural style andarchitecture-based middleware that support development of highly distributed, mobile,and resource-constrained applications that are representative of the ubiquitous comput-ing domain. The style and the middleware in concert provide a set of specialized capa-bilities required for ubiquitous applications (efficiency, distribution, mobility,application context awareness, deployment, graceful degradation, and so on). ThePrism style and Prism-MW have been successfully tested and evaluated on a numberof applications to date. They have also been used as an educational tool in courses onsoftware architectures and embedded systems at USC. Two major industrial organiza-tions, the first developing military ground vehicle systems and the second developingavionics systems, have completed independent preliminary evaluations of the workdescribed in this paper. In addition, the Java implementation of Prism-MW has beensuccessfully stress-tested by the second organization for use in one of their key embed-ded, dynamic, highly distributed avionics systems. Of particular interest to them wasPrism-MW’s “on the wire protocol” implemented in border connectors, which wasdeemed “very efficient and flexible”.

While our experience thus far has been very positive, a number of pertinent ques-tions remain unexplored. Our future work will span issues such as extending context

awareness to include self-reconfiguration (e.g., by observing the message traffic andco-locating the components that communicate frequently), extending the graceful deg-radation support (e.g., by exploiting Prism’s explicit architectural topologies to re-route communication paths during disconnection), and investigating the trade-offbetween message compression to reduce network traffic and the computational over-head the compression introduces.

9. References 1. Batory D., and O'Malley S., The Design and Implementation of Hierarchical Software Systems

with Reusable Components. ACM TOSEM, 1(4), October 1992.2. A. Birrell and B. Nelson. Implementing Remote Procedure Calls. ACM Transactions on

Computer Systems, Feb. 1984.3. T. H. Cormen, et al. Introduction to Algorithms. MIT Press, 2000.4. P. Crescenzi, and V. Kann. A Compendium of NP Optimization Problems.

http://www.nada.kth.se/nada/theory/problemlist.html5. F. DeRemer and H. Kron. Programming-in-the-Large Versus Programming-in-the-Small.

IEEE TSE, June 1976.6. W. Emmerich. Software Engineering and Middleware: A Roadmap. In The Future of Software

Engineering, pp. 119-129, ACM Press 2000. 7. M. E. Fiuczynski and D. Grove. A Programming Methodology for Disconnected Operation.

Technical Report, University of Washington, March 1994.8. A. Fuggetta et al. Understanding Code Mobility. IEEE TSE, May 1998.9. A. Goldberg. Smalltalk-80: The Language. Addison-Wesley, 1989.10. I. F. Haddad. X/Motif Programming. Linux Journal, May 2000.11. J. S. Heidemann et al., Primarily Disconnected Operation: Experiences with Ficus. Second

Workshop on Management of Replicated Data. IEEE, November 1992. 12. R.E. Johnson. Documenting Frameworks as Patterns. OOPSLA’92. Vancouver, Canada, 1992.13. J. J. Kistler and M. Satyanarayanan. Disconnected Operation in the Coda File System. ACM

Transactions on Computer Systems Feb. 1992, Vol. 10, No. 1, pp. 3-25.14. T. Lindholm and F. Yellin. The Java Virtual Machine Specification. 2nd Edition Java Series.

Addison Wesley 1999.15. D. C. Luckham and J. Vera. An Event-Based Architecture Definition Language. IEEE

Transactions on Software Engineering, September 1995.16. J. Magee, et al., Specifying Distributed Software Architectures. ESEC 95. 1995. Sitges, Spain. 17. N. Medvidovic and M. Mikic-Rakic. Software Architectural Support for Distributed, Mobile,

and Resource-Constrained Environments. TR USC-CSE-2002-502.18. N. Medvidovic, et al. A Language and Environment for Architecture-Based Software

Development and Evolution. ICSE’99, Los Angeles, CA, May 1999.19. N. Medvidovic and R. N. Taylor. A Classification and Comparison Framework for Software

Architecture Description Languages. IEEE TSE, vol. 26, no. 1, January 2000. 20. M. Mikic-Rakic and N. Medvidovic. Middleware for Software Architecture-Based

Development in Distributed, Mobile, and Resource-Constrained Environments. TR USC-CSE-2002-508.

21. P. Oreizy, et al. An Architecture-Based Approach to Self-Adaptive Software. IEEE IntelligentSystems and Their Applications, 14(3), May/Jun 1999.

22. D.E. Perry, and A.L. Wolf. Foundations for the Study of Software Architectures. SoftwareEngineering Notes, Oct. 1992.

23. J. Prosise. Programming Windows with MFC. Microsoft Press, 2nd Edition. 1999.24. H.C. Rao. Distributed Application Framework for Large-Scale Distributed Systems. ICDCS-

13, pp. 31-38, 1993. 25. P. Reiher, et al. Resolving File Conflicts in the Ficus File System. USENIX, pp. 183-195.

Boston, MA, USENIX. June, 1994.26. B. Shannon, et al. Java 2 Platform, Enterprise Edition: Platform and Component Specifications.

Addison Wesley 2000. 27. M. Shaw, and D. Garlan. Software Architecture: Perspectives on an Emerging Discipline.

Prentice Hall, 1996.28. R.N. Taylor, N. Medvidovic, et al. A Component- and Message-Based Architectural Style for

GUI Software. IEEE TSE, 22(6), June 1996.29. Y. Weinsberg, I. Ben-Shaul. A Programming Model and System Support for Disconnected-

Aware Applications on Resource-Constrained Devices. To Appear at ICSE’02.30. Sun Microsystems. K Virtual Machine (KVM). http://java.sun.com/products/kvm.