a systems integration framework for process analysis and ...jainra/research_papers/r6.pdf ·...

A Systems Integration Framework for ProcessAnalysis and ImprovementRashmi Jain,* Anithashree Chandrasekaran, and Ozgur Erol

Stevens Institute of Technology, Castle Point on Hudson, Hoboken, NJ 07030SYSTEMS INTEGRATION FRAMEWORK FOR PROCESS ANALYSIS AND IMPROVEMENT

Received 16 July 2007; Revised 7 October 2008; Accepted 5 May 2009, after one or more revisionsPublished online in Wiley InterScience (www.interscience.wiley.com)DOI 10.1002/sys.20148

ABSTRACT

Systems Integration (SI) is an important element of Systems Engineering. It involves the integration ofhardware, software, products, services, processes, and humans. The ever-increasing scale of complexityof systems and its impact on the business requires that we revisit the processes involved in thedevelopment and integration of a system. This paper proposes a Systems Integration Process Model (SIPM)based on a comprehensive lifecycle view of systems integration. As part of the ongoing SI research atStevens Institute of Technology, the authors have developed a Systems Integration Framework (SIF) whichincorporates the relevant aspects of integration from a lifecycle perspective and sets a foundation to anend-to-end approach to SI. Our end-to-end approach focuses on how integration issues can be addressedup-front to minimize integration related complexities and challenges later on in the system engineeringprocess. This paper discusses the merits and benefits of applying the SIPM to evaluate and improve currentSI processes in organizations. The paper provides, in addition to an overview of the SI framework, theactivities included in the model. The model was pilot tested to evaluate the SI processes at a governmentagency. The results were used to provide recommendations for SI process reengineering. © 2009 WileyPeriodicals, Inc. Syst Eng

Key words: system integration; verification and validation; interface; COTS; legacy; integration process;integration activities

1. INTRODUCTION

The challenge of addressing systems integration complexityand qualifying a system for deployment or implementationdemands new innovative methods for qualification, integra-tion, and testing. While researching for methodologies foreffective systems integration, we realized that, in order to

develop a new methodology, a clear and concise under-standing of the processes and activities involved in SI is muchneeded.

Systems integration activities are highly coupled and sup-ported by activities across a system lifecycle. This compre-hensive view of SI integrates both the products and theprocesses to achieve the required objectives. A SI frameworkapproach was taken to address the challenge. Based on thisframework, five process areas of SI were identified, andactivities under each process area were studied. The firstiteration of developing a SIPM used the literature review of theexisting and available process models in SI and SystemsEngineering (SE) as its foundation. This model was thenrefined by the study of industry best practices for SI and SE.This model was then piloted at a government agency to

*Author to whom all correspondence should be addressed (e-mail:[email protected]; [email protected];[email protected]).

Systems Engineering© 2009 Wiley Periodicals, Inc.

1

Regular Paper

evaluate its current integration process effectiveness and pro-pose gaps and recommendations to improve the effectiveness.

The next several sections will discuss the details of SIFramework and the lifecycle view, SI processes model, andrelated activities, and the results from the application of themodel at a government agency.

2. SYSTEMS INTEGRATION AND SYSTEMSENGINEERING

Systems Integration is an important element of Systems En-gineering which involves the integration of hardware, soft-ware, products, services, business processes, and humans[Grady, 1994; Jain, 2007]. From a process perspective, thesystems integration process creates the links within the Sys-tems Engineering process from requirements collection toverification and validation and ultimately to operation of thesystem. It would be accurate to state that the SI process andactivities occur throughout the SE lifecycle. SI should beimplemented from the beginning and throughout the systemdevelopment rather than being implemented only as a “down-stream” event.

The existing standards, models, and guidelines of systemsengineering and software engineering address systems inte-gration issues partially. They tend to view systems integrationfrom a perspective of integrating physical components. Thesestandards and models lack a holistic end-to-end approach toSI. For example, 15288:2002 [ISO/IEC, 2002] systems inte-gration process activities include:

a. Define an assembly sequence and strategy that mini-mizes systems integration risks.

b. Identify the constraints on the design arising from theintegration strategy.

c. Obtain integration enabling systems and specified ma-terials according to the defined integration procedures.

d. Obtain system elements in accordance with agreedschedules.

e. Assure that the system elements have been verifiedagainst acceptance criteria specified in an agreement.

f. Integrate system elements in accordance with applica-ble interface control descriptions and defined assemblyprocedures.

g. Use the specified integration facilities.h. Record integration information in an appropriate data-

base [ISO/IEC, 2002].

This definition of systems integration does not include theimportant issues of systems integration such as interoperabil-ity, interface control and management, business process inte-gration, and traceability. Due to the emerging SE challengesand the increasing importance of systems integration, the needfor a holistic approach to Systems Integration has becomecritical.

3. SYSTEMS INTEGRATION FRAMEWORK (SIF)

As a part of the ongoing systems integration research atStevens Institute of Technology, a Systems Integration Frame-work (Fig. 1) was developed based on the literature reviewand evaluation of systems integration processes and models.This framework was developed as a baseline to identify acomprehensive set of end-to-end activities that may constituteand define the scope of systems integration processes. Thedetails on this framework can be obtained from Jain, Erol, andChandrasekaran [2009.].

This Systems Integration Framework illustrates a compre-hensive end-to-end systems integration approach. This ap-proach is based on the premise that integration occurs

Figure 1. Jain’s Systems Integration Framework. [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]

2 JAIN, CHANDRASEKARAN, AND EROL

Systems Engineering DOI 10.1002/sys

throughout the lifecycle of a system. Systems integration isnot a one-time activity. Our current focus of SI processes andactivities looks at the following three dimensions of SI inter-actions as illustrated in the SI framework. (1) Interoperabilityis a prerequisite to achieving successful systems integration.If the subsystems or systems are not interoperable, theycannot be integrated. Standards or other forms of guide-lines—international, national, or industry-specific—providean opportunity to design systems which can be “open” andsimple to integrate. On the other hand, these guidelines canalso constrain the design by requiring additional design ele-ments that may lead to more complex systems integration. (2)The other processes of system development lifecycle such asrequirements management, interface control and manage-ment, testing, verification and validation, and configurationmanagement should include and address SI issues. (3) Legacysystems integration and COTS integration issues are inevita-ble in integration of most systems. Interoperability and inter-face management issues of legacy and COTS systems affectthe overall success of systems integration to a significantextent.

Due to the shift within the SE community towards consid-ering “enterprises” as systems [Rouse, 2005zaq;1; Carlockand Fenton, 2001zaq;1; Kosanke et al., 1999zaq;l], EnterpriseIntegration (EI) has become an important aspect of SI. EI isrelated to integrating technology, processes, and people tofacilitate a greater flow of information and effective decisionmaking across the enterprise with the goal of improvingefficiency and competitive advantage. Integration of businesssystems, or for that matter any system, requires a good under-standing of business processes and business process analysis.Systems are designed, developed, and engineered to supportthe business processes within an enterprise and alignment ofprocesses and systems is crucial to better meet the needs ofstakeholders for whom the systems are built. This approachsets our foundation of the end-to-end (life cycle) approach toSI. Therefore, business process integration (BPI) is an impor-tant element of our SI Framework. An understanding of theconcept of a business process and the need to conduct inte-grated business process analysis is a prerequisite for systemsintegration. Though the authors believe both EI and BPI areessential for an end-to-end integration the scope of the re-search study of the agency’s SI process does not include themdue to programmatic limitations.

4. SYSTEMS INTEGRATION LIFECYCLE ANDPROCESSES

Extending the framework, we define the Systems Integrationprocess as a set of activities that transforms the stakeholderrequirements into an operational system by unifying the proc-ess components and product components into a whole whileensuring compliance to the specified levels of componentoperations and interoperability. The scope and objectives ofsystems integration should be clearly identified to ensure thatnew systems and components are able to work with theexisting systems and components. This can be achieved byapplying [Mische, 1998] four states of systems integration:

• Interconnectivity is the initial and the fundamental statein the systems integration. It requires all new and exist-ing system components and equipment to connect andwork together.

• Interoperability means that all interconnected informa-tion system components and equipment should be ableto function and interact with each other. Interoperabilityis considered as the key state of systems integration.

• Semantic consistency refers to the concern of consis-tency at data level.

• Convergent integration involves the amalgamation ofcomponents and technology with business processes,people, skills, and knowledge.

A lifecycle view of SI will help us in understanding thecontext of SI and its scope across system engineering lifecycle phases. It also helps in identifying and addressing SIissues as and when they happen throughout the system devel-opment, implementation, and operation.

Figure 2 shows how the five Systems integration subproc-esses fit within the Systems Engineering Process. These proc-esses are embedded within the SE lifecycle processes inplaces where SI activities become critical and important. TheSI lifecycle has five high level subprocesses. These subproc-esses are identified as Integration Requirements; IntegrationArchitecture; Integration Planning; Integration Implementa-tion; and Integration Validation and Verification as shown inFigure 2.

Once a conceptual design for a system is chosen and alloperational scenarios (use cases) to understand the context areanalyzed, the broader category of stakeholder requirementsare then refined and derived to form system requirements orspecifications. During this phase of SE life cycle, it is criticalto identify requirements that will impact systems integration.These requirements termed as integration requirements ad-dress the required level of integration and quality. Theseintegration requirements fall under the categories of compli-ance, interoperability, qualification, COTS (commercial off-the-shelf), and legacy. This subprocess of identifying,refining, deriving, and managing the above categories ofrequirements is called Integration Requirements subprocess.

The integration requirements are then used to develop thedesign (architecture) to address the integration issues such asCOTS, legacy, interfaces, testability, qualification, and com-pliance. These design decisions are based on the functionaland physical architecture of the system. Tradeoff decisionsare based on these architectures and requirements. This sub-process of developing architectures to integrate the system asa whole using tradeoffs and decision making is called theIntegration Architecture subprocess.

Based on the optimized system architecture design, plansare developed to manage and implement interfaces and tests.The plans are based on the interface architecture and qualifi-cation architecture. The planning decisions are made concur-rently based on the system model optimization sub-process.This subprocess of harmonizing the interface management,verification, and validation of the integrated subsystems andthe system with its environment is called Integration Planningsubprocess.

SYSTEMS INTEGRATION FRAMEWORK FOR PROCESS ANALYSIS AND IMPROVEMENT 3


Once the detailed designs are implemented and subsys-tems are built, verification and validation of the subsystemsis done as per the requirements, architecture, and the plan.These subsystems are then integrated and the process ofverification and validation occurs to determine the complianceand qualification of the integrated system. This process iteratesuntil the completed system integrates with its operational envi-ronment. The subprocess of integrating the subsystems andsystem as a whole based on the architectures is called IntegrationImplementation subprocess. The subprocess of testing and quali-fying the subsystems and the integrated system is called Integra-tion Verification and Validation subprocess.

Based on the system development lifecycle, these subproc-esses can feed into each other and iterate to mitigate andmanage the unknown factors of the system and optimizeintegration effectiveness. These five subprocesses of systemsintegration form the five fundamental processes of systemsintegration process model. SIPM identifies all the subprocessesthat are critical for systems integration. It further explains theinteraction of activities within each subprocess, and betweenthe five subprocesses. The following section discusses theactivities in each subprocess.

5. SYSTEMS INTEGRATION PROCESS MODEL(SIPM)

SIPM shows how each of the sub-processes interacts and howthe process flow occurs in each. For the purpose of clarity and

readability, the process model is shown in four process flowdiagrams (Figs. 3, 4, 6, and 7). The interconnections betweenthe processes are shown in a rounded rectangle with theactivity number listed. The activity numbers are shown alongwith the activity name in each process flow. SIPM has fivesubprocesses and 45 activities. This model was developedbased on literature reviews on systems integration, standards,and integration best practices. The following subsectionsprovide a brief discussion on each subprocess and their proc-ess flows.

The process flows are used only for the understanding ofthe dependence between each activity in systems integration.The SI process execution sequence or concurrence is notshown in the process flows as included in this paper. However,the direction and nature of process flows forms the basis oftraceability between all the activities related to each of theprocesses (artifacts, resources, etc.). Traceability facilitatesvalidation in each subprocess. Validation helps us determinethat these systems integration processes have produced theright design and related artifacts. The process flows includedin this paper do not explicitly show the traceability acrossthem and the validation at the end of each sub-process.

5.1. Integration Requirements Subprocess

Integration Requirements is one of the most important sub-processes of the systems integration process. The process flowof the requirements sub-process is shown in Figure 3. In order

Figure 2. SI process activities within SE Lifecycle (SE Lifecycle as adapted from SYS 625 Course Notes, Stevens Institute of Technology,Hoboken, NJ). [Color figure can be viewed in the online issue, which is available at www.interscience.wiley.com.]



to avoid issues and surprises during the implementation, it isimportant to elicit system requirements with a systems inte-gration focus early on. The delays in product development aremainly attributed to errors and rework that result due to theerrors during the final phases. The effort and money spent onthese errors and fixes are huge [McConnell, 1996]. Theseerrors could have been avoided with early consideration ofintegration issues. It is a good practice to identify systemsintegration stakeholders and gather requirements from themearly in the SE life cycle. The systems integration processbegins with these requirements subprocess activities. In gen-eral, systems integration requirements fall under six majorcategories or types

5.1.1. Interoperability RequirementsInteroperability is the ability of systems to provide servicesto and accept services from other systems, and to use theservices so exchanged to enable them to operate effectivelytogether. Interoperability can be achieved by designing andbuilding systems against a defined interoperability require-ment, and maintaining that interoperability throughout as thesystem changes and upgrades through configuration manage-ment, and testing for interoperability against those require-ments. Interoperability Requirements address or help addressan operationally recognizable activity or sequence of activi-ties that has a definable starting action, a definable concludingaction, and which involves the exchange of items like data,information, material, or energy between two or more systems(subsystems or platforms). Such an exchange may be interac-tive and may involve the use of more than one transfermedium; however, the information content on all transfermedia must be definable. These requirements are related to anoperational capability. In most cases, few interoperabilityrequirements are identified and interoperability often be-comes an issue when a system is deployed.

5.1.2. Interface RequirementsInterface is a connection resource for linking to anothersystem’s interface or to another system’s component. Inter-face is the functional and physical characteristics required toexist at a common boundary or connection between systemsor items. Interface requirements must address total systemperformance, the fidelity of the interface, and any systemrequirements meant to constrain interface design [Buede,2000]. Interface requirements are statements on the interfacedesigns, protocols used, data formats, entity relationships, andprocessing rules. They define the interfaces and their inputsand outputs. Interface Interactions between elements[Pimmler and Eppinger, 1994] are:

1. Spatial: A spatial-type interaction identifies needs foradjacency or orientation between two elements.

2. Energy: An energy-type interaction identifies needs forenergy transfer between two elements.

3. Information: An information-type interaction identifiesneeds for information or signal exchange between twoelements.

4. Material: A material-type interaction identifies needsfor materials exchange between two elements.

In cases where systems integration is highly linked to theorganizational planning and operations and in cases whereend-user computing is involved, organizational interfaces be-come critical. Organizational interfaces are the commonboundaries between user, system, and organization. The na-ture of the interfaces can be procedures, data, personnel,hardware, and software [Trauth and Cole, 1992]. Organiza-tional interfaces mainly include communication betweenuser/system, and organization and its subunits, user/system,and organizational vendors and subcontractors, organiza-tion/team coordination, training, documentation, support andservices, business alignment planning, organizational innova-tion [Beise, 1994; Trauth and Cole, 1992]. The quality of suchinterfaces also determines system effectiveness [Beise, 1994].Implications on system design and development due to organ-izational interfaces in terms of organizational forms of sup-port are discussed in Trauth and Cole [1992] and in otherliterature discussed in the former.

Interface requirements for all external and internal inter-faces are gathered in the early phases of systems integration.Interface requirements gathering and analysis help in under-standing the critical variables of all internal and externalinterfaces and their predicted variations but only to someextent—those that can be foreseeable. However, in the opera-tional environment there is always a challenge to deal withthe impact of interaction of these critical variables that wasnot foreseen. An upfront and early-on focus on interfacerequirements and architecture helps in addressing these is-sues. Interface requirements help in architecting interfaces toachieve robustness and specified level of “ilities.”

5.1.3. Qualification/Test RequirementsQualification requirements address the needs to qualify thesystem as being designed right, the right system, and anacceptable system [Buede, 2000]. Qualification is the processof verifying and validating the system design and then obtain-ing the stakeholder’s acceptance of the design. Qualificationis associated with testing, acceptance, verification, and vali-dation. The four elements of the qualification requirementsare (i) observance—how the expected qualification data foreach input/output and system-wide requirements will be ob-tained, that is, test, analysis and simulation, inspection, ordemonstration; (ii) verification plan—how the qualificationdata will be used to determine that the real systems conformsto the design that is developed; (iii) validation plan—how thequalification data will be used to determine that the realsystem complies with the originating requirements; and (iv)acceptance plan—how the qualification data will be used todetermine that the real system is acceptable to the stakehold-ers.

5.1.4. Operational Readiness RequirementsOperational readiness requirements address requirements thathelp to ensure that the solution can be correctly deployedwithin the enterprise or the operational environments. Theserequirements identify the entire roll out procedures and pro-grams that are required and production support to install ordeploy the system or release in its operational environments.These requirements are obtained from ConOps and servicelevel agreements of the system. ConOps describes the result



of the system conceptual analysis process. They include all ofthe information needed to describe the users’ needs, goals,expectations, operational environment, processes, and char-acteristics for the system under consideration. Operationalreadiness requirements focus on the sections of ConOps thatinclude modes of operation for the proposed system, userclasses and user characteristics, operational features of theproposed system, and operational scenarios for each opera-tional mode defining the system’s interaction with other sys-tems. These requirements ensure that all operating scenarioscan be supported by the proposed system.

5.1.5. Integration Technology RequirementsIntegration technology requirements address systems integra-tion with technical solutions. They address details on mecha-nisms that need to be implemented which will allow thetransfer of data between systems, and mechanisms for initiat-ing actions in other systems. Integration technology require-ments address feasibility, availability, and relevance oftechnical solutions for optimal integration. In most casesintegration technology focuses on the interconnectivity aspectof the comprehensive systems integration. Some examples ofintegration technology are data transfer, transport services,file transfer protocol, document protocols, remote procedurecalls, etc.

5.1.6. Standards, Guidelines, and RecommendationsRequirementsStandards, guidelines, and recommendations requirementsare the constraining requirements that have to be compliedwith for various reasons specific to each system and itsdomain, for example, security and safety requirements. Thesestandards, guidelines, and recommendations can be organiza-tional specific, technology specific, domain specific, auditand regulatory requirements, and international standards forquality, safety, and security. These include all formal, de jure,and de facto standards relevant to the technology and theproposed system. In most cases these requirements are usedto achieve interoperability, interchangeability, and portability.Systems tend to have longer operational life if they are basedon universally accepted standards and have been time-tested.These requirements can be both functional and nonfunctional.

The above six categories of Integration Requirements aregathered from the stakeholders and are further analyzed andrefined as shown in Figure 3. The derived requirements arerequirements that help us better understand and support theseoriginating requirements (requirements obtained from stake-holders). They are detailed system requirements for the cho-sen system concept. These requirements need to be iteratedwith the stakeholders of systems integration and signed off sothat they can be used as a baseline for systems integrationprocess. These requirements have to be documented well andmaintained with good configuration management (CM). Con-figuration management plays a key role in achieving systemsintegration effectiveness and is part of every subprocess ofintegration. Four classic operational aspects of ConfigurationManagement (CM) are identification of the structure of theproduct/process, control of their releases, accounting for theirstatus, and audit and review of their completeness.

5.2. Integration Architecture Subprocess

The integration requirements are addressed through the inte-gration architecture which is embedded in both the physicaland functional architectures of the system. Figure 4 illustratesthe process model of the integration architecture subprocess.Integration architecture includes legacy systems integration,COTS integration, interface definition, control and manage-ment, and qualification architecture. This section briefly ex-plains each of these act ivi t ies. [Jain, Erol, andChandrasekaran, 2009] provides detailed discussion on theseactivities.

We identify the strategies for integrating legacy systemsduring the conceptual design phase. These legacy systems areusually mission/business critical, and should remain func-tional at all times [Konstantas, 1996]. They fully satisfysystem functional requirements and support current businessfunctionality and have been thoroughly tested in the actualoperational environment. They are also coupled with the restof the operational infrastructure. The legacy systems providea set of design constraints which have to be identified anddefined. The limitations of legacy systems such as pollution,

Figure 3. Integration requirements activities flowchart. [Color fig-ure can be viewed in the online issue, which is available at www.interscience.wiley.com.]



embedded knowledge, poor lexicon, coupling, layered archi-tectures, frequency of failures and breakdowns, obsolescence,and maintenance cost are discussed in detail in Bianchi et al.[2003] The requirements for legacy integration are gatheredand analyzed. They are further derived to understand therequired level of interoperability and interfaces for integratingthe legacy system/subsystem. These requirements constrainthe architectural decisions for the proposed system.

Once the system requirements are elicited and a baselinearchitecture is developed then build or buy related designdecisions for subsystems and components are made. Thecommercial-off-the-shelf (COTS) subsystems or componentsare identified. A COTS item is one that is sold, leased, orlicensed to the general public; offered by a vendor trying toprofit from it; supported and evolved by the vendor whoretains the intellectual property rights; available in multiple,identical copies; and used without modification of the inter-nals [OSD, 2002]. The requirements on the level of interop-erability and interfaces for COTS integration are derived forfurther architectural refinement and decisions. A simultane-ous COTS approach is recommended to addresses COTSintegration issues. In this approach convergent decisions aremade considering the following tradeoffs simultaneously[SEI, 2002]:

1. Stakeholder needs and business processes: Require-ments (including quality attributes such as perform-ance, security, and reliability), end-user business

processes, business drivers, and operational environ-ment.

2. Marketplace: Available and emerging COTS technol-ogy and products, nondevelopment items (NDI), andrelevant standards.

3. Architecture and Design: Essential elements of thesystem and the relationships between them. Elementsinclude structure, behavior, usage, functionality, per-formance, resilience, reuse, comprehensibility, eco-nomic and technologic constraints and tradeoffs, andaesthetic issues.

4. Programmatics and risk: The management aspects ofthe project. These aspects consider the cost, schedule,and risk of building, fielding, and supporting the solu-tion to include the cost, schedule, and risk for changingthe necessary business processes.

The interface architecture defines the interface specifica-tion based on all the system internal and external interfacerequirements. A baseline interface definition must be agreedupon before the beginning of implementation activities, andthe user interface must be defined and maintained as anintegral part of the system specification. Interface control andmanagement is an important part of the integration architec-ture that results in the management of communication, coor-dination, and responsibility across a common boundarybetween two organizations, phases, or physical entities whichare interdependent. It ensures that internal and external inter-faces are properly identified, integrated, stabilized, and con-trol led early in order to prevent expensive andtime-consuming fixes later.

Semantic integration addresses the semantic content ofdata in different systems. It is more important to be aware ofinconsistencies in the semantics rather than to eliminate thedifferences, which may be difficult, especially in systemsfrom different vendors. Semantic consistency refers to theconcern for consistency at the data level. Once the informationsystem components and equipment are interconnected andoperational, users are able to access systems and manipulatedata (create, retrieve, modify, and delete) across various op-erational environments. For this reason, the implementationof semantic integration is essential to prevent data duplica-tion, redundancy, and instability [Mische, 1998].

The qualification architecture involves defining and man-aging the test activities as shown in Figure 5. Test approachesare developed to provide the objectives, schedule, environ-ment requirements, and entry and exit criteria for the teststage. Based on these approaches tests are planned and pre-pared to identify test conditions, and test cycles, and to definethe input data and expected results. It is important to establishthe appropriate test environments and to ensure that they aretested prior to the execution. Test cases and sequence for testexecutions are derived based on the system requirements andthe test approaches.

5.3. Integration Planning and ImplementationSubprocess

Figure 6 shows all the activities in the integration planningand implementation subprocesses. The integration require-

Figure 4. Integration architecture activities flowchart. [Color figurecan be viewed in the online issue, which is available at www.interscience.wiley.com.]



ments and architecture are analyzed to identify the risksassociated with systems integration. The risk analysis occursconcurrently as the requirements and architectures evolve. Allthe factors that impact integration and testability of the systemare identified. These risk factors and their mitigation activitiesneed to be included in the integration plan. An integration planis based on the integration specification elicited from therequirements and architecture. The integration plan includesdetailed integration implementation, verification, and valida-tion activities, execution sequences, dependencies, resources,tools, executions states and modes, assessment of integration,

and management of these activities. During this subprocesstraceability across all integration process related activities areverified. The integration plan and all specifications are docu-mented. Configuration management of the plan and the speci-fications plays a critical part in the success of integration.Verification and validation of the integration plan is per-formed by testing the system prototype. In most of the systemdevelopment efforts, the proposed system is modeled andoptimized by developing prototypes of the system. Theseprototypes are based on the derived requirements and archi-tecture. They are used to understand the behavior of the

Figure 6. Integration planning and integration implementation activities flowchart. [Color figure can be viewed in the online issue, which isavailable at www.interscience.wiley.com.]

Figure 5. Test activities.



system in its development and operational environment. Thishelps in identifying and mitigating the risks at an early stage.Once the subsystems and components are built or acquired,the integration activities are implemented as per the integra-tion plan. Some other subactivities that will be involved in thisIntegration Planning and Implementation Subprocess will be:monitor status of components and subsystems at lower levelsof procurement or integration, receive components and sub-systems, check conformance of components and subsystemsto their specifications, prepare integration environment, de-velop integration tools and facilities, and develop integrationprocedures and certify integration personnel.

5.4. Integration Validation and VerificationSubprocess

Figure 7 shows the activities of the verification and validation(V&V) subprocess.{FIG7} Verification and validation of thesubsystems and components that are built or acquired followsthe qualification plan developed in the earlier phase. Verifica-tion establishes the truth of correspondence between a prod-uct/system and its specification. The activities in this V&Vsubprocess verify and validate if we are building the rightproduct/system and at the same time building it right. Valida-tion is the act of ensuring that the system works as per itsintended functionality and that the users are satisfied andwilling to accept the system. An example is the comparisonof the actual system response to an online transaction to whatwas originally expected, requested, and finally approved for

an online transaction processing system. Validation estab-lishes the suitability of a product/system for its operationalmission in a given environment based on operational or fieldtesting. Both verification and validation activities should betraced back to the system requirements.

During this subprocess the test scripts are verified andvalidated, test environment is developed and the tests areexecuted in its sequence. These activities are based on thedeveloped qualification plan. There are two types of testing,Functional/Life Cycle Approach and Structural. In functionaltesting, test the functionality of the system and ensure that theuser functional requirements and specifications are met. Testconditions are generated to evaluate the correctness of theapplication. These include unit test, assembly test, systemsintegration test, operational readiness test, and user test. Instructural testing, we test the structural and physical capabil-ity of the system and ensure that the system is structurally andtechnically sound, can perform the intended tasks, and thatthe components integration works cohesively. These includefatigue testing, strain and impact testing, branch and pathtesting, control flow testing, data flow testing, security testing,and stress/volume/scalability testing. The test results arelogged and documented. The errors are fixed through reworkand change control management.

The SIPM process model has been primarily proposed tofacilitate baselining of as-is SI processes in an organization,benchmarking them against targeted optimal levels, and reen-gineering them for better integration effectiveness as a resultof controlling complexity of SI. The model would help or-

Figure 7. Integration implementation, integration verification, and validation activities flowchart. [Color figure can be viewed in the onlineissue, which is available at www.interscience.wiley.com.]



ganizations identify areas of SI that are strong and others thatrequire more attention. It would also highlight critical attrib-utes of SI in their specific organizational context.

6. APPLICATION OF THE SYSTEMSINTEGRATION PROCESS MODEL (SIPM) FORPROCESS ANALYSIS AND IMPROVEMENT

This section describes our experience of applying the SIPM toanalyze, evaluate, optimize, and recommend improvementsto the current systems integration process and activities of agovernment agency (as observed and reported on nine differ-ent development projects). We will be referring to this gov-ernment agency as the “Agency” in the rest of this paper.

A critical component of undertaking process analysis andredesign is to identify the gaps and redundancies and imple-ment improvements in order to make processes efficient,effective, and adaptable [Harrington, 1995]. This paper illus-trates our findings of using the SIPM as a baseline to analyzeand improve the current SI process of the Agency. The nextsection will cover our methodology for conducting this workfor the Agency.

7. RESEARCH METHODOLOGY

This section provides an overview of the research methodol-ogy used for the pilot application of SIPM for SI processreengineering. The research sample was nine system develop-ment projects in the agency. The project data were providedby the Systems Engineering Leads of each of the nine projectson the scope of the SI process and related activities that werebeing conducted.

We used the survey research method to identify and ana-lyze the Agency’s current Systems Integration process andactivities. The survey includes 45 questions based on theactivities of SIPM. Each question is asked to determine if thespecific activity is currently performed on each of the nineprojects. Respondents were asked to select one of the threegiven answers of “Yes”; “Not exactly but we perform similaractivities”; or “No”. A free-form comments section was alsoprovided at the end of the survey. The survey responses wereconsolidated, reviewed, and analyzed.

The as-is analysis of the Agency’s current SI process wasbased on five focus areas (research questions): (1) gap analy-sis between the Agency’s current SI process and SIPM; (2)strengths and weaknesses of the Agency’s current SI process;(3) effectiveness of the integration requirements subprocessin terms of completeness, refinement, and traceability; (4)effectiveness of the Agency’s current SI process to addresscritical SI issues such as COTS, legacy, interoperability, con-figuration management, interface control and management,and adherence to standards and regulations; and (5) quality ofthe integration verification and validation. The survey resultswere analyzed based on these research questions. The re-search questions, survey results, and the analysis are dis-cussed in detail in the following section.

8. RESEARCH FINDINGS

The survey results and their analysis not only demonstratedto us the level of effectiveness of the Agency’s current SIprocess but also revealed the areas which need improvement.Based on these findings, a set of recommendations was pro-vided to the Agency to improve its current SI process andactivities that require attention. This section is organized byresearch questions and their relevant analysis and findings.

The results are shown as percentage values which indicatethe relative difference between the proposed SIPM andAgency’s current process and activities. This difference iscalculated by assigning a weight of 2 to “Yes” answers whichwere given for a specific activity, a weight of 1 to “Not exactly,but we perform similar activities’ answers,” and assigning aweight of 0 to “No” answers. A weighted sum of the answerswere calculated for a given a specific activity. We then iden-tified the gaps based on an assumption that if all projectsreported a “Yes” for an activity then that activity will beconsidered as having a 0% gap or, in other words, the activitysatisfies requirements of out SIPM completely (at 100%).

8.1. Gap Analysis of the Agency’s Current SIProcess and SIPM

Based on the survey results, we found that the overall gapbetween the Agency’s current SI process and the proposedSIPM is 28% (Fig. 8). This indicates that the Agency’s currentSI process is not comprehensive enough (28% deficient) inscope to include all aspects of an end-to-end systems integra-tion approach. A subprocess by subprocess gap analysis re-flected that the Integration Architecture Subprocess inparticular requires more reengineering. This subprocess has a40% variance with respect to the integration architecturesubprocess in SIPM. Addressing integration issues and designdecisions earlier in the development lifecycle will reduce theerror and fault rate and as well as rate of rework. This will inturn provide programmatic advantage in terms of cost, sched-ule and resources. [Jain et al., 2008] shows that the systemarchitecture and requirements impact the systems integrationcomplexity and quality of verification and validation. Theintegration architecture subprocess is highly dependent on thesystem architecture and requirements subprocesses. Thetradeoff decisions critical to systems integration are addressedduring this subprocess of SI. To improve the effectiveness ofthe SI process, the Agency needs to provide more emphasison the integration architecture activities.

The gap analysis also shows that the Agency’s current SIprocess was relatively stronger in the Integration Implemen-tation sub-process and Integration Verification and Validationsub-process activities (variance with respect to the corre-sponding sub-process in SIPM less than 15%). This shows thatthe Agency’s current SI practices focus mainly on the integra-tion implementation, verification and validation. In otherwords, the Agency’s significant focus is on the downstreamactivities of systems integration. This finding resonates withour early discussion on how the current SE practices considersystems integration as the physical integration of subsystemsand components and their verification and validation. How-ever, SIPM takes a different stand and suggests a comprehen-



sive end-to-end approach which requires emphasis on integra-tion starting from the earlier front-end sub-processes of SElifecycle.

8.2. Strengths and Weaknesses of the Agency’sCurrent SI Process

We analyzed the survey results in order to identify thestrengths and weaknesses of the Agency’s current SI process.This was done to better understand the current state of systemsintegration at the Agency as reported by the nine projects.Identification of the weak areas within the Agency’s currentSI process helped us to focus on the areas which required moreattention and which needed to be improved in order to achievea more effective integration process.

We identified the different SI activities in the Agency andtheir corresponding gaps. The size of the gap is indicative ofthe strengths and weaknesses of the Agency’s SI process. Thebigger the gap the weaker is the Agency on those SI activitiesand vice versa.

Figure 9 shows strengths and weaknesses of the Agency’scurrent SI process. The overall SI process gap of 28% isshown by the horizontal line. The x-axis shows the SI processsub-processes and the y-axis shows the level of gap. Thehigher the vertical placement of the bubble above the 28%line, the bigger is the gap. The size of the bubbles indicatesthe number of activities which fall within the level of differ-ence on the y-axis. For example, if we look at the lower leftquadrant of the figure, where the x-axis shows “IntegrationRequirements Activities,” we see a bubble marked with “2”at the 11% level on the y-axis. This bubble indicates thatcurrently there are 2 activities within the Integration Require-ments Activities which have 11% gap from the SIPM level.

Our focus in this research was to recommend improve-ments for optimizing the current state of Agency’s SI processand related activities. Understanding the Agency’s strengthshelped us to develop and recommend a process model whichwould be suitable with its given strengths. We also identified

and analyzed the deficiencies of the Agency’s SI process andpointed out the risks arising out of these deficiencies. As aresult, the areas which were identified as weaknesses ofAgency’s current SI process were rated as highest priority forimprovement. Our approach was to develop a model for theAgency that eliminated the inefficiencies resulting from theweak areas of the Agency’s current SI process. This wouldhelp contain the risks resulting from such SI process ineffi-ciencies.

8.3. Integration Requirements Issues

As discussed earlier, integration requirements subprocess isone of the most important subprocesses within SI process. Itprovides the front-end basis for effective systems integration.We analyzed the survey results to identify integration require-ments issues within Agency’s current SI process. We lookedat two main issues within integration requirements: (i) em-phasis on different types of integration requirements and (ii)refinement of different types of requirements. We found thatthe Agency is relatively good in identifying stakeholders andgathering and analyzing the systems integration require-ments. Overall 78% of the requirements integration relatedactivities are currently being conducted at the Agency.

8.3.1. Emphasis on Different Types of RequirementsWe analyzed the survey results to identify the level of empha-sis (or lack thereof) given to the different types of require-ments. Figure 10 shows the level of emphasis given todifferent types of requirements. The three areas on which theagency focuses were qualification, standards, and COTS re-quirements.

Interface, integration technology, interoperability, and leg-acy requirements received the least emphasis. This was in-dicative of a substantial potential for improvement.Identification, analysis, and management of interoperability,legacy, integration technology, and interface requirements areextremely critical in defining, constraining, and facilitatingsystems integration. The lack of emphasis on these types of























Figure 8. Gap analysis of the Agency’s current SI subprocesses. [Color figure can be viewed in the online issue, which is available atwww.interscience.wiley.com.]



Figure 9. Strengths and weaknesses of the Agency’s current activities. [Color figure can be viewed in the online issue, which is available atwww.interscience.wiley.com.]

Figure 10. Level of emphasis on different types of requirements. [Color figure can be viewed in the online issue, which is available atwww.interscience.wiley.com.]



requirements can lead to integration issues and jeopardize theeffectiveness and efficiency of systems integration process.

8.3.2. Refinement of Originating RequirementsOur SIPM goes beyond the gathering of originating integrationrequirements and emphasizes on their refinement as well. Wefound that the Agency’s current SI process included a fairlystrong requirements refinement process. The deriving re-quirements process received only 4% less emphasis than thecollecting originating requirements (Fig. 11). This findingmight indicate that some of the requirements categories werenot identified in the beginning or not gathered from stakehold-ers. As a result they had to be derived at the later stages.Another reason for this difference might reflect that sometypes of originating requirements could not be collected com-pletely and therefore had to be derived at the later stages. Weassume that a significant portion of this gap is contributed bythe missing design-tradeoff requirements primarily related toCOTS and legacy integration.

8.4. The Role of Systems Integration IssuesImpacting Agency’s Current SI Process

Some of the common systems integration related issues areinteroperability, COTS and Legacy Integration, InterfaceControl and Management, Standards, Guidelines and Recom-mendations, and Configuration Management. This section

discusses how effectively the Agency’s current SI processaddresses these issues. Our findings are summarized in Figure12. Recommendations on adoption of these integration issuesrelated SIPM activities were provided to the Agency for opti-mization and reengineering.

8.4.1. Interoperability IssuesInteroperability issues directly impact the success of systemsintegration process. The 38% variance (Fig. 12) between theAgency’s current interoperability related activities and SIPM

shows a lack of focus on interoperability issues. Interoperabil-ity of a system is addressed through the interfaces, compli-ance, and design tradeoffs. SIPM proposes early identificationof interoperability requirements. These requirements help inidentifying the respective compliance requirements and otherdesign constraints at an early stage of development. Interfacesare also directly impacted by these requirements. SIPM alsofocuses on traceability of these requirements from their originto their implementation, verification and validation.

8.4.2. COTS and Legacy Integration IssuesIn today’s world, the common challenges faced when engi-neering a system are related to the integration and interoper-ability with COTS systems and existing legacy systems. Thegap analysis (Fig. 12) shows that the Agency’s focus on theCOTS and legacy integration issues are not adequate. Adop-tion of COTS and legacy systems are driven by the time-to-

Figure 11. Gap between SIPM and As-Is Requirements Subprocess. [Color figure can be viewed in the online issue, which is available atwww.interscience.wiley.com.]

Figure 12. Issues impacting Agency’s current systems integration process. [Color figure can be viewed in the online issue, which is availableat www.interscience.wiley.com.]



market constraints and the higher level of technology readi-ness of the COTS products. But the longevity and lack ofdesign flexibility with COTS and legacy systems constrain thesystem integration. Upfront COTS and legacy integrationdecisions can help in identifying the appropriate interface andperforming the adequate V&V. SIPM supports early COTS andlegacy system integration.

8.4.3. Role of Interface Control and ManagementThe gap analysis (Fig. 12.) shows that the Agency’s focus oninterface control and management has lesser variance com-pared to other integration issues. Good interfaces enablebetter system integration and interoperability. The SIPM in-cludes activities which foster adaptable and scalable interfacecontrol and management. Interface control and managementensures that internal and external interfaces are properly iden-tified, integrated, stabilized, and controlled early-on in orderto prevent expensive and time-consuming fixes later.

8.4.4. Role of Standards, Recommendation, and Guidelinesin the SI ProcessThe usage of standards, recommendations and guidelines isimportant for SI process to ensure interoperability, to mini-mize interface issues and to streamline development efforts.According to survey results, standards, recommendations,and guidelines play an important role in the Agency’s currentSI process. This reflected the regulatory nature of develop-ment that was unique to the Agency. Though the variance wasonly 22% (Fig. 12) between the Agency’s current state andthe SIPM, we still recommended the Agency focus more onthem as the consequences of not including such constrainingrequirements in the SI process could be severe. SIPM empha-sizes the use of standards, recommendations, and guidelinesto achieve uniformity and consistency in design thereby lead-ing to simple systems integration.

8.4.5. Role of Configuration Management in SI ProcessConfiguration Management (CM) is another important aspectof SIPM. The gap analysis (Fig. 12) shows a 26% variancebetween the Agency’s configuration management and thecorresponding SIPM activities. SIPM includes activities formanaging the configuration of requirements, specification,and test results. Weak configuration management activitiesmake it difficult to control and manage not just the systemsintegration process but also the other development subproc-

esses. The variance in configuration management is a resultof the divulged focus on configuration due to very high focuson documentation in the Agency’s development environment.The Agency was recommended to provide more focus onchange controls and thereby improve its configuration man-agement.

8.5. Quality of Integration Verification andValidation

The quality of integration verification and validation (V&V)directly impacts the overall outcome of SI Process. In the SIPM

the prerequisites of the integration verification and validationsubprocess are addressed in the upstream activities of SIprocess. These upstream activities provide the required arti-facts and requirements for an upfront and effective planningof the V&V sub-process. Ten such supporting activities ofV&V from other subprocesses of SIPM were identified: (i)Integration Requirements Subprocess: (Derive QualificationRequirements, Derive Test Cases); (ii) Integration Architec-ture Subprocess: (Develop Semantics for Integration, De-velop Semantic Specification, Develop QualificationArchitecture, Develop Test Sequence, Develop Test Environ-ments Specifications, Test Architecture); and (iii) IntegrationPlanning Subprocess: (Develop System Prototype, Test Sys-tem Prototype). These 10 activities along with the five V&Vactivities aid in achieving better quality of integration verifi-cation and validation.

Figure 13 shows the minimum, maximum, and average gapvalues for these activities between our SIPM stipulations andthose that were reported by the Agency’s projects for each ofthese activities. This analysis shows us that the downstreamactivities that result in better quality of V&V are providedmore emphasis than the upstream activities. The average gapvalues were less than 15% in the Integration Planning andIntegration Verification and Validation subprocesses. Thisanalysis also shows that more importance needs to be givento the Integration Architecture subprocess (higher average gapvalue of 45%) and thereby improve the quality of V&V.

9. CONCLUSIONS AND FUTURE WORK

The Systems Integration Process Model (SIPM) is developedand applied to analyze one organization’s current Systems

Figure 13. Activities impacting quality of systems integration verification and validation. [Color figure can be viewed in the online issue, whichis available at www.interscience.wiley.com.]



Integration process and activities, and recommend improve-ments. Most available standards, maturity models, and meth-odologies address systems integration issues partially and donot take a holistic end-to-end approach to systems integration.Due to the emerging challenges, the need for a holistic ap-proach to systems integration has become critical. Our Sys-tems Integration Process Model is developed as a result ofextensive research in the field and intends to provide a com-prehensive approach to systems integration. The applicationof our SIPM to evaluate, analyze, and optimize one organiza-tion’s current systems integration process and activities pro-vided us an opportunity to implement and test our Model. TheModel is in its early phases of being piloted. There are at leasttwo doctoral-level researchers who are working on furthertesting and refining it. Our hope is that in the future applica-tions we will be able to define metrics related to each of the45 SI activities. Our future work will focus on definingdifferent levels of SI process maturity based on the metricsfor each of the SI activities. The maturity level of the SIprocess will indicate the likelihood for success or failure ofthe product or project.

REFERENCES

C. Beise, A model of the is/organizational interface and users’perceptions of is effectiveness, Computr Personnel (July1994).zaq;2

A. Bianchi, D. Caivano, V. Marengo, and G. Visaggio, Iterativereengineering of legacy systems, IEEE Trans Softw Eng 29(3)(2003), 225–241.

D. Buede, The engineering design of systems, Wiley, New York,2000.

J.O. Grady, Systems integration, CRC Press, Boca Raton, FL, 1994.

H.J. Harrington, The new model for improvement: Total improve-ment management, Bus Process Re-Eng Management J 1(1)(1995)zaq;3.

ISO/IEC, 15288:2002, Systems engineering—system life cycleprocesses, International Organization for Standardization/Inter-national Electrotechnical Commission, Geneva, 2002.

R. Jain, Systems integration course notes, Stevens Institute of Tech-nology, Hoboken, NJ, 2007.

R. Jain, O. Erol, and A. Chandrasekaran, A framework for end-to-end approach to systems integration, Int J Indust Syst Eng 4(6)(2009), to appear.

R. Jain, A. Chandrasekaran, G. Elias, and R. Cloutier, Exploring theimpact of systems architecture and systems requirements onsystems integration complexity, IEEE Syst J (June 2008)zaq;2.

D. Konstantas, Migration of legacy applications to a corba platform:A case study, Proc IFIP/IEEE Int Conf Distributed Platforms:Client/Server and Beyond: DCE, CORBA, ODP and Adv Dis-tributed Appl, 1996, pp. 100–112.

S. McConnell, Software quality at top speed, Software Dev 4(8)(1996), 38–42.

M.A. Mische, “Defining systems integration,” Reengineering: Sys-tems integration success, M. A. Mische (Editor), CRC Press,Boca Raton, FL, 1998.

OSD, Commercial item acquisition: Considerations and lessonslearned, Office of the Secretary of Defense, Washington, DC,2002.

T.U. Pimmler and S.D. Eppinger, Integration analysis of productdecompositions, ASME Conf Des Theory Methodol, 1994, pp.343–351.

SEI, Evolutionary process for integrating cots-based systems: Aoverview, Carnegie Mellon, Software Engineering Institute,Pittsburgh, 2002.

E. Trauth and E. Cole, The organization interface: A method forsupporting end users of packaged software, MIS Quart 16(1)(1992), 35–53.

Rashmi Jain is an Associate Professor of Systems Engineering at Stevens Institute of Technology. Dr. Jain has over 15years of experience of working on Information Technology (IT) systems. Prior to joining Stevens she was with Accenture(formerly known as Andersen Consulting). Over the course of her career she has been involved in leading theimplementation of large and complex systems engineering and integration projects. She has given invited lecturesinternationally at Keio University, and Shibaura Institute of Technology, Japan, Overseas Chinese Institute of Technology(OCIT), Taiwan, Indian Institute of Technology—Delhi etc. She is a visiting professor for System Architecture andIntegration at Keio University. Her teaching and research interests include systems integration, systems architecture anddesign, business process reengineering, and rapid systems engineering. Dr. Jain has authored several papers on thesetopics. She holds Ph.D. and M.S. degrees in Technology Management from Stevens Institute of Technology.

Anithashree Chandrasekaran is a Doctoral Candidate in the School of Systems and Enterprises at Stevens Institute ofTechnology. She is also working as a technology risk manager. Her research interests includes rapid systems developmentand its processes, development process reengineering, risk management and modeling, system integration, system designand architecture. She is currently working on developing a dynamic risk assessment model for rapid system developmentprocess. She obtained her B.E. in Electrical and Electronics Engineering from P.S.G. College of Technology, India. Sheobtained her M.S. in Systems Engineering from Stevens Institute of Technology. She served as president and also as afounding member of the Stevens INCOSE student chapter.



Ozgur Erol is a Ph.D. student in Systems Engineering and Engineering Management at Stevens Institute of Technology.Her research interests include systems integration process evaluation, management, and improvement. She received herbachelors and masters degrees in Industrial Engineering from Istanbul Technical University, Turkey and her MBAdegree from Saint Joseph’s University, Philadelphia, PA. She has worked in several information technology andorganizational reengineering projects prior to joining the Ph.D. program at Stevens.

<enote>AQ1: Please list in References<enote>AQ2: Please give volume number and page range<enote>AQ3: Please give page range



a systems integration framework for process analysis and ...jainra/research_papers/r6.pdf ·...

Documents