feasibility of a rapid systems engineering framework: …jainra/research_papers/r55.pdffeasibility...

Int. J. Industrial and Systems Engineering, Vol. 7, No. 1, 2011 45

Copyright © 2011 Inderscience Enterprises Ltd.

Feasibility of a rapid systems engineering framework: an exploratory study

Rashmi Jain*, Anithashree Chandrasekaran, Lymari Castro and Mary VanLeer School of Systems and Enterprises,Stevens Institute of Technology, Hoboken, NJ 07030, USAFax: +1 201 216 5541 E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] E-mail: [email protected] *Corresponding author

Abstract: Systems Engineering (SE) has been traditionally viewed as an extremely rigorous approach for having resource intensive processes often perceived to involve bureaucratic decision making; therefore, deemed to be affordable only by large companies and government institutions. However, in recent years, the commercial industry is trying to leverage the benefits of SE by selecting the relevant aspects that apply to them and customising SE to a leaner and compressed version. This paper explores the suitability and the feasibility of rapid approaches and techniques to the existing SE processes. More specifically, this paper describes 22 techniques of rapid systems engineering (RSE) during the design and implementation processes, relevant to those processes and lessons learned. These techniques are applied to the 14 SE processes as illustrated by the SE standard – ISO 15288. An exploratory survey was developed by the authors based on the SE process activities as per ISO 15288 to evaluate application utilising a rapid approach in current projects from various industries. This paper provides conclusions to applying rapid techniques to the SE processes based on existing literature and the experiences of the projects surveyed. It concludes with a discussion of potential research projects for the evaluation of RSE.

Keywords: RSE; rapid systems engineering; systems integration; validation; verification; life cycle management; ISO 15288; risk management; COTS integration; requirements; architecture; configuration management; system design and architecture.

Reference to this paper should be made as follows: Jain, R., Chandrasekaran, A., Castro, L. and VanLeer, M. (2011) ‘Feasibility of a rapid systems engineering framework: an exploratory study’, Int. J. Industrial and Systems Engineering, Vol. 7, No. 1, pp.45–65.

Biographical notes: Rashmi Jain is an Associate Professor of Systems Engineering at Stevens Institute of Technology. She has over 15 years of experience in Information Technology (IT) systems. Over the course of her career, she has been involved in leading the implementation of large and complex systems engineering and integration projects. She has done lectures internationally at Keio University, Overseas Chinese Institute of Technology (OCIT), Indian Institute of Technology – Delhi. Her teaching and research

46 R. Jain et al.

interests include systems integration, systems architecture and design, business process reengineering and rapid systems engineering. She holds PhD and MS in Technology Management from Stevens Institute of Technology.

Anithashree Chandrasekaran is a Doctoral Candidate in the School of Systems and Enterprise at Stevens Institute of Technology. Her research interests include rapid systems development and its processes, development process reengineering, risk management and modelling, system integration and system design and architecture. Currently, she is working on developing a dynamic risk assessment model for rapid system development process. She obtained her BE in Electrical and Electronics Engineering from P.S.G. College of Technology, India. She obtained her MS in Systems Engineering from Stevens Institute of Technology. She is the President of Stevens International Council of Systems Engineering student chapter.

Lymari Castro is a Doctoral Candidate in the School of Systems and Enterprise at Stevens Institute of Technology. She received a MEng in Engineering Physics from Cornell University and a BS in Theoretical Physics from the University of Puerto Rico. She is a Lead Systems Engineer for the Department of Defense and previously worked as a Systems Engineer at Raytheon Company. Current responsibilities involve testing and integration of rapid development projects. She is a Member of the Institute of Electrical and Electronics Engineers (IEEE) and the International Council of Systems Engineering (INCOSE). Her research interests are rapid systems development and systems integration.

Mary VanLeer has over 28 years of experience in defining, designing and supporting IT solutions for high-end data centre environments. At International Game Technology, She held the position of Director, Engineering Operations, where she brought together the centralised functions of engineering and created a centre of excellence in quality management, project management and systems engineering. At Sun Microsystems, she held the position of Director, Software Engineering in both the Chief Technologist’s Office and the Enterprise Systems Software Organization. In those positions, she introduced new verification methods to test the robustness of the products’ system recovery capabilities and led numerous initiatives in the quality efforts for diagnosability, serviceability and fault management.

1 Introduction

Leadership in today’s markets is acquired and sustained through innovative products and services that appear to be available simply to the consumer at will and at a price point suitable for their use (Golovatchev and Budde, 2007; Lee et al., 2005; Onuh et al., 2007). The pressures on suppliers in healthcare, retail, communications, automotive (Sumantran, 2005) and networked services are relentless and capable of driving any particular supplier to leadership or bankruptcy within a single product life cycle. In contrast, the Industrial Revolution provided opportunities for entrenchment in transportation, construction and raw materials where time scales for change and delivery of new products to market were measured in decades. In today’s knowledge, economy time scales are measured in months (Dereli and Baykasoglu, 2008; Ekman and Angwin, 2007; Onuh et al., 2007) or more rarely in a few years, leaving companies unable to respond at this pace and merely

Feasibility of rapid systems engineering framework 47

surviving (Acharya and Mahanty, 2008; Jamali and Keshishian, 2006). The challenge for these companies to apply the rigour and methodology of systems engineering (SE) and yet be able to meet the time-to-market targets, creates an opportunity to research and introduce variations of best practices of the SE disciplines for a rapid environment. Perceptions exist that there are only two choices to implement SE (Eisenhardt and Tabrizi, 1995):

1 implement structured, disciplined and well planned throughout the entire life cycle then evaluate opportunities to compress the activities

2 create an environment that utilizes an intuition experience rather than planning.

These options provide challenges in rapidly changing environments where companies must address the three sides of the ‘iron triangle’ (Ambler, 2008) of cost, quality (Savino et al., 2007), and schedule with equal consideration (Kim, 2007; Rizzi and Regazzoni, 2007). Given the situation, companies should examine what is relevant for their business environment, and then implement the appropriate SE activities with approaches that allow them to achieve the desired targeted cycle times. In Section 2, this paper introduces the definition of rapid systems engineering (RSE) and some common success factors associated with RSE. In Section 3, this paper introduces research and ideas that are relevant to this discussion. The authors have based their views of the challenges on an examination of applied rapid techniques to the SE process defined in ISO 15288. ISO 15288 was used as the baseline process framework since it offers a comprehensive view of the processes key to successful SE. In Figure 1, ISO 15288 is presented showing the use of an iterative methodology for the technical processes and the infrastructure and enabling processes, which are foundational to the project across the bottom. On either side of the iterative diagram are the processes focused on key areas of acquisition, supply chain management and deployment. Given this as an initial focused research on the applicability of rapid techniques for SE, the authors selected only a set of key processes to evaluate shown as the shaded boxes in Figure 1. This provides a starting point to encourage future research in the application of rapid techniques to these and the remaining processes.

In section 4, the authors introduce the next level of research through the creation and administration of a focused survey applicable to the process techniques presented in Section 3. The survey was administered to individuals responsible for the SE processes in both the commercial and government industry segments. The survey asked the participants to evaluate the importance of these techniques, as they are applicable to RSE. Based on the responses, Section 5 provides an overview of the authors’ analysis of the data. Sections 6 and 7 provide the authors’ implications associated with this research and concluding statements including potential research topics to expand the capabilities of RSE.

48 R. Jain et al.

Figure 1 ISO/IEC 15288:2008 (E) evaluation of process rapidity (see online version for colours)

2 What is rapid systems engineering?

While RSE is an evolving term that will mature in scope as SE is implemented in rapid environments, the authors have chosen to define RSE as: adopting methodologies, tools and techniques that can introduce rapidity into the SE process while optimising the success factors during the development of projects. The success factors are specific to the system-of-interest being developed, and depend on the type of system, its related product line and/or platform, organisation structure and processes, and market forces such as competition and window of opportunity, innovation and customers. Some of the common success factors across projects are return-on-investment (ROI) (Oke et al., 2008), cost of ownership, time to market (Muthiah and Huang, 2008), productivity (Muthiah and Huang, 2006) quality (Amuthakkannan et al., 2008) and customer satisfaction. Another way to view RSE is as a set of tools, methodologies and people and management techniques that results in a SE life cycle that reduces the time to market a product, from concept to implementation, without sacrificing the quality of products.

3 Research on rapidity of key processes

In order to identify the scope of SE processes that apply and may be described as being most relevant for implementing rapidity to any development project, the authors selected 14 processes (as illustrated in the shaded boxes in Figure 1), sub-processes and activities from ISO 15288. This section discusses the scope of these SE activities as it applies to rapid development projects. The discussion is based on extensive doctoral-level research on existing literature, provisions of the ISO 15288 standard and the experience of the authors in successfully implementing rapidity of the SE processes on several projects.


3.1 Life cycle model management

Rapidity in SE has to be addressed throughout the life cycle of a system in order to optimise its impact and benefits. A basic pre-requisite for addressing rapidity is that the life cycle processes support smaller and incremental (Conforto and Amaral, 2008; Larman, 2004; Shore and Warden, 2008) development and frequent releases. This technique was originally introduced in software development to respond to demands of shorter time scales at a lower cost while maintaining high-quality standards (Cockburn, 2006; Highsmith, 2004). Although large formal enterprise releases could run as long as 12–18 months, this does not preclude a release cycle of small iterations in the 1–2 week time frame. In the first evolution of rapid development, the initial focus is placed on breaking the project into smaller pieces, the second evolution overlaps in a phased approach and the third evolution shortens the phases into an iteration that is focused on value (Cottmeyer, 2008).

3.2 Human resource management

In order to build a product within the shortened time frames, the organisation of the team and the utilisation of skills is critical (Butcher, 2007; Cockburn, 2006; Alssamadisy, 2008). There are three main functions recommended for a team involved in rapid development activities: the product owner or customer proxy, development team and quality assurance (Alssamadisy, 2008). Customer involvement or a customer proxy is critical to the success of the team in a rapid environment by providing continuous feedback on feature/functionality of the product under development (Shore and Warden, 2008). The development team creates the product according to customer requirements and their feedback. Quality assurance then tests the product (Leffingwell, 2007) to ensure compliance to customer needs and specifications.

Rapid techniques encourage close collaboration among team members creating a shared ownership in the project. As a team, they look for improvements to streamline the process and implement appropriate changes to ensure success. Teams should be made up of a mix of senior and junior people who are risk takers, results oriented and willing to receive feedback on design and delivery. Throughout the life cycle process, there is an evaluation of the skills and competencies necessary to complete the iteration. Within the iteration, everyone is given an opportunity to learn new technologies and become proficient in different areas; however, there may be a point when no one in the team has the particular skill set (Gero and Kannengiesser, 2004). As in any major project, planning and budgeting should allow necessary training to ensure that the skill levels are raised to meet the demands.

3.3 Project planning and project assessment

ISO 15288 presents project planning and project assessment as separate processes. For this paper, the authors combined these processes in their research and evaluation. By definition, rapid techniques employ an iterative and incremental process providing customer value. Traditional project plans would start with a waterfall approach and assume that everything was stable from the requirements input, the team members allocated, to the level of executive support. When the project was finally released, it would meet the needs of all the stakeholders. Unfortunately, these conditions are rare in

50 R. Jain et al.

today’s development environment. Requirements change based on the customer input, environmental changes and executive support. Team members may come and go based on the skill levels and the business needs. Executive support can change based on new business challenges. These realities often create the conditions for project cost and schedule overruns.

The challenge for project managers in today’s environment is to anticipate the changes in conditions and then create plans that strive for predictability within shorter cycles (Cohn, 2006). Iterative and incremental development are not simply a review of overlapping waterfall. Instead, it is designed to focus on delivery of value added functionality by having all the necessary skill sets focused and working in parallel at all times.

At the beginning of the project, a kick-off meeting is held between the product management team and the development team to create the shared vision of the project and develop a project schedule and resource allocation plan divided into a series of iterations defining a sequence of project releases. Every new iteration starts with a planning session which reviews the total feature set, prioritises the feature set, establishes the schedule, identifies the resources, organises the work and gains agreement from the team on deliverables (Leffingwell, 2007). Daily meetings or ‘stand ups’ provide the communication mechanism and keep track of the completed progress. The end of the iteration provides the team the opportunity to demonstrate their success with completed features and a forum for retrospective, which encourages the team to reflect and make the necessary changes to the process, making the next iteration more efficient. The project manager becomes an integrator of the team’s knowledge and the project management processes throughout the project life cycle (Morandotti, 2007).

3.4 Risk management

Risk is a function of the anticipated frequency of occurrence of an undesired event, the potential severity of resulting consequences and the uncertainties associated with the frequency and severity (NASA-STD-8719. 13A, 1997). Risk is a combination of an abnormal event or failure and the consequences of that event or failure to a system’s operators, users or environment (McManus, 2001). Risk management is the practice of identifying, analysing, assessing and acting to eliminate or mitigate risk (Stein et al., 2002). The risk management paradigm shown in Figure 2 (Carr et al., 1993) provides emphasis on the iterative nature of the risk management processes.

Figure 2 Risk management paradigm (see online version for colours)

Source: CMU/SEI-93-TR-6 (1993).


Determining an initial set of known risks and identifying relevant check points throughout the RSE project aids in faster and continuous risk management. Identifying when and where the information about a risk factor becomes critical in SE process helps in doing risk analysis more frequently and in avoiding negligence due to lack of risk analysis (Conrow and Shishido, 1997). Identifying the initial set of known risks upfront and focusing on gathering information on unknown risks increases the risk awareness and saves time throughout the process. As unknown risks emerge are identified, they can be added to knowledge management and then applied to other RSE efforts.

Risk management cannot be performed without measurement. The depth (what extent of accuracy) and the width (how many metrics) of measurement determine how much effort and resources are required to perform risk analysis. In RSE, the role of risk management is to guide in continuous decision making (Schroeder, 2000). The objective is to identify the minimum information that is required to make the best decision (Barki et al., 2001; Browning et al., 2002). The required level of information to make a SE decision can be obtained by using relative or indirect measurement. By doing so, the time taken to estimate measurements will be significantly reduced and still provide adequate information to make good decisions. For example, within the process, there is a need to understand the task completion time to develop a module of software. One approach would be to expend time and effort in accurately analysing and estimating the amount of time to complete. Instead, by using relative or indirect measurements, tasks of equal size and complexity which have been completed are used as the input into the estimation process. This approach provides the estimate much faster with less detailed analysis significantly reducing the time it takes to provide exact measurements.

3.5 Configuration management

Configuration Management (CM) is the process of controlling the evolution of families of systems and provides techniques, methods and procedures to maintain a product history, to uniquely identify and locate each version of a system, and to initiate, evaluate and control change to the product during development and after release (Feiler and Dart, 1990; Mette and Hass, 2003; Stevens et al., 1998). The primary objective of CM is to ensure effective management of the evolving configuration of a system, both hardware and software, during its life cycle (INCOSE, 2006). CM effectiveness can be measured in terms of consistency, stability and traceability (Stevens et al., 1998).

Rapidity in CM can be achieved in a number of ways, from establishing configuration controls to the choice of tools and automation. The main focus of CM, which is also highly critical for RSE, is traceability. Systems need to adapt to the rapid change driven by the global dynamic marketplace, technological evolution and variety of environments (Feiler and Dart, 1990; Fricke and Schulz, 2005) . Traceability of changes is critical in controlling and managing their impact on the SE life cycle (Jain et al., 2008, 2009). Two such common ways of establishing traceability are maintaining functional and physical relationships and tracking attributes of configuration items (Mette and Hass, 2003). They are also the most important among the essential activities in CM for RSE.

Relating and maintaining functions and services to subsystems and components results in better change control and configuration management. When there are changes in services and functions due to the changing customer or market needs, the established traceability between services and functions helps in identifying the right configuration items that are impacted by the changes. The study of the impact of changes based on the

52 R. Jain et al.

CM dependency mappings assists in making informed decision about the feasibility and required level of effort, and thereby the value of the change. This technique enables and supports the successful estimation and prioritisation activities within the project. A configuration item can have any number of attributes associated with it based on the level of detail. Identifying and tracking these attributes are important but also time consuming. The one approach is to optimise the time spent on identifying and tracking the attributes. Service level objectives are attributes that provide the most valuable information and should be tracked within the CM process for RSE.

3.6 Requirements and requirements analysis

The time necessary to identify, define and analyse the stakeholder’s requirements and transform them into system requirements can be significantly reduced by introducing RSE practices in the requirements process. RSE will reduce the time to complete the requirements definition and analysis process by providing flexibility in addressing unidentified or changing stakeholder’s requirements.

Since RSE uses an iterative and incremental development approach, the multiple iterations offer rapid feedback on the progress of the project and the functionalities of the system (Callahan and Moretton, 2001). Frequent interactions with the stakeholders enable rapidity during the requirements definition and analysis process because SE can obtain early feedback from the stakeholder and manage new, unidentified or changing requirements. Continuous stakeholder feedback is an essential element of RSE (De Beer et al., 2009). When using a linear-sequential approach, the project is at risk of taking too long to complete and not meeting the stakeholder needs at the time of delivery, because the original requirements or business needs have fundamentally changed (Maner, 1997). RSE overcomes this risk by obtaining stakeholder feedback at the end of the iteration and by allowing the stakeholder to prioritise the desired functionalities or requirements at the beginning of the next iteration. This ensures that the delivered product satisfies customer needs and reduces the cost associated with rework after the product has been delivered. RSE enforces customer involvement throughout the projects’ life cycle to ensure requirements volatility is managed throughout the project. This can be accomplished by collocating teams. When collocation of the whole team is not easily achieved, then stakeholders’ representatives are collocated with the engineering team to ensure the stakeholder needs are addressed in the iteration and that any misunderstandings about the system requirements are resolved early in the project’s lifecycle (Teasley et al., 2002a).

3.7 Architectural design

In RSE, projects that utilise familiar architectures or small improvements to the architecture, less integration for new technologies or the introduction to the market with familiar technologies can move through the process quickly. Projects that require novel technologies or innovative architectures can introduce uncertainty which may require experimentation, thus creating longer development times (Meyer and Utterback, 1995). To lessen the risk created by novel technologies, rapid prototyping can be used (Andrews and Goeddel, 1994; De Beer et al., 2009). Time boxed research spikes can be used to determine architecture novelty as long as they do not distract the team from completing the project (Isham, 2008).


In addition to prototyping, selection of less complex architectures can speed the development times. Some projects have found that by trading off technology novelty for greater complexity allows for faster time to market (Tatikonda and Rosenthal, 2000). When possible, postpone architectural decisions until the ‘last responsible moment’ allowing an interleaving of development with design creating a just in time architectural process (Krunic, 2007).

3.8 Implementation

In RSE, implementation focuses on developing systems in short time scales, at low cost, while maintaining high standards of quality (Woi Hin, 2006). RSE achieves rapidity in the implementation/development process by allowing the customer to identify and prioritise the functionalities that add more value to the system at the start of the iteration. The basic premise of this model is that it is easier to add capability to a small well-developed product than to build a complete application at once. For customers, managers and developers, this method can offer a potential payoff in minimum defect injection and leakage rates (Constantine, 2001). A technique used originally in the software development process is Test Driven Development. It encourages the development and implementation processes to be driven in a structured methodology using simple designs and continuous delivery of highly reliable functionality (Beck, 2003).

3.9 Integration

Systems integration is an important element of SE, which involves the integration of hardware, software, products, services, business processes and human beings (Grady, 1994; Jain, 2007). The systems integration process is a set of activities that transforms the stakeholder’s requirements into an operational system by unifying the process components and product components into a whole while ensuring compliance to the specified levels of component operations and interoperability (Jain et al., 2010). The scope and objectives of systems integration should be clearly identified to ensure that new systems and components are able to work with the existing systems and components.

Within the process, integration requirements are gathered and analysed along with other requirements ensuring that everything supports the integration aspects of the system. This process supports traceability of requirements and verification that the requirements can be tested or qualified. Prior to the design phase, systems integration evaluations provide guidance to the design team(s) of any constraints and/or challenges for integration that may be determined through the analysis of modelling and simulation. During the design of the system, there are various assumptions about the systems that can have an impact on the integration aspects of the process. Utilising experts to validate the design decisions provides the checks and balances to reduce the possibility of integration issues later forcing a redesign when changes are both time consuming and costly. These approaches can save cost and effort on rework later by reducing the probability of redesign for developed or procured interfaces due to integration defects.

RSE promotes continuous integration whereby components and modules are physically or virtually integrated and tested as they are being developed. As the components and subsystems evolve over a period of time, changes are made to the overall system, where integration of these components or changes is necessary at a system level. A best practice is to perform integration (physically or virtually) as these

54 R. Jain et al.

new developments or changes are made. Continuous integration increases quality of the product and reduces the development risks (Duvall et al., 2007; Holck and Jorgensen, 2003). A well-integrated system will increase the quality throughout the life cycle.

3.10 Verification and validation

Verification and validation (V&V) is a collection of analysis and testing activities across the full life cycle and complements the efforts of other quality-engineering functions (Wallace and Fujii, 1989). Verification establishes the truth of correspondence between a product/system and its specification. The activities in this V&V sub-process verify and validate if the system is being built right and the right system is being built to address the customer needs. The purpose of verification is to confirm that the specified design requirements are fulfilled by the system (ISO/IEC – 15288, 2008). Validation establishes the fitness of a product/system for its operational mission based on operational or field testing. The purpose of the validation is to provide objective evidence that the services provided by a system when in use comply with stakeholders’ requirements (ISO/IEC – 15288, 2008). This process performs a comparative assessment and confirms that the stakeholders’ requirements are correctly defined. All V&V activities should be traced back to the system requirements (Jain et al., 2008).

A systematic approach using different testing methods to predict the defects and performance is proposed by Amuthakkannan et al. (2008) to improve the quality of real-time software by identifying the defects in software product and improving the development processes. Qualification requirements address the needs to qualify the system as being designed right, the right system and an acceptable system (Beude, 2000). The four elements of the qualification requirements (Beude, 2000) are:

1 observance – how the estimates (qualification data) for each input/output and system-wide requirements will be obtained, that is, test analysis and simulation, inspection or demonstration

2 verification plan – how the qualification data will be used to determine that the real system conforms to the design that is developed

3 validation plan – how the qualification data will be used to determine that the real system complies with the originating requirements

4 acceptance plan – how the qualification data will be used to determine that the real system is acceptable to the stakeholders.

RSE builds in verification activities in an iterative method throughout the life cycle of the project to confirm the conformity and identify gaps between system specifications and the final systems (Forte, 1997; Larman, 2004). By doing continuous verification activities, errors and defects are found in a timely manner and contained in the initial stages thus reducing the cost to fix (Boehm and Basili, 2001; Crispin and Gregory, 2009; Gilb and Graham, 1993; Leffingwell, 2007). Risk-based decisions on acceptable deviation and effort of rework can be made at the right time and can be followed through iteratively. Figure 3 presents a sample iterative and adaptive testing process using a design structure matrix.


Figure 3 Iterative vs. traditional verification and validation process shown using design structure matrix

Source: Steward (1981), Browning (2001) and Levardy and Browning (2005).

In RSE, validation is performed through continuous and frequent prototyping (McConnell, 1996) with customer involvement early in the life cycle. This helps in timely validation that the developed or to be procured system is exactly what the customers want (Buxton, 2007; Grimm, 2004; Noorani, 2006). The general challenge of miscommunication of a conceptualised system between customers and engineers is well addressed. Prototypes also help in customers getting familiarised with the system as it evolves (Venkateswaran and Son, 2004). In RSE, identifying constraints and expected results would enable rapidity in validation for confirming the system to achieve its goals within the intended operational environment. Inadequate input from the design activities or use of the wrong verification and validation methods can cause unnecessary rework and retest (Levardy and Browning, 2005). The optimal information flow means that the right verification and validation activities deliver the right information using the right input data from the design process at the right time and place in a most effective and efficient manner.

3.11 Transition

The purpose of the RSE transition process is to establish a capability to provide services specified by stakeholder’s requirements in the operational environment. The activities conducted during this process verify the system’s readiness for deployment as well as the readiness of enabling systems, for example, operating systems, support systems, operator training systems, user training system, etc.

To introduce rapidity in the transition process, RSE relies on an iterative approach, continuous testing, integration and customer involvement and feedback through iterative or rapid prototyping. Deployment and operational requirements are continuously revised as the project progresses. RSE techniques ensure changes in deployment requirements are effectively managed. Rapid prototyping is used to obtain early systems validation and accelerated product development (Andrews and Goeddel, 1994). The most obvious benefit of this technique is the ability to evaluate requirements for applicability and unanticipated errors. By using iterative prototypes, RSE can monitor the rate of change of deployment and operational requirements and modify the system under development to satisfy the new requirements. The customer needs and desires may change when they begin operating the system and gain a deeper understanding of how it will support their

56 R. Jain et al.

mission (Boehm, 2000). Thus, requirements tend to emerge with continued use and mission understanding rather than be pre-specifiable. RSE techniques reduce total project costs because the delivered product tends to be more stable since most of the errors have been already discovered and fixed in early iterations of the system.

4 Research methodology

Based on the literature research, the authors have included and discussed within the scope of this paper, 14 SE processes from the ISO 15288 standard and 2 relevant rapid implementation techniques for each SE process evaluated. To enable a more focused research activity, some processes were combined, such as human resources and life cycle management, thus reducing the total number of process focus areas from 14 to 11. The 11 process combinations and techniques are described in Table 1. Based on this information, the authors created a survey to evaluate the applicability of these techniques in RSE projects. The survey was administered to individuals within both the commercial and defence sectors where respondents were asked to rate each technique statement on a scale of 1–5 (1 being highly disagree and 5 being highly agree) with regard to enabling rapidity within the SE life cycle. The survey consists of 22 questions along with an introduction on the meaning of RSE and a few questions on demographics of the respondents. One survey question was a false statement – a rapid SE technique which would not have relevance in application of rapidity. The authors’ expectation was that all respondents would disagree with the statement.Table 1 Techniques by process

Process Techniques Human resources and life cycle management

Create a team with a combination of disciplines and creative/innovative flexibility Plan for an increased number of release cycles that include customer approved value-added functionality

Project planning and project assessment

Develop project plans and update them continuously throughout the life cycle At inception, plan for scope and project definition changes throughout the project life cycle

Risk management Determine the initial set of known risks and identification of relevant checkpoints throughout the project life cycle Use relative and derived metrics instead of direct metrics

Configuration management Relate and maintain functions and service to subsystems and components Define service level objectives that provide the minimum set of attributes that need to be tracked and maintained

Requirements definition and requirements analysis

Provide mechanisms for initial and continuous collaboration from the customer and stakeholders Use iterative prototypes to manage project dynamism


Table 1 Techniques by process (continued)

Process Techniques Architectural design Use of architectural modelling to analyse critical technical

issues and evaluate alternatives Utilise prototypes to allow the best architectures to evolve iteratively

Implementation Selection of methods, tools and techniques that eliminate waste Use an incremental development approach to break-up large functions into smaller manageable pieces focusing on core and highly prioritised functions initially

Integration Place focus on integration early in the life cycle to evaluate and analyse designs which may create integration issues Continuously integrating and testing components and modules physically or virtually to support traceability will result in cost, time and risk containment

Verification Perform integration activities throughout the project to confirm conformity between system specifications and the final system Use an iterative verification process to identify gaps between the system specifications and the final system functionality early in the life cycle.

Transition Ensure the readiness of the system and the readiness of the enabling systems , for example, operating system, support system, operator training, user training, etc. Identify and include deployment requirements in the requirements phase

Validation Use frequent prototypes with customers to validate the system will meet the objectives Identify constraints and expected results to confirm the system can achieve its intended use in its intended operational environment

5 Research analysis and survey results

The survey was administered to 22 respondents representing both the commercial and defence sectors. The respondents experience ranged from 9 to 31 years creation and implementation of systems in their industry. The average experience across the respondents was 20 years with an average gap (SD) of 8 years from the mean. Of the 22 techniques, the respondents agreed that 85% were applicable to RSE as seen in Figure 4. As a second level of decomposition, the total responses were then categorised by three process areas:

1 infrastructure processes which include human resource and life cycle management

2 enabling processes which include project planning, project assessment, risk management and configuration management

58 R. Jain et al.

3 technical processes which include requirements definition, requirements analysis, architectural design, implementation, integration, verification, transition and validation.

Figure 5 presents the data in terms of percentages of strongly agree/agree, neutral and disagree/strongly disagree against the three process areas. The data presents the respondents belief that the techniques identified for the technical processes as most applicable while the enabling processes were believed to be least applicable when implementing rapidity into the SE process. The survey results indicated a clear agreement of 21 out of 22 techniques when applied in rapid environments supporting ISO 15288.

The authors then calculated mean and SD using weighted averages ( 2 to 2 for Strongly Disagree to Strongly Agree) to determine any significant differences between the individual techniques. The weighted means and SD can be reviewed in Figure 6. It illustrates the respondents’ level of agreement to the techniques where the closer to the centre the higher the agreement (indicated by a high Mean score) to the technique – the smaller was the difference in their level of agreement (indicated by a small SD) across all respondents. This is an extremely important indication of two things:

1 the respondents’ support for the selection of the relevant SE processes and techniques

2 that the SE processes and techniques that were rated as relevant for achieving rapidity had very strong and consistent support across the respondents.

Figure 4 Percentage of agreement of techniques applicable to RSE (see online version for colours)

Figure 5 Percentage of agreement within processes (see online version for colours)


Figure 6 Survey mean and SD of techniques to RSE

The findings show six rapid techniques representing four SE processes to have a weighted means of 6.2 and above, therefore being the most relevant. Those processes and techniques are found in Table 2. The respondents clearly identified upfront and early focus on systems integration issues, continuous and build in verification activities throughout the life cycle of the project and an iterative process in the integration and verification activities to be very suitable and relevant for implementing rapidity in SE.

There were two SE processes (integration and verification), where the responses to two RSE techniques:

1 continuously integrate and test components and modules physically or virtually to support traceability will result in cost, time and risk containment

2 perform verification activities throughout the life cycle of the project to confirm conformity between the system specification and the final system, varied the most with the highest SD.

This could be a reflection of the diversity of the industry domains represented in this exploratory study. The perception of integration and verification activities can be very different in the defence and non-defence (commercial) domains. Moreover, since these SE activities are extremely critical in the life cycle considerations, respondents tended to have strong views on them, thereby leading to significant differences.

The survey respondents only somewhat agreed (neutral) to the use of relative and derived metrics instead of exact and direct metrics, for reducing the time and resources spent on risk management. This could be explained by the need for the projects to have accuracy in risk management through more direct and exact metrics. Most projects

60 R. Jain et al.

following RSE perform detailed management of issues arising out of oversight in risk planning and management. Also in some of the RSE projects, the nature of the project requires close to 100% accuracy in determining the risks. In projects where human lives depend on the product risks, performing a relative metric risk assessment will not be an option. Hence, this statement cannot be generalised for all RSE projects.

Table 3 presents the four rapid SE techniques associated with the three SE processes that rated the least relevant in their application to the SE life cycle. The one that created the biggest surprise was the technique to plan for scope and project definition changes. This question was worded in the negative to create additional thought in the answers. Instead, a majority of the respondents rated it as not relevant with 13 responding as expected and 9 responding either neutral or negative. Based on the weighted means, the two techniques that had the lowest means and the lowest SD were

1 plan for an increased number of release cycles that include customer approved value-added functionality

2 use relative and derived metrics instead of direct metrics. Table 2 Most relevant techniques for rapidity based on the weighted means of the survey

Process Techniques

Requirements definition and requirements analysis

Provide mechanisms for initial and continuous collaboration from the customer and the stakeholders.

Implementation process Use an incremental development approach to break up large functions into smaller manageable pieces focusing on core and highly prioritised functions initially Continuously integrate and test components and modules physically or virtually to support traceability will result in cost, time and risk containment

Verification process Continuously integrate components and modules as they are developed Use an iterative verification process to identify gaps between the system specifications and the final system functionality early in the life cycle

Table 3 Least relevant techniques applied to rapid systems engineering processes

Process Techniques

Human resources and life cycle management

Plan for an increased number of release cycles that include customer approved value-added functionality

Quality management and project management

Develop project plans and update them continuously throughout the life cycle At inception, plan for scope and project definition changes throughout the project life cycle

Risk management Use relative and derived metrics instead of direct metrics


6 Discussions and implications

Some of the survey participants indicated that they found it difficult to answer the questions in terms of RSE since they were not familiar with the concept. They expressed it would have been easier for them if the questions were expressed in terms of reducing time to market, for example, some respondents indicated that the practices proposed by RSE are good SE practices that should be implemented in any SE project, therefore they agreed with most of the questions. The majority of the respondents commented that although they agree with the fact that the scope of the project should change throughout the life cycle of the project to allow rapidity, they did not like the question because it was not compatible with the SE processes used in their working environments.

7 Conclusion

While this exploratory study indicates a good beginning towards application of some time-tested SE processes in a rapid environment, there is caution in drawing conclusions from its limited scope. The strength of the paper comes from its empirical basis in credible research and an attempt to design a framework for application of rapidity in SE processes. It also provides a common repository of techniques supporting the SE processes within the ISO 15288 standard. Many of the outcomes in this survey could be skewed due to interpretation of the term RSE, structure of the questions and also the familiarity of the SE processes as described in the 14 that were covered in this study. Our findings generally indicate that activities related to technical processes are viewed as relevant techniques for implementing rapidity. Some aspects of the management processes as enablers were almost unanimously viewed with skepticism in assigning priority in fast-paced and resource critical rapid environments and were also considered as overheads and not essential components of rapidity. These provide an initial guidance on areas of SE where rapidity can be implemented and those where skepticism in general on the benefits of these SE processes for rapid development will have to be first removed. Further work needs to be done to initiate some larger studies addressing more specific questions on the role of SE in rapid environments.

Areas of future research

Based on the initial analysis, the following areas provide additional research opportunities:

development and administration of a more refined survey that crosses multiple industry and project boundaries with domains focused on RSE opportunities and consequences

systems analysis techniques that support RSE

62 R. Jain et al.

consequences of design decisions on deployment (certain design decisions may enable rapidity through the early phases of the life cycle, yet introduce an increase of cost in the deployment and sustaining processes)

supportability, the cost of ownership long term, training and implications of technology changes required to sustain a RSE process.

References Acharya, P. and Mahanty, B. (2008) ‘Effect of business growth on software project management

issues in Indian IT industry’, Int. J. Industrial Systems Engineering, Vol 3, No. 4, pp.407–422. Ambler, S. (2008) The “Broken Iron Triangle” Software Development Antipattern, Viewed on 28

April 2009, Available at: http://www.ambysoft.com/essays/brokenTriangle.html. Amuthakkannan, R., Kannan, S.M., Selladurai, V. and Vijayalakshmi, K. (2008) ‘Software quality

measurement and improvement for real-time systems using quality tools and techniques: a case study’, Int. J. Industrial and Systems Engineering, Vol. 3, No. 2, pp.229–256.

Andrews, B. and Goeddel, W. (1994) ‘Using rapid prototypes for early requirements validation’, IEEE/AIAA Digital Avionics Systems Conference Proceedings, IEEE, Phoenix, AZ.

Barki, H., Rivard, S. and Talbot, J. (2001) ‘An integrative contingency model of software project risk management’, Journal of Management Information Systems, Vol. 17, No. 4, pp.31–69.

Beck, K. (2003) Test-Driven Development By Example, Boston, MA: Addison-Wesley. Beude, D. (2000) The Engineering Design of Systems, New York, NY: John Wiley. Boehm, B. (2000) ‘Requirements that handle IKIWISI, COTS, and rapid change’, IEEE Computer-

Software Management, Vol. 33, No. 7, pp.99–102. Boehm, B. and Basili, V. (2001) ‘Software defect reduction Top 10 list’, IEEE Computer-Software

Management, Vol. 34, No. 1, pp.135–147. Browning, T. (2001) ‘Applying the design structure martix to system decomposition and

integration problems: a review and new directions’, IEEE Transactions on Engineering Management, Vol. 48, No. 3, pp.292–306.

Browning, T., Deyst, J., Eppinger, S. and Whitney, D. (2002) ‘Adding value in product development by creating information and reducing risk’, IEEE Transactions on Engineering Management, Vol. 49, No. 4, pp.443–458.

Butcher, T. (2007) ‘Supply chain knowledge work: should we restructure the workforce for improved agility’, Int. J. Agile Systems and Management, Vol. 2, No. 4, pp.376–392.

Buxton, B. (2007) Sketching User Experiences: Getting the Design Right and the Right Design, 1st edition, San Francisco, CA: Morgan Kaufmann.

Callahan, J. and Moretton, B. (2001) ‘Reducing software product development time’, Int. J. Project Management, Vol. 19, No. 1, pp.59–70.

Carr, M., Konda, S., Monarch, I., Ulrich, F. and Walker, C. (1993) ‘Taxonomy based risk identificatioon’, Technical Report, CMU/SEI-93-TR-6: Pittsburgh. PA.

Cockburn, A. (2006) Agile S/W Development: The Cooperative Game, 2nd edition, Boston, MA: Addison-Wesley.

Cohn, M. (2006) Agile Estimating and Planning, 1st edition, Upper Saddle River, NJ: Pearson Education.

Conforto, E.C. and Amaral, D.C. (2008) ‘Evaluating an agile method for planning and controlling innovative projects’, Project Manaement Journal, Vol. 9999, No. 9999, Unreleased (Evaluation).

Conrow, E. and Shishido, P. (1997) ‘Implementing risk management on software intensive projects’, IEEE Software, Vol. 14, No. 3, pp.83–89.


Constantine, L. (2001) Methodological Agility, Viewed on 30 April 2009, Available at: http://www.ddj.com/architect/184414743.

Cottmeyer, M. (2008) The Evolution of a Project Schedule: Agile Development Practices,Orlando, FL, Viewed on 19 May 2009, Available at: http://blog.versionone.net/blog/2008/11/heading-to-orlando-for-agile-development-practices.html.

Crispin, L. and Gregory, J. (2009) Agile Testing: A Practical Guide for Testers and Agile Teams,1st edition, Boston, MA: Addison-Wesley.

De Beer, D., Campbell, R., Truscott, M., Barnard, L. and Booysen, G. (2009) ‘Client-centered design evolution via functional prototyping’, Int. J. Product Development, Vol. 8, No. 1, pp.22–41.

Dereli, T. and Baykasoglu, A. and Buyukozan, G. (2008) ‘An affordable reverse engineering framework for innovative rapid product development’, Int. J. Industrial and Systems Engineering, Vol. 3, No. 1, pp.31–37.

Duvall, P., Matyas, S. and Glover, A. (2007) Continuous Integration: Improving Quality and Reducing Risk, 1st edition, Boston, MA: Addison-Wesley.

Eisenhardt, K.M. and Tabrizi, B.N. (1995) ‘Accelerating adaptive processes: product innovation in the global computer industry’, Administrative Science Quarterly, Vol 40, No. 1, pp.84–110.

Ekman, A. and Angwin, D. (2007) ‘Industry patterns of agility: a study of the role of information systems and infromation technology as an antecednt of strategic agility within European organizations’, Int. J. Agile Systems and Management, Vol. 2, No. 4, pp.360–375.

Alssamadisy, A. (2008) ‘Self organizing teams’, Agile Adoption Patterns: A Roadmap to Organizational Success (1st ed.). Boston, MA: Addison-Wesley.

Feiler, P. and Dart, S. (1990) Software Configuration Management: Advances in Software Development Environments, Viewed on 30 April 2009, Available at: http://www.sei.cmu.edu/ legacy/scm/abstracts/absscm_in_sde.html.

Forte, G. (1997) ‘Managing change for rapid development’, IEEE Software, Vol. 14, No. 2, pp.120–122.

Fricke, E. and Schulz, A. (2005) ‘Design for changeability (DfC): principles To enable changes in systems throughout their entire lifecycle’, Systems Engineering Journal, Vol. 8, No. 4, pp.342–359.

Gero, J.S. and Kannengiesser, U. (2004) ‘Modelling expertise of temporary design teams’, Journal of Design Research, Vol. 4, No. 2. http://mason.gmu.edu/~jgero/publications/2004/04Gero KannengJnlDR.pdf.

Gilb, T. and Graham, D. (1993) Software Inspections, 1st edition, London, England: Addison-Wesley.

Golovatchev, J.D. and Budde, O. (2007) ‘Next generation PLM – an integrated approach to product development in the service industry’, Proceedings of the International Conference on Product Lifecycle Management PLM’07, Inderscience, Stezzano, Italy, 11–13 July.

Grady, J.O. (1994) Systems Integration, 1st edition, Boca Raton, FL: CRC Press. Grimm, T. (2004) User’s Guide to Rapid Prototyping, 1st edition, Deerborn, MI: Society of

Manufacturing Engineers. Highsmith, J. (2004) ‘Guiding principles: customers and products’, Agile Project Management:

Creative Innovative Products, 1st edition, Boston, MA: Pearson Education. Holck, J. and Jorgensen, N. (2003) ‘Continuous integration and quality assurance: a case study of

two open source projects’, Australian Journal of Information Systems, Vol. 11, No. 1, pp.40–53.

INCOSE (2006) Systems Engineering Handbook V.3, 3rd edition, Seattle, WA: International Council on Systems Engineering.

Isham, M. (2008) ‘Agile architecture IS possible – you first have to believe!’, Agile 2008 Conference, Toronto, Canada.

64 R. Jain et al.

ISO/IEC – 15288 (2008) ‘Systems and software engineering – systems lifecycle process’, Standard, ISO/IEC 15288:2008, ISO/IEEE, Piscataway, NJ.

Jain, R. (2007) ‘SYS systems integration course notes’, SDOE 605, Systems Integration, Hoboken, NJ: Stevens Institute of Technology.

Jain, R., Chandrasekaran, A., Elias, G. and Cloutier, R. (2008) ‘Exploring the impact of systems architecture and systems requirements on systems integration complexity’, IEEE Systems Journal, Vol. 2, No. 2, pp.209–223.

Jain, R., Chandrasekaran, A. and Erol, O. (2009) ‘A framework for end-to-end approach to systems integration’, Int. J. Industrial and Systems Engineering, Vol. 4, No. 6, TBD, accepted for publication.

Jain, R, Chandrasekaran, A and Erol, O (2010) ‘A systems integration framework for process analysis and improvement’, SE Journal, Vol. 13, No. 3, pp.TBD.

Jamali, D. and Keshishian, T. (2006) ‘Agility, knowledge management and organisational learning: synergies and inter-relationships’, Int. J. Agile Systems and Management, Vol. 1, No. 2, pp.133–145.

Kim, S.J. (2007) ‘Introduction to PLM of Hyundai motor company’, Proceedings of the International Conference on Product Lifecycle Management PLM’07, Inderscience, Stezzano, Italy, 11–13 July.

Krunic, V. (2007) ‘Agile architecture – changing application servers’, Agile 2007,Washington, DC.

Larman, C. (2004) Agile and Iterative Development : A Manager’s Guide, 1st edition, Boston, MA: Addison-Wesley.

Lee, C.K.M., Lau, H.C.W. and Yu, K.M. (2005) ‘A generic model to support rapid product development: an XML schema approach’, Int. J. Product Development, Vol. 1, Nos. 3/4, pp.323–340.

Leffingwell, D. (2007) Scaling Software Agility: Best Practices for Large Enterprises, 1st edition, Boston, MA: Addison-Wesley.

Levardy, V. and Browning, T. (2005) ‘Adaptive test process – designing a project plan that adapts to the state of a project’, Proceedings of the 15th Annual International Symposium of INCOSE,INCOSE, Rochester, NY.

Maner, W. (1997) Rapid Application Development, Viewed on 5 December 2008, Available at: http://csweb.cs.bgsu.edu/maner/domains/RAD.htm.

McConnell, S. (1996) Rapid Development – Taming Wild Software Schedules, 1st edition, Redmond, WA: Microsoft.

McManus, J. (2001) ‘Risks in software projects’, Management Services, Vol. 45, No. 10, pp.6–10, Viewed on 29 April 2009, Available at: http://www.highbeam.com/doc/1P3-86017068.html.

Mette, A. and Hass, J. (2003) Configuration Management Principles and Practice, 1st edition, Boston, MA: Addison-Wesley.

Meyer, M. and Utterback, J. (1995) ‘Product development cycle time and commercial success’, IEEE Transactions on Engineering Management, Vol. 42, No. 4, pp.297–304.

Morandotti, D. (2007) ‘PLM design and delivery models: key issues and lessons learned from projects on the field’, Proceedings of the International Conference on Product Lifecycle Management PLM’07, Inderscience, Stezzano, Italy.

Muthiah, K.M. and Huang, S.H. (2006) ‘A review of literature on manufacturing systems productivity measurement and improvement’, Int. J. Industrial Systems Engineering, Vol. 1, No. 1, pp.461–484.

Muthiah, K.M. Huang, S.H. (2008) ‘Overall cycle time effectiveness metric for system-level service performance measurement and bottleneck detection’, Int. J. Industrial Systems Engineering, Vol. 3, No. 5, pp.594–609.


NASA (1997) ‘NASA-STD-8719. 13A’, Technical Standard, Safety and Risk Management Division, NASA, 8719A. 13A, Office of the Associate Administrator for Safety and Mission Assurance, Washington, DC.

Noorani, R. (2006) Rapid Prototyping: Principles and Applications, 1st edition, Hoboken, NJ: John Wiley.

Oke, S.A., Oyedokun, I.O., Akanbi, G.O. and Oyawale, F.A. (2008) ‘Development of a performance measurement system for manufacturing systems’, Int. J. Industrial Systems Engineering, Vol. 3, No. 4, pp.498–513.

Onuh, S., Popov, I. and Bennett, N. (2007) ‘Trends in agility for rapid product development and manufacturing – a review’, Int. J. Agile Systems and Management, Vol. 2, No. 2, pp.135–146.

Rizzi, C. and Regazzoni, D. (2007) ‘Conceptual design knowledge management in a PLM framework’, Proceedings of the International Conference on Product Lifecycle Management PLM’07, Inderscience, Stezzano, Italy, 11–13 July.

Savino, M., Nicchiniello, G., Bouras, A. and Vigilante, L. (2007) ‘Toward a structured approach for the integration of lifecycle requirements in quality management systems’, Proceedings of the International Conference on Product Lifecycle Management PLM’07, Inderscience, Stezzano, Italy.

Schroeder, K. (2000) Continuous Stategy: Early Lessons on Accelerating Strategic Decision Making, Corporate Strategy Board, Corporate Executive Board, Washington, DC.

Shore, J. and Warden, S. (2008) The Art of Agile Development, 1st edition, Sebastopol, CA: O’Reilly.

Stein, J., Freeman, K., Bastable, M., Jacobs, G. and Bedocs, J. (2002) An Integrated System Life Cycle Based Risk Management Methodology, Viewed on 8 December 2008, Available at: http://www.sae.org/technical/papers/2002-01-0145.

Stevens, R., Brook, P., Jackson, K. and Arnold, S. (1998) Systems Engineering, Coping with Complexity, 1st edition, Upper Saddle River, NJ: Prentice-Hall.

Steward, D. (1981) ‘Design structure system: a method for managing the design of complex systems’, IEEE Transactions on Engineering Management, Vol. 28, No. 3, pp.71–74.

Sumantran, V. (2005) ‘Accelerating product development in the automobile industry’, Int. J. Manufacturing Technology and Management, Vol. 6, Nos. 3/4, pp.361–371.

Tatikonda, M. and Rosenthal, S. (2000) ‘Technology novelty, project complexity, and product development project execution success: a deeper look at task uncertainty in product innovation’, IEEE Transactions on Engineering Management, Vol. 47, No. 1, pp.74–87.

Teasley, S., Covi, L., Krishnan, M. and Olson, J. (2002a) ‘Rapid software development through team collocation’, IEEE Transactions On Software Engineering, Vol. 28, No. 7, pp.671–683.

Venkateswaran, J. and Son, Y-J. (2004) ‘Design and development of prototype distributed simulations for evaluation of supply chains’, Int. J. Industrial Engineering: Theory Applications and Practice, Vol. 11, No. 2, pp.151–160.

Wallace, D.R. and Fujii, R.U. (1989) ‘Software verification and validation: an overview’, IEEESoftware, Vol. 6, No. 3, pp.10–17.

Woi Hin, K. (2006) ‘Future implementation and integration of agile methods in software development and testing’, Proceedings of the Conference on Innovations in Information Technology, 2006, IEEE, Dubai, 19–21 November, 2006.

feasibility of a rapid systems engineering framework: …jainra/research_papers/r55.pdffeasibility...

Documents