seven systems engineering myths and the corresponding realities

Copyright Joseph Kasser 2010 1

Seven systems engineeringmyths and the corresponding

realitiesJoseph E. Kasser

Visiting Associate ProfessorTemasek Defence Systems Institute

National University of SingaporeEmail [email protected]


Topics

The state of systemsengineering 2010

Seven systems engineeringmyths


INCOSE Fellows Briefing on SEBoK, July 2009

Systems engineering

Means whatever the speaker intends Subjective, no anchor points Endless pronouncements of positions

Overlaps with other ways of doing things Management, design, problem solving, OR

Confusing information in text books Opinions of authors (mine included)

Poorly practiced Defects in paradigm Promises big things Reports of successes and failures Snake oil salesmen?


State of Systems Engineering atINCOSE IS Academic Forum 2009

Electrical engineering before Ohm’s law waspostulated

Electrical engineering before Maxwell’sequations were stated engineers built motors by winding coils but had no

theory upon which to predict the behaviour of themotor before powering it up for the first time

Chemistry before the periodic table of elementswas discovered

Medicine in the 1800’s before medical science provided a theory of why

some medications work and why some don’t


The discipline

Systems engineering is in its early stages. A discipline in these stages is characterized

by debates based on subjective opinions participants talking past each other a lack of listening contradictory and confusing information a number of myths



Myth 1: There are Standards for systems engineering Myth 2: The “V” model of the systems engineering

process Myth 3: Follow the systems engineering process and all

will be well Myth 4: Complexity needs new tools and techniques Myth 5: Systems of systems are a different class of

problem and need new tools and techniques Myth 6: Changing requirements are a cause of project

failure so get the requirements up front Myth 7: The single systems engineering process


Bear with me: helicopter view

Systems engineers at work in

“Else” is doing hersystems

engineering here


The Hitchins-Kasser-Massie Frameworkfor understanding systems engineering*

* Kasser and Massie, 2000


Myths of systems engineering

Myth There are Standards for systems engineering

Reality There are no such Standards Standards cover

Process for engineering systems different parts of the process

Engineering Management Moreover, Standards focus on wrong aspect

MIL-STD’s freely available at http://www.everyspec.com


499 Systems engineeringmanagement

Purpose to develop aSystems EngineeringManagement Plan Not to do systems engineering

Two templates Generally not tailored

MIL-STD-299A SystemsEngineering Management


EIA-632

Process forengineering asystem

Not process forsystemsengineering


IEEE-1220

Management of thesystems engineeringprocess

Not doing systemsengineering

The systems engineering process providesa focused approach for productdevelopment that attempts to balance allfactors associated with product life cycleviability and competitiveness in a globalmarketplace.”


ISO-IEC 15288

Systems EngineeringProcess

Purchase price* CHF 168,000

Current version15288:2008

Revised from 2002version

* http://www.iso.org/iso/catalogue_detail?csnumber=43564


Committed costs vs. Lifecycle

DAU, 1993 quoted in INCOSE Systems Engineering Handbook 3.1 (2nd Printing) modified


Generic perspective:Common content of standards?

Common content?


Focus of Standards –chronological perspective

Based on Table 5 in Honour E.C., Valerdi R., “Advancing an Ontology for SystemsEngineering to Allow Consistent Measurement”, CSER 2006

Conceptualizing problem andalternative solutions

No

IEEE-1220

No

ANSI/ EIA632

Verification & validation

Technical management/leadership

Technical analysis

System implementation

System architecting

Requirements engineering

Mission/purpose definition

SE Categories

No

ISO-15288CMMI

No

No

MIL-STD-499C

No NoNoNoNo



Myth 1: There are Standards for systems engineering. Myth 2: The “V” model of the systems engineering






The “V” Model

18


Defects in the V Model

Lacks ‘prevention of defects*’ Definition of successful test? Design works from requirements T&E work from the need T&E identify defects and plan to find them

after they have been built into the system Why not prevent the defects?

Does not cope with change* Kasser, J. E., "Eight deadly defects in systems engineering and how to fix them ", Proceedings of the17th International Symposium of the INCOSE, San Diego, CA, 2007. 19


Waterfall representation ofseries of activities

Design

Requirements

Test

Operate

Implement

Redraw Waterfallmoving these

blocks up

20


Waterfall representation in Vformat

Design

Requirements

Implement

Test

Operate

Shows functionalrelationships

V is a representation of the Waterfall model,

Does not cope with change

V is forView

[not processmodel]

21




process Myth 3: Follow the systems engineering process and

all will be well Myth 4: Complexity needs new tools and techniques Myth 5: Systems of systems are a different class of




Focus on systemsengineering process

The successful implementation of proven,disciplined systems engineering processes resultsin a total system solution that is-- Robust to changing technical, production, and operating

environments; Adaptive to the needs of the user; and Balanced among the multiple requirements, design

considerations, design constraints, and programbudgets.*

A single process, standardizing the scope, purposeand a set of development actions, has beentraditionally associated with systems engineering.**

* United States Department of Defense 5000 Guidebook 4.1.1** Arnold, 2000 quoting (MIL-STD-499B, 1993) and (IEEE 1220, 1998)


Which process- 632?

Systems Analysis& Control

• Analyse Missions & Environments• Identify Functional Requirements• Define/Refine Performance & Design

Constraint Requirements

Functional Analysis/Allocation• Decomposition to Lower-Level Functions• Allocate Performance & Other Limiting

Requirements to Lower-Level Functions• Define/Refine Functional Interfaces (Internal/External)• Define/Refine Functional Architecture

Synthesis• Transform Architectures (Functional to Physical)• Define Alternative Product Concepts• Define/Refine Physical Interfaces (Internal/External)• Define Alternative Product & Process Solutions

• Select Preferred Alternatives• Trade-off Studies• Effectiveness Analysis• Risk Management• Configuration Mgmt• Interface Management• Data Management• Performance Based Progress• Performance Measurement

–SE Master Schedule–Tech Perf Measurement–Technical Reviews

Verification

Requirements Loop

Design Loop

Requirements Analysis

Process Input

PROCESS OUTPUT

Systems Analysis& Control

• Analyse Missions & Environments• Identify Functional Requirements• Define/Refine Performance & Design

Constraint Requirements

Functional Analysis/Allocation• Decomposition to Lower-Level Functions• Allocate Performance & Other Limiting

Requirements to Lower-Level Functions• Define/Refine Functional Interfaces (Internal/External)• Define/Refine Functional Architecture



• Select Preferred Alternatives• Trade-off Studies• Effectiveness Analysis• Risk Management• Configuration Mgmt• Interface Management• Data Management• Performance Based Progress• Performance Measurement

–SE Master Schedule–Tech Perf Measurement–Technical Reviews

Verification

Requirements Loop

Design Loop

Requirements Analysis

Process Input

PROCESS OUTPUT


Which process- 1220?


Which process- DERA?

O pe ratio na lsy ste m

S u p p liedsystem

C o m p o n en treq u irem en ts

C o m p o n en td es ig n

C o m p o n en tb u ild & tes t C o m p o n en ts

S u p p liedco m p o n en ts

L o ca lreq u ire m e n ts& c o n s tra in ts

a 11 1

P ro p o se dc h a ra c ter is tic s

A llo ca te dreq u ire m e n ts

In sta lla tio n &v alid atio n

U s erre q u ire m en ts

d e fin itio n

S ys te mre q u ire m en ts

d e fin itio nA rc h ite ctu ra l

d e sig nL o ca l

req u ire m e n ts& c o n s tra in ts

In te g ratio n &v erific atio n

A llo ca te dreq u ire m e n ts

In te gra tedsy ste m

P ro p o se dc h a ra c ter is tic s

P rob le m A p prec ia tion

S o lu tio n D eve lop m en t

S o lu tio n A bstrac tion S o lu tio n R ea lisa tion


Blanchard and Fabrycky’ssystems engineering functions

Blanchard and Fabrycky, Systems Engineering and Analysis 1981

Functional view not a process.Note as a process time seems to be running

backwards?

27


The Systems Engineering Process from A. T. Bahill and B. Gissing, Re-evaluating systemsengineering concepts using systems thinking, IEEE Transaction on Systems, Man andCybernetics, Part C: Applications and Reviews, 28 (4), 516-527, 1998.http://www.incose.org/practice/fellowsconsensus.aspx

Terry Bahill’s systemsengineering functions

Functional view not a process.Note as a process time seems to be running backwards?

28

SIMILAR – acronym from first letter of each box


Two external perspectives:The problem solving process

1. Problem Definition*2. Problem Analysis.3. Generating possible

Solutions.4. Analyzing the Solutions.5. Selecting the best

Solution(s).6. Planning the next course of

action (Next Steps)

1. Identify and Select theProblem**

2. Analyze the Problem3. Generate Potential Solutions4. Select and Plan the Solution5. Implement the Solution6. Evaluate the Solution

* http://www.gdrc.org/decision/problem-solve.html (accessed 11 Jan 2009)

** http://www.c-pal.net/course/module3/pdf/Week3_Lesson21.pdf (accessed 11 Jan 2009)


Something’s wrong here

The systems engineering processseems to be the problem solvingprocess Semantics - levels of detail in each step

differ Similar process steps in other

professions Were they doing

Systems engineering, or problem solving?


What systems engineeringprocess?

Each process seems to be different Some overlap the problem solving process

Mar, Hitchins, etc. Some cover the whole system lifecycle

Blanchard and Fabrycky, Bahill and Gissing, etc. Some cover the ‘realization of the solution’ part

of the system lifecycle MIL-STD 499, EIA-632, IEEE 1220, etc.

Which one is “the” process?


Standish Report 1994*Top 10 reasons for …

Project Success1. User involvement2. Executive management support3. Clear statement of requirements4. Proper planning5. Realistic expectations6. Smaller project milestones7. Competent staff8. Ownership9. Clear vision & objectives10. Hard-working, focused staff

1. Incomplete requirements2. Lack of user involvement3. Lack of resources4. Unrealistic expectations5. Lack of executive support6. Changing requirements and

specifications7. Lack of planning8. Didn’t need it any longer9. Lack of IT management10. Technology illiteracy

* http://www.cs.nmt.edu/~cs328/reading/Standish.pdf

Project Failure

Where is “process” mentioned?Focus is on people!


The focus is on people notprocess

Literature Is full of advice as to

how to make projectssucceed

Has little if anythingto say about theproliferating processstandards


Myths of systems engineering

“Complexity” and “Systems of Systems” Dichotomy of views

In general: commercial world copes,Defence has problems First view

Difficult, need new tools and techniques Type II?

Second view “What’s the problem, get on with it”

Type V?


Complex systems

Quotes Chaos 1995 study suggesting systematic reason for projectfailure

Suggests inherent complexity is reason for difficulties – wrong! complexity was not a cause of project failure in Chaos study – poor management was

Quotes own prior work “for all practical purposes adequate testing ofcomplex engineered systems is impossible” Only applies to the way they are designed today [JEK]

Suggests evolutionary process for engineering large complex systems –right! But it applies to engineering any type of system

Published in Systems, Man and Cybernetics, 2003. IEEE International Conference on, 2003necsi.edu/projects/yaneer/E3-IEEE_final.pdf


Two Types of Complexity*

Real world complexity - in which elements of thereal world are related in some fashion, and made upof components. We try to abstract out real world complexity Complexity is in the eye of the beholder

Artificial complexity – elements of the real worldthat should have been abstracted out whendrawing the internal and external systemboundaries, since they are not relevant to thesystem (problem at hand). Artificial complexity gives rise to complicatedness

See cartoons by Rube Goldberg and W. Heath Robinson

* Kasser J.E., Palmer K., “Reducing and Managing Complexity by Changing the Boundaries of the System", Proceedings of the Conferenceon Systems Engineering Research, Hoboken NJ , 2005.


Representation of the system

Processes and productsare systems

Complicated example inRube Goldberg cartoon

http://www.rubegoldberg.com/gallery_02.php


Building artificial complexity


Focus in on a toolbox ofmethodologies*

Problem solver needs a methodology for [selecting the appropriatemethodology for] solving a problem

“Classification of a system as complex or simple will depend on theuser and on the purpose he has for considering the system”

“Systems engineering has been defined by Jenkins as ‘the science ofdesigning complex systems in their totality to ensure that the componentsubsystems making up the system are designed, fitted together, checkedand operated in the most efficient way’.”**

OR not SE!

* M.C. Jackson and P. Keys, 1984, J. Operations Research Society, Volume 35, Number 6, p 473-486, Published in Great Britain** G.M. Jenkins, (1969) The systems approach. In Systems Behaviour, J. Beishon and G Peters, Ed., 2nd Edn. P 82, Harper & Row, London

So why dowe needcomplexsystems

engineering?


System or system ofsystems?

Fighter Wing

Red Leader

Aircraft

Pilot

Ordnance

Airframe

Navigation

Propulsion

Guidance

Incomplete


System or system ofsystems?

World War II Allied convoy in North Atlantic ocean Logistics suppliers for imported equipment






problem and need new tools and techniques Myth 6: Changing requirements are a cause of

project failure so get the requirements up front Myth 7: The single systems engineering process


Incremental ModelThe incremental life cycle

Userrequirements

Systemrequirements

Architecturaldesign

Installation &validation

Part 1 Operations


Part 2Operations


Part 3Operations

Time

Operationalsystem

r135A

1

2

3

Componentdevelopment

Part 1


Part 2


Part 3

Integration &verification

Part 1


Part 2


Part 3

Get the requirements up front, stillno consideration of change in needs

Students are used to seeing time running horizontally


Evolutionary DevelopmentThe evolutionary lifecycle

Userreqs

Operations

Operations

Operations

TimeOperational

system

Any part of the system may changebut project discipline is followed

r136A

1

2

3

Systemreqs Architectural

design Componentdevelopment Integration &

verification Installation& validation

Userreqs System

reqs Architecturaldesign Component

development

Userreqs System

reqs Architecturaldesign Component

development

Feedback from system 1

Feedback from system 2Installation& validation


Installation& validation


First consideration of some changes in requirements,concept of external changes not shown


Myth and reality Origins

The failure to capture the entire problem/needand create the full set of matching specificationsfor the solution system in the early phases ofsystems engineering

Confusion between the original uncapturedrequirements and those requirements that arisedue to changes

Reality Overlooking the fact that requirements change

continuously and failure to manage that changeis a cause of project failure


HKMF Area processes

Area contains activities Process consists of activities Parallel processes producing products

Systems engineering Non-systems engineering

Project management Engineering Logistics Etc.

Interdependent

Start End

Systems engineering

Non-systems engineering


Insight

Each area in the HKMF contains a set ofactivities from which a process may beconstructed systems engineering (SEP) non-systems engineering

Project management Engineering Logistics Etc.


Why the various versions ofthe SEP are different

“the systems engineer creates a unique processfor his or her particular development effort” Biemer and Sage, 2009, page 153, Kasser and Palmer, 2005

Consider each published version of the SEP* asthe unique process created for their particulardevelopment effort by someone or some group at some point in time, at some point in the system lifecycle

* In text books, Standards, web pages, etc.


Two systems engineeringprocesses

1. Unique systems engineering process(USEP) to a project

• Constructed from set of appropriate activities by someone or some group at some point in time, at some point in the system lifecycle

• Doing process• Instantiated in Standards

2. Process that constructs the USEP forrealizing a system

• Planning process


10-Step systems engineeringprocess to construct the USEP

1. Plan the project Review lessons learned, locate industry best practices

2. Explore/survey the what needs to be done.3. Conceive at least two feasible processes

Explore industry best practices4. Identify ideal selection criteria for evaluation of the processes.

Cost, schedule, resources, technology5. Perform tradeoffs to determine and select the best process.6. Fine tune selected process.

Use best parts of each alternative from Step 37. Formulate strategies and plans to create preferred process.8. Create preferred process using activities as building blocks

Document them in the PP, SEP, SEMP, TEMP etc.9. Verify the preferred process can realize the system

Stakeholder consensus10. Terminate the project.


Summary

The state of systems engineering 2010 Seven systems engineering myths

Further details Read paper See CSER, EUSEC, and INCOSE 2010

publications on http://therightrequirement.com E.g. Kasser, J. E. and Hitchins, D. K., "Unifying the different

systems engineering processes", proceedings of theConference on Systems Engineering Research, Hoboken,NJ., 2010.


Any questions? "It ain't what you don't know that gets you into trouble. It's

what you know for sure that just ain't so." Mark Twain 1835-1910 Myth 1: There are Standards for systems engineering. Myth 2: The “V” model of the systems engineering process Myth 3: Follow the systems engineering process and all will be well Myth 4: Complexity needs new tools and techniques Myth 5: Systems of systems are a different class of problem and

need new tools and techniques Myth 6: Changing requirements are a cause of project failure so get

the requirements up front. Myth 7: The single systems engineering process.

seven systems engineering myths and the corresponding realities

Business

systems of systems

discipline systems engineering

systems engineering

systems engineering

systems engineering

systems engineeringmyths

dontcopyright joseph

process engineering