research profile systems engineering in the enterprise

RESEARCH PROFILE

Systems Engineering in the Enterprise

October 16, 2007

Dr. Donna H. Rhodes

Massachusetts Institute of Technology

[email protected]

seari.mit.edu © 2007 Massachusetts Institute of Technology 2

Research Cluster-Portfolio Mapping

XSE Strategic Guidance

XSE in the Enterprise

XSE Economics

XDesigning for

Value Robustness

XXXXSocio-Tech Decision Making

SE-SynthesisSE-FieldR-STARSV-STARS


Research Portfolio (4)

SYSTEMS ENGINEERING in the ENTERPRISE

This research area involves empirical studies and case based research for the purpose of understanding how to achieve more effective systems engineering practice in context of the nature of the system being developed, external context, and the characteristics of the associated enterprise.

– Engineering systems thinking in individuals and teams

– Collaborative, distributed systems engineering practices

– Social contexts of enterprise systems engineering

– Alignment of enterprise culture and processes

– Socio-technical systems studies and models

Empirical Studies and case based research for understanding of how to achieve more effective practice


Motivation

Many program failures attributed to inadequate execution of sound processes

– Reality is that this often relates to factors beyond process execution and cost/schedule pressures

– Insufficient post-program assessment, particularly of soft factors

– Governance not always clear in SoS type programs

Problem Statement for MITRE/MIT Joint Research in Social Contexts of

Enterprise Systems Engineering

The Government programs that MITRE supports are suffering changes in

requirements, cancellations, and shifting work areas. These difficulties reflect

shifting interactions among powerful stakeholders who have competing interests,

with no one effectively in control. While MITRE has always managed social,

organizational, cultural, and political aspects of its business in tandem with the

technical, these needs exceed our existing skill set.


Motivation

• High variation in the scale of and nature of enterprises

• Increased diversity of stakeholders involved in and impacted by a program

• Systems engineering often involves collaboration of teams across geographies and organizations

• New methods and technologies are available for model-based SE and other advanced practices, but we lack empirical data to show under what conditions and within what enterprise contexts these can be successfully used

• Successful engineering depends upon systems thinking but ambiguity in what this means and how to develop it


Evolution of Practice of Systems Engineering

Over the past five or six decades, the discipline known as “Systems Engineering” has evolved. At one time, many years ago, development of a capability was relatively simple to orchestrate.

The design and development of parts, engineering calculations, assembly, and testing was conducted by a small number of people. Those days are long gone.

Teams of people, sometimes numbering in the thousands are involved in the development of systems; and, what was previously only a development practice has evolved to become a science and engineering discipline.

Saunders, T., et al, System-of-Systems Engineering for Air Force Capability Development: Executive Summary and Annotated Brief, AF SAB TR -05-04, 2005


Four Perspectives for Engineering Systems Thinking

1. A very broad interdisciplinary perspective, embracing technology, policy, management science, and social science.

2. An intensified incorporation of system properties (such as sustainability, safety and flexibility) in the design process. – Note that these are lifecycle properties rather than first use properties. – These properties, often called “ilities” emphasize important intellectual

considerations associated with long term use of engineering systems.

3. Enterprise perspective, acknowledging interconnectedness of product system with enterprise system that develops and sustains it. – This involves understanding, architecting and developing organizational structures,

policy system, processes, knowledgebase, and enabling technologies as part of the overall engineering system.

4. A complex synthesis of stakeholder perspectives, of which there may be conflicting and competing needs which must be resolved toserve the highest order system (system-of-system) need.


Socio-Technical Case Study on Micro Air Vehicle

(Bartolomei, PhD 2007)

• Identify system of interest

– System Type

– System Boundary

– Context

• Define Modeling Objectives

• Collect Data

• Code Data

• Organize Coded Data in Modeling Framework

• Examine Model for missing/conflicted data

• Resolve Missing Data

• Perform Analysis

• Iterate

Qualitative Knowledge Construction

Technical DomainSocial Domain

Environmental Domain

Functional Domain

Process Domain

System Type: Engineering System

MAV Product Development System (MAV-PD)

Context

Create a Dynamic, End-to-End Representation of the MAV-PD

Modeling Objectives


Qualitative Knowledge Construction using Engineering Systems MatrixJason Bartolomei, PhD 2007

Jason Bartolomei, PhD 2007, [email protected]

Descriptive Analysis(Evolutionary Dynamics of a Sociotechnological System)

Time 1 Time 2

Time 3 Time 4

E n v iro n m e n t

Sociotechnological Systems

S o c ia l O rg a n iz a tio n

C u ltu re

In d iv id u a ls

V a lu e P ro p o s itio n s

P ro c e s s e s

R u le s

S o c ia l/P o litic a l:

S ta k e h o ld e r C h a n g e s

S o c ia l In flu e n c e s

C o m p e titio n

L a w s /P o lic ie s /R e g u la tio n s

P h y s ic a l:

R e s o u rc e s

W e a th e r

E c o n o m ic :

R e s o u rc e s

Ma rk e t

T e c h n o lo g ic a l:

In n o v a tio n

O b s o le s c e n c e

O th e r S y s te m s

S ta n d a rd s

T e c h n ic a lA r ti fa c ts

In fra s tru c tu re

P ro c e s s e s

F u n c tio n s

P h y s ic a l L a w s

Sociotechnological System Decomposition

A C T IV IT IE S D S M

O B J E C T S D S M

F U N C T IO N S D S M

O B J E C T IV E S D S M

S T A K E H O L D E R S D S M

S Y S T E M D R IV E R S D S M

S Y S T E M B O U N D A R Y

• A ll diagonal matrices are n2—share the same column and row headings

• Decomposition begins northwest to southeast

• Off-diagonal cells represent relationships between variables

• Goal is to represent traceability between multiple “views” of the system

• The framework maps “physical” and “non-physical” relations

Prescriptive Analysis(Hot/Cold Spot Analysis)

Sensitivity Analysis

Benefit Calculation:

Network Analysis

Cost Calculation:

Cost (effort,$)

Benefit (utils,$)

Measure of Uncertainty/Volatility

Uncertainty/VolatilityMeasure:

Forecast

Low p

Low Cost

Low Benefit

Low p

High Cost

Low BenefitLow p

High Cost

High Benefit

High p

Low Benefit

Low Cost

Tail Connector

Wing Connector

Interchangeable

Battery Module

Low pHigh Benefit

Low Cost

High p

High Cost

Low BenefitHigh p

High Benefit

Low Cost

High p

High Benefit

High Cost


The understanding of the organizational and

technical interactions in our systems,

emphatically including the human beings who

are a part of them, is the present-day frontier of

both engineering education and practice.

Dr. Michael D. Griffin, Administrator, NASA

Boeing Lecture, Purdue University

28 March 2007


MITRE/MIT Research on Social Contexts of Systems Engineering

Develop social science capabilities and products complementing MITRE’s technical capabilities in order to meet the challenges of Systems Engineering at the Enterprise level• Transform practical field experience of MITRE site staff

into social-scientific understanding that is usefully transferable

• Leverage experience and approaches from MIT partners

Technical Approach� Case Studies� Workshops� 2nd Round of Case Studies� Communicate Lessons Learned


Recent Studies of SE in the Enterprise Indicate Organizations Need to Develop “Situational Leadership” Capabilities

Develop ‘situational leadership’ abilities of engineers in regard to

making decisions at multiple levels – component, system, SoS

– Includes an improved understanding of the decisional trade off process for local versus global system value delivery

…leading development of a product?

…leading development of a system family?

…a system?


This MUST be decomposed, since understandings can be contradictory

Before designing an intervention, know what you are trying to produce

System-of-Systems SE Traits

Not detail focused

Thinks out-of-the-box

Creative

Abstract thinking

Process-Centered SE Traits

Detail oriented

Structured

Methodical

Analytical

Define the Goal then Design the Intervention

“Systems Thinking Mindset”LAI Funded Research by Davidz (2006)


RESEARCH PROJECT

Enabling Systems Thinking to Accelerate the Development of Senior Systems Engineers

Consensus on primary mechanisms that enable or obstruct systems thinking development in engineers

1. Experiential learning2. Individual characteristics3. Supportive environment

Even though systems thinking definitions diverge, there is

consensus on primary mechanisms that enable or obstruct systems thinking development in engineers

Dr. Heidi Davidz, PhD 2006

LAI Funded Research


Experiential Learning Develops Systems Thinking (Davidz, 2006)

Remarkable Consensus for Data Solicitation Format

Q: What were key steps in your life that developed your systems Q: What were key steps in your life that developed your systems thinking abilities?thinking abilities?


Collaborative Systems Thinking Aligning Culture and Standardized Process

CultureStandardized

Process

Collaborative

Systems Thinking

How do

engineering

processes interact

with culture?

How do culture and

process enable

collaborative systems

thinking?

Examines the development of systems thinking within teams of engineers.

Emphasis placed on the role of standard process and its interactions with organizational culture.

Research motivated by desire to better understand systems thinking at the team level within engineering.

Focuses on the role of standardized process, its artifacts and associated tools, in enabling or promoting team level systems thinking —termed collaborative systems thinking.

Caroline Twomey Lamb, LAI PhD Student, 2009


RESEARCH PROJECT

Collaborative Distributed Systems Engineering in the Aerospace Industry

Darlene Utter, S.M. 2007

Social Factors:

Local/Company Culture DifferencesCareer Development Advancement

Distributed TeamworkCommunications

Working w/ IT and toolsProject Management

Technical Factors:

SE Process/ArchitectureDistributed Decision MakingTools/Information TechnologyKnowledge Management

Cost & ScheduleProduct Impact

CDSE

Develop heuristics for successful CDSE resulting from case studies

Recommendations to overcome barriers to successful CDSE

Recommendations for future work in this area


Data Analysis Overview

Collaboration Situation and Management

AA BB

Knowledge, Data and Decision Management

SE Processes and Practices

Collaboration Tool Use

CDSE Social and Cultural Environment

CDSE Benefits and Motivation

Description RecommendationLesson Learned

Issue or Barrier

Success Factor

Irrelevant

OR

Tool TrainingNetwork Reliability

Tool VersionsTool AccessLearning Curves

Classified Data

Interview Heading Topics

Company

Transcripts

Subtopic

Interviewee Experience

Data Analysis

Example


Exploratory Study (Utter 2007)Collaborative Distributed SE vs

Traditional SE: Observations of Two ProgramsCDSE vs. Traditional SE Environment A B A and B

1There is a great deal more "up-front" work to coordinate SE efforts, teams, resources,

etc.X

2Communications in a CDSE environment are in general more difficult and facilitated by

the introduction of and reliance on collaboration tools.X

3CDSE meetings are more formal, thus there is less brainstorming and social

interactions amongst teams.X

4 New and different processes are standardized, mandated and followed. X

5There are additional obstacles and complexities: company proprietary data sharing, corporate fire-walls, non-disclosure agreements, classified data transfer.

X

6Untraditional organizational channels are used to enforce all developers to use the

agreed upon processes.X

7Centrally collected raw data metrics are used to measure relative company

performance.X

8 New and different SE management positions are created to coordinate efforts. X

9Collaboration creates a "one team" or "one goal" work arrangement, where all

contractors are working toward the same final, integrated product.X

10

It is more difficult to allocate or re-allocate resources as changes occur, since formal

contracts with schedules and resource allocations are typically done way in advance of program execution.

X

11 There are more discussions and more frequent interactions among teams. X


Exploratory Study (Utter 2007)CDSE vs Traditional SE - Hypotheses to be Explored May Lead to CDSE Maturity Model

CDSE Topic Proposed Successful CDSE Theme

Collaboration Situation and ManagementNeed well-defined SE and program organizational structure, with additional middle-management to

coordinate efforts across and companies.

Collaboration Situation and Management Need management buy-in and wide-spread enforcement of the processes.

Collaboration ToolsCollaboration tools are critical ; the better the tools and the processes in place for their use, the

more "distributed'' the work and the less resources that are wasted.

Collaboration ToolsWidely available collaborative product development tools are also needed to support successful SE

and development.

Knowledge, Data, and Decision ManagementTransference and sharing of classified data is an issue that affects almost all aspects of CDSE. Need to dedicate resources.

Knowledge, Data, and Decision Management

Company proprietary data development and sharing affects almost all aspects of CDSE, including

trust, product integration, system cohesiveness, and information dissemination. Need to dedicate

resources.

Knowledge, Data, and Decision ManagementBetter methods are needed to facilitate dissemination of information and decisions to teams. Need

to dedicate resources.

Knowledge, Data, and Decision ManagementTools and processes are needed for successful knowledge, data, and decision management practices to address many issues.

SE Processes and PracticesFormal, contractually obligated, and publicized SE processes are needed to control all aspects of

systems development.

SE Processes and PracticesWidely available and platform independent system modeling and simulation tools are needed to

confine system defects in phase and facilitate system-wide integration.

SE Processes and Practices

System interfaces especially those that cross company boundaries, are a problem area. Recognize this issue, the importance of system interfaces, and dedicate resources early on to

better define and monitor interfaces.

Social and Cultural EnvironmentProgram kick-off face-to-face, and regularly scheduled face-to-face meetings are necessary to build and maintain relationships and trust between teams.

Social and Cultural Environment

Have team-building activities or social events - the CDSE social environment is believed by many to be more formal, and almost all interviewees suggested team social events as a mechanism to

improve relationships.


Future Research Directions

• More extensive and rigorous studies to understand collaborative distributed systems engineering – Is there a preferred system architecture to support CDSE?

– What incentives and performance measures should be used?

– Development of an “SE Collaboration Maturity Factor”

• Additional research related to development of systems competencies in the workforce

• Field research to motivate theory and principles for developing and managing enterprises for context-harmonized interactions

• Understand the factors for effective systems engineering in commercial product and service enterprises


Summary

• Modern systems involve more complex enterprises

• Enterprises are in themselves systems – Methodologies from social sciences, complexity science and systems science are important

– Must go beyond organizational and behavior approaches from management sciences

• Leverage for systems engineering effectiveness lies at the intersection of social and technical

• Empirical and case-based studies are needed to put the “science”behind systems engineering in the enterprise

• Next segment– Research Report: Caroline Lamb, Leveraging Organizational Culture, Standard

Process, and Team Norms to Enable Collaborative Systems Thinking

– Brief Overview: David Broniatowski, Latent Semantic Indexing of Committee Preferences in Healthcare Innovation Systems


References

Bartolomei, J.E., Qualitative Knowledge Construction for Engineering Systems: Extending Design Structure Matrix Methodology in Scope and Procedure, Doctor of Philosophy Dissertation, Engineering Systems Division, MIT, June 2007

Davidz, H., Nightingale, D., and Rhodes, D.H., “Enablers and Barriers to Systems Thinking Development: Results of a Qualitative and Quantitative Study,” 3rd Conference on Systems Engineering Research, Hoboken, NJ, March 2005

Davidz, H., Nightingale, D., and Rhodes, D.H., “Enablers, Barriers, and Precursors to Systems Thinking Development,” 17th International Conference on Systems Engineering, Las Vegas, NV, September 2004

Lamb, C.T., and Rhodes, D.H., “Standardized Process as a Tool for Higher Level Systems Thinking,”INCOSE International Symposium 2007, San Diego, CA, June 2007

Lamb, C.T., and Rhodes, D.H., “Promoting Systems Thinking Through Alignment of Culture and Process: Initial Results,” 5th Conference on Systems Engineering Research, Hoboken, NJ, March 2007

Rhodes, D.H, “INCOSE SEANET – Systems Engineering & Architecting Doctoral Research Network,”3rd Conference on Systems Engineering Research, Hoboken, NJ, March 2005

Utter, D.A., Collaborative Distributed Systems Engineering, Master of Science Thesis, Engineering Systems Division, MIT, January 2007

QUESTIONS

http://seari.mit.edu

[email protected]

research profile systems engineering in the enterprise

Documents