systems of systems: characteristics and challenges
DESCRIPTION
Systems of Systems: Characteristics and Challenges. Barry Boehm, [email protected] USC Center for Systems & Software Engineering http://csse.usc.edu October 25, 2006. Overview. Definitions, Examples & Motivation SoS Characteristics and Challenges Conclusions. What is a “System-of-Systems”?. - PowerPoint PPT PresentationTRANSCRIPT
Systems of Systems: Characteristics and Challenges
Barry Boehm, [email protected] Center for Systems & Software Engineeringhttp://csse.usc.edu October 25, 2006
2
Overview
Definitions, Examples & Motivation
SoS Characteristics and Challenges
Conclusions
3
What is a “System-of-Systems”? Very large systems developed by creating a framework or architecture
to integrate component systems SoS component systems independently developed and managed
– New or existing systems Some outsourced Some externally evolving
– Have their own purpose– Can dynamically come and go from SoS
SoS exhibits emergent behavior not otherwise achievable by component systems
SoS activities often planned and coordinated by a Lead System Integrator (LSI)
Typical domains– Business: Enterprise-wide and cross-enterprise integration to support core
business enterprise operations across functional and geographical areas– Military: Dynamic communications infrastructure to support operations in a
constantly changing, sometimes adversarial, environment
4
What Does an SOS Look Like: Future Combat Systems
6
The Need for Net-Centric Systems of Systems (NCSOS)
Lack of integration among stove-piped systems causes– Unacceptable delays in service– Uncoordinated and conflicting plans– Ineffective or dangerous decisions– Inability to cope with fast-moving events
Increasing NCSOS benefits– See first; understand first; act first– Network-centric operations coordination– Transformation of business/mission potential– Interoperability via Integrated Enterprise Architectures
7
Overview
Definitions, Examples & Motivation
SoS Characteristics and Challenges
Conclusions
8
SoS Characteristics and Challenges - I
Complexity: Size and number of organizations, interfaces, suppliers, coordination groups– Scalability of processes, methods, and tools
Dynamism: Number of changes/month, average time to process change– Rapid change impact analysis, change synthesis,
negotiation, development, validation, implementation Emergence: requirements not pre-specifiable Build-to-spec processes, contracts infeasible
– C2ISR a better metaphor for SoS acquisition than purchasing-agent
– Command, control, intelligence, surveillance, reconnaissance
9
Complexity of SoS Solution Spaces
Size: 10-100 MLOC Number of external interfaces: 30-300 Number of “Coopetitive” suppliers: 20-200
– Even more separate work locations Depth of supplier hierarchy: 6-12 levels Number of coordination groups: 20-200
– Reviews, changes, risks, requirements, architecture, standards, procedures, technologies, -ilities, integration, test, deployment, personnel, infrastructure, COTS,…
– Key personnel spend 60 hours/week in meetings Unprecedentedness Emergence Rapid change
Necessarily software-intensive…
10
Average Change Processing Time: 2 SoSs
Average workdays to process changes
020406080
100120140160
WithinGroups
AcrossGroups
ContractMods
11
Acquisition C2ISR Via Spiral OODA Loops
Life Cycle Architecture Milestone for Cycle
Decide on next-cycle capabilities, architecture upgrades, plans
• Stable specifications, COTS upgrades• Development, integration, V&V, risk management plans• Feasibility rationale
Act on plans, specifications
• Keep development stabilized• Change impact analysis, preparation for next cycle (mini-OODA loop)
Orient with respect to stakeholders priorities, feasibility, risks
• Risk/Opportunity analysis• Business case/mission analysis• Prototypes, models, simulations
Observe new/updated objectives, constraints, alternatives
• Usage monitoring• Competition, technology, marketplace ISR
Operate as current system
Accept new system
Example: ARPANet/Internet Spiral
12
SoS Characteristics and Challenges - II
COTS/Reuse/Legacy diversity– Many sources of incompatibility, changes– COTS: average 8-10mo/release; unsupported after 3 releases
Multiple missions and stakeholders to support– Increment and change content requires negotiation
Independently evolving systems– Often with “coopetitive” suppliers, interoperators
More time needed for systems definition– Before and after source selection
More time needed for teambuilding, partner coordination, supplier management, change management , integration and test
13
How much Architecting is Enough: COCOMO II Analysis
0
10
20
30
40
50
60
70
80
90
100
0 10 20 30 40 50 60
Percent of Time Added for Architecture and Risk Resolution
Perc
ent o
f Tim
e A
dded
to O
vera
ll Sc
hedu
le
Percent of Project Schedule Devoted to Initial Architecture and Risk Resolution
Added Schedule Devoted to Rework(COCOMO II RESL factor)
Total % Added Schedule
10000KSLOC
100 KSLOC
10 KSLOC
Sweet Spot
Sweet Spot Drivers:
Rapid Change: leftward
High Assurance: rightward
14
SoS Integrationand Testing
Size Drivers• # SoS interface protocols• # SoS scenarios• # unique component systems
Cost Drivers• Requirements understanding• Architecture maturity• Level of service requirements• SoS team capability• Maturity of LSI processes• Tool support• Cost/schedule compatibility• SoS risk resolution• Component system maturity and
stability• Component system readiness
COSOSIMO: I&T Sub-Model
LSI I&TEffort
15
Conclusions
Individual SoS attributes are highly challenging– Complexity, dynamism, emergence, uncontrollables,
stakeholder diversity– Their combinations are even more challenging
Acquisition management and negotiation skills are at least as important as systems engineering skills– C2ISR a better metaphor for SoS acquisition than
purchasing-agent
More time needed for systems definition– Before and after source selection
Backup Charts
17
Complexity of Solution Spaces- Breadth, Depth, and Length
Platform N
• • • Platform 1
Infra
C4ISR
Command and ControlSituation AssessmentInfo FusionSensor Data ManagementSensor Data IntegrationSensorsSensor Components:
2008 2010 2012 2014 2016
…1.0 2.0 3.0 4.0 5.0
Width
Length
Depth
DOTMLPF
Legend: DOTMLPF Doctrine, Organization,
Training, Materiel, Leadership, Personnel, Facilities
C4ISR Command, Control, Communications, Computers,
Intelligence, Surveillance, and Reconnaissance
18
Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May 2004
1. Acquisition management and staffing2. Requirements/architecture feasibility3. Achievable software schedules4. Supplier integration5. Adaptation to rapid change6. Quality factor achievability and tradeoffs7. Product integration and electronic upgrade8. Software COTS and reuse feasibility9. External interoperability10.Technology readiness
19
$100M
$50M
Arch. A:Custommany cache processors
Arch. B:ModifiedClient-Server
1 2 3 4 5
Response Time (sec)
Original Spec After Prototyping
Available budget
Effect of Unvalidated Requirements-15 Month Architecture Rework Delay
20
Effect of Unvalidated Software Schedules
Original goal: 18,000 KSLOC in 7 years– Initial COCOMO II, SEER runs showed infeasibility – Estimated development schedule in months for
closely coupled SW with size measured in equivalent KSLOC (thousands of source lines of code):
– Months =~ 5 * 3√KSLOC- KSLOC 300 1000 3000 10,000
- Months 33 50 72 108
•Solution approach: architect for decoupled parallel development;Schedule As Independent Variable (SAIV) process
21
The SAIV* Process Model
1. Shared vision and expectations management2. Feature prioritization3. Schedule range estimation and core-capability determination
Top-priority features achievable within fixed schedule with 90% confidence
4. Architecting for ease of adding or dropping borderline-priority features And for accommodating past-IOC directions of growth
5. Incremental development Core capability as increment 1
6. Change and progress monitoring and control Add or drop borderline-priority features to meet schedule
*Schedule As Independent Variable; Feature set as dependent variable. Also works for cost, schedule/cost/quality as independent variable.
22
Top-10 Risks: Software-Intensive Systems of Systems - CrossTalk, May 2004
1. Acquisition management and staffing2. Requirements/architecture feasibility3. Achievable software schedules4. Supplier integration5. Adaptation to rapid change6. Quality factor achievability and tradeoffs7. Product integration and electronic upgrade8. Software COTS and reuse feasibility9. External interoperability10.Technology readiness
23
COTS UpgradeSynchronization and Obsolescence Many subcontractors means an increasing number of evolving COTS
interfaces Aggressively-bid subcontracts can deliver obsolete COTS
– New COTS released every 8-9 months (GSAW)– COTS unsupported after 3 releases (GSAW)– An actual delivery: 120 COTS; 46% unsupported
Emphasize COTS interoperability in source selection process Develop contract/subcontract provisions/incentives to ensure
– Consistency and interoperability across contractor and subcontractor-delivered COTS software products
– Such products are recent-release versions Develop a management tracking scheme for all COTS software products in
all NCSOS software elements Develop a strategy for synchronizing COTS upgrades
24
Emerging Scalable Spiral Process- For 21st century systems engineering and acquisition
The C4ISR Metaphor for NCSOS Acquisition– Role of OODA loops– Example: Internet Spiral– Example: FCS Win Win Spiral Model
Risk-Driven Scalable Spiral Model– Increment view– Life-cycle view– Role of anchor point milestones
Acquisition management implications People management implications
25
Risk-Driven Scalable Spiral Model:Increment View
Increment N Baseline
Rapid Change
High Assurance
Short, Stabilized Development of Increment N
Increment N Transition/O&M
Short Development Increments
Stable Development Increments
Foreseeable Change (Plan)
Increment N Baseline
Rapid Change
High Assurance
Short, Stabilized Development of Increment N
Increment N Transition/O&M
Short Development Increments
Stable Development Increments
Foreseeable Change (Plan)
26
Risk-Driven Scalable Spiral Model:Increment View
Increment N Baseline
Future Increment Baselines Rapid Change
High Assurance
Agile Rebaselining for Future Increments
Short, Stabilized Development of Increment N
V&V of Increment N
Increment N Transition/O&M
Current V&V
Short Development Increments
Future V&V Stable Development Increments
Continuous V&V
Concerns Artifacts
Deferrals Foreseeable Change (Plan)
Resources Resources
Increment N Baseline
Future Increment Baselines Rapid Change
High Assurance
Agile Rebaselining for
Short, Stabilized Development of Increment N
V&V of Increment N
Increment N Transition/O&M
Current V&V
Short Development Increments
Future V&V Stable Development Increments
Continuous V&V
Concerns Artifacts
Deferrals Foreseeable Change (Plan)
Resources Resources
Unforseeable Change (Adapt)
27
Risk-Driven Scalable Spiral Model:Life Cycle View
SystemInception
SystemElaboration
Agile DI2 (OO&D) Rebaselining
Plan-Driven DI1 Construction (A)
DI1 V&V
Plan-Driven DI2 Construction (A)
DI2 V&V
System LCA System, DI1 LCA DI2 B/L LCA
DI2 LCA
Changes
LCA: Life Cycle ArchitectureIOC: Initial Operational CapabilityOO&D: Observe, Orient and DecideV&V: Verification and ValidationDI: Development IncrementB/L: Baselined
28
Risk-Driven Scalable Spiral Model:Life Cycle View
SystemInception
SystemElaboration
Agile DI2 (OO&D) Rebaselining
Plan-Driven DI1 Construction (A)
DI1 V&V
Agile DI3 (OO&D) Rebaselining
Plan-Driven DI2 Construction (A)
DI2 V&V
Agile DI4 (OO&D) Rebaselining
Plan-Driven DI3 Construction (A)
DI3 V&V
DI1 Trans’n
DI1 Usage
DI2 Trans’n
DI2 Usage
DI3 Trans’n
DI3 Usage
System LCA
DI3 LCA
System, DI1 LCA DI2 B/L LCA DI3 B/L LCA DI4 B/L LCA
Update
Update
Update
DI2 LCA
Changes
Changes
Changes
DI4 LCA
. . .
. . .
DI1 IOC
DI3 IOC
DI2 IOC
LCA: Life Cycle ArchitectureIOC: Initial Operational CapabilityOO&D: Observe, Orient and DecideV&V: Verification and ValidationDI: Development IncrementB/L: Baselined
29
LCO (MS A) and LCA (MS B) Anchor Points Pass/Fail Criteria A system built to the given architecture will
– Support the operational concept– Satisfy the requirements– Be faithful to the prototype(s)– Be buildable within the budgets and schedules in the
plan– Show a viable business case– Establish key stakeholders’ commitment to proceed
LCO: True for at least one architectureLCA: True for the specific life cycle architecture; All major risks resolved or covered by a risk management plan
30
Spiral Feasibility Rationale Deliverable
LCO, LCA reviews not just UML/PowerPoint charts Need to show evidence of product and process feasibility Evidence provided by prototypes, production code, benchmarks,
models, simulations, analysis– Sizing and cost/schedule model results for process feasibility
Evidence provided in advance to LCO/LCA review team– Key stakeholders, specialty experts
Lack of evidence risks destabilizing the process– Needs coverage by viable risk mitigation plan
Key new progress metric– Feasibility evidence progress vs. plans
31
Acronym Definitions
B/L BaselinedC4ISR Command, Control, Communications,
Computers, Intelligence, Surveillance, and Reconnaissance
CCD Core Capability Drive-ThroughCOTS Commercial Off the ShelfCRACK Collaborative, Representative, Authorized,
Committed, KnowledgeableDI Development IncrementDODAF Department of Defense Architectural
FrameworkDOTMLPF Doctrine, Organization, Training,
Materiel, Leadership, Personnel, FacilitiesERP Enterprise Resource PlanningFEAF Federal Enterprise Architectural
FrameworkGSAW Ground System Architectures WorkshopIESG Internet Engineering Steering GroupIETF Internet Engineering Task ForceIKIWISI I’ll Know It When I See ItIOC Initial Operational Capability IPT Integrated Product TeamIRR Inception Readiness Review
LCA Life Cycle ArchitectureLCO Life Cycle ObjectivesLSI Lead System IntegratorMLOC Million Lines of CodeMS MilestoneNCSOS Net-Centric System of SystemsOO&D Observe, Orient, and DecideOODA Observe, Orient, Decide, ActPM Person-Month/Program ManagerPRR Product Release ReviewSAIV Schedule As Independent VariableSE System EngineeringSoS System of SystemsSOW Statement of WorkSW SoftwareSys Engr System EngineeringV&V Verification and ValidationWMI War-fighter machine interface