headquarters u.s. air force - architecture · pdf filei n t e g r i t y - s e r v i c e - e x...
TRANSCRIPT
I n t e g r i t y - S e r v i c e - E x c e l l e n c e
1
Headquarters U.S. Air Force
MODELING AND SIMULATION FOR ARCHITECTURE ASSESSMENT
Alexander H. LevisAF/ST
4/20/04
ASC - 2I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Outline• Introduction
• Architecture Design
• Architecture Evaluation
• Modeling and Simulation Challenges
• Conclusion
ASC - 3I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Need
“As DoD enters into an era of Net-Centric Operations and Warfare(NCOW), the ability to portray and understand complex many-to-many relationships becomes even more important. Capabilities must be able to“plug and play” in a Joint, global, multimedia, and multilingualenvironment. To achieve this ability, there must be a mechanism for incorporating information technology consistently, controlling the configuration of technical components, ensuring compliance with technical “building codes,” and ensuring efficient processes.Architectures provide this mechanism by serving as a means for understanding and managing complexity.”
- DOD Architecture Framework, Version 1.0, Volume 1, p. 3-5, August 2003
ASC - 4I n t e g r i t y - S e r v i c e - E x c e l l e n c e
DODArchitecture Framework v. 1.0
15 August 2003
Architecture: The fundamental organization of a system embodied in its components, their relation-ships to each other, and to the environment and the principles guiding its design and evolution
ASC - 5I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Components of the Framework
JOA
Operational
Systems
Technical
Standard1999 +
Standard1998
NodeA
NodeB
NodeC
IER
CADM
DDDS
DDDS
LISI
LISI
TRM
SHADE
JTA
COE
UJTL
COELISI
CADM
SystemX
SystemY
SystemXSystemZ
SystemX
NodeA
NodeB
NodeC
SystemY
SystemY
IERIER
Activity 1 Activity 2Activity 3
Activity 1Activity 2
♦ Common Definitions
♦ Common Products and Data
♦ Common References
ASC - 6I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Three Views (DODAF v1.0)
The operational view is a description of the tasks and activities, operational elements, and information exchanges required to accomplish DOD missionsThe systems view is a set of graphical and textual products that describes systems and interconnections providing for, or supporting, DOD functions
The technical standards view is the minimal set of rules governing the arrangement, interaction, and interdependence of system parts or elements
Each view is described through a set of “products”, i.e., diagrams, tables, graphics, narrativesEach of the 26 products contains “data” that characterize some aspect of the architecture
ASC - 7I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Systems Engineering Process
The traditional Systems Engineering process is focused on system designAt a conceptual level, it is a sequential process with clearly defined steps and milestones (in reality, many interactions and feedback loops)It is exemplified by the “V” Process modelTo cope with uncertainty in requirements and the complexity of Systems-of-Systems, an architecture based approach is now mandatedThe architecture based approach, while more complex up front, facilitates integration and interoperabilityIt forces users, acquirers, and developers to work together at all stages
ASC - 8I n t e g r i t y - S e r v i c e - E x c e l l e n c e
System of Systems** After Maier, Sage, …
A system will be called a System-of-Systems (SOS) when:
– The component systems achieve well-substantiated purposes in their own right even if detached from the overall system, and
– The components systems are managed in large part for their own purposes rather than the purposes of the whole.
A “System-of-Systems” exhibits behavior not achievable by the component systems acting independently
The System of Systems concept is really at the heart of systems engineering architecting, design and integration
The key Air Force issues are related to Systems-of-Systems problems
– Focus on Capabilities
– Focus on Integration of conventional and information operations
– Focus on further integration of air and space assets
– Focus on legacy systems
ASC - 9I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Architecture-based System Design Process
Architecture Design
OperationalArchView
System Design
MISSION
SystemsArchView
OperationalC
oncept
TechnicalArch View
The Architecture-based System Design Process is actually a continuum – from mission to system design
– The Operational Architecture view expresses the functional requirements that are then satisfied by construction in the systems architecture view.
– The process is iterative – with iterations representing refinement levels until actual designs are obtained
ASC - 10I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Architecture Design Guidance
3 4 5 6
1Determine the intended use
of the architecture
Determine views and
products to be built
Build the requisite products
Use architecture for intended
purpose
2Determine
characteristics to be captured
Determine scope of
architecture
Defense Organizations Industry
Six Steps
ASC - 11I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Architecting…
ArchitectureDesign
Executable Model
Construction
ArchitectureAnalysis andEvaluation
MISSION
C4ISR AFProduct
Generation
DOD Arch.Framework
OperationalC
oncept
The Complete Process
Alternative approaches and tools can be used to design the C4ISR/DOD Architecture views – structured analysis or object orientation
The executable model is used for behavior and performance evaluation
Architectural efforts should be goal or problem based and not “architecture product based”
ASC - 12I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Total Architecture Design Process
To Develop an Architecture:• Determine the intended use of the architecture• Determine the architecture scope
Establish the Point of View Establish the Boundary of the architectureEstablish the Layer (Domain, Conops,…)Establish the Time Frame (as is, to be, …)
• Determine the characteristics to be captured• Determine views and products to be built• Build the requisite products• Carry out Analysis, Evaluation, and Comparison• Use architecture for intended purpose
To make acquisition decisionsTo design systemsTo migrate systems
ASC - 13I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Process For Product Development
A six stage process has been developed for generating the 23 products for the Operational and Systems architecture views and the Integrated Dictionary
The process has been derived by approaching the problem from thesystems engineering point of view and using (a) Structured Analysis; (b) Object Orientation
Each product has been perceived as an Entity containing data; a formal Data Model was derived showing the relationship among the various entities (a simplified version of volume II of DODAF)
The relationships among the entities induced a partial ordering which led to a series of steps for their production
The processes utilize existing tools and techniques to derive the requisite products and are both compatible with the development of an Executable model
ASC - 14I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Key Entities
Operational Nodes and Operational ElementsActivitiesNeedlinesOperational Information ElementsOrganizationAssetSystem, System Element, System ComponentSystem FunctionSystem NodeLinkSystem Information ElementPerformance Parameter SetNetworks and Interfaces
(Operational view)
(Organizational view)
(Systems view)
Key Classes Data Model
Operational Node
Operational Element
Needline Operational Information
Element
System Node
System Link
System Data
Element
System Function
(Operation)
Activity
LAN/WAN
Performance Parameter
SetInterface
Asset
Organization
Has
Represents
is Associated with
Performs/ Is performed by
Implements/ is implemented by
Performs/ Is performed by
Implements/ Is implemented by
Implements/ Is implemented by
Contains
Is Attached To
Is Attached To
Contains
Enables
OPERATIONAL VIEW
SYSTEMS VIEW
Has
Maps To
Is a
May Represent
System Element
System Component
1..*
1..*
1..*
1..*
1..*
0..*
1..*
1..*
1..*
1..*
1..*
1..*
1..*
1..*
1..*
1..*
1..*1..*
1..* 1..*1..*
Generates or Consumes
1..*
Generates or Consumes
1..*
AFAMS - 15
The Six Stage Process: Structured Analysis
Define Operational
Nodes
Define Operational Elements
Determine Needlines
Define Operational Information Elements
Define System Nodes
Determine Systems, Elements,
Components
Define Links
Define System
Information Elements
Allocate Activities to systems to
Define System Functions
Allocate Activities to Operational Elements
Select LAN/WAN Networks
Define Parameter
Performance Set
Define Interfaces
Determine Assets
Select Organizations
OPERATIONAL VIEW
SYSTEMS VIEW
Create High Level Operational
Concept Graphic with Textual Description
Create Functional
Decomposition
Activity Model (OV-5)
STD (OV-6b)
Rule Model (OV-6a)
Logical DataModel (OV-7)
Integrated Dictionary
(AV-2)
System Desciptions
(D12)
Tactical Description of Doctrine, Tactics,
Operational Procedures
(D5)
Operational Information Elements
(D6)
Universal Joint Task
List (D2)
Operational Concept
(AV1 and D1)
Organization List (D3)
Organizatinal Relationships
(D4)
Operational Concept Graphic OV-1
Operational Node
Connectivity Description
(OV-2)
Select Functions
Create Activity Model
Create Logical Data
Model
Define Logical Rules
Create Operational State
Transition Description
Ensure Concordance
Determine Organizational Relationships
Command Relationship
Chart (OV-4 )
Create System
Functionality Description
Create Physical Data
Model
Create System2
Matrix
Create System
Communications Description
Create System Interface
Description
Create System
Information Exchange
Matrix
Create System
Performance Parameter
Matrix
Create System
Evolution Description
Create Operational
Node Connectivity Description
Create Operational Information Exchange
Matrix
Operational Information Exchange
Matrix (OV-3)
Create Operational Activity to System
Function Traceability
Matrix
Operational Activity to
System Function Traceability
Matrix (SV-5)
System Interface
Description (SV-1)
System Communications
Description (SV-2)
Systems2 Matrix (SV-3)
Systems Functionality Description
(SV-4)
System Information
Exchange Matrix (SV-6)
System Evolution
Description (SV-8)
Physical Data Model (SV-11)
System Performance
Parameter Matrix (SV-7)
Communication Systems
Description (D9)
Migation Systems
(D11)
System Functions
(D8)
System Performance
Attributes (D10)
STAGE 0 STAGE 1 STAGE 2 STAGE 3 STAGE 4 STAGE 5
Maintain Integrated Dictionary
All Processes
States and Events
(D7)
Create System
Technology Forecast
Systems Technology
Forecast (SV-3)
Created by: System Architectures Laboratory George Mason University Fairfax, VA 22020 [email protected]
AFAMS - 16
ASC - 17I n t e g r i t y - S e r v i c e - E x c e l l e n c e
RemarksThe process appears to be complex; this is due to the tight coupling among products
(This tight coupling is to be expected: all views and products describe aspects of a single architecture)
The process requires concurrent, but coordinated activities to take place
(It is the architect’s role to plan and direct these activities)
Products are produced in all six stages
(This allows for continuous review and evaluation of the architecture description process)
The process diagram (flow chart) indicates what products are needed in each case; the choice is not arbitrary because of interdependencies
Operational Concept
(AV1 and D1)
Create High Level Operational
Concept Graphic with Textual Description
Operational Concept
Graphic OV-1
Organization List (D3)
Operational Concept
(AV1 and D1)
Organizatinal Relationships
(D4)
Universal Joint Task
List (D2)
Select Use Cases
Create Use Case Diagrams
Define Operational
Elements
Determine
Assets
Select Organizations
Determine Organizational Relationships
Command Relationship
Chart (OV-4 )
Define Operational
Nodes
Allocates Activities to Operational
Classes
Determine Needlines
Rule Model
(OV-6a)
Logical Data
Model (OV-7)
STD (OV-6b)
Define System Nodes
Determine Systems, Elements,
Components
Define Links
System Desciptions
(D12)
Activity Model (OV-5)
Doctrine, Tactics, and Operational Procedures
(D5)
States and Events
(D7)
Create Activity
Diagrams
Create Interaction Diagrams
Create Class
Diagrams
Create Operational
State Transition Description
Define Logical Rules
Create Logical
Data Model
Ensure Concordance
Operational Event Trace Description
(OV-6c)
Define Operational Information Elements
Create Operational Infromation Exchange
Matrix
Create Operational
Node Connectivity Description
Operational Infromation Exchange
Matrix (OV-3)
Operational Node
Connectivity Description
(OV-2)
Operational Information Elements
(D6)
Create Operational Activity to
System Function Traceability
Matrix
Operational Activity to System
Function Traceability Matrix
(SV-5)Create System Functionality Description
Physical Data Model
(SV-11)Logical Data Model
Systems Functionality Description
(SV-4)
Create Activity
Diagrams
Create Interaction Diagrams
Ensure Concordance
and Conformance
Create System State
Transition Description
Define System Rules
Create Physical
Data Model
Create Class
Diagrams
Systems State Transition
Description (SV-10c)
Allocations
Allocate Activities to systems and Defined System
FunctionsSystem/
Functions (D8)
System Event/Trace Description
(SV-10b)
OO System Arch View
Create System
Technology Forecast
Create System
Information Exchange
Matrix
Define System
Information Elements
Define Interfaces
Create System2
Matrix
Systems Technology
Forecast (SV-9)
System Information Exchange
Matrix (SV-6)
Systems2 Matrix (SV-3)
System Communications
Description (SV-2)
System Interface
Desciption (SV-1)
System Evolution
Description (SV-8)
Create System
Evolution Description
Create System Interface
Description
Create System
Communications Description
Select LAN/WAN Networks
Define Performance
Parameter Set
Create System Performance
Parameter Matrix
System Performance
Parameter Matrix (SV-7)
Communication Systems
Description (D9)
Migation Systems
(D11)
System Performance
Attributes (D10)
Systems View
Operational View
Created by: System Architectures Laboratory George Mason University Fairfax, VA 22020 [email protected]
All ProcessesMaintain
Integrated Dictionary
Integrated Dictionary
(AV-2)
Stage 0 Stage 1 Stage 2 Stage 3 Stage 4 Stage 5
OV-1
OV-4
OV-5,6,7
OV-2, 3
SV- 4,10,11
SV- 1,2,3,6
SV-7,8,9
SV-5
AV-2
AV-1
AFAMS - 18
The Six Stage Process – Object Orientation
ASC - 19I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Layered Process
Architecture development is a top-town process through layers of abstraction – a process of refinement. The process is, of course, iterative. However, we must complete a layer first before drilling down to the next layer.
Develop the “High Level” Architecture first –
• While all three views are developed, emphasis is placed on the Operational View
Develop the “Intermediate” Architecture
• Refine the Operational view and begin elaborating the Systems and Technical Standards views
Develop the “Detailed” Architecture
• Refine the Systems and Technical Standards views
ASC - 20I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Layered Process
OPERATIONALCONCEPT
OPERATIONALARCHITECTURE
SYSTEMSARCH. VIEW
EXECUTABLEMODEL
DYNAMICSMODEL
OPERATIONALCONCEPT
OPERATIONALARCHITECTURE
EXECUTABLEMODELDYNAMICS
MODEL
“High Level”
LOGIC
BEHAVIOR
concrete/specific ORGANIZATION
MODELSYSTEMSARCH. VIEW
EXECUTABLEMODEL
PERFORMANCE
ORGANIZATIONMODELSYSTEMS
ARCH. VIEW
OPERATIONALCONCEPT
OPERATIONALARCH. VIEW
“Detailed”
“Intermediate”
TECHNICALARCH. VIEW
abstract/general
ASC - 21I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Time Phased Process
Repeat Layered Process for different future times
Small changes, if any, in the High Level architecture
Some changes in the Intermediate Architecture to reflect evolution of doctrine and warfighting capabilities
More substantial changes (and multiple alternatives) at the Detailed Architecture level to reflect future systems and options under different budget projections (cost analysis)
Requirement: Performance and Effectiveness measures to improve over time
Trade-off alternative phasing-in and phasing out strategies for systems
ASC - 22I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Time-phased Process
OPERATIONALCONCEPT
SYSTEMSARCHITECTURE
OPERATIONALARCHITECTURE
EXECUTABLEMODELTECHNICAL
ARCHITECTURE
ANALYSISPHASE
SYNTHESISPHASE
MOPs, MOEs
EVALUATIONPHASE
MISSION
OPERATIONALCONCEPT
SYSTEMSARCHITECTURE
OPERATIONALARCHITECTURE
EXECUTABLEMODEL
TECHNICALARCHITECTURE
ANALYSISPHASE
SYNTHESISPHASE
MOPs, MOEs
EVALUATIONPHASE
MISSION
OPERATIONALCONCEPT
SYSTEMSARCHITECTURE
OPERATIONALARCHITECTURE
EXECUTABLEMODEL
TECHNICALARCHITECTURE
ANALYSISPHASE
SYNTHESISPHASE
MOPs, MOEs
EVALUATIONPHASE
MISSION
OPERATIONALCONCEPT
SYSTEMSARCHITECTURE
OPERATIONALARCHITECTURE
EXECUTABLEMODELTECHNICAL
ARCHITECTURE
ANALYSISPHASE
SYNTHESISPHASE
MOPs, MOEs
EVALUATIONPHASE
MISSION NOW
NOW + 1
NOW + 2
NOW + kPLANNED INPUTS: SYSTEMS AND PROCEDURES
Time
ASC - 23I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Remarks
A key product of the architecture development process is the Integrated Dictionary - a data base containing all of the information describing an architectureAll this information is necessary but not sufficient for evaluating an architectureNote that the DODAF 1.0 does not address temporal issues – none of the 26 products contain time information necessary for simulationFor evaluation, we also need scenarios, key threads, and metrics
ASC - 24I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Navy’s ASN (RDA) CHENG Approach
The DoDAF artifacts can be aligned to the classical systems engineering process.
The DoDAF can provideTraceability from:
Capability Needs to Functional Analysis to Interoperability
Assessments to Performance andBehavior Predictions.
With proper organization, the DoDAFartifacts can support Analysis of Alternatives for Acquisition Decision Making.
ASC - 25I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Focus is on Capabilities
EXECUTABLE
USER INTERACTION
TechnicalArch. view
OperationalArch. view
SystemsArch. view
IntegratedDictionary
USER SPECIFIEDCAPABILITIES BEHAVIORS
ARCHITECTURE SPECIFIED BEHAVIORS
User behavioral and performance requirements vs. architecture enabled behavior and performance
ASC - 26I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Air Force CONOPSAir Force CONOPSGlobal Vigilance Global PowerGlobal Reach
GlobalPersistent
AttackCONOPS
GlobalPersistent
AttackCONOPS
GlobalMobility
CONOPS
GlobalMobility
CONOPS
Nuclear ResponseCONOPS
Nuclear ResponseCONOPS
Space & C4ISRCONOPS
Space & C4ISRCONOPS
Global Strike
CONOPS
Global Strike
CONOPS
Homeland Security CONOPS
Homeland Security CONOPS
ASC - 27I n t e g r i t y - S e r v i c e - E x c e l l e n c e
A Precondition: Concordance
A total view of the architecture (all views and products) is necessary from the beginning so that the various products are developed in the context of the other ones (this is one of the key Architect’s tasks)
Concordance among products must be ensuredA tedious, but essential activity
While the relationships among the various products are well understood, maintaining Concordance throughout the design process is an Architect’s responsibilityIf a CASE tool allows the creation of all the models using the same data dictionary (the integrated dictionary), the concordance is facilitated
StartWait for incoming
threat
Threat detected
Intercepting order issued
Interceptor en route
Interceptor in firing
range
Weapons away
Contact message sent
Threat replied and complied
Threat silent
incoming threat ««398»»
Generate intercepting order ««399»»
Interceptor takes off ««400»»
Interceptor close to the threa ««401»»
state=war ««405»»
response from the threa t««403»»
state=peace ««402»»
no response ««404»»
order=fire ««406»»
Return to base««407»»
Return to base««408»»
Function :S1: if (Surv-Dir-Content=default) and (Threat-Status=incoming)
then (TP-Update=new) and (Threat-Status=sensed)S2: if (Surv-Dir-Content=track-interc) and(Threat-Status=engaged)
then (TP-Update=in_firing_range)S3: if (Surv-Dir-Content=assess_kill) and (Threat-Status=destroyed)
then (TP-Update=killed)
Function C1: if (TP-Update=new) then (Order-Content=intercept)C2: if (Report-Content=interceptor_away) then (Surv-Dir-Content=track_interc)C3: if (TP-Update=in_firing_range) and (State=war)
then (Order-Content=fire)C4: if (TP-Update=in_firing_range) and (State=peace)
then (Order-Content=make_contact)C5: if (Enemy-Response=silence) and (Report-Content=contacted) then (Order-Content=fire)C6: if (Report-Content=weapons_away) then (Surv-Dir-Content=assess_kill)
Function A1: if (Order-Content=intercept) and (TP-Update=new)
then (Action=take_off) and (Report-Content=interceptor_away)A2: if (Order-Content=fire)
then (Action=launch_weapons) and (Report-Content=weapons_away)A3: if (Order-Content=make_contact) then (Action=contact) and (Report-Content=contacted)
Rules for the scenario driver:if (Action=take_off) then (Threat-Status=engaged) after delayif (Action=launch_weapon) then (ThreatStatus=destroyed) after delayif (Action=contact) and (Threat-Behavior=reply_comply) then
(Enemy-Response=reply_and_comply)if (Action=contact) and (Threat-Behavior=silent) then (Enemy-Response=silence)
Additional rule for Sense Function:- Associate threat and interceptor tracks:
if (Surv-Dir-Content=track_interc) and (Threat-Status=sensed) and (THREAT.Interceptor-ID=0)then (THREAT.Interceptor-ID=SURVEILLANCE-DIRECTIVE.Interceptor-ID)
Sense
Command
Act
inc
or
Surv-Dir-Content««298»»: default««241»», track_interc««299»»««327»»««422»», assess_kill««287»»««337»»Threat-Status««242»»: incoming««245»», sensed««244»»««423»», engaged««240»»««345»», destroyed««305»»««347»»TP-Update««306»»: new««307»»««325»»««339»», in_firing_range««308»»««328»»««331»», killed««309»»Order-Content««296»»: intercept««311»»««338»», fire««312»»««335»»««340»», make_contact««313»»««341»»Action««314»»: take-off««315»»««343»», launch_weapon««316»»««346»», contact««317»»««348»»««350»»Report-Content««318»»: , interceptor_away««319»»««326»», weapons_away««320»»««336»», contacted««321»»««334»»Enemy-Response««322»»: reply_and_comply««323»», silence««324»»««352»»State««333»»: peace««332»», war««329»»Threat-Behavior: reply_comply««349»», silent««351»»THREAT.Interceptor-ID««420»»: 0««421»», 1, 2, ...
new
Track-IDThreat-StatusInterceptor-ID (FK)
THREAT /1
Threat-Status
THREAT
Interceptor-ID
Track-ID (FK)Interceptor-ID (FK)Surv-Dir-Content
SURVEILLANCE-DIRECTIVE /2
Surv-Dir-Content
SURVEILLANCE-DIRECTIVE
Track-ID (FK)Interceptor-ID (FK)TP-Update
TACTICAL-PICTURE /3
TP-Update
TACTICAL-PICTURE Track-ID (FK)Interceptor-ID (FK)State (FK)Order-ContentOrder-NB (AK1)
ORDER /4
Order-Content
ORDER
State
RULES-OF-ENGAGEMENT /5RULES-OF-ENGAGEMENT
State
Track-ID (FK)Interceptor-ID (FK)Report-ContentReport-NB (AK1)
REPORT /6REPORT
Report-Content
is reported in
is used for the generation of
is consistent with
Track-ID (FK)Interceptor-IDAction
CONTROL-TO-INTERCEPTOR /7CONTROL-TO-INTERCEPTOR
Action
is reported into engage
results in
can trigger
constrains the construction of
constrain the generation of
Track-ID (FK)Interceptor-ID (FK)Enemy-Response
INTERCEPTOR-REPORT /9
Enemy-Response
results in
can trigger
INTERCEPTOR-REPORT
ACTIVITY
DATA
RULEDOMAINS
DYNAMICS
Sense««274»»
A1
Command««276»»
A2
Act««277»»
A3
ORDER««284»»
SURVEILLANCE-DIRECTIVE««279»»
TACTICAL-PICTURE««281»»
REPORT««286»»
NEW-TRACK««283»»
RULES-OF-ENGAGEMENT««280»»
THREAT««278»»
CONTROL-TO-INTERCEPTORx««285»»
INTERCEPTOR-REPORT««90»»
ICOM/Entity
Activity/Rule
Attribute/Domain
Transition/Rule
Value/Rules
ASC - 29I n t e g r i t y - S e r v i c e - E x c e l l e n c e
ObservationsEvaluation requires an understanding of the purpose and desired use of the systems or System-of-Systems that would be built conformant to the architecture. Generally, this is expressed as requirements. These can be decomposed into internal requirements and external requirements.External requirements have to do with the ability of systems built conformant to the architecture in question to interface with other systems that are external to the architecture itself. One of the main goals of DOD in using architectures is to ensure interoperability between systems to enable the quick synthesis of “go to war” capabilities.To evaluate architectures with respect to interoperability, DOD has instituted processes for comparing the static views of architectures of different programs to check the proposed interfaces between systems. However, interoperability goes beyond just ensuring that communications system and messages are compatible. The dynamic behavior and performance of the systems also must be considered to ensure interoperability.C4I Support Plan
ASC - 30I n t e g r i t y - S e r v i c e - E x c e l l e n c e
ObservationsThe evaluation of the internal requirements of an architecture is a complex undertaking with many aspects. These aspects can be classified as follows:
Verification: Is it really an architecture? Are the views and the products that make up each of the three views (Operational., Systems, and Technical) mutually consistent (the concordance problem)?Logical Consistency: Is the architecture logically correct? Is the operational concept represented correctly for the specified mission or set of missions? Are the Information Exchange Requirements satisfied?Behavioral Correctness: Does the architecture exhibit the desired behaviors? Performance Requirements: If a set of systems are chosen to implement the architecture, will the System-of-Systems meet the performance requirements of the mission?
ASC - 31I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Executable Model
The executable model has several characteristicsIt is derived from the architecture design in a traceable wayIt has an underlying mathematical model that enables the application of analytical tools (algorithms)It enables simulation
It is important to understand that building the executable model does not provide in itself an evaluation.
Verification and logical consistency outline the steps of a process for evaluation, and the executable model becomes an important tool in that process.
Each step requires more effort and additional information.
One advantage of this staged approach is that one does not need to enter the details of the systems, a very laborious and costly undertaking, until the previous three stages are completed satisfactorily.
ASC - 32I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Challenge of Architecture Evaluation
The real goal: The operators define capabilities that they require in order to achieve the desired effects and technologists work with them to develop alternative solutions that include the legacy systems.
The Problem: Traditional Requirements based acquisition has become problematic
The Solution: Operators and designers are expected (or being forced) to work together from the beginning and must continue working together to the end – the demarcation lines between requirements and specs and development are disappearing
The use of architectures is only beginning to yield the expectedimprovements in acquisition and in war fighter capability
ASC - 33I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Problem Statement
Given that an organization develops an architecture description that is conformant to the DOD Architecture FrameworkGiven that this architecture description is contained in an Integrated DictionaryWe need to address the following layered questions:
Is the architecture logically correct?Does the architecture exhibit the desired behavior?Do instantiations of this architecture exhibit the desired performance characteristics?Do systems built in conformance to this architecture provide the desired capability?Can we analyze alternatives?
The questions pose strong analytical challenges (metrics and algorithms) and Modeling and Simulation challenges
ASC - 34I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Layer 1
Is the architecture logically correct?This issue has to do with the Operational Architecture View.
Is the architecture a correct implementation of the CONOPS?Does the CONOPS work or are there logical inconsistencies?
In a way, this can be seen as a validation of the CONOPS; a necessary first step since the architecture approach is driven by the CONOPS.Both analytical tools and simulation can be used for this evaluation. The executable model (discrete event dynamical system) is essential for this analysis.
ASC - 35I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Layer 2
Does the architecture exhibit the desired behavior?
This issue has to do with the Operational Architecture View and with the Systems Architecture View.
The CONOPS usually specifies some required behaviors. Are these behaviors (sometimes expressed as key threads) included in the operational view?Does the system architecture view that represents an instantiation of the operational architecture view enable these behaviors?
In a way, this can be seen as a validation of the Systems Architecture view with respect to the Operational Architecture view. At issue here is that the systems architecture view may contain many legacy systems. Analytical/algorithmic approaches as well as Modeling and Simulation approaches are appropriate.
ASC - 36I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Layer 3
Do instantiations of this architecture exhibit the desired performance characteristics?This issue has to do with the Systems Architecture View.
To evaluate performance, system characteristics (performance parameters) need to be included In many cases, we have crossed the hard-to-define boundary between architecture and system design
We may need to go to this level for a sound analysis of alternativesThis level often requires the use of discrete event dynamical system models (event driven) as well as time-driven models
This poses a major unresolved technical challenge: how to interconnect time driven and event driven models
∆t
ASC - 37I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Layer 4
Do systems built in conformance to this architecture provide thedesired capability?
This is the hard one, but fortunately some progress has been achieved at the research level, but not implemented yet at the industrial levelMuch work needs to be done in articulating capabilities and expressing them in measurable terms (Measures of Merit for a capability): Questions range from “Can I do this?” to “How well can I do this?”Issue: Formal construction of key threads
ASC - 38I n t e g r i t y - S e r v i c e - E x c e l l e n c e
AHL1
Layer 5
Analysis of AlternativesA host of sub-problems are included in this one
How do I compare two Operational Architecture views that represent the same CONOPS? Given the same operational architecture view, how do I compare two alternative Systems Architecture views? A key acquisition issue when DOD provides the Operational Architecture?How do I compare two very different key threads that
represent the same capability? What about unintended consequences?How do I trace to architectural issues the differences in performance of two alternative large proposed systems?
Slide 38
AHL1 Alexander H. Levis, 10/13/2003
ASC - 39I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Conclusion
We have all began on a journey of discovery
We understand the fundamentals
There is sufficient theory in mathematics and computer science and modeling & simulation technology that needs to be exploited for this class of problems; there are also several key theoretical problems with temporal issues that need to be resolved
We need an M&S program (probably Joint)
But we need to build on those and develop applications that willtake us to the desired end
ASC - 40I n t e g r i t y - S e r v i c e - E x c e l l e n c e
A Simple Example: Air Intruder (AI) Operational Concept Description
A very simple example is used to illustrate the various Architectural productsReal examples run into 100’s of pages The AI system will protect a restricted area against intrudersWhen and intruder (threat) is detected, an aircraft is sent to interceptAccording to the level of hostility, the engagement takes different forms;
During war time (war), the interceptor shoots the intruder without warningDuring peace time, the interceptor attempts to contact the intruder and shoots at it, if it does not reply
Intruders must be killed within a specified time, otherwise theywill be leakers, able to attach targets
ASC - 41I n t e g r i t y - S e r v i c e - E x c e l l e n c e
The Air Intruder Example (OV-1)
ISR COMMANDCENTER
CONTROL
INTERCEPTORTHREAT/INTRUDER
HIGHERHEADQUARTERS
System Boundary
ASC - 42I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Node Connectivity Description (OV-2)
Threat
ISR Node
HHQ
Interceptor Node
Control Node
Command Node
ThreatStatus
Rules of Engagement
Tactical Picture
Surveillance Directive
OrderReportTactical Picture
Control to Interceptor
Interceptor Report
ThreatStatus
[Sense][Command]
[Control][Engage]
Architecture Boundary
ASC - 43I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Information Element Exchange Matrix (OV-3)
Information Source Information Destination
Operational Information Media Size Unit Operational Activity Operational ActivityElement Element Element
Threat-Status Microwave 10K Byte Intruder N / A ISR SenseThreat-Status Microwave 10K Byte Interceptor Engage N/A N/AInterceptor Control Microwave 10K Byte Controller Control Interceptor EngageInterceptor Report Microwave 10K Byte Interceptor Engage Controller ControlROE WAN 10K Byte HHQ N / A Command Center CommandTactical Picture Microwave / WAN 1M Byte ISR Sense Command Center CommandTactical Picture Microwave / WAN 1M Byte ISR Sense Controller ControlSurveillance-Directive Microwave / WAN 10K Byte Command Center Command ISR SenseOrder WAN 10K Byte Command Center Command Controller ControlAct-Report WAN 10K Byte Controller Control Command Center CommandInterceptor-ID WAN 10K Byte Interceptor Engage Controller Control
ASC - 44I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Command Relationship Chart (OV-4)
Higher HQ
ISR Asset
Command Center
Controller
Interceptor
ASC - 45I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Functional Decomposition
Three main Functions: Sense, Command, Act
SENSE COMMAND ACT
CONDUCT AI
CONTROL ENGAGE
ASC - 46I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Activity Model (OV-5)CONTEXT DIAGRAM
Purpose: illustrate IDEF0 and the relationships between the different types of models: process, data, rules, and dynamics
Viewpoint: Architect
RULES-OF-ENGAGEMENT
THREAT
NON_ENGAGED_THREAT
DESTROYED_THREAT
LEAKED_THREAT
ConductAir Intrusion
(AI)A0Operations
ASC - 47I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Activity Model (OV-5)FIRST LEVEL OF DECOMPOSITION
SENSE
A1
COMMAND
A2
ACT
A3P. 3
RULES-OF-ENGAGEMENT
THREAT
NON_ENGAGED_THREATDESTROYED_THREATLEAKED_THREAT
TACTICAL-PICTURE
ORDER
REPORT
NEW-TRACK
SURVEILLANCE-DIRECTIVE
ASC - 48I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Activity Model (OV-5)DECOMPOSITION OF ACT (A3)
CONTROL
A31
ENGAGE
A32
ORDER
REPORTNEW-TRACK
NON_ENGAGED_THREAT
DESTROYED_THREAT
LEAKED_THREAT
THREAT
INTERCEPTOR-ID (RTB)
ACT_REPORT
INTERCEPTOR_REPORT
CONTROL-TO-INTERCEPTOR
ASC - 49I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Logical Data Model: IDEF1x (OV-7)
Nine Entities: (Corresponding to the ICOM of the IDEF0 Model)ThreatSurveillance DirectivesTactical PictureOrdersRules of EngagementReportControls to InterceptorAct ReportInterceptor Report
ASC - 50I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Logical Data Model (OV-7)
Track-IDThreat-Status Interceptor-ID (FK) Time_sensed Time_engaged
THREAT /1THREAT
Interceptor-ID
Track-ID (FK) Interceptor-ID (FK)
SURVEILLANCE-DIRECTIVE /2
Surv-Dir-Content
SURVEILLANCE-DIRECTIVE
Track-ID (FK) Interceptor-ID (FK)
TACTICAL-PICTURE /3
TP-Update
TACTICAL-PICTURETrack-ID (FK) Interceptor-ID (FK) State (FK) Order-Content Order-NB (AK1)
ORDER /4ORDER
RULES-OF-ENGAGEMENT /5RULES-OF-ENGAGEMENTState
Track-ID (FK) Interceptor-ID (FK) Report-NB (AK1)
REPORT /6REPORT
Report-Content
is reported in
is used for the generation of
is consistent with
Track-ID (FK)
Action
CONTROL-TO-INTERCEPTOR /7
ActionInterceptor-ID
CONTROL-TO-INTERCEPTORto engage
results in
can trigger
constrains the construction ofconstrain the generation of
Report_Type
Track-ID (FK) Interceptor-ID (FK) Enemy-Response
INTERCEPTOR-REPORT /9INTERCEPTOR-REPORT
Track-ID (FK) Interceptor-ID (FK)
ACT-REPORT /8ACT-REPORT
triggers
Owned Attributes in Italics
1 1
1
p
1
1
p
1
1
ASC - 51I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Rule Model (OV-6a)
• The first step is to define the values taken by the attributes of interest
• Attributes of interest for the AI example:• Surv-Dir-Content: default, track-interc, assess-kill• Threat-Status: incoming, sensed, engaged, destroyed• TP-Update: new, in-firing-range, killed• Order-Content: intercept, fire, make-contact• Action: take-off, launch-weapons, contact• Report-Content: interceptor-away, weapons-away, contacted• Enemy-Response: reply-and-comply, silent• State: war, peace
ASC - 52I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Rule Model (OV-6a)
Function SENSE
S1: if (Surv-Dir-Content=default) and (Threat-Status=incoming) then (TP-Update=new) and (Threat-Status=sensed)
S2: if (Surv-Dir-Content=track-interc) and(Threat-Status=engaged) then (TP-Update=in_firing_range)
S3: if (Surv-Dir-Content=assess_kill) and (Threat-Status=destroyed) then (TP-Update=killed)
S4: if (Surv-Dir-Content=track_interc) and (Threat-Status=sensed) and (THREAT.Interceptor-ID=0)then (THREAT.Interceptor-ID=SURVEILLANCE-DIRECTIVE.Interceptor-ID)
ASC - 53I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Rule Model (OV-6a)
Function COMMAND
C1: if (TP-Update=new) then (Order-Content=intercept)C2: if (Report-Content=interceptor_away) then (Surv-Dir-
Content=track_interc)C3: if (TP-Update=in_firing_range) and (State=war)
then (Order-Content=fire)C4: if (TP-Update=in_firing_range) and (State=peace)
then (Order-Content=make_contact)C5: if (Enemy-Response=silence) and (Report-Content=contacted)
then (Order-Content=fire)C6: if (Report-Content=weapons_away) then (Surv-Dir-
Content=assess_kill)C7: if (Enemy-Response=reply_and_comply) and (Report-
Content=contacted) then delete Report
ASC - 54I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Rules Model (OV-6a)
Function CONTROL
CT1: if (Order-Content=intercept) and (TP-Update=new)then (Action=take_off) and (Interceptor-ID=int) after delayand (Report-Content=interceptor_away)
CT2: if (Order-Content=fire) then (Action=launch_weapons) and (Report-Content=weapons_away)
CT3: if (Order-Content=make_contact) then (Action=contact) and (Report-Content=contacted)
Function ENGAGE
E1: if (Action=take_off) then (Threat-Status=engaged)) after delayE2: if (Action=launch_weapon) then (ThreatStatus=destroyed) and
(ThreatStatus = (if time<400 then destroyed, else leaked) after delayE3: if (Action=contact) and (Threat-Behavior=reply_comply) then (Enemy-
Response=reply_and_comply) and (ThreatStatus=not_engaged)E4: if (Action=contact) and (Threat-Behavior=silent) then (Enemy-
Response=silence)
ASC - 55I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational State Transition Description (OV-6b)
HIGH LEVEL
StartWait for incoming
threat
Threat detected
Intercepting order issued
Interceptor en route
Interceptor in firing
range
Weapons away
Contact message
Threat replied and
complied
Threat silent
incoming threat
Generate intercepting order
Interceptor takes off
Interceptor close to the threat
state=war
response from the threat
state=peace
no response
order=fire
Return to base
Return to base
sent
ASC - 56I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational State Transition Description (OV-6b)
MORE DETAILED LEVEL
Threat Status=engaged
Wait for incoming
threat
Evaluating Threat
Do: TP-update
Generating Order
Generating Interceptor Command Do: Select Interceptor
Generating Launch Weapon Message
Generating Look
Command
Queuing Sensor
Do: Surveillance
Directive
Waiting for Interceptor
Report
Sensing
Threat Not Engaged
Threat Killed
Threat Leaked
Start
incoming threat
TP=new
Order=Intercept
Interceptor away
Surveillance Directive Sent
TP= in_firing_range
Order=contact [state=peace]
Order=fire [state=war]v[silence]
Control=contact
reply_and_comply Enemy Response=silence
Control=Launch_weapons
Threat Status=destroyed
TP=killed [time<400]
TP=killed [time>=400]
ASC - 57I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Operational Event Trace Description (OV-6c)
ThreatISR Node Command Node
HHQControl Node
Order
IncomingSurveillance Directive
ROE
Engage Node
Surveillance DirectiveControl to Interceptor
Tactical Picture (New)
Report
Weapon away
Tactical Picture (update)Order
In firing range
Interceptor ReportSurveillance DirectiveDestroyed
Tactical Picture (update)X
ASC - 58I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Requirements
Required Measures of Performance:
Average response time < 400 secondsNo more than two leakers
A leak occurs if an intruder is not intercepted within 400 seconds
No constraint on throughput rate
ASC - 59I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Variable Specifications
Need to identify variables to characterize:Inputs:
Number of threats (integer)Input Interarrival Time (seconds per threat)
System:Number of interceptors (integer)Processing delays (seconds)Number of processing resources (integer)
Output:Average time to kill (seconds)Throughput rate (kills per seconds)Number of leakers (integer)
PARAMETERS
MEASURESOF
PERFORMANCE
ASC - 60I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Performance Evaluation
Varying parameters:Input interarrival time: from 0 to 100 secondsNumber of interceptors: 3, 4 or 5
Fixed parameters:Number of threats = 101 Resource for each functionProcessing delays...
ASC - 61I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Parameter Locus - Discrete
0 1 2 3 4 5 6 70
10
20
30
40
50
60
70
80
90
100
Number of Interceptors
Inpu
t Int
erar
rival
Tim
e
SelectedValues
ASC - 62I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Parameter Locus - Continuous
Number of Interceptors
0 1 2 3 4 5 6 70
10
20
30
40
50
60
70
80
90
100
Inpu
t Int
erar
rival
Tim
e
To visualize the transformation from Parameters to Measures ofPerformance better, consider the shaded space.
ASC - 63I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Collecting MOPS
AFCEA SL 1100© A. H. Levis Lect. 11 - 63
Executable model using Colored Petri Nets
ASC - 64I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Performance Locus
0200
400600
0
0.010.02
0.030
2
4
6
8
10
Average Response TimeThroughput Rate
Leak
s
Three dimensional surface of Measures of Performance
ASC - 65I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Performance Locus
0 100 200 300 400 500 600 7000
0.005
0.01
0.015
0.02
0.025
0.03
Average Response Time
Thro
ughp
ut R
ate
Projection of the Performance Locus on the Throughput Rate / Av. Response Time plane
ASC - 66I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Performance Locus
0 100 200 300 400 500 600 7000
1
2
3
4
5
6
7
8
9
10
Average Response Time
Leak
s
Projection of the Performance LocCost / Av. Response Time p
us on the lane
ASC - 67I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Performance And Requirement Loci
Average Response Time0 100 200 300 400 500 600 700
Lp
Lr
RequirementsLocus
0
1
2
3
4
5
6
7
8
9
10Le
aks
Performance Locus
MOE =S(Lp∩ Lr)Measure of
S(Lp)= 30%
Effectiveness
ASC - 68I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Back To The Parameter Locus
0 1 2 3 4 5 6 70
10
20
30
40
50
60
70
80
90
100
Number of Interceptors
Inpu
t Int
erar
rival
Tim
e
MOE = 75 %
As the number of interceptors increases, higher rate attacks can be handled –the relationship is nonlinear
ASC - 69I n t e g r i t y - S e r v i c e - E x c e l l e n c e
Conclusion
This example illustrates the process of using an architecture to do trade-off analyses such as the ones done for the CRRA
The Operational View products instantiated the Operational Concept
The Systems View was very simple and was suppressed (no SV products) but the information was used in the scenario and the executable model (performance parameters of the assets)
The logical analysis established that the Rules were consistent and coherent
The behavioral analysis established key threads
The Performance evaluation allowed the Analysis of Alternatives
These enables the use of the architecture in the assessment of Systems-of-Systems
I n t e g r i t y - S e r v i c e - E x c e l l e n c e
70
Headquarters U.S. Air Force
MODELING AND SIMULATION FOR ARCHITECTURE ASSESSMENT
Alexander H. LevisAF/ST
4/20/04