Modeling for Verification and Validation WorkshopOverview and Scope

Tuesday, September 23, 2015FAA William J. Hughes Technical Center

William D. MillerStevens Institute of Technology

INCOSE INSIGHT Magazine Editor-in-ChiefFormer INCOSE Technical Director

Focus

• Why Model• Some Definitions• Types of Models• Challenges of Modeling• Some Successes• Model-Based Integration and Test• Projected Growth in Computing Capability

2

Why Model

• Cost avoidance• Validate

– Requirements– Architecture– System– Performance

• Verify systems against requirements• What if

3

Some Definitions

4

• Models – physical, analytical, or logical representation of a system, entity, phenomenon, or process

• Simulation – implementation of a model over time– Virtual … represent systems both physically and electronically, e.g.,

flight simulator– Constructive … use of mathematical and decision-based modules and

statistical techniques– Live … simulated operations conducted by real operators using real

equipment• Fidelity – degree to which aspects of the real world are

represented• Resolution – degree to which physical (appearance) aspects

of the real world are represented … does it look like the real thing.

Types of Models

5

Complex SystemComplex System

Physical ModelsPhysical Models

Abstract ModelsAbstract Models

ScaledWooden Models

ScaledWooden Models

Prototype Models

Prototype Models

Simulation Models

Simulation Models

Virtual RealityVirtual Reality

Logical ModelsLogical Models

Event DrivenSystem DynamicsMonte Carlo

Analytic ModelsAnalytic Models

e.g., Systems of Equations

DeterministicStochastic

RequirementsStructureBehaviorParametrics

Human centric

Challenges of ModelingGeorge Box, published in proceedings of a 1978 statistics workshop: •Now it would be very remarkable if any system existing in the real world could be exactly represented by any simple model. However, cunningly chosen parsimonious models often do provide remarkably useful approximations. For example, the law PV = RT relating pressure P, volume V and temperature T of an "ideal" gas via a constant R is not exactly true for any real gas, but it frequently provides a useful approximation and furthermore its structure is informative since it springs from a physical view of the behavior of gas molecules.•For such a model there is no need to ask the question "Is the model true?". If "truth" is to be the "whole truth" the answer must be "No". The only question of interest is "Is the model illuminating and useful?".

https://en.wikipedia.org/wiki/All_models_are_wrong

6Beware of emergent behaviors in socio-cyber-physical systems!

Some Successes• Manhattan Project (1940s) from Richard W. Hamming, The Art of Doing

Science and Engineering– Design options modeled and simulated on IBM accounting machines until a

design was chosen to test– Last minute assessment of probability that the first live test would ignite the

atmosphere

• Boeing 777 from Karl Sabbagh, 21st Century Jet– Computer-graphics Aided Three-dimensional Interactive Application (CATIA)– Electronic Preassembly in the Computer (EPIC) replaced mock-ups – Flight control system models

• Semiconductors– Formal methods to verify designs driven by Intel’s Pentium chip design defect

• Lithographic Machines from Jan Tretmans, editor, Embedded Systems Institute, Tangram: Model-based integration and testing of complex high-tech systems– Reduction in testing interval for next gen type systems driven by Moore’s Law

7

Model-Based Integration and Test

8

Requirements R, designs D, models M, realizations Z of a system with n components and infrastructure that allows integration of models and realizations

integrate

designR D

R1

Rn

define

define

design

design

D1

Dn

M1

Zn

model

realize

define

Mnmodel

Z1

infr

ast

ruct

ure

I

integraterealize

Model-Based System Testing

Model-BasedSystemIntegration

Integration

DesignR D

R 1

R n

Subsystem Requirements

Subsystem Requirements

Subsystem Design

Subsystem Design

D1

Dn

M1

Zn

Model

Realization

MnModel

Z1

infr

astr

uctu

re

I

IntegrationRealization

System Requirements

ESI

Projected Growth in Computing Capability

9

Petaflops

Baseline: China’s Tianhe-2 computer rated at 33.86 petaflopsAssumption: Moore’s Law holds up for the next 9 years

Presenters

1. Bill Miller (Stevens Institute and INCOSE)2. Mark Flanigan/Simon Daykin (NATS UK)3. Don Firesmith (SEI Carnegie Mellon) 4. Paul Miner (NASA)5. David Allsop (Boeing)6. Jonathan Hammer (Noblis)

10