Calhoun: The NPS Institutional Archive

Faculty and Researcher Publications Faculty and Researcher Publications

2011-10

A Modeling Approach for Developing

System Performance Requirements (presentation)

Green, John M.

http://hdl.handle.net/10945/34360

A Modeling Approach for Developing System Performance

Requirements

John M. Green

Naval Postgraduate School

jmgreen@nps.edu

Issues to be Addressed

• The concept of system performance and how to measure effectiveness has been the topic of numerous papers of over the years.

• Typically the focus is on one system characteristic such as reliability (R) or operational availability (A0) though the Air Force Weapons System Effectiveness Industry Advisory Committee (WSEIAC) recommended that both are required along with system capability (C).

• This is in recognition that performance measures are extremely useful to the system engineer in five key areas:

1. Establishing requirements; 2. Assessing successful mission completion; 3. Isolating problems to gross areas; 4. Ranking problems relative to their potential to impact the mission; and 5. Providing a rational basis for evaluating and selecting between proposed

problem solutions and their resulting configurations.

2

Goal

• This presentation will present a top-down modeling approach based on functional flow block diagrams that shows how the system engineer can develop an overall system performance measure that is inclusive of R, A0, and C.

• It starts with the system concept and allows the system engineer to allocate performance at each layer of analysis, from system to components, ultimately providing detailed performance requirements which will provide a basis for evaluating candidate solutions.

3

Prediction

• Effectiveness calculations are about prediction

• Objective of prediction is twofold:

1. System effectiveness predictions form a basis for judging the adequacy of system capabilities

2. Cost-effectiveness predictions form a rational basis for management decisions.

4

Outline

• Three key studies

• Overview of the approach

• An example

• Summary

• References

5

Three Key Studies

– WSEIAC (Weapons System Effectiveness Industry Advisory Committee) Study (1964)

– MORS C2 Measures of Effectiveness Study (1986)

– Paper by John Marshall (1991)

– Other support work listed in references

6

# 1: WSEIAC Study

• Developed for the Air Force in 1964 and follows AFSC-375 series

• Looked at two approaches: 1. Immediately commit resources to an intuitively

plausible (re)design and surmount the problems as they arise, or

2. Explore in the “minds eye” the consequences of the (proposed) system characteristics in relation to mission objectives before irrevocably committing resources to any specific approach

• It is a framework for evaluating effectiveness

7

System Effectiveness

• Concluded that system effectiveness can be defined as a measure of the extent to which a system may be expected to achieve a set of specific mission requirements.

• System effectiveness is a function of three primary components: availability (A), dependability (D), and capability (C).

• Definition allows one to determine the effectiveness of any system type in the hierarchy of systems

8

Definitions

• Availability (A) – a measure of the condition of the system at the start of a mission, when the mission is called for at some random point in time.

• Dependability (D) – a measure of the system condition during the performance of the mission given its condition (availability) at the start of the mission.

• Capability (C) – a measure of the results of the mission given the condition of the system during the mission (dependability)

9

Mathematical Formulation

11 12 1

1 2

21 22 2

(0)

(0)

d d cE a a

d d c

1 2A a aSystem state (up/down)

1

0

2

(0)

(0)

cC

c

Capability at t0

11 12

21 22

d dD

d d

d11 = probability of operational at end given operational at start d12 = probability of fail at end given operational at start d21 = probability of operational at end given fail at start d22 = probability of fail at end given fail at start

10

#2: MORS C2 Study

DEC ISION

MAKER

IMPLEMENT

RESULTS

ANALYSIS R ESULTS

VALUES OF MEASURES

MEASUR ES FOR FUNCTIONS

SYNTHESIS OF STATICS

AND DYNAMICS

FUNCTIONS

SYSTEM ELEMENTS

PROBLEM STATEMENT

MODULAR COMMAND AND CONTROL

EVALUATION STRUCT URE (MCES)

AGGR EGATION

OF MEASURES

DATA GENERATION

EX, EXP, SIM, SUB JECTIVE

SPECIFICATION OF MEASURES

(CR ITERIA) MOP, MOE, MOFE

INTEGRATION OF SYSTEM

ELEMENTS AND FUNCTIONS

C2 PR OCESS

DEFINITION

C2 SYSTEM

BOUNDING

PROBLEM

FOR MULATION

Dimensional

Parameters

System

Subsystem

Environment

Force

MOPs

MOFEs

MOEs

1 2 3, , ,......., nMOP f p p p p

11

#3: John Marshall Paper

• Marshall developed a mathematical relationship between D, A, C, and S based on the work of Ball and Habayeb

• Related concept to an operational characteristics curve – Initial Curve is based solely on

the physics involved – Subsequent shape of the

curve is defined by variance in system design, operational usage, and environmental conditions

Threat Activity

Probability

Pta

Threat

Detect/Control/

Engage

Pdce

=Pcd*

Pcl

Threat

Attack

Pca

Ship Susceptability

Ph=P

ta*P

dce*P

ca

Ship Vulnerability

Pk/h

Ship Killability

Pk=P

h*P

k/h

Ship Survivability

Ps=1-P

k

Range

Derived

Performance

Curve

R

P

P

Cumulative

Probability

t1

Available

Performance

Curve1.0

12

A FFBD Example

http://en.wikipedia.org/wiki/Functional_flow_block_diagram

13

14

Aggregating Processes

Process A Process B Process CStart Outcome

t A B CP P P P

Process A

Process B

Start Outcome

t A B A BP P P P P

Process A

Process B

Process CStart Outcome

( )t c A B A BP P P P P P

Overview of the Approach

1. Establish the intended purpose of the system 2. Establish those system characteristics which

contribute to the designed ability of the system to accomplishment of the system purpose.

3. Measure/compute the numerical value that describes the degree to which each of these characteristics affects the accomplishment of the system purpose

4. Combine all computed/measured values into a form suitable to obtain a system operational value.

15

A SIMPLIFIED EXAMPLE

16

From the Ship’s Perspective

Prime Directive: Actively Defend Ship

Functions: Detect Control Engage

Track TEWA

Search

Radar

Optical

Search

Track

Radar

Allocated to:

Search

Auto

Track

Auto

TEWA

Weapons

Manual

Assign

Manual

Track

OR

OR

OR

Launch

OR

Functions

Process

Defend Ship against Cruise Missile Threats

17

Sensor Operational Objectives

• Required functions to be performed:

– Detection, Tracking, Classification, ID, Ranging

• Target characteristics and separation

• Coverage volume or area and background

• Atmospheric and weather conditions

18

19

Pd

R

Pd

R

Pd

R

Rmax

Rmax

IR

Radar

IFF

MIN

MAX

MIN

+

+

Topen fire

T

1.0

Tmax

T

1.0

TFC

T

1.0

TmaxT

min

Fire ControlC2

Max wait for

IFF response

Baseline Network

IFF Raleigh

Radar Swerling

IR Exponential

C2 Uniform

FC Fixed Time Delay

IFF Wait Fixed Time Delay

20

System Reaction Time

Distribution Reaction Time Distribution

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0 5 10 15 20 25

Range (NM)

Cu

mu

lativ

e P

robabil

ity

Shown without effects of D, A, or S

Sensor Operational Objectives into Effectiveness

Performance Parameters (C) • Detection range • Tracking range • Classification range • ID range • Pd • SNR minimum • Spatial resolution • Sensitivity • Total FOV (look angle) • False alarm time • Frame time • Physical characteristics

Reliability Parameters

• Dependability

• Availability

• Survivability

21

Parameters Drive IR Sensor Design

• Optics

• Detectors

• Signal processing

• Display and recording

22

In Summary

• Presented a top-down modeling approach based on functional flow block diagrams that shows how the system engineer can develop an overall system performance measure that is inclusive of R, A0, and C.

• It started with the system concept which allows the system engineer to allocate performance at each layer of analysis, from system to components, ultimately providing detailed performance requirements which will provide a basis for evaluating candidate solutions.

• Approach can be useful to the system engineer in five key areas: 1. Establishing requirements; 2. Assessing successful mission completion; 3. Isolating problems to gross areas; 4. Ranking problems relative to their potential to impact the mission; and 5. Providing a rational basis for evaluating and selecting between proposed

problem solutions and their resulting configurations

23

24

Functional and Non-functional Performance

25

References

• Ball, Robert. The Fundamentals of Aircraft Combat Survivability Analysis and Design, AIAA Press, 1985

• Bard, Jonathon. An Analytical Model of the Reaction Time of a Naval Platform. IEEE Vol. SMC-11, No. 10 Oct. 1981, pp. 723-726

• DARCOM-P 706-101 CH. 24 • Habayeb, A. R. Systems Effectiveness, Pergamon Press, 1987 • Hitchens, Derek. • Friddell, Harold G. and Herbert G. Jacks. System Operational

Effectiveness (Reliability, Performance, Maintainability), 5th National Symposium on Reliability and Quality Control, January 1959

• Marshall, John. Effectiveness, Suitability & Performance, 59th MORS, 12 June 1991

26