analysis process
DESCRIPTION
Systems Engineering Program. Department of Engineering Management, Information and Systems. EMIS 7305: Systems Reliability, Supportability and Availability Analysis. Analysis Process. UPDATED 01.15.08. Dr. Jerrell T. Stracener, SAE Fellow. Leadership in Engineering. - PowerPoint PPT PresentationTRANSCRIPT
Analysis Process
Dr. Jerrell T. Stracener, SAE Fellow
UPDATED 01.15.08
Leadership in Engineering
EMIS 7305: Systems Reliability, Supportability and Availability Analysis
Systems Engineering ProgramDepartment of Engineering Management, Information and Systems
Purpose of Analysis
To impact the decision-making process
The Analysis Process
1. Define the problem and formulate an objective
2. Identify the analysis options for accomplishing the objective and down-select to the preferred option.
3. Perform the analysis and draw conclusions
4. Present the analysis results, both technically and from the layperson perspective
Problem Solving Approach
• Define the problem• Visualize• Diagram• Review the problem definition & objective• Formulate possible alternates for solving theproblem and down select to the preferred method• Solve the problem• Review the results and prepare report• Present report
AFSAA TRAINING -- “STEP #1 OF THE SCIENTIFIC
METHOD:DEFINING THE PROBLEM”
byRoy E. Rice, Ph.D., P.E.
AF OR FAM Course25 Apr 2005
DEFINING THE PROBLEM
• Introduction• The Scientific Method• The Role of the People involved in the Analysis• Explanation of established procedures• The Template• Examples
Outline
INTRODUCTION
Decision Making = choosing among alternatives
Objective / Question
Analyst (analysts, scientists, engineers, etc)
Models (analytical, simulations, etc.)
Data / Assumptions
En
sure
ad
equ
ate
ran
ge
of
alte
rna
tiv
es
Est
ab
lish
m
ean
ing
ful
mea
sure
s
En
su
re t
rac
ka
bil
ity
/tr
ac
ea
bil
ity
Def
ine
issu
es/
qu
esti
on
s
The product of decision makers
Choices will be made with or without analysis
Key pillars of analysisHelping the decision maker understand problems and
candidate solutions
The foundations of credible analysis
Tools, techniques, and infor-mation surrounding the art of timely, informative support to
decision makers
Role of Analysis
INTRODUCTION
The EssentialIngredients of
Analysis
Objective/Question
Analyst
Model(s)Data/Assumptions
THE SCIENTIFIC METHOD
• Define the Problem• Develop Alternatives• Develop Criteria to distinguish among alternatives• Develop “measures” of the criteria• Determine Analytic Methodology (to include DOE)• Determine “tool(s)” to execute the methodology• Gather data• Exercise “tool(s)”• Analyze results• Report Results
Typically we jump to these steps
Typically we spend the least amount of our time in this step
“Operations research is a scientific method of providing executive departments with a quantitative basis for decisions regarding the operations under their control.” -- Morse and Kimball
THE NATO C2 CODE OF BEST PRACTICES (COBP)- DRAFT
PerformProblem
Formulation
DevelopSolutionStrategy
EstablishMoM
I dentifyScenarios
EstablishMethods & Tools
& Req.
StudyProducts
RepresentHuman/ Org.
I ssues
R
R
I
StudyRisk
Data Collection& Analysis Plan
Risk & Uncertainty
Study
R External Review
I I nternal Review
What How
Flawed
Acceptable
Acceptable Risk
Unacceptable Risk
FormulatedProblem
SolutionStrategy
DEFINE THE PROBLEM
• Step 1 – “Define Problem”
• Step 1a (Roy’s Rule)– “Define the Problem again…because you didn’t do it
right the first time!”
• Step 1b (Vince Roske’s Rule)– “First find out what the question is - then find out
what the real question is!”
• EXAMPLE: Close Air Support Study in J8
“It often occurs that the major contribution of the operations research worker is to decide what is the real problem.” -- Morse and Kimball
DEFINE THE PROBLEM- WHY? -
• Perfectly right answer to the totally wrong question!• Probably waste resources• Probably lose Credibility• NATO COBP - DRAFT
– “In the initial problem formulation iteration, it is critical to begin with an understanding of the REAL problem rather than a determination to apply readily available tools, scenarios, and data.”
– “Explicit problem formulation must precede construction of concepts for analysis or method selection…Proper resourcing of problem formulation activities will improve the overall efficiency and quality of the study.”
– “An understanding of the decisions to be supported by the analysis and the viewpoints of the various stakeholders is essential to clarifying the study issues.”
THE ROLE OF THE PEOPLE INVOLVED IN PROBLEM DEFINITION
• The Decision Maker– Communicate the “context” of the Problem– “Buy into” the Problem and Solution Approach– Provide “Heading Checks”– Have Open Door– Commitment vs. Participation - “ham & eggs”
• The Mid-level Manager– Be a Conduit - Major General Robinson (J8)
• “What does your FINAL chart look like?”
– Must understand analysis - what it can/cannot do
• The Analyst– Articulate the Problem back to D.M.– Must put himself/herself in the D.M.’s shoes
Problem Definition is an Iterative Process
EXPLANATION OF ESTABLISHED PROCEDURES
• Initial Diagnosis– Identify critical factors– What the principle decisions are– What the MOEs are– Tradeoffs to compare alternatives
• Problem Elements– Controllable or decision variables– Uncontrollable variables– Restrictions or constraints on the variables– Objectives for defining a good or improved solution– “Confines” of the analysis - multifold impacts
Principles of Management Science - Wagner
EXPLANATION OF ESTABLISHED PROCEDURES
• Decision Objectives - goal of the decision• Problem Context
– Stakeholders– Triggers– Influences
• Problem Boundaries– Timeframe– Rule sets– Facts– Assumptions
• Analytic Objectives
Naval War College Pub
EXPLANATION OF ESTABLISHED PROCEDURES
• Context of the study– Geopolitical context that bounds the problem space– Political, social, historical, economic, geographic, technological
environments– Actors– Threats– Aim and objectives of the analysis, including the decisions to be supported– Generic C2 issues– Relevant previous studies
• Aspects of the problem– Issues to be addressed– Assumptions– High-level MoMs– Independent variables (controllable and uncontrollable)– Constraints on the values of the variables (domain and range)
NATO COBP - DRAFT
EXPLANATION OF ESTABLISHED PROCEDURES
• Rice’s Method (from Dr. Bill Lesso, University of Texas at
Austin)
Given:
The problem is to:
By choice of:
Subject to:
RICE’S METHOD
Given: set the environment; state your assumptions
The problem is to: clear statement of the specific problem to be solved
By choice of: determine the decision variables
Subject to: determine the constraints and restrictions
RICE’S METHOD
Given:- what is the timeframe of the study?- what scenario(s) are applicable?- what alternatives are to be used?- what are “fixed” parameters?- who are the players?- what are the threats?
RICE’S METHOD
Given: The problem is to: - not “to analyze” or “to study”- not a statement of fact (e.g., “we have a
pilot shortage”)- address the causes not the symptoms- what are you really trying to quantify or
measure?- probably related to operational
objectives- what...
TYPES OF MILITARY OR PROBLEMS
- Combat Analysis- - Land- - Air- - Naval- - Amphibious
- Resource Allocation
- Military Operational Testing
- Logistics Analysis
- Transportation Analysis
- Operations Planning
- Military Weapon System Acq.
Math methods of Attrition Fire AllocationKill-chainManeuverSearch TheorySimulationNumerical Methods
Probability TheroyMarkov ChainsStoch Processes
OptimizationMicroeconomicsDecision Analysis
D.O.E.SamplingStats/ANOVA/Hypothesis TestingData Analysis
Reliability & MaintainabilitySupply Support
- Neg Binomial- Poisson Proc.
Network ProgrammingQueuingThroughput Analysis
Game TheoryDeliberate Planning - TPFFD Feasibility
Rqts. DefinitionCost-benefit AnalysisAoACost Analysis
RICE’S METHOD
Given: The problem is to: By choice of: - what are the decision variables?- what does the decision maker get to
“play with”?- what does he/she have control over?- what...
RICE’S METHOD
Given:The problem is to: By choice of:Subject to: - constraints - hard; inviolable (e.g., distances,
24 hours in a day, ramp space)- restrictions - man-made (e.g., start times)- what are the constraints on the variables?- what might you want to perform sensitivity
analysis on?
EXAMPLES
EXAMPLE #1: Mobility Requirements Study - 1991
Given: 2005 Scenario with delivery schedulesUnits to be movedLift assets, capacities, and cycle timesAnnualized costsCost of prepositioning
The problem is to: determine whether a proposed mix of lift assets is adequate to meet requirements to close the force on time, and if not, determine the minimum cost mix which will meet requirementsBy choice of: the number & types of additional lift assets
Prepositioning Lift timing & taskings
Subject to: Time windows (ALD’s & RDD’s)Air & sea throughputSustainment build-up policyInitial inventory of lift assetsPrepositioning limitsAir/sea mode mixPAX arrival limits
EXAMPLE #2: Strategic Force Mix Study
Given: the most probable definition of the nuclear threat over the planning horizon and the best estimate of the effectiveness of all possible nuclear weapons to counter that threat,
The problem is to: develop the minimum cost strategic force structure (Triad platforms and weapons)
By choice of: the number & types of nuclear weapon systems that will make up the strategic arsenal in each year of the period under examination,
Subject to: constraints on sufficiency goals by which to measure force effectiveness; START limits, arsenal size and shape; production-destruction-and conversion rates; and the requirement to maintain or improve effectiveness during the transition period.
SUMMARY
• Follow the Scientific Method• Step 1: Problem Definition is the most critical
step• Problem Definition is an iterative process
– Must get Decision Maker commitment
• Procedures:– NATO C2 COBP– Rice’s Method
“In section 3.4.4 is reported a case where it was nearly decided that it was not worth while to put antiaircraft guns on merchant vessels because they did not shoot down enemy planes. It took an operations research worker to point out that, even though the enemy planes were not shot down, the antiaircraft guns were valuable because they decreased the accuracy of the enemy planes enough to lessen the chance that the merchant vessel be sunk..” -- Morse and Kimball