lesson12: supportability design reviews - dau · web view2012/12/12 · log 211 supportability...
TRANSCRIPT
LOG 211 Supportability Analysis
Student Guide
LOG 211 Supportability Analysis
Student Guide
Lesson12: Supportability Design Reviews
Content
Slide121. Lesson 12: Supportability Design Reviews
Welcome to Lesson 12 on Supportability Design Reviews.
Introduction
Content
Slide 122. Topic 1: Introduction
Comment by PDallosta: Slide 12-3 page 12-3 update TMRR
Technology Maturation & Risk Reduction
Slide 123. Life Cycle Management Framework: Where Are You? What Influence Do You Have?
Supportability design reviews offer Life Cycle Logistician’s (LCL’s) opportunities to impact a system, from “Influencing the Design” to “Supporting the Design” in the life cycle. There are numerous design reviews throughout the Life Cycle Management Framework, but for the purposes of this lesson, the Supportability design reviews listed below will be covered.
Materiel Solution Analysis Phase
Alternative Systems Review (ASR)
Technology Development Maturation and Risk Reduction (TMRR) PhaseComment by PDallosta: Updated TMRR
System Functional Review (SFR)
Preliminary Design Review (PDR)
Engineering & Manufacturing Development Phase
Critical Design Review (CDR)
Developmental Test and Evaluation (DT&E)
Functional Configuration Audit (FCA)
Production Readiness Review (PRR)
Production & Deployment Phase
Initial Operational Test and Evaluation (IOT&E)
Physical Configuration Audit (PCA)
Content
Slide 124. Topics and Objectives
Content
Slide 125. Supportability Design Reviews Lesson Approach
Key questions to ask for each review in this lesson are:
What inputs are necessary for a Supportability design review at this phase in the Life Cycle Management Framework?
How does this particular design review impact the system’s Supportability?
What must be achieved before this review is considered complete?
Overview of Supportability Design Reviews
Content
Slide 126. Topic 2: Overview of Supportability Design Reviews
Comment by PDallosta: Slide 12-7 page 12-7 update to TMRR
Technology Maturation & Risk Reduction
Slide 127. What Are Supportability Design Reviews?
Before delving into each specific Supportability design review, it is helpful to understand how reviews are used at the program level. From the perspective of the Program Manager, these reviews are an opportunity to ascertain whether or not certain criteria (described as “exit criteria” in this lesson) have been met and whether a decision to proceed is warranted. In general, the LCL supports these reviews through the Supportability analyses discussed throughout this course. The LCL should participate in a review to explain how those analyses support a decision as to whether the exit criteria have been met and the program can proceed.
In summary, these reviews may:
Serve to confirm major technical efforts within the Life Cycle Management Framework
Verify the outputs from each phase within the Life Cycle Management Framework
Ascertain progress in defining system technical requirements
Ensure that the system under review has a reasonable expectation of being judged operationally effective and suitable
Be required by contract and/or DoD policy to include review processes and requirements
Content
Technology Maturation & Risk Reduction
Slide 128. Supportability Design Reviews: Overview
In general, Supportability design reviews follow the same basic pattern:
1. An event serves as an impetus for a review conference to convene. Note: Although most reviews are part of a master schedule, the reviews are conducted to determine if the criteria have been met, not when the criteria have been met. This can be refered to as “event-driven” as opposed to “schedule-driven.”
The review consists of the IPTs and other stakeholders applying various criteria and Supportability questions.
Upon successfully completing all actions associated with a particular review, the system moves to the next review, development phase and/or project milestone. Often the outputs are used in the next review.
Although each design review requires specific entry/exit criteria to make particular decisions, in general, they include the inputs, processes and outputs described below.
Supportability Design Review Requirements
Entry Criteria: Phase-specific accomplishments required to be completed before a program is allowed to enter a particular phase in the Life Cycle Management Framework
The program may have to fulfill criteria, including:
Maturity
Performance
Documentation
Inputs: Standup Integrated Product Teams (IPTs)
Identify role: Who is doing what?
Define design review goal
Define schedule/timeline
Supportability Considerations/Questions:
Affordability?
Effective Design?
Supportability?
Functionality?
Design Readiness?
Design Capability?
Producibility?
Outputs: Integrated Product Teams
Assess the system’s Supportability requirements and characteristics for compliance to the review’s entrance and exit criteria
Identify Supportability deficiencies, identify potential impacts to the design and make recommendations regarding their resolution
Identify requirements and mechanism for exchanging information, to include analyses, reports, and data
Assign action items, specifying responsibilities, schedule and expected outcome(s)
Exit Criteria: Program-specific accomplishments that must be satisfactorily demonstrated before a program can progress further in the current Life Cycle Management Framework or transition to the next phase.
Specific Supportability Design Review Requirements, the Defense Acquisition Guidebook and Assessment Checklists
Inputs/outputs, entry/exit criteria, and Supportability questions/considerations specific to a particular review are discussed in the next topic. In addition Chapters 4, 5, and 9 of the Defense Acquisition Guidebook (DAU) provide detailed discussion of technical reviews requirements for Systems Engineering, Life Cycle Logistics and the Test & Evaluation communities.
Content
For example, Section 4.3, Systems Engineering Activities In the Life Cycle, outlines the Technical Review process. To assist in the preparation for and conduct of technical reviews technical review risk assessment checklists are available for each of the reviews. These checklists are designed as a technical review preparation tool, and should be used as the primary guide for the risk assessment during the review. The checklist itself can be both an input to, and an output of, the review. The Integrated Product Team can do a self-assessment as input, but the review participants should take ownership of the assessed risk as an output of the review. These checklists are available on the Systems Engineering Community of Practice and are accessible in the DAU Technical Reviews Continuous Learning Module, CLE-003.
Content
Slide 129. Design Reviews and Affordability
The logistician’s Supportability and life cycle goals during Supportability design reviews should:
Leverage and use performance-based logistics strategies to reinforce optimization of total system availability while minimizing cost and logistics footprint
Merge design effectiveness and process efficiency to gain the maximum mission effectiveness as the goal for performance-based logistics
Offer opportunities to check how the system is developing along effective lines of Affordability and Supportability
Supportability Design Reviews and Criteria
Content
Slide 1210. Topic 3: Supportability Design Reviews and Criteria
Technology Maturation & Risk Reduction
Slide 1211. Alternative Systems Review (ASR)
Definition
In the Materiel Solutions Analysis phase, the Alternative Systems Review (ASR) is a technical review that demonstrates the preferred concept is cost effective, affordable, operationally effective, and suitable, and can be developed to provide a timely solution to a need at an acceptable level of risk. This review also determines whether the system is ready to proceed to the next phase, Technical Development.
By reviewing alternative materiel solutions, the ASR helps ensure that sufficient effort has been given to conducting trade studies that consider and incorporate alternative system designs that may more effectively and efficiently meet the defined capabilities. A successful review is predicated on the Integrated Product Team's determination that the operational capabilities, proposed solution(s), available technologies, and program resources (funding, schedule, staffing, infrastructure, and processes) form a satisfactory basis for proceeding into the Technology Development Maturation and Risk Reduction (TMRR) phase.Comment by PDallosta: Updated TMRR
Key ASR Question
The Alternative Systems Review answers many questions that will be addressed next. The key ASR question asks whether one or more of the proposed systems are cost-effective, affordable, operationally effective and suitable with an acceptable level of risk. The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1212. ASR Process
Inputs
The Alternative Systems Review begins with the following inputs:
Successful completion of:
Initial Technical Review (ITR)
Program System Review (PSR)
A data repository that includes:
Preferred System Solution(s) concept/description
Draft Request for Proposal (RFP)
Analyses results and definitions
System Requirements document
Including interoperability and system distributed services requirements
Initial Capabilities Document (ICD)
Alternative maintenance and Sustainment concepts
Updated cost and schedule data
In addition to these inputs, the following key entry questions and/or issues need to be addressed:
Is there a plan to conduct an ASR, or has one been completed in preparation for a Milestone A Decision?
In preparation for the ASR, what alternative systems were evaluated during the Capabilities Assessment phase?
Do the stakeholders have an understanding of how available system concepts meet capabilities described in the ICD, and of the Affordability, operational effectiveness and technology risks inherent in each alternative concept?
Supportability Considerations/Questions
During the ASR, you should consider questions or criteria under the following key questions:
1. Affordability? Including support costs, is the system cost effective?
Effectively Designed? Is it operationally effective and suitable?
Supportability? Can it be supported in the field to do its mission?
You can find more specific Supportability questions and considerations for the Alternative Systems Review in the Lesson 12 Job Aid that is located after the lesson Summary.
Outputs
ASR is successful when it accomplishes and/or produces the following exit criteria and outputs:
Preliminary system specifications
Systems Engineering Plan (SEP)
Support and maintenance concepts
The following questions must also be successfully answered:
1. Is the system software scope and complexity sufficiently understood and addressed in the Technology DevelopmentAcquisition Strategy to enable low software technical risk?
Has the system technical baseline been captured in a preliminary system specification that is consistent with technology maturity and the proposed program cost and schedule?
(Source: DAG, p. 230)
Successful completion of the Alternative Systems Review leads to the passing of MS A and the start of the Technology Development Maturation and Risk Reduction (MRR) phase.Comment by PDallosta: Updated TMRR
Content
Technology Maturation & Risk Reduction
Slide 1213. System Functional Review (SFR)
Definition
In the Technology Development Maturation and Risk Reduction (TMRR) phase, the System Functional Review (SFR) is a formal review of the conceptual design of the system to establish its capability to satisfy requirements. It establishes the functional baseline and seeks to confirm that the system has a reasonable expectation of satisfying requirements.
Key SFR Question
The System Functional Review answers many questions that will be addressed next. The key SFR question asks whether the system can reasonably satisfy the requirements of the Initial Capabilities Document.
The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1214. SFR Process
Inputs
The System Functional Review begins with the following entry criteria and inputs:
Draft Capability Development Document (CDD)
Requirements traced from CDD to system specifications, functional specifications and subsystem specifications
Approved Materiel Solution
Preliminary Technical Performance Measurements (TPMs) identified and traceable to Measures of Effectiveness (MOE) and Measures of Performance (MOP)
Functionality assigned to Key Performance Parameters (KPPs)
Verification and validation methodology defined for each specification requirement
System functional specification completed to subsystem level
Functional requirements for operations and maintenance assigned to subsystems, hardware, software, or support after detailed reviews of the design’s architecture
Updated
Systems Engineering Plan
Software Development Plan
Logistics and training requirements identified in system and subsystem specifications
Additional inputs:
Updated Integrated Architecture Viewpoints
Updated use case analysis
Completed Preliminary subsystem specification and Interface Design Documents
Allocated Information Assurance Controls over to system components
Created Technology maturation plans for critical technologies.
Defined Technology Maturation and Risk Reduction (TMRR) Development Exit Criteria.
Supportability Considerations/Questions
During the SFR, you should consider questions or criteria under the following key questions:
1. Affordability: Is the program with the approved functional baseline executable within the existing budget?
Functionality: Are the system functional requirements sufficiently detailed and understood to enable system design to proceed?
Supportability: Are the Supportability requirements to achieve the support strategy included in the performance specifications?
You can find more specific Supportability questions and considerations for the System Functional Review in the Lesson 12 Job Aid.
Outputs
SFR is successful when it accomplishes and/or produces the following exit criteria and outputs:
Establishes:
System functional baseline with traceability to lower-level performance requirements
Preliminary system-level maintenance plan
Updates:
Cost Analysis Requirements Description (CARD), or equivalent
Test and Evaluation Management Plan (TEMP), or equivalent
Program development schedule including system and software critical path drivers
Risk assessment for the EMD phase
(DAG, pp. 243-5)
Successful completion of the System Functional Review leads to the Preliminary Design Review.
Content
Technology Maturation & Risk Reduction
Slide 1215. Preliminary Design Review (PDR)
The Preliminary Design Review (PDR) is the next Pre-Milestone B review we cover in the Technology Maturation and Risk Reduction (TMRR) Development phase. The PDR influences system design and support by ensuring that the allocated baseline is established. Each function in the functional baseline is allocated to one or more system configuration items to ensure traceability.
Definition
The PDR is a formal review that confirms the preliminary design logically follows the SFR findings and meets the requirements. It normally results in approval to begin detailed design.
Key PDR Question
The Preliminary Design Review answers many questions that will be addressed next. The key PDR question asks whether the system has a reasonable chance of being judged operationally effective, suitable and affordable before becoming a Program of Record (POR) at Milestone B.
The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1216. PDR Process
Inputs
The Preliminary Design Review begins with the following entry criteria and inputs:
Approved Life Cycle Sustainment Plan with updated program Sustainment development efforts and schedules
Subsystem specifications for hardware, software, and associated Interface Control Documents. These enable detailed design or procurement of subsystems.
System requirements/capabilities
Test and Evaluation Master Plan (TEMP), including:
Sell-off criterion for each product
Witness requirements
Documentation
Cost Analysis Requirements Description (CARD), updated as required
Risk and opportunity management, reviewed and managed regularly
Configuration Management Board running with metrics
In addition to these inputs, the following key entry issues need to be addressed. In preparation for the PDR, are the following activities/actions completed or documents/information available?
Requirements
Design Reliability
Design Maintainability
Requirements Traceability Matrix
Data
Supplier data describing specific components
Design data defining major subsystems, equipment, software and other system elements
Design & Analyses
Reliability & Maintainability Analyses, reports, trade studies
Supportability analysis data/Logistics Product Database
Design documentation
Equipment and part standardization
Costs/Risks
Developmental Test & Evaluation plans
Spares and Government-Furnished Property (GFP)
Supportability Considerations/Questions
During the PDR, you should consider questions or criteria under the following key questions:
1. Affordability: Is the program with the approved functional baseline executable within the existing budget?
Design Readiness: Have the majority of manufacturing processes been defined and characterized?
Supportability: Does the system meet all logistics and Supportability requirements?
You can find more specific Supportability questions and considerations for the Preliminary Design Review in the Lesson 12 Job Aid.
Content
Outputs
PDR is successful when it accomplishes and/or produces the following exit criteria and outputs:
Establishes:
Preliminary Design Review (PDR) Report
A system allocated baseline
The Requirements Traceability Matrix (RTM)
Updates:
Cost Analysis Requirements Description (CARD) or CARD-like document based on the system allocated baseline
Risk assessment for Engineering and Manufacturing Development (EMD)
Approved Life Cycle Sustainment Plan updating program
Sustainment development efforts and schedules
Program schedule, including system and software critical path drivers, and hardware, software, human/support systems
(DAG, pp. 247-50)
Successful completion of the Preliminary Design Review is a prerequisite for a successful Milestone B decision, which permits the system to enter the Engineering and Manufacturing Development phase.
Technology Maturation & Risk Reduction
Slide 1217. Critical Design Review (CDR)
Definition
In the Engineering and Manufacturing Development phase, the Critical Design Review (CDR) is a formal review conducted to evaluate the completeness of the design and its interfaces. In addition to establishing the initial product baseline and placing it under configuration control, the CDR also ensures the system under review has a reasonable expectation of satisfying the requirements of the Capability Development Document (CDD) within the current budget and schedule.
Key CDR Question
From a Supportability perspective, the Critical Design Review answers the question of whether the system meets all logistics and Supportability requirements from the Capability Development Document within the current allocated budget and schedule.
The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1218. CDR Process
Inputs
The Critical Design Review begins with the following entry criteria and inputs:
Approved Life Cycle Sustainment Plan that updates program Sustainment development efforts and schedules
System requirements and capabilities
CARD, updated as required
Risk and opportunity management reviewed and managed regularly
Product baseline under configuration control
In addition to these inputs, the following key entry issues need to be addressed. In preparation for the CDR, are the following activities/actions completed or documents/information available?
Design
Prepare production baseline (“Build To” documentation)
Determine if the system design documentation (product baseline, including Item Details Specs, Material Specs, Process Specs) is satisfactory to start initial manufacturing
Capture product baseline in the detailed design documentation
Review test plans to assess if test efforts are developing sufficiently to indicate the Test Readiness Review will be successful
Costs/Risk
Produce published agenda (several days prior to the conference)
Prior Reviews
Completed all action items related to the previous conference (PDR) successfully
Supportability Considerations/Questions
During the CDR, you should verify the achievement of the following criteria:
1. Affordability: Is the program executable with the existing budget and the approved product baseline?
Design Capability: Does the detailed design satisfy the Capability Development Document or any available draft Capability Production Document?
Supportability: Does the system meet all logistics and Supportability requirements?
You can find more specific Supportability questions and considerations for the Critical Design Review in the Lesson 12 Job Aid.
Outputs
CDR is successful when it accomplishes and/or produces the following exit criteria and outputs:
Establishes:
Post-CDR Report
System initial product baseline is established and places it under configuration control
Issues and Actions Summary
Completed FMECA/FTA/ Safety and Maintainability analyses
Estimates of System R&M
Supportability requirement enablers including Human Systems Integration (HSI) design features, with Environment, Safety and Occupational Health (ESOH) risk reduction features included in the design
Updates:
Cost Analysis Requirements Description (CARD) or CARD-like document based on the system product baseline.
Risk Assessment for the EMD phase, including issues/risks that could result in a breach to the program baseline or substantively impact cost, schedule or performance.
Approved Life Cycle Sustainment Plan, including:
Updated program Sustainment development efforts
Schedules based on current budgets
Test evaluation results
Supportability design features Developmental Testing Results confirmed for key Sustainment enablers or drivers requirements are complete. If the test results to date do not indicate the operational test is likely to succeed or risk has increased, new developmental and operational testing criteria and plans should be considered, along with fallback plans.
(DAG, pp. 262-5)
Successful completion of the Critical Design Review leads to the next review in the Engineering and Manufacturing Development phase.
Content
Technology Maturation & Risk Reduction
Slide 1219. Developmental Test and Evaluation (DT&E)
Developmental Test and Evaluation (DOT&E) activities are principally conducted during the Technology Maturation and Risk Reduction (TMRR) Development (TD) and Engineering and Manufacturing Development (EMD) phases to ensure the system design satisfies desired capabilities.
Definition
DT&E is an engineering test process conducted to provide data for the assessment of a design’s achievement of critical system performance parameters. The testing is performed at the component, subsystem and system-level configurations of hardware and software, and ensures the verification and validation of the Systems Engineering process.
DT&E is conducted as part of the design process. The testing may be conducted by the Contractor under Government oversight, independently by the Government, or jointly by the parties. DT&E differs from Initial Operational Test and Evaluation (IOT&E) in that the IOT&E is conducted by the Service’s Operational Test Agency (OTA) and determines the system’s operational effectiveness and suitability in the operational environment.
Key DT&E Question
Developmental Test and Evaluation (DT&E) answers the question of whether the system design solution satisfies desired capabilities. The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1220. DT&E Process
Inputs
Developmental Test and Evaluation begins with the following entry criteria and inputs:
Areas needing test to complete FMECA/FTA/RCM Analysis
Testing required as part of Reliability improvement program
Performance estimates and specifications
In addition to these inputs, the following key entry questions need to be addressed:
Are Modeling and Simulation (M&S) uses in Developmental Test and Evaluation (DT&E), Operational Test and Evaluation (OT&E), Live Fire Test and Evaluation (LFT&E), System of Systems (SoS) interoperability testing and information assurance testing integrated in an efficient continuum?
Has the system demonstrated sufficient technical maturity into Initial Operational Test and Evaluation (IOT&E) regarding Critical Technical Parameters (CTPs), including interoperability, and is it documented in the Test and Evaluation Master Plan (TEMP)?
Supportability Considerations/Questions
During DT&E, you should focus on the following key question:
Supportability: Are Supportability metrics being met?
You can find more specific Supportability questions and considerations for the Developmental Test and Evaluation in the Lesson 12 Job Aid.
Outputs
DT&E is successful when it accomplishes and/or produces the following exit criteria and outputs:
Areas requiring redesign
Improvements in Reliability
Successful completion of the Developmental Test and Evaluation leads to the next review in the Engineering and Manufacturing Development phase, the Functional Configuration Audit (FCA).
(DAG, pp. 802-3)
Content
Technology Maturation & Risk Reduction
Slide 1221. Functional Configuration Audit (FCA)
Definition
The Functional Configuration Audit (FCA) is the second review in the EMD phase and may be conducted concurrently with the System Verification Review (SVR). SVR is a multi-disciplined product and process assessment that ensures the system under review can proceed into Low-Rate Initial Production.
The FCA is a formal review conducted to verify that all subsystems can perform all of their required design functions in accordance with their functional and allocated configuration baselines.
Key FCA Question
The Functional Configuration Audit answers the question of whether the functionality or performance of hardware and software configuration items meets their specifications.
The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1222. FCA Process
Inputs
The Functional Configuration Audit begins with the following entry criteria and inputs:
Accomplishment of program Supportability objectives detailed in acquisition documents (CDD, LCSP)
A completed system Technical Data Package
Developmental Test & Evaluation (DT&E) results
In preparation for the FCA, are the following activities/actions completed or documents/information available?
Requirements:
Functional and allocated baselines
Verification is comprehensive and complete
Testing:
Test procedures and results
Preproduction and production test results
Design & Analyses:
Configuration audits, including completion of all change actions, have been completed for all configuration items
Systems engineering planning is updated for production
Costs/Risks:
Critical achievements, success criteria and metrics have been established for production
Risk management planning has been updated for production
Readiness issues for continuing design, continuing verifications, production, training, deployment, operations, support and disposal have been resolved
Supportability Considerations/Questions
During FCA, you should focus on the following key question:
Functionality: Do all the subsystems do what they are supposed to do according to the functional configuration?
Outputs
FCA is successful when all FCA-related actions and risks are identified, resolved or effectively mitigated to an acceptable level, and the EMD product is sufficiently mature for entrance into Low-Rate Initial Production.
Successful completion of the Functional Configuration Audit leads to the next review in the Engineering and Manufacturing Development phase, the Production Readiness Review (PRR). (DAG, pp. 267-8)
Content
Technology Maturation & Risk Reduction
Slide 1223. Production Readiness Review (PRR)
Definition
The Production Readiness Review (PRR) is the last review covered in the EMD phase. The PRR is a formal examination of a program to determine if:
The design is ready for production
Production engineering problems have been resolved
The producer has accomplished adequate planning for the Production and Deployment phase
Key PRR Question
The Production Readiness Review answers the question of whether the system design is ready for production.
The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1224. PRR Process
Inputs
The Production Readiness Review (PRR) begins with the following entry criteria and inputs:
Long lead-time items identified and ordered
Mature software
Manufacturing processes identified and validated
Special tooling: jigs, fixtures and other manufacturing aids available
Quality assurance procedures and methods in place
Manufacturing personnel trained
In addition to these inputs, the following key entry issues need to be addressed. In preparation for the PRR, are the following activities/actions completed or documents/information available?
Design
The developer’s design is complete.
Product design is low risk from the standpoint of producibility and has stabilized.
The design is in consonance with the operational, maintenance and support concepts, including meeting inter-service and foreign interoperability requirements.
Costs/Risk
Verification of the design has been accomplished, including qualification of subsystems and components as appropriate, and demonstration of performance and R&M characteristics.
Long lead-time materials identified, and action initiated for advance procurement where appropriate. Sole source items are identified, and continuity of supply is ensured.
Production cost projections have been made and are well supported.
Data
The Technical Data Package will permit competitive acquisition and domestic and foreign co-production, where appropriate.
Prior Reviews
A Critical Design Review has been accomplished and discrepancies resolved.
Training and Manpower
Skilled production manpower will be available in sufficient numbers for the planned term of production.
Supportability Considerations/Questions
During the PRR, you should focus on the following key question:
Producibility: Are we ready to go into production? Producibility is the degree to which “Design for Manufacturing” concepts have been used to influence system and product design. Producibility facilitates timely, affordable, and optimum-quality manufacture, assembly, and delivery of system to the field. You can find more specific Supportability questions and considerations for the Production Readiness Review in the Lesson 12 Job Aid.
Outputs
PRR is successful when it accomplishes and/or produces the following exit criteria and outputs:
IPT reviews readiness of manufacturing process, including:
Quality management system
Production planning (i.e., facilities, tooling and test equipment capacity, personnel development and certification, process documentation, inventory management, etc.)
IPT determines:
System requirements are fully met in the final production configuration.
Production capability forms a satisfactory basis for proceeding into Low-Rate Initial Production (LRIP) and Full-Rate Production.
Remaining risks have been captured and risk mitigation plan is in place.
Successful completion of the Production Readiness Review leads to the next review in the Engineering and Manufacturing Development phase. (DAG, pp. 268-9)
Content
Technology Maturation & Risk Reduction
Slide 1225. Life Cycle Management Framework: Where Are You?
What Influence Do You Have?
You have reached the Production and Deployment phase of the course. You will learn about two Supportability design reviews that occur during this phase. First, let’s examine the general characteristics of the Production and Deployment phase.
Slide 1226. Production and Deployment Phase
The Production and Deployment phase commences at Milestone C. During the Production and Deployment phase, the system should achieve operational capability that satisfies mission needs.
Inputs to Production and Deployment include:
Evaluation results from testing
Identification of:
Exit criteria for the Production and Deployment Phase
Entry criteria for the Operations and Support Phase
Acquisition program baseline
Capability Development Document and Capability Production Document
Systems Engineering Plan
Test and Evaluation Master Plan
Programmatic Environment, Safety, and Occupational Health Evaluation
Product support element requirements
System Safety Analyses to include updated ESOH Risk Analysis (section 4.4.7.5), update to Safety Requirements Criteria Analysis, System Safety Hazard Analysis
Content
Two work efforts take place during the Production and Deployment phase:
Low-Rate Initial Production (LRIP)
Full-Rate Production and Deployment (FRP&D)
(DAG, p. 271)
Content
Technology Maturation & Risk Reduction
Slide 1227. Initial Operational Test and Evaluation (IOT&E)
Definition
Initial Operational Test and Evaluation (IOT&E) is the last review covered in this lesson. Earlier we discussed DT&E, which the design team conducted. IOT&E is an evaluation of operational effectiveness and Suitability made by an independent operational test agency, with user support as required.
Key IOT&E Question
The Initial Operational Test and Evaluation answers the question of whether the system performs in an operationally effective and suitable manner under realistic operational conditions.
The LCL can address this key question using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1228. IOT&E Process
Inputs
The Initial Operational Test and Evaluation (IOT&E) begins when the developing organization provides the following inputs to an independent testing authority:
System Reliability, Maintainability and Supportability performance requirements, demonstrated in as realistic an operating environment as possible
Completed:
Life Cycle Sustainment Plan
Systems Engineering Report
Logistics Product Database
Test and Evaluation Master Plan
Technical manuals
Training plan/program
Provisioning plan
Product support strategy
Content
Supportability Considerations/Questions
The Initial Operational Test and Evaluation process works differently than the reviews discussed previously. In IOT&E, the developing organization hands off the inputs listed above to an independent agency who reports to Congress. The independent agency will compile a list of deficiencies and trouble tickets. The LCL must address and resubmit the fixed deficiencies to the independent agency.
During the IOT&E, the independent agency will be answering the following questions:
Supportability: Does the system satisfy the user’s needs?
Suitability: Does it work as advertised including Supportability?
You can find more specific Supportability questions and considerations for the Initial Operational Test and Evaluation in the Lesson 12 Job Aid.
Outputs
When the independent agency finds that the deviances and related issues have been solved the system can proceed out of IOT&E with the following outputs:
Issued the Beyond Low Rate Initial Report (BLRIR) to Congress
Measured the system’s operational effectiveness and suitability performance in operational use.
Successful completion of the Initial Operational Test and Evaluation leads to the next review in the Production and Deployment phase. (DAG, pp. 811-3)
Content
Technology Maturation & Risk Reduction
Slide 1229. Physical Configuration Audit (PCA)
Definition
The Physical Configuration Audit (PCA) is a formal audit that establishes the final product baseline as reflected in an early production configuration item, and is conducted before the decision for Full-Rate Production.
A configuration item (CI) is any item that satisfies an end-use function and is designated for separate configuration management. Any item required for Product Support and designated for separate procurement is a CI
The Physical Configuration Audit answers the question of what is the actual configuration of an item being produced; it is the final check before hand-off to Sustainment.
Please note: The PCA risk assessment checklist is designed as a technical review preparation tool, and should be used as the primary guide for assessing risk during the review. This checklist is available on the Systems Engineering Community of Practice.
Key PCA Questions
What is the actual physical configuration of the item being produced?
Is the item accurately reflected in the LPD?
The LCL can address these key questions using the inputs, entry criteria, Supportability questions, exit criteria and outputs discussed with the following slide.
Content
Slide 1230. PCA Process
Inputs
PCA begins with the following inputs, entry questions and/or issues:
Complete Technical Data Package, describing the product baseline
Completed manufacturing and quality control plans
Supportability Considerations/Questions
During PCA, you should focus on the following key questions:
Producibility: Do we have a clear picture of the total configuration of the system? Do we know all the parts in the system? Can we produce at full rate production?
Outputs
PCA is successful when it accomplishes and/or produces the following exit criteria and outputs:
All PCA-related actions and risks are identified, resolved or effectively mitigated to an acceptable level.
The design and manufacturing documentation match the item as specified in the contract.
Signed PCA certificate of completion or approval between the developing and the sustaining organizations that is submitted to the Milestone Decision Authority (MDA) for Full Rate Production(FRP)
(DAG, pp. 274-5)
Content
Slide 1231. IPS Elements and IPTs
The Integrated Product Teams conduct Supportability Design Reviews to:
Assess the system’s Supportability requirements and characteristics for compliance to the review’s entrance and exit criteria
Identify Supportability deficiencies, identify potential impacts to the design and make recommendations regarding their resolution
Identify requirements and mechanism for exchanging information, to include analyses, reports, and data
Assign action items, specifying responsibilities, schedule and expected outcome(s)
The IPTs most involved in developing the design interface for Supportability include:
Human Systems Integration
Test & Evaluation
Product Support Management
Systems Engineering
These teams will be responsible for providing updates to documents such as the Life Cycle Sustainment Plan, the Systems Engineering Plan, the Test and Evaluation Master Plan, the RAM-C Rationale Report and others.
Exercise
Content
Slide 1232. Topic 4: Exercise
Slide 1233. Exercise
Purpose
The purpose of the exercise is for students to apply their understanding of logistics Supportability issues during design reviews. Students will differentiate impacts that Supportability design reviews have on Supportability and Supportability Analysis.
Instructions
1. Log into Blackboard and navigate to the Lesson 12 Exercise.
Answer the questions.
Submit your answers in Blackboard.
Summary
Content
Slide 1234. Topic 5: Summary
Content
Slide 1235. Takeaways
Content
Slide 1236. Summary
Supportability Design Reviews Job Aid
The purpose of this job aid is to provide the student with an organized collection of specific Supportability questions and considerations for each of the Supportability Design Reviews covered in Lesson 12: Supportability Design Reviews.
Supportability questions and considerations for the following Supportability design reviews are covered:
Materiel Solution Analysis Phase
Alternative Systems Review (ASR)
Technology Maturation and Risk Reduction (TMRR) Development Phase
System Functional Review (SFR)
Preliminary Design Review (PDR)
Engineering & Manufacturing Development Phase
Critical Design Review (CDR)
Developmental Test and Evaluation (DT&E)
Functional Configuration Audit (FCA)
Production Readiness Review (PRR)
Production & Deployment Phase
Initial Operational Test and Evaluation (IOT&E)
Physical Configuration Audit (PCA)
Alternate Systems Review
Affordability: In addition to answering the question about the cost-effectiveness of the system, you should also ask:
Are the risks for competitive prototyping and initial development (through to the allocated baseline) known and manageable?
Have required investments for technology developmentmaturation, to mature design and manufacturing related technologies, been identified and funded?
Effectively Designed: In addition to asking whether the system is operationally effective and suitable, you should also ask:
Is the proposed materiel solution(s) sufficiently detailed and understood to enable entry into Technology Maturation and Risk Reduction (TMRR) Development with low technical risk?
Are the system software scope and complexity sufficiently understood and addressed in the planning for the Technology Maturation and Risk Reduction (TMRR) Development phase to enable an acceptable/manageable level of software technical risk?
Is the Technology Maturation and Risk Reduction (TMRR) Development work effort properly staffed?
Is the Technology Maturation and Risk Reduction (TMRR) Development work effort executable within the existing budget and schedule?
Has a preliminary system specification, consistent with technology maturity and the proposed program cost and schedule, been captured in the system technical baseline?
Supportability: In addition to asking whether the system is supportable in the field, you should also ask:
Have the preliminary manufacturing processes and risks been identified for prototypes?
Have initial producibility assessments of design concepts been completed?
Is the program schedule for the Technology Maturation and Risk Reduction (TMRR) Development Phase executable (technical/cost risks)?
Are the hazards sufficiently understood and addressed to achieve an acceptable/manageable level of ESOH risk in the Technology Maturation and Risk Reduction (TMRR) Development phase?
(Source: DAG P. 230)
System Functional Review
Affordability: In addition to answering the question about the existing budget of the system, you should also ask:
Does the updated cost estimate fit within the existing budget?
Are adequate processes and metrics in place for the program to succeed?
Are the risks known and manageable for development?
Is the program schedule executable (technical/cost risks)?
Is the updated Cost Analysis Requirements Description (CARD) consistent with the approved functional baseline?
Functionality: In addition to asking questions about whether system design can proceed you should also ask:
Has the system Functional Baseline been established to enable preliminary design to proceed with proper configuration management?
Can the system functional requirements, as disclosed, satisfy the draft Capability Development Document?
Has a draft preliminary Software Requirements Specification been defined, with complete verification requirements?
Has an initial software architecture design been defined?
Is the software functionality in the approved functional baseline consistent with the updated software metrics and resource-loaded schedule?
Supportability: In addition to asking questions about whether Supportability requirements are included in specifications you should also ask:
Are the program development efforts required to achieve the sustainment key performance parameters, key system attributes, and enabler metrics, along with their corresponding schedules, included in the program documentation and Life-cycle Sustainment Plan?
Have the system-level hazards been reviewed and mitigating courses of action been identified?
Is the program properly staffed?
(DAG, pp. 243-5)
Preliminary Design Review
Affordability: In addition to asking questions about the existing budget you should also ask:
Are the risks known and manageable for integrated testing and developmental and operational evaluation?
Is the program schedule executable (technical/cost risks)?
Is the program properly staffed?
Has the program‘s cost estimate been updated?
Is the preliminary system level design producible within the production budget?
Is the updated CARD consistent with the approved allocated baseline?
Design Readiness: In addition to asking questions about manufacturing processes you should also ask:
Does it follow functional review requirements?
Does the status of the technical effort and design indicate operational test and evaluation success (operationally effective and suitable)?
Can the preliminary design, as disclosed, satisfy the draft Capability Development Document?
Has the system allocated baseline been established and documented to enable detailed design to proceed with proper configuration management?
Have the majority of manufacturing processes been defined and characterized?
Are initial manufacturing approaches documented?
Have producibility assessments of key technologies been completed?
Has a production cost model been constructed?
Supportability: In addition to asking questions about meeting logistics requirements you should also ask:
Are adequate processes and metrics in place for the program to succeed?
Have sustainment and human integration design factors been reviewed and included, where needed, in the overall system design?
Can the industrial base support production of development articles?
Have long-lead and key supply chain elements been identified?
Can the risks associated with ESOH hazards be mitigated to an acceptable risk level within the existing budget?
(DAG, pp. 247-50)
Critical Design Review
Affordability: In addition to asking questions about the existing budget you should also ask:
Is the detailed design producible within the production budget?
Is the updated Cost Analysis Requirements Description (CARD) consistent with the approved product baseline?
Does the updated cost estimate fit within the existing budget?
Design Capability: In addition to asking questions about the design satisfying the Capabilities Development Document you should also ask:
Does the status of the technical effort and design indicate operational test and evaluation success (operationally effective and suitable)?
Has the system product baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management?
Is the software functionality in the approved product baseline consistent with the updated software metrics and resource-loaded schedule?
Have key product characteristics having the most impact on system performance, assembly, cost, Reliability, and sustainment or safety been identified?
Have the critical manufacturing processes that affect the key characteristics been identified and their capability to meet design tolerances determined?
Have process control plans been developed for critical manufacturing processes?
Have manufacturing processes been demonstrated in a production representative environment?
Are detailed trade studies and system producibility assessments underway?
Are materials and tooling available to meet pilot line schedule?
Has the system production cost model been updated, allocated to subsystem level, and tracked against targets?
Are long lead procurement plans in place and has the supply chain been assessed?
Are the ESOH residual risks known and manageable?
Supportability: In addition to asking questions about meeting logistics requirements you should also ask:
Supportability requirement enablers, such as Human Systems Integration (HSI) design features, inclusive of the environment, safety and occupational health risk reduction features, are included in the design.
Failure Mode Effects and Criticality Analysis have been completed and any remaining subsystem requirements for the product support package elements design are complete.
Key sustainment characteristic drivers (including critical manufacturing processes to achieve them) have been identified, and an estimate of system Reliability based on demonstrated Reliability rates and other sustainment drivers are being used in developing the product support package.
Development testing results are used to update the sustainment metric projection estimates and any planned corrective actions to hardware/software deficiencies have been identified.
Test success criteria for any remaining development testing and operational testing plans (for testing both operationally effective and suitable) for key sustainment enablers or drivers requirements are complete. If the test results to date do not indicate the operational test success is likely or risk has increased, new developmental and operational testing criteria and plans should be considered, along with fallback plans.
Program sustainment development efforts with corresponding schedules, including system fabrication, test, and software critical path drivers, are included in LCSP updates.
Has the detailed design satisfied sustainment and Human Systems Integration requirements?
Are adequate processes and metrics in place for the program to succeed?
Are the risks known and manageable for testing in support of developmental and operational evaluation objectives?
(DAG, pp. 262-5)
Developmental Test and Evaluation
Are critical system design requirements, including Supportability metrics, being met?
Is there a negative or positive influence on Supportability from the Reliability predictions and assessments based on the test data
(DAG, pp. 802-3)
Functional Configuration Audit
Functionality: Do all the subsystems do what they are supposed to do according to the functional configuration?
(DAG, pp. 267-8)
Production Readiness Review
Additional Producibility questions you should consider are:
Do we have long lead items?
Are all technologies mature enough for production?
Is the supply chain established and stable with materials available to meet planned low rate production?
Have we identified the skills and machines?
Is the program properly staffed?
Are the production facilities ready and required workers trained?
Are the ESOH residual risks known and manageable?
Have we laid out processes and process controls?
Has the system product baseline been established and documented to enable hardware fabrication and software coding to proceed with proper configuration management?
Are adequate processes and metrics in place for the program to succeed?
Are the risks known and manageable?
Is the detailed design producible within the production budget?
(DAG, pp. 268-9)
42 of 58
January 2013
Final v1.3
January 2013
Final v1.3
43 of 58