Presented to:

Presented by:

Autotestcon 2017

Tony Conard

Schaumburg, Illinois

NAVAIR, Automated Test Systems (ATS)

Program Manager

TPS Life Cycle Panel ‘TPS Requirements and Procurement’

TPS Acquisition Challenges

Potential conflicts: some

say “do this” and some

say “do that”

Need to sort through policies and

processes and initiatives to balance

the scale to meet mission needs

2

Challenge: Balancing Policies, Processes, Guidance, with successful program

execution to deliver products and services that meet the warfighters needs

2

3

AIR-0.0

Vice Admiral Paul Grosklags

Commander, Naval Air Systems Command

• Increase Readiness, Affordability and Speed

• Reduce Cost and Increase Speed of Products to Fleet

• Smaller empowered program / project teams

• Significantly reduce “derived requirements” – Across the Board!

• Well understood risk assessments

• Tailoring in vs. out and “right sizing” program structures and requirements

“Top priorities for 2017 have not

changed; we still are focused 100%

on readiness of our existing aircraft

and how to get new capabilities to

the fleet more quickly.”

Delivering Rapid Capability

Challenge: Delivering Rapid Capabilities at value to the Warfighter

Acquisition programs are managed by

application of Systems Engineering (SE) in

accordance with DoDI 5000.02, Enclosure 3 Optimizes operational capability

Total system performance

Minimizes total ownership costs

Shall develop a Systems Engineering Plan

(SEP) to describe the program’s overall

technical approach Tailored to the application of SE

NAVAIRINST 4355.19E defines System

Engineering Technical Reviews (SETR) as an

integral part of process “…SETR process is tailorable in scope such that it

can be “right sized”…

Independent technical “health” assessment of the

design throughout the development cycle

Systems Engineering

Challenge: Use SEP To “tailor/ right-size” SETR to fit complexity of Project

4

• Requirements / Technical Baseline Management

– Define high level requirements (ORD/CDD/CPD/RAD) and develop System

Performance Specification (SPS)

– Requirement Analysis Document performance parameters and the SPS are

derived from the legacy peculiar support equipment (PSE) and unit under test

(UUT) specifications

– Systems Engineering Plan (SEP) developed & tailored to efforts

– SOW will define engineering and logistics requirements, reviews, and audits

• Risk, Opportunity & Issue Management

– Risks and issues are captured/managed during risk assessment meetings held

by the Risk Management Board (RMB) IAW the Risk Management Plan (RMP)

5

4

3

2

4 3 2 1

1

LIK

EL

IHO

OD

CONSEQUENCE

5

4

3

2

1

Requirements & Risk Management

5

Challenge: Efficient Requirements, Risks, Issues, and Opportunity Management

Systems Engineering

6

Tailor, “right size” for program

Cybersecurity & OTPSs

• Cyber process replaces Information Assurance (IA) or DIACAP process

with Risk Management Framework (RMF)

• Incorporate cybersecurity early and robustly in the OTPS acquisition and

development to reduce risk of a cyber threat

• Ensure mission risks and resilience are central to program and

operational decisions

• NAVAIR OTPS Cyber Impacts- What we know for now… All OTPSs are considered Platform IT (PIT)

Per OPNAVINST 5239.1D, an Assess & Authorize (RMF steps 4&5) are

required for OTPS procurements

Using RMF process will ultimately determine if an Authority to Operate (ATO)

is required for the TPS effort

If low complexity design (wires, passive devices, simple Ics)-then Assess &

Authorize only

If Hi complexity, external transmissions, processing of none CASS FoT

station controlled S/W , requires ATO & full RMF process

Pursue use of reciprocity, standard controls and overlays for OTPS

7

RMF Process Overview

8

RMF Process

NGOR Overview

• The NAVAIR Generic OTPS RFP (NGOR) is a crucial component of the NAVAIR

acquisition process that provides a standard, tailorable RFP for the procurement of

Operational Test Program Sets (OTPS)

• NGOR is policy, NAVAIRINST 13630.5A states that the Aviation Support Equipment

PMA260 is responsible for developing and maintaining a generic OTPS RFP package

• NGOR follows the Uniform Contract Format:

SECTIONS B-M

ATTACHMENTS: (1) STATEMENT OF WORK (SOW)

(2) SYSTEM PERFORMANCE SPECIFICATION/MIL-PERF-32070A Addendum

(3) GENERAL ACCEPTANCE TEST PROCEDURE (GATP)

(4) TECHNICAL DATA PACKAGE (TDP) CONTRACT REQUIREMENTS

(5) TECHNICAL MANUAL CONTRACT REQUIREMENTS (TMCR)

(6) PROVISIONING STATEMENT OF WORK (PSOW)

(7) ADDRESSEE LIST

(8) DISTRIBUTION STATEMENTS

(9) DD FORM 254 (CONTRACT SECURITY CLASSIFICATION SPECIFICATION)

(10) UUT List

(11) STORAGE CASE DRAWINGS

CONTRACTS REQUIREMENTS DATA LIST (CDRLs)

• Current baselines: NGOR v17.1; tailoring options provided

• NGOR Link can be found on DoD ATS website: http://www.acq.osd.mil/log/mpp/ats.html

• For assistance, questions & comments contact the NGOR IPT at NGOR@navy.mil

9

NGOR in the Acquisition Lifecycle M

at

ur

it

y

Le

ve

l

Time

SRR-II

PDR

CDR

TRR

SVR-PRR

SRR-I

Performance Baseline CCA PPP CARD PESHE RFP IMP TEMP Spec SEP RMP

RFP

MS-C

FRPDR

Initial Product Baseline Est 2D Drawings Reviewed Test Plan Approved (K) H/W-S/W CDRLs Reviewed IMS Updated SEP Updated Spec Updated

Operational Config. Est. Test Plan Approved (Govt) Test Procedures Reviewed Test Schedule Delivered

ASR

SRB

RAD

Final Product Baseline Est. Test Reports Signed

Contractor Sys Baseline Spec Updated IMS Baselined

Allocated Baseline Est. IMS Updated SEP Updated

AoA

IOC

Desk

Audit Desk

Audit

Requirements/Need Established

APBA

Prog

Designation

Spec

(Draft)

10

NGOR program events are tailored to fit program scope

Cybersecurity & OTPSs

11

Fosters interoperability amongst ATS(modular and open systems)

Promote innovation and international technology transfer

Allows information to be passed across TPS Life Cycle

boundaries (design, other levels of maintenance)

Promotes Industry development of COTS related tools

Can be a cost effective approach due to data reuse across ATS

interfaces

TPS Related Standards • IEEE 1641.1 Standards for Signals and Test Definitions

• IEEE 1671.1 ATML for Test Information

• IEEE 1671.2 ATML for Instrument Description

• IEEE 1671.3 ATML for Unit Under Test

• IEEE 1671.4 ATML for Test Configuration

• IEEE 1671.5 ATML for Test Adapter

• IEEE 1671.6 ATML for Test Station

• IEEE 1445 Digital Interchange Format (DTIF)

• IEEE 1636.1 Test Results

• IEEE 1636.2 Maintenance Action Information

Need to understand barriers that impact application of standards

Standards In TPS Lifecycle

Cybersecurity & OTPSs

12

Benefits of Standards to TPS Lifecycle

ATEs meeting emerging avionics complex technologies

• Testing High Speed busses, LVDS, JTAG interfacing, Ethernet,

Firewire

Sustainment of multiple ATS environments

• Atlas, TestStand/Lab Win/View, Direct test

• ATLAS is standard signal definition language, but has limitations

• Lack of trained programmers that are efficient with ATLAS

• Transitioning older, outdated languages to today’s environments

• Managing obsolescence across many COTS development/runtime

environments

Interoperability between TPS environments and ATE

Style guides for modern TPS languages

TPS sustainment QA and development tools

Cyber in TPS and ATE environments

Lack of consistent use of IEEE Standards in TPS Life Cycle

TPS Life Cycle Challenges & Opportunities

Summary

Challenge: Balancing Policies, Processes, Guidance, with successful program execution to deliver

products and services that meet the warfighters needs

Overarching DoD and Services specific TPS Acquisition policies provide guidance

System Engineering, SETRs and Cyber approaches should be “tailored” via SEP and

program docs to fit the program’s complexity and risks

NAVAIR’s NGOR provides a standard tailorable RFP for procurement of OTPSs − Can also be used for other customers

Purse Continuous Process Improvement (CPI)

Identify Risk and Opportunities via IPT to stakeholders

Balance all factors to provide rapid- “Speed to the Fleet” Products to warfighters within

C/S/P baselines

13