Test Readiness Review

Title

MSR #

1

Testing at MCTSSA

What, How, Why

06 February 2019

Kimberly ReynoldsTest & Certification Group (TCG)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

MCTSSA

2DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

What We Test

3

•C4I

•Sensors

•Tactical Networking

•Communications

•Cybersecurity

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

C4I

4

Combat Operations Center (COC)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

C4I

5

Advanced Field Artillery Tactical Data System

(AFATDS)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

C4I

6

Joint Tactical Common Operational Picture

(COP) Workstation (JTCW)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

C4I

7

Tactical Voice Switching System (TVSS)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

C4I

8

Networking On-The-Move (NOTM)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

C4I

9

Common Aviation Command and Control

System (CAC2S)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

SENSORS

10

AN/TPS-80 Ground/Air Task Oriented Radar

(G/ATOR)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

SENSORS

11

AN/TPS-59 3-D Air Search Radar

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

12

Tactical Data Network (TDN)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

13

Joint Enhanced Core Communication System

(JECCS)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

14

Data Distribution System – Modular (DDS-M)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

15

Multifunctional Information Distribution

System (MIDS)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

16

Defense Advanced Global Positioning System

(GPS) Receiver (DAGR)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

17

Marine Corps Enterprise Network (MCEN)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

TACTICAL NETWORKING

18

Wireless

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

COMMUNICATIONS

19DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CYBERSECURTY

20DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

What We Test

21

•C4I

•Sensors

•Tactical Networking

•Communications

•Cybersecurity

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

How We Test

22

Testing is conducted in one or more of the laboratories that

comprise the Marine Air Ground Task Force (MAGTF) Systems

Integration Environment (SIE) also known as VII MEF.

The SIE is a dynamic, scalable, and instrumented hardware-

in-the loop distributed facility that represents each

component of the MAGTF, to include a high fidelity afloat

facility that integrates Navy communications networks.

The SIE promotes the ability to perform operationally relevant

testing through the use of tactical systems and mission

based test threads.

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

ORIENTATION

23

SATCOM

Lab

GCE Lab

RF/NOTM

LabsACE Lab OFTSSC

Test &

Certification

Engineering &

Infrastructure

RadarsNetwork

Efficiency

Lab

MC2S

Lab

LFOC15th MEU

MAGTF SIE

NDF Lab

NGEN LabMPY

Tactical

Networking

Lab

Cloud

Computing-

Mobility Lab

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

How We Test

24

We use detailed processes for each phase of a test event.

These processes, originally developed by Test & Certification

Group (TCG) for testing, are now used by the Activity for all

events.

• Planning

• Execution

• Analysis

• Reporting

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

PLANNING

25DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

PLANNING

26DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

EXECUTION

27DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

ANALYSIS

28DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

REPORTING

29DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

How We Test

30

We use several Scientific Test and Analysis Techniques

(STAT) and tools to assist the test team.

• STAT (Design of Experiments (DOE), Observational

Analyses, Hypothesis Testing, etc.)

• Automation

• Virtualization

• Simulation/Stimulation (SIM/STIM)

• Network Tools

• Analysis Tools

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

31

• To find/prevent defects

• Find problems before they get fielded

• Enable informed decision making

Ultimately – to Make Marines More Capable

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

32DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

33DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

34DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

35DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

36DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

37DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

38DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

Why We Test

39DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

40

The majority of our testing challenges stem from lack of

personnel.

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

41

The majority of our testing challenges stem from lack of

personnel with the following skill sets

• Data/System Analysts (TDL Data, Sensor Data, Interface

Data)

• System Operators (CAC2S, COC, MIDS, JRE, JTCW, AFATDS)

• Test Tool Operators (Network Tools, MLST3, DTS, GTE)

• Subject Matter Experts (MAGTF, Air Operations, Ground

Operations)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

42

Data/System Analysts

(TDL Data, Sensor Data, Interface Data)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

43

• System Operators (CAC2S, COC, MIDS, JRE, JTCW, AFATDS)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

44

• Test Tool Operators (Network Tools, MLST3, DTS, GTE)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

45

• Subject Matter Experts (MAGTF, Air Operations, Ground

Operations)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

46

• Subject Matter Experts (MAGTF, Air Operations, Ground

Operations)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

47

• Subject Matter Experts (MAGTF, Air Operations, Ground

Operations)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

CHALLENGES

48

The majority of our testing challenges stem from lack of

personnel.

• Data/System Analysts (TDL Data, Sensor Data, Interface

Data)

• System Operators (CAC2S, COC, MIDS, JRE, JTCW, AFATDS)

• Test Tool Operators (Network Tools, MLST3, DTS, GTE)

• Subject Matter Experts (MAGTF, Air Operations, Ground

Operations)

DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.

SummaryA test organization with operational relevance

• What we test

– USMC Programs of Record

• How we test

– MAGTF SIE

– PEAR Processes

– STAT

• Why we test

– Make Marines More Capable

• Challenges

– SMEs (MAGTF, Air, and Ground Operations)

– Data/System Analysts

– System and Test Tool Operators

49DISTRIBUTION STATEMENT A. Approved for public release. Distribution is unlimited.