412th test engineering group vision for future knowledge ... · 412th test engineering group vision...

412th Test Engineering Group Vision for Future

Knowledge Management (KM)

Paul Waters412 TENG Tech Director

[email protected]/24/2018 1

DISTRIBUTION STATEMENT A. Approved for public release; Distribution is unlimited

412-TW-PA-18268

mailto:[email protected]

Overview• Objective

– Describe the TENG vision for Knowledge Management and get feedback on this approach

• Outline– Driving Factors– Requirements– Specific Characteristics– Solution Approach

• BLUF– Knowledge Management has the potential to be a game

changer for the test community and the T&E community is behind industry in this area

5/24/2018 2

Value

Volume

Velocity

Variety

Veracity


412-TW-PA-18268

Mission• 412th Test Wing

– Mission:• Test and Evaluate Weapons Systems to Ensure War-

Winning Combat Capabilities– Vision:

• The World Leader in Developmental Test and Evaluation ... Agile, Ready, and Right!

• 412th Test Engineering Group Mission– Manage T&E Engineering Core Competencies in

Airframe-Propulsion-Avionics (APA), Range, Instrumentation

• Operate and maintain the Edwards Flight Test Range• Provide airborne instrumentation systems and technical

expertise5/24/2018 3


412-TW-PA-18268

Customer Base

5/24/2018 4

Global Vigilance452 FLTS

Global Hawk, UAVs

Joint Strike Fighter-461 FLTS

F-35 Global PowerFighter-416 FLTS

F-16, T-38

Hypersonics

Global PowerBomber-419 FLTS

B-1, B-2, B-52

Global Reach418 FLTS

KC-46, KC-135,C-17, C-5, C-130

Raptor411 FLTS

F-22A

DT

Combined/ Integrated Test Force

Additional Future

Programs

Platform Avionics Cyber Security Testing

Emerging TechnologyCTF

sUAS, C-UASDISTRIBUTION STATEMENT A. Approved for public release; Distribution is unlimited

412-TW-PA-18268

Driving Forces• Decision Support

– Technical Reports needed in < 42 days– Often requiring a Preliminary Report of Results

much faster (sometimes within a week of test)• More non-Deterministic Systems under Test

– Sensor Fusion– Health Monitoring– Autonomy

• Data Quantity– Current Test Programs already exceeding 1 TB

of data per flight 5/24/2018 5


412-TW-PA-18268

Today’s Data Center• Aircraft Configuration Info

– Instrumentation must be configured with parameter information, calibration coefficients, data collection instructions and aircraft interface information

• Data Ingest – Data from the aircraft instrumentation system must be transferred

using removable media, converted into engineering units and stored• Initial Data Processing

– Generate standard data products from engineering units for further analysis

• Data Analysis– Engineers use engineering unit files, standard data products and

results from off-aircraft range systems to perform detailed system analyses

– Primarily use Matlab, Python, or custom code

5/24/2018 6DISTRIBUTION STATEMENT A.

Approved for public release; Distribution is unlimited 412-TW-PA-18268

Data Center Vision• Requirements

– Software must be open-source and/or government owned; must be adaptable to meet unique user requirements

• Must support both production and data investigation– Hardware should utilize commodity and/or open architecture components

• Must be expandable– Functional requirements

• Ingest the data (analog, bus data, video, audio, external sources)• Initial data processing/production• Data analysis investigation and production

• Envisioned Solution– Ingest all of the source

data for a test program– Process the data and make

it available in a secure datalake environment

– Perform Production Analysis and Discovery Analytics

– Archive Data to a Secure Cloud for long-term storage



Data Ingest• Requirements

– Support multi-removable memory modules (RMM) (minimum 3); Goal: vendor independence

– Offload from RMM 1TB in less than 30 minutes; Goal: less than 15 minutes; Stretch: Offload during taxi back

– Utilize standards for: • Airborne Instrumentation Systems data; • Instrumented Ground Systems data; • Time Space Position Information; • Time; • Video; etc

– Ingest unformatted data about the test:

• Aircraft Configuration;• Test Plans;• Test Cards;• Pilot and Test Team Notes;• Interface Control Documents; etc



Data Processing• Requirements:

– Quick look data available in 15-30 minutes• Support post-mission debrief with test team and aircrew

– Prepare the data for analysis and analytics• Decommutate digital and analog data • Convert analog data to engineering units• Apply meta-data• Integrate multiple data

sources– Catalog the data for

future analytics • Specifics dependent on evolution

of data analytic tools5/24/2018 9


412-TW-PA-18268

Data Analysis Tools• Requirements

– Environment must support Engineers developing tools• Likely requires having local data scientists

– Data Retrieval must be Intuitive• Must be able to query across flights and

aircraft– Security for multiple classifications and caveats– Incorporate Analytics Environment

• Open-Source tools: Hadoop, etc• Proprietary tools: Tableau, etc• Government Owned tools: JPL and others

– AUDREY (Assistant for Understanding Data through Reasoning, Extraction, & sYnthesis);

– RAID (Real-time Automated Insight Engine for Data to Decision)

– Support Existing Analysis Tools• Commercial Environments: Matlab, Python, etc• Commercial Tools: IADS, DIVA, ILIAD, etc• Toolboxes: Custom developed



Challenges to KM• Security, Contracts, Culture are Initial Barriers

– Security at the parameter/file level needs to be verified • Need to have cheaper data diodes to go from low to high

– Ownership of data and ICDs to interpret data is infrequently with the government

• Prime contractors view data as proprietary information– Time/resources are required to perform detailed analytics

• Timely answers are critical to meet decisions• Charging policies do not support consolidation of data

• Expertise is limited– Need to increase data scientist support staff– Need to increase training on current and future toolsets

• Investment is needed– Need to create “Data Lakes”

• Storage and availability of data is inconsistent and generally inadequate

• Use Cases are required– Demonstrate Value– Define operational constraints and policies



Bomber Instrumentation/Data Upgrades

RAID R&D Proposal

Roadmap

5/24/2018 12

• Demonstrated Value• Defined Use Cases• Data Ownership Clarification

• AF Investment• Defined Security Plans• Analytics Toolsets Available

•JSF Knowledge Management technology prototyping•S&T investments in Analytic toolsets•Development of Experimental Analytic environments•Increasing Python and Statistics capabilities

Engineering Data Center –Lab Environment

JSF Knowledge Management

•Documenting Requirements•Developing S&T capabilities•Leveraging Existing Data Centers

•Implementing AF and DoD Investments in KM

•Continued Tool development

Proposed AFTC Investment

AUDREY R&D Project

Cross Platform Data CenterCur

rent

Orange Flag SAPF Lab


412-TW-PA-18268

QUESTIONS

5/24/2018 13