Copyright © 2018 Raytheon Company. All rights reserved. Customer Success Is Our Mission is a registered trademark of Raytheon Company.

Defense applications using Machine Learning/Artificial

Intelligence (ML/AI) and GPUsDr. Shubha KadambePhone: (310)334-1604 (O), (310)291-0025(C)Email: shubha.kadambe@raytheon.com

Overall framework

3/16/2018 2

Definitions Situation Assessment

– Identify, locate entities of interest and their relationships: “what is where now and who it is connected with”

– Describe what entities are doing: “behavior models” Behavior Recognition & Inference:

– Intent: infer based on situational change over time: “what are they up to”– Saliency: identify objects of interest based on type of behavior

Entities behavior prediction– Detect/Identify objects or behaviors associated with current or future risks; rank threat

severity New mission adaption

– Adapt the learned models for new missions using a small set of training data rather than re-training the entire models

3/16/2018 3

Types of Sensors Radar

– SAR– GMTI– Early warning– Search/acquisition/track– AESA

E-O– Visible– IR– Hyperspectral– multispectral

EW– Jammer/effectors– Receivers– Spectrum management

GPS/Precision guidance Laser (LIDAR) Acoustic

– Sonar Forward looking Side scan Towed array Synthetic aperture

Seismic

3/16/2018 4

Radar– ISR– Tactical– A2A/A2G/G2A– Airborne– Space

E-O– ISR– Tactical– A2A/A2G/G2A– Airborne– Space

Situation Assessment A Situation is a high-level representation of the state of the world Situation Assessment is a process of gathering high-level information using low-level data to help in

better decision making. Input: entities, ids, events and tracks Output: behavior models “behavior” – variation in object characteristics (e.g. location, velocity) and its relationship with other

entities/objects over time. “behavior models” – relate ID and behavior – expectations/descriptions about how various types of

objects behave– Can also be group behaviors

Describes behaviors of individual and/or group of entities

3/16/2018 5

1

Examples of Learning algorithms for situation assessment Fusion

– Dempster-Shafer– Information theory– Dezert-Samrandche

Deep learning– Feedforward networks– Convolutional networks– Recurrent networks

Finite state machine– Natural language processing based approaches– Petri net

Graph approaches Bayesian Reasoner Case based reasoner

3/16/2018 6

1

Behavior Classification & Inference Inputs: behavior models Output: saliency and intent Classifies behaviors; detects changes in behavior Considers behavior of multiple entities, e.g.

– Flying in formation– Converging on a location– Shadowing– Anomalous behavior

“Classification” - map from features to classes– Implies features associated w/behavior models– Classes of saliency and inferring intent (e.g., fast moving boats towards a ship indicate the intent of hitting

the ship)

Example Learning algorithms for behavior classification and inference

Support vector machine Neural network Gaussian mixture models Bayesian classifier/reasoner Hidden Markov Model Decision tree Classification and Regression Trees (CART)

3/16/2018 8

2

Entities behavior prediction Inputs: Track Saliency and Intent; Entities, Tracks and behavior models Outputs: Resource Recommendations Provide guidance for resource planning and management Determine which objects/entities to focus based on:

– characterization of past/current behavior– prediction of future behavior– threat potential– Threat level

Example Learning Algorithms for Entities Behavior prediction

Anticipatory learning Reinforcement learning Markov Decision Processing Viterbi decoding Finite state machine Bayesian prediction Dynamic stochastic network

3/16/2018 10

3

New Mission/Task/Environment Adaptation Inputs: new mission type or priorities; previously learned behavior models; some sensor data Output: updated behavior models Description: leverage existing models to:

– Accelerate learning process by adapting certain parts of the learned models using a small set of new training data

– Improve performance in changing situations/environments

3/16/2018 11

4

Example Learning algorithms for new mission/task/environment adaptation

Instance based transfer learning Feature based transfer learning Inductive transfer learning Self taught transfer learning

– Deep learning which layers are transferrable, co-adaptable and needs specialization

3/16/2018 12

4

Adaptive planner/adaptive resource manager

Inputs: resource recommendations; legacy system priorities and pilot input Outputs: sensor control Description: learns to allocate system assets to optimally realize goals

– In particular allocating assets to manage/track important entities/objects/targets

3/16/2018 13

5

Example Learning algorithms for Adaptive planner/adaptive resource manager

Partially Observable Markov Decision processing Online policy learning Markov decision processing Q-learning Fuzzy logic Task allocation by branch and bound Task allocation/team formation based on weighted synergy, ad hoc team formation Game theory Monte Carlo Tree Search (MCTS) Reinforcement learning and Monte Carlo planning

3/16/2018 14

5

What aspects of sensor we can control smartly? Type of sensor to use Receiver

– beam steering– Dwell– bandwidth

Transmitter– beam steering, – dwell, – bandwidth, – frequency, – waveform

FOV, Azimuth, elevation, tilt (EO/IR) Wavelength (IR, multispectral, hyperspectral) Radar modes

3/16/2018 15

What algorithm to use when? Depends on the

– Functionality– Decision time– Computation complexity– Amount of training data needed– Learning rate– Learning accuracy– Decision accuracy– Mission type Tactical Strategic

– Mission differences between training and test– Environmental changes

3/16/2018 16

Architecture vs. decision time vs. algorithms

3/16/2018 17

Part of the Architecture/framework

Algorithms Decision time requirements

Situation Assessment

Deep learning • Pre-mission learning using lots of training data; learning time can be in hours.

Finite state machine • Non-real-time trained model update learning during strategic mission or intel gathering; learning time in minutes

Graph approaches • Non-real-time trained model update learning during strategic mission or intel gathering; Learning time in minutes

Bayesian reasoner • Real-time assessment of threat situations during tactical missions (varies from micro seconds to seconds depending on the application)

Case based reasoner • Real-time assessment of threat situations during tactical missions (varies from micro seconds to seconds depending on the application)

Fusion approaches • Real-time assessment of threat situations during tactical missions (varies from micro seconds to seconds depending on the application)

1

Architecture vs. decision time vs. algorithms

3/16/2018 18

Part of the Architecture/framework Algorithms Decision time requirements

Behavior recognition & Inference Support vector machine

• Used during tactical missions and has to be in real-time (varies from micro seconds to less than a second depending on the application and type of mission)

Neural network

Gaussian mixture model

Bayesianclassifier/Reasoner

Hidden Markov Model

Decision tree

Classification and Regression Trees (CART)

2

Architecture vs. decision time vs. algorithms

3/16/2018 19

Part of the Architecture/framework Algorithms Decision time requirements

Entities Behavior prediction Anticipatory learning • Non-real-time model learning prior to a mission or during strategic missions.

• Instantaneous/real-time (as it is happening – less than micro second) prediction with high confidence during tactical missions.

Reinforcement learning Markov Decision Processing

Viterbi decoding

Finite state machine

Bayesian prediction

Dynamic stochastic network

3

Architecture vs. decision time vs. algorithms

3/16/2018 20

Part of the Architecture/framework Algorithms Decision time requirements

New mission adaptation Instance based transfer learning

Less than a second of training data to adapt previously learned models to new tasks and domains.Feature based transfer

learning

Inductive transfer learning

Self taught transfer learning

4

Architecture vs. decision time vs. algorithms

3/16/2018 21

Part of the Architecture/framework Algorithms Decision time requirements

Adaptive planner/resource manager

Partially Observable Markov Decision processing

• Seconds to minutes during strategic missions and adaptive threats are nonlethal and slowly changing their behavior

• In dynamically changing environments and in the presence of lethal threats decisions need to be made in nanoseconds to milliseconds.

Online policy learning

Markov decision processing

Q-learning

Fuzzy logic

Task allocation by branch and bound

Task allocation/team formation based on weighted synergy,

ad hoc team formation

Game theory

Monte Carlo Tree Search (MCTS)

Reinforcement learning and Monte Carlo planning

5

Summary of Architecture-ML/AI algorithms-decision time

22Decision time minutes to seconds Decision time ms to ns

Example Technologies Being Developed at Raytheon/SAS

Technologies being developedDeformable ATR using deep learning Suite of machine learning algorithms for adaptive threat response

Neural nets for sensor control Architecture for wideband receivers

POMDP for optimum course of action selection Learning algorithms for signal classification

Finite state machine based situation awareness Learning algorithms for human preference models for plan selection and team formation

Reinforcement learning based online policy learning for course ofAction Selection.

3D super structure reconstruction and classification for object recognition

3D ATR Predictive analytics

POMDP for sense making Electromagnetic battle management framework and tools

Dynamic stochastic network based anomaly detection

Distributed battle management framework

Adaptive team formation with user preference model

Suite of machine learning algorithms for information management

Suite of machine learning algorithms for adaptive threatcharacterization

3/16/2018 23

Specific Benefits From the TechnologiesApplicable technologies Specific benefit/enhancement from the Technologies

Deformable ATR, 3D ATR, neural nets Improved ATR performance

Fusion and Finite state machine based situation awareness Improved situation awareness by Combining information(object ids ad tracks) from different sensors

POMDP, Reinforcement learning based online policy learning Adaptive appropriate sensor selection and control to improve the quality of situation awareness of important objects

POMDP, Adaptive planner Adaptive AESA beam selection and Control to improve the target detection/id accuracy

Reinforcement learning based online policy learning Protect fire control radar against cognitive Jammer

3D ATR, finite state machine based situation awareness and a suite of machine learning algorithms

Precise actionable information extraction and dissemination. Precise multiple target tracking and detection.

Dynamic stochastic network based anomaly detection Proactive analytics and detection of Emergent anomalous threat behaviors

Finite state machine based situation awareness and online policy learning based strategy selection

Cyber security against peer adversary.

3/16/2018 24

Specific Benefits From the Technologies

3/16/2018 25

Applicable technologies Specific benefit/enhancement from the technologies

Finite state machine based situation awareness and online policy learning based strategy selection

High accuracy communication network identification and low power, low latency countermeasure strategies at Phy, MAC and network layer.

Finite state machine based electronic situation awareness Predictive situation awareness that helps to proactively jam a threat radar or a cyber node

POMDP, Reinforcement learning based online policy learning

Optimum attack strategy selection for both electronic attack and cyber attack. Optimum resources management for electromagnetic battle management

Suite of machine learning algorithms for adaptive threat characterization

Accurate signal classification for early warning radar

Suite of machine learning algorithms for adaptive threat response

Reactive counter measures

Machine Learning Tool Sets Matlab toolboxes

– Statistics and Machine Learning– Neural Networks

Open source frameworks:– R– TensorFlow– Theano– Caffe– Torch– Microsoft CNTK (supposed to be faster than Theano, Torch 7 and TensorFlow -

https://blogs.microsoft.com/next/2016/01/25/microsoft-releases-cntk-its-open-source-deep-learning-toolkit-on-github/#sm.000008k0wuiqi5fedv9smfnn85lc9)

– Microsoft Distributed Machine Learning toolkit (https://www.dmtk.io/)– Scikit-learn: machine learning in Python (http://scikit-learn.org/stable/)

3/16/2018 26

Process Flow For Object Recognition

Raw Data

Training Data

Deep NN Weights Analysis

Data/Processing Management

State-of-the-Art Deep Learning Software

COTS/FOSS Fast training Flexible NN

architectures

Analysis PID performance /

Confusion matrix Visualizations of generated

convolutions / Gain insight into NN learned features

Overfitting / Underfitting

GPU Background (1/2)

Different architecture of GPU requires decomposition to parallel computation; Use of small local memory needed for fastest execution speed

SIMT per row (Single Instruction Multiple Thread)

GPU Background (2/2)Single precision calculations can

be very fast on GPUsGPU Architecture

(From NVIDIA CUDA Programming Guide;Benchmark not specified)

SIMT

Use of Graphical Processing Units (GPUs) When to Use:

– Algorithms have parallelizable structures that match capabilities of GPUs– Available CPUs don’t provide the execution speed needed– Size, Weight, Power, or Cost constraints drive selection of GPUs instead of

CPUs– Capabilities / limitations of target GPUs are understood

NVIDIA Quadro K5000 4GB:1536 cores; 122 W;

GeoForce GTX 7802304 cores; 250 W;

When to Avoid:– Algorithm execution times <~ I/O times– SW development and maintenance costs are constrained– GPU capabilities are not understood / in flux

CUDA Background CUDA = Compute Unified Device Architecture Extensions to C/C++ to support NVIDIA GPUs Supports more than graphical programming Runs on Windows, Linux, Mac OS

– x86 and ARM based processors supported– Plug-ins for MS Visual Studio

COTS libraries exist for some common processing Evolving “Compute Capabilities”

– 1.0: Initial release– 1.3: Double precision supported– 2.0: Additional atomics, printing, concurrent kernel exec– 3.5: Dynamic parallelism

Backwards compatibility supported Development tools and docs at https://developer.nvidia.com/

CUDA vs Open CL CUDA:

– NVIDIA proprietary architecture– More mature technology and programming constructs– More fully exploits processing capabilities

Open CL:– Open standard– Supported by multiple vendors– Code is more portable, but is not likely to run as fast– Code may need to be reworked for different targets– Can run on CPU if not GPU is present

Summary ML/AI exploding

– There is lot of myth as well– ML/AI no magic

Defense systems will have ML/AI to make smart decisions– Described a framework with needed functionalities/capabilities for this– Briefly described different ML/AI algorithms to use for each functionality/capability

Mentioned some of the algorithms we are developing Described briefly

– what tools and resources are available– what factors need to be considered while selecting an algorithm

It is an exciting time to be working on ML/AI algorithms that enable systems/products make smart decisions

For speed and processing hardware that is small size and weight using GPGPU with multi-core– Programing them using CUDA

3/16/2018 33