Radio Deep Learning EffortsShowcase Presentation
November 2016
Tim O’SheaSenior Research Associate
Program Overview
• Program Objective: Rethink fundamental approaches to how we do radio signal processing from a data-centric
machine learning driven perspective – provide large steps forward in system performance, generalization and autonomy!
• High Level Motivation: We have seen major changes in how signal processing is done in the vision, voice and text
fields over the last 5 years due to major advances of techniques in the field of deep learning Learn from and adopt many of these methods to greatly advance our radio capabilities!
Move from expert based features to learned features Move from rule based control system to learned control systems Focus on methods which generalize well and learn directly from data Leverage architectures which work well in vision/voice and adopt them to the radio
domain Convolutional networks, Recurrent networks, Attention Models, Deep
Reinforcement Learning • Program Personnel
Tim O’Shea, Research Faculty Kayla Brosie, Graduate Student Seth Hitefield, Graduate Student
Technical Focus Areas
DOD S&T
• Specific Focus Areas Signal Sensing and Spectrum Awareness
Signal Detection and Separation Signal Modulation and Protocol Identification Control System Learning and Resource Optimization Anomaly Detection and Semi-Supervised Learning of Emitters
Learning to Communicate Learning New Physical Layer Representations for Radio Transport Learning from Sparse Structure on Legacy Physical Layers
Network Optimization Methods Hyper-parameter Optimization, architecture search methods Network parameter learning, probabilistic alternatives to back-prop
Current deep dive
areas
Signal Modulation and Protocol Classification
DOD S&T
• Dataset Generation GNU Radio based labeled synthetic dataset with known ground
truth and realistic channel effects (fading, Doppler, noise) Working to expand modulations, variation and channel effects More information available from GRCon Paper and at
http://radioml.com http://pubs.gnuradio.org/index.php/grcon/article/view/11
• Modulation Recognition Demonstrated Conv-Net based classifier learning on raw RF data
which outperforms most current day expert classifiers With some tuning, complexity vs state of the art DARPA expert
system is 4-6x lower in FLOP count for forward classification (no on-line learning)
Full paper @ https://arxiv.org/abs/1602.04105 Looking at leveraging more recent advances from vision domain
Signal Modulation and Protocol Classification
DOD S&T
• Semi-Supervised Learning How can we learn from non-labeled radio data? Since
this is most of the world? Organize and identify classes autonomously based on
sparse representations and clustering methods Bootstrap from similar supervised learned features
and unsupervised feature learning• Raw RF Anomaly Detection
Applications of reconstruction-based anomaly detection to raw wide-band radio data to detect anomalies (malicious users, interference, failures, etc)
Leverage recurrent sequence prediction models in place of Kalman predictor In error vector distribution based novelty detection method
Hyper-Parameter Optimization Methods
DOD S&T
• Hyper-Parameter Search In neural networks we typically have two kinds of parameters to define the
network Weight Parameters: Used in neuron to define the transfer function from
input to out, tuned using back-propagation of loss function through network gradients
Fully connected layer weight matrix, bais matrix Convolutional layer filter weights Recurrent layer weights and biases
Hyper-Parameters: Chosen to define how the network is structured typically before training begins, defines how many actual parameters gradient based training must optimize
Network depth, width, layer types, connections, activation functions, convolutional filter configurations, weight initializations, learning rate, dropout rate, regularization rates
• Since gradient descent for a single model can be very expensive we need a lot of concurrency to be able to efficiently search the parameter space and the hyper-parameter space
Ongoing work on developing an open distributed architecture for performing hyper-parameter searches https://github.com/osh/dist_hyperas
Runs on top of Keras (Theano and TensorFlow backends)
Hyper-Param Search Controller
Node (SGD
Model Optimization)
Node (SGD
Model Optimization)
GPU GPU GPU GPU
Meta-Model
+ Dataset
Performance
Ranking