backup: machine learning - unitrento enver sangineto... · 2020. 6. 9. · backup: machine learning...
TRANSCRIPT
![Page 1: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/1.jpg)
Backup: Machine Learning
Enver Sangineto DISI
University of Trento, Italy
![Page 2: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/2.jpg)
• My (general) research interests
• Backup: Hybrid biological-artificial networks
• Backup: Neuromorphic computing
Overview
![Page 3: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/3.jpg)
• My (general) research interests
• Backup: Hybrid biological-artificial networks
• Backup: Neuromorphic computing
Overview
![Page 4: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/4.jpg)
My Research Area
• Deep Learning:
– Discriminative methods
– Generative methods
![Page 5: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/5.jpg)
Current (non-Backup) Research Interests
• Discriminative training with minimal human supervision – Weakly-supervised Object Detection
– Anomaly Detection
– Few-Shot Learning
– Domain Adaptation
• GAN-based Image Generation – GANs conditioned on structured input
– Improving GAN stability
![Page 6: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/6.jpg)
• My (general) research interests
• Backup: Hybrid biological-artificial networks
• Backup: Neuromorphic computing
Overview
![Page 7: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/7.jpg)
Goals: 1. Use an ANN to predict what a biological net "thinks"
2. Perform hybrid, joint artificial-biological computations
Hybrid biological-artificial nets
![Page 8: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/8.jpg)
Goals: 1. Use an ANN to predict what a biological net "thinks"
2. Perform hybrid, joint artificial-biological computations
Hybrid biological-artificial nets
![Page 9: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/9.jpg)
Predicting the behaviour of a biological network
• Can we read and predict what a brain thinks?
• There are some in-vivo experiments using human beings
9
![Page 10: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/10.jpg)
"Mind reading"
• Reconstructing the brain signal (e.g., fMRI) using a neural network
![Page 11: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/11.jpg)
Image reconstruction examples
![Page 12: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/12.jpg)
• Common problems of the brain-signal reconstruction approaches: – Voxels have a low spatial resolution
– Small datasets
• A possible alternative: in-vitro experiments
• Goal: – To access each biological neuron
– To collect large training datasets
Brain reading in the Backup project
![Page 13: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/13.jpg)
Photonic circuits and optogenetics used to stimulate and read biological neuron activations
Accessing individual neurons
![Page 14: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/14.jpg)
• Let B be the biological network with n neurons
• Let A be the ANN with m neurons
• A and B do NOT need to share the same structure
Biological net’s activation prediction: a possible schema
![Page 15: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/15.jpg)
• Let st be the light-based stimulus of B at time t
• E.g., st Rn represents the individual-neuron stimulation
• In a sparse stimulus, for most i (1 <= i <= n), st,i = 0
Biological net’s activation prediction: a possible schema
![Page 16: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/16.jpg)
• After a time delay (k), let rt+k, be the activation state of B, i.e., B’s "response" induced by st as measured at time t + k.
• rt+k Rn
• rt+k is the light-based “readout” of B
Biological net’s activation prediction: a possible schema
![Page 17: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/17.jpg)
• We can collect a virtually unlimited dataset
D = {(st , rt+k)}
• D is used to train A
Biological net’s activation prediction: a possible schema
![Page 18: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/18.jpg)
• A learns to predict rt+k from st
• Formally: A(st) = rt+k
• A is a "functional copy" (a backup…) of the memories of B
Biological net’s activation prediction: a possible schema
![Page 19: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/19.jpg)
Goals: 1. Use an ANN to predict what a biological net "thinks"
2. Perform hybrid, joint artificial-biological computations
Hybrid biological-artificial nets
![Page 20: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/20.jpg)
Hybrid computational systems
• Can we develop hybrid computational systems?
• Can we replace arbitrary parts of the biological network with an artificial network, still preserving the same functional behaviour?
![Page 21: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/21.jpg)
Hybrid computational systems: a possible schema
• B is arbitrarily split in two sub-nets
• Analogously:
st = (st(1), st
(2))
rt+k = (rt+k(1), rt+k
(2))
![Page 22: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/22.jpg)
• B1 is inhibited or removed
• B1 is replaced with an ANN A
• A is connected with B2
Hybrid computational systems: a possible schema
![Page 23: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/23.jpg)
• A and B2 exchange information
• Goal: B2 ‘s response (rt+k
(2)) should be statistically similar to what is obtained without amputation
Hybrid computational systems: a possible schema
![Page 24: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/24.jpg)
• My (general) research interests
• Backup: Hybrid biological-artificial networks
• Backup: Neuromorphic computing
Overview
![Page 25: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/25.jpg)
Neuromorphic Computing
Goal: to implement ANNs using photonic circuits
This is motivated by the much higher speed and lower power consumption of a photonic circuit w.r.t. an electric circuit
![Page 26: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/26.jpg)
Our current solutions
• MultiLayer Perceptron (MLP)
• Reservoir Computing
![Page 27: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/27.jpg)
Our current solutions
• MultiLayer Perceptron (MLP)
• Reservoir Computing
![Page 28: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/28.jpg)
MLP using silicon photonics
• Our MLP is based on a microring resonator whose (thermal) nonlinear response corresponds to the neuron activation function
![Page 29: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/29.jpg)
MLP using silicon photonics
• Only light intensity is used (no phase information)
![Page 30: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/30.jpg)
MLP using silicon photonics
• Constraints:
– All net’s weights and activations should be positive
– The sum of the weights associated with the connections exiting from a neuron should be limited by 1
![Page 31: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/31.jpg)
MLP using silicon photonics
• We solved this constrained-optimization problem using a technique called Projected Gradient Descent
• In each SGD step, w is projected onto the admissible area defined by our constraints.
![Page 32: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/32.jpg)
MLP using silicon photonics
Preliminary Results:
• We used a software-based simulation
• MNIST dataset (60,000 28X28 digit images)
• MLP structure: 784-200-10
• Our simulation: 92% accuracy
• Standard, non-constrained ANN (same structure): 97%
• MNIST “linear regime”: 88%
![Page 33: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/33.jpg)
Our solutions
• MultiLayer Perceptron (MLP)
• Reservoir Computing
![Page 34: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/34.jpg)
Reservoir Computing (RC): Introduction
• It is an RNN with:
– input-to-hidden layer weights randomly fixed
– hidden-to-hidden layer weights randomly fixed
– hidden-to-output layer weights (readout) learned
• Main advantage: it is easy to train
• Can be implemented using photonic circuits
![Page 35: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/35.jpg)
Reservoir Computing (RC): Introduction
![Page 36: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/36.jpg)
• Sparsity is obtained by setting most of the connection weights to 0:
Reservoir Computing (RC): Introduction
![Page 37: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/37.jpg)
• Difference with a standard RNN:
– Testing: no difference
– Training: only Who needs to be trained
Reservoir Computing (RC): Introduction
![Page 38: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/38.jpg)
RC: Our solution
• Our trainable last layer (Who) is a Perceptron
• Specifically, using both phase and intensity, we have complex-valued activations and weights (CV-Perceptron)
![Page 39: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/39.jpg)
Our CV-Perceptron
• The output neuron activation function is the squared light intensity
• All computations, except the last activation function are performed using photonic circuits
• This is different from common implementations [1], in which Who ht is computed electronically
[1] Amin et al., Single-layered complex-valued neural network for real-valued classification problems. Neurocomputing (72) 2009
![Page 40: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/40.jpg)
Our CV-Perceptron: Simulation Results
Real datasets (normalized test error):
Symmetry detection (@ 0 error):
[1] Amin et al., Single-layered complex-valued neural network for real-valued classification problems. Neurocomputing (72) 2009
Dataset Amin [1] (act-fun. 1)
Amin [1] (act-fun. 2)
RV Perceptron
Ours
Fisheriris 0.19 - 0.15 0.11
Diabetes 0.37 0.47 0.23 0.27
Cancer 0.026 0.025 0.014 0.08
Dataset Amin [1] (act-fun. 1)
Amin [1] (act-fun. 2)
RV Perceptron
Ours
Longest seq. 7 6 1 6
![Page 41: Backup: Machine Learning - UniTrento Enver Sangineto... · 2020. 6. 9. · Backup: Machine Learning ... –Weakly-supervised Object Detection –Anomaly Detection –Few-Shot Learning](https://reader034.vdocuments.us/reader034/viewer/2022051908/5ffc7f82135b346db7563215/html5/thumbnails/41.jpg)
Thank you!
Questions?